To remove a member from a styleguide, you'll need to visit the "About" page of your styleguide and then click on the member avatars from the right panel to open the Members list. From this list, click on the "..." options menu next to the name and select "Remove from styleguide."
☝️ To remove a user from your invite suggestions, the user's membership should be removed from all projects and styleguides (including both active and archived ones).