Making certain categories of CPT not publicly queryable

I am developing a custom CMS for a client and one of the Custom Post Types is ‘contact’, these contacts are organized into categories by a custom taxonomy; my question is whether there is a way to set only certain categories to be publicly queryable. I only seem to be able to get solutions working where either make all posts of a certain post type non-public and not-publicly queryable or the other way around but not just those in a certain category.

Some additional information as suggested by @kaiser:
The custom taxonomy set up the same way normal categories are set up for posts and pages. It is just the one custom taxonomy which handles this aspect. I want to work with one custom post type only for this as i would otherwise be forced to duplicate a lot of custom fields, for example: all ‘contacts’ have a lot in common (contact details, etc..) but out of these ‘contacts’ i only want certain ‘categories’ to be visible and accessible from outside of admin; as an example, a contact with category ‘artist’ should show publicly and show up in category and sitemap pages but not those under ‘providers’.

I do not wish to use multiple CPTs or multiple taxonomies if i can avoid it.

Thank you all for your input.

Solutions Collecting From Web of "Making certain categories of CPT not publicly queryable"

Sounds like you really just need a filter on pre_get_posts to remove the unwanted terms. Something like this, but I am not sure I understand all the conditions you need so think of it as proof of concept only.

function exclude_terms_wpse_117242($qry) {
  if (is_user_logged_in()) {
    $tq = array(
            'taxonomy' => 'your_tax',
            'field' => 'id',
            'terms' => array(1,2,3),
            'operator' => 'NOT IN',