implement facet search

[vlandham]

Power tool component of client requires more sophisticated search capabilities.

Current strategy for dealing these searches:

There are 3 facets to orient search around:

• location
• client isp
• transit (server) isp

Each facet allows for searching of all 3 entities but with different constraints.

• location facet allows for searching of client isp and transit isp limiting the results of these searches to only include client isps and transit isps found in this location
• client isp likewise allows searching for location and transit isp with locations only listing locations the selected client isps are active in. And only transit isps with data coming from the selected client isp(s)
• transit isp allows searching for location and client isp. This is the location from data from which client isps come from that terminate with this transit isp. So this is still client location. Also, client isps are filtered by showing only client isps that have data terminating with this transit isp.

  Filters

With the facets Here are the different entity/filtering combinations we have:

• location with no filters
• location filter by client isp
• location filter by transit isp
• client isp with no filters
• client isp filter by location
• client isp filter by transit isp
• transit isp with no filters
• transit isp filter by location
• transit isp filter by client isp

A tradeoff with this approach limits us to filtering by a single entity. This works well with our facet type UI - but means that client isp, for example, could only be filtered by location - not location and transit isp

Possible Implementation

We have an existing solution to filtering by one entity in place:

client_location_client_asn is a table with a compound key of 2 values:

• location_key
• client_asn_number

This provides a solution for filtering a client isp by a location. If we expand this solution to include the additional filtering we need, it would involve making these tables:

• client_location_search ✔️
  • keys: reverse_location_key
• client_location_client_asn ✔️
  • keys: location_key, client_asn_num
• client_location_transit_asn
  • keys: location_key, transit_asn_num
• client_asn_search ✔️
  • keys: client_asn_name_lookup
• client_asn_client_loctation
  • keys: client_asn_num, client_location
• client_asn_transit_asn
• transit_asn_search
• transit_asn_client_location
• transit_asn_client_asn

where ✔️ indicates already existing tables.

Our current implementation of location_search only provides starts with name matching - due to bigtable query capabilities. This is a limitation of client_asn_search This would probably be a limitation for transit_asn_search as well.

For filtering. the keys are all built from asn keys and location keys. So the easiest thing to do would be to pass all results matching a filter (for example, all transit isps with client location of 'new york') to the api. If the resulting dataset was large, further filtering could occur at the api level (example: only pull out values with name containing 'lev').

The UI has a need for allowing multiple values in the initial filter. We will use these as union values for the filterable dependent entities.

For example. On facet by location, new york and london are selected. Now the user is selecting a client_isp. The API in this scenario must return client isps that appear in new york or london.

This could be implemented at the API level by acquiring both client isp lists from bigtable, and then performing the union there.

[pbeshai]

Thanks for writing this up Jim, looks good to me :+1:

[vlandham]

done.