Search + Taxonomy

[kinlane]

This has been migrated from Ohana (https://github.com/codeforamerica/ohana-api/issues/417#issuecomment-295782865), and worth considering as part of specification:

As part of Code for Miami's involvement with the Knight Cities Challenge, we recently conducted a Civic User Testing Group in Miami with a reskinned version of the Ohana web search. You can find our thoughts here.

We then built a follow-up prototype and performed CUTGroup testing for that prototype as well.

While developing this follow up project, we noticed the API is not configured to return taxonomy data in a keyword search. To get around this, our app ran an initial keyword search to get category data, iterate through each result, and then a second search for that particular organization’s details.

As a result, if a search for the keyword “food” returns 50 organizations, the app must make 51 calls to the API (one for the initial search, and one for each of the 50 organizations in the results). As a result, the app suffered from noticeable performance issues that would not be sustainable at scale.

We can solve this in two ways:

Refactor the API to return taxonomy data in keyword search results; or Create a second API that allows third parties to copy the vendor provider’s entire data set into their database, which they can then customize for more efficient searches.

[greggish]

FYI, here is the CUTGroup report, and here is the analysis from the front-end development test.

There was some back and forth about this in the Ohana issue thread, but I don't think the issue was resolved. cc @ErnieAtLYD @davidjamesknight @monfresh

[kinlane]

I'm proposing that we leave taxonomy out of v1.1 of the specification, and save it for consideration in v1.2 or beyond.

While the schema is straightforward, I think the access around this needs more discussion and thought -- also keeping v1.1 a simpler release.

Schema - https://openreferral.readthedocs.io/en/latest/reference/#taxonomy

[NeilMcKLogic]

To me this just illuminates the need to be careful about what to return (and not return) in search results so that a multitude of additional round trips through the API, in practical applications, are not necessary. The spec should assert an "opinion" about that so all implementers are aware of this and expose methods that are practically both useful and scalable.

[kinlane]

@NeilMcKechnie agreed. So much to think about when loading up a single API path, or spreading out across many specially designed API paths. Key is how to keep intuitive and simple, but allowing consumers to easily scratch and get more. Allow for wide use cases from bulk upload system to system, all the way to single phone number lookup using Amazon Alex. I'd like to see simple all the way to complex views of /search/, as well as taxonomy based dimensions where you can quickly filter, and navigate a database based upon many different taxonomies. I'd like to get a coherent /search plan articulated as part of next version v1.2.

[kinlane]

Another data point I'd normally have from existing API management operations, and would like to gather from this group is mobile vs web, and increasingly messaging (slack, fb), and voice (alexa). With web connectivity returns everything someone desires is not really a problem, offloading on the web interface to sort it out. With mobile, returning everything becomes a problem. We need a precise schema to minimize network latency. The same is going to be true for the newer channels like messaging and voice.

[NeilMcKLogic]

@kinlane , so where are we landing on this very important topic?

[dsurrao]

Is Taxonomy ready to try out in the v1.2 draft API? https://openreferral.github.io/api-specification/hsda-taxonomy/

[kinlane]

The taxonomy has been separated out into its own service, that works with HSDA v1.2. The definition has been reset has v1.0, and is up for feedback.

It is currently in a feedback phase, so I'd give another 30 days before it is ready, but I will be moving forward the demo code alongside the feedback, and happy to ping you when in a functional state (https://github.com/human-services/hsda-taxonomy).

[mrshll1001]

We will be archiving this repository soon. We're closing this issue because we believe it has been conclusively addressed as part of work on the main HSDS standard.

In this case, HSDS 3.0's API specification provides endpoints for taxonomies and taxonomy terms; they also have search parameters on the endpoint, too.