investigate if API code can be simplified

[vineetbansal]

From @msalganik on April 10, 2018 15:32

SQL -> JSON connector is this possible? desirable?

_Copied from original issue: fragilefamilieschallenge/metadataapi#13

[vineetbansal]

Assume is http://api.metadata.fragilefamilies.princeton.edu

@msalganik We talked to two functions on the api:

1. Get all or a specific attribute of a variable, given it's name This most naturally maps to urls of the form:

   <api_site>/variable/<name>
   e.g.
   <api_site>/variable/ce3datey

   and

   <api_site>/variable/<name>?<field>
   e.g
   <api_site>/variable/ce3datey?data_source

The rationale of having variable in the path is that it makes it explicit that we're interested in a "variable" (as opposed to a "topic" or an "umbrella"), and leaves us the flexibility of providing similar paths to these other entities in the future using a similar scheme, if we choose to.

The rationale of having <field> as a query string parameter (instead of something like ?fieldName=data_source is that it lends itself naturally to supporting multiple fields, e.g.

<api_site>/variable/ce3datey?data_source&data_type

2. Search for a variable given search parameters.

I like Flask-Restless's approach to this, where your search queries would be:

<api_site>/variable?q={"filters":[{"name":"scope","op":"eq","val":20}]}
<api_site>/variable?q={"filters":[{"name":"data_type","op":"eq","val":"bin"}]}
<api_site>/variable?q={"filters":[{"name":"data_type","op":"like","val":"b%"}]}
<api_site>/variable?q={"filters":[{"name":"data_type","op":"like","val":"%b"}]}

Although a little verbose, the 'q' in the query string makes it clear that we wish to search. The 'filters' for our search would be a list of dictionaries (each having name/op/val). Having a list means that the design is extensible if down the road we wish to incorporate narrowing down the query by providing multiple filters (which would combine using AND):

<api_site>/variable?q={"filters":[{"name":"data_type","op":"eq","val":"bin"}, {"name":"scope","op":"eq","val":20}]}

I do like separating the operator op (your original suggestion) instead of merging it with the val (as I suggested in our discussion earlier), which leaves open the possibility of adding operators like 'neq' (not equal), 'lt' (less than), 'gt' (greater than) etc.

Note that we get almost all of this for free right away with Flask-Restless, though we may wish to expose only a subset of things for now to keep the api simple, as long as we agree on the general format above.

[vineetbansal]

If (2) above is too far-fetched for our use case, then for searches we can go with simple search patterns specified by 'name' and 'val' (with val being either exact or fuzzy) like:

/variable?name=data_type&val=bin (searches for 'bin' in the data_type field for variables) /variable?name=label&val=%father% (searches for variables with 'father' somewhere in the label field) /variable?name=topic&val=paradata (searches for variables with topic=paradata)

[vineetbansal]

From @msalganik on June 1, 2018 19:32

@vineetbansal I talked about this with @katejaeger @atkindel and @thartshorne. We all like the new style (the one proposed in your original post) much better than what we have now. Thanks.

The next step seem to be to implement it. Are you suggesting that we use Flask-Restless or just copy their syntax? Can you explain the trade-offs of using Flask-Restless versus us just doing it ourselves?

[vineetbansal]

For now, I'd suggest we copy the syntax, with support for operators 'eq' and 'like'. Flask-Restless hasn't been updated since Feb 2015, which is not a good sign.

[vineetbansal]

From @msalganik on June 7, 2018 3:6

@vineetbansal OK sounds good.

[vineetbansal]

The new API is now in effect. We can create/fix issues on an individual basis as they arise.