search

EOL / tramea

A lightweight server for denormalized EOL data
Other
2 stars 1 forks source link

[3] Better data structure returned by Pages API batch mode: hash not array #232

Open hyanwong opened 8 years ago

hyanwong commented 8 years ago

As discussed in gitter. At the moment a pages API batch mode call (https://github.com/EOL/tramea/issues/40) returns an array of hashes (dictionaries), with each containing a single item [{ID1:{data}}, {ID2:{data}}]. I suggest we should simply return a single dictionary, keyed by page id, i.e. {ID1:{data}, ID2:{data}}. This also prevents the JSON array return vulnerability too (http://stackoverflow.com/questions/3503102/what-are-top-level-json-arrays-and-why-are-they-a-security-risk)

In addition, we should ensure that the ID keys are identical to the IDs passed in to the API. At the moment, where multiple IDs map to the same page, the API returns the lowest value. So if you pass in e.g. id=2201694,... the return value is [{1049441:{"identifier": 1049441,...}},{...}]. I suggest for this example we should return {2201694:{"identifier": 1049441,...}} which will tell users that the id maps to a new value.

hyanwong commented 8 years ago

@JRice likes this :)

hyanwong commented 8 years ago

NB, although not backwards compatible, the batch mode is very new, and afaik I'm the only one really using it, so this cane could probably be pushed without bumping the API number (although we should probably notify @iimog and Jorrit)

hyanwong commented 8 years ago

Any ETA on this one

jhammock commented 8 years ago

good question. We've taken a break from data services to put out fires, resurrect harvesting, and make progress on v3. We should be able to pick up one or two. Is this your top item in data services, @hyanwong ? @jhpoelen what's your favorite at the moment?

hyanwong commented 8 years ago

I think this is an easy one, but the sooner the better, before anyone else starts using the batch API. I suspect mine (and @jhpoelen 's) top item is database dumps (e.g. the id mapping that @JRice managed to get working, and the one I've just suggested in gitter about taxa with at least one image).

jhpoelen commented 8 years ago

@jhammock @hyanwong If I would have to choose, I'd pick https://github.com/EOL/tramea/issues/162 - taxon hierarchies dump including related page id, external taxon ids (e.g. itis ids), ranks, common names and a thumbnail image. An example can be found here: https://github.com/jhpoelen/fishbase_taxon_cache .

jhammock commented 7 years ago

I believe the work on this is complete. @JRice is it awaiting deploy?

hyanwong commented 7 years ago

If someone could post here when it is live on the main site, that would be great, as I'll need to switch my JSON code to conform to the new format.