search

NCBI-Hackathons / Metadata_categorization

A crowdsourcing/expert curation platform for metadata categorization.
Creative Commons Zero v1.0 Universal
5 stars 0 forks source link

Fix broken queues by not requiring Solr fields to be in doc when empty #29

Closed eweitz closed 8 years ago

eweitz commented 8 years ago

Adding a "note" value in the UI breaks the queue. Upon page refresh, the queue displays a 500 Internal Server error, e.g. KeyError at /queue/7: 'note'. The error makes further work on the affected queue impossible.

This critical bug is caused by the Solr annotation core not using a kludge established at my request during the hackathon: using "0" to indicate when a field has no value. (For development speed in the web backend, it was quicker to have empty fields present in the Solr JSON than to omit them. Due to a nuance of Solr, empty strings were not possible to use as values, so we settled on "0".) Specifically, this bug is caused by the Solr core simply omitting the "note" field when it has no value. Although the bug could be fixed by using "0" in Solr docs' empty note field like we do with all other fields, the problem could also be fixed by abandoning that peculiar convention.

Given that, update the Django web backend to not require empty fields to be present in the JSON responses from Solr. Such a change would not only fix this critical bug, but also avoid the need for inconvenient repopulations (e.g. https://github.com/NCBI-Hackathons/Metadata_categorization/issues/26#issuecomment-191916395).

@lepons, if you see any problems with me getting rid of the need for <fieldName>: "0" in Django, please let me know.

eweitz commented 8 years ago

Fixed. @lepons, this eliminates the need for "0" in empty Solr fields.