search

usnistgov / jarvis

JARVIS-Tools: an open-source software package for data-driven atomistic materials design. Publications: https://scholar.google.com/citations?user=3w6ej94AAAAJ https://www.youtube.com/watch?v=2-XHeC8gbeY
https://pages.nist.gov/jarvis/
Other
313 stars 124 forks source link

Invalid OPTIMADE API responses #221

Open ml-evs opened 2 years ago

ml-evs commented 2 years ago

Describe the bug The responses from the JARVIS-DFT OPTIMADE API have several outstanding issues, as reported by the dashboard and pasted into this gist (https://gist.github.com/ml-evs/89f7bacac1c8a266074b82c89b78a11a).

I can't find a repo for the actual API (the code is not in this repo or https://github.com/usnistgov/JARVIS-DFT). Is there anywhere I can help fix this?

knc6 commented 2 years ago

Hi @ml-evs Thanks for checking this. The JARVIS web-app is maintained using a private repo and is not part of the jarvis-tools package necessarily. I have kept a copy of jarvis_optimade in the private repo: https://github.com/knc6/jarvis_optimade and added you as a collaborator. Please feel free to check out.

Currently, after a few bug fixes (not pushed to production yet), I see the following output

Testing entire implementation at http://localhost:8000/optimade/jarvisdft/v1
.....✖......✖✖✖.✖.✖✖✖✖✖✖✖✖✖✖✖✖✖......

FAILURES
========

http://localhost:8000/optimade/jarvisdft/v1 - _test_page_limit - failed with error
        ResponseError: Field `meta->more_data_available` was missing.
http://localhost:8000/optimade/jarvisdft/v1/structures?page_limit=5&response_fields=assemblies,chemical_formula_hill,last_modified,chemical_formula_descriptive,elements,dimension_types,structure_features,species_at_sites,nsites,nperiodic_dimensions,species,chemical_formula_anonymous,nelements,chemical_formula_reduced,cartesian_site_positions,elements_ratios,immutable_id,lattice_vectors - _deserialize_response - failed with error
        ValidationError: 1 validation error for ValidatorStructureResponseMany
        meta
          field required (type=value_error.missing)
http://localhost:8000/optimade/jarvisdft/v1/info/structures - _test_must_properties - failed with error
        ResponseError: Some 'MUST' properties were missing from info/structures: {'structure_features'}
http://localhost:8000/optimade/jarvisdft/v1/structures?filter=id STARTS "JVASP-13526" - _construct_queries_for_property - failed with error
        ResponseError: Required field `meta->more_data_available` missing from response for structures?filter=id STARTS "JVASP-13526".
http://localhost:8000/optimade/jarvisdft/v1/structures?filter=last_modified >= "2022-02-03T17:14:39.696262+00:00" - _construct_queries_for_property - failed with error
        ResponseError: Required field `meta->more_data_available` missing from response for structures?filter=last_modified >= "2022-02-03T17:14:39.696262+00:00".
http://localhost:8000/optimade/jarvisdft/v1/structures?filter=chemical_formula_reduced STARTS "Br3Y" - _construct_queries_for_property - failed with error
        ResponseError: Required field `meta->more_data_available` missing from response for structures?filter=chemical_formula_reduced STARTS "Br3Y".
http://localhost:8000/optimade/jarvisdft/v1/structures?filter=chemical_formula_anonymous STARTS "A3B" - _construct_queries_for_property - failed with error
        ResponseError: Required field `meta->more_data_available` missing from response for structures?filter=chemical_formula_anonymous STARTS "A3B".
http://localhost:8000/optimade/jarvisdft/v1/structures?filter=nelements = 16 - _construct_queries_for_property - failed with error
        ResponseError: Required field `meta->more_data_available` missing from response for structures?filter=nelements = 16.
http://localhost:8000/optimade/jarvisdft/v1/structures?filter=elements HAS ANY "Br","Y" - _construct_queries_for_property - failed with error
        ResponseError: Required field `meta->more_data_available` missing from response for structures?filter=elements HAS ANY "Br","Y".
http://localhost:8000/optimade/jarvisdft/v1/structures?filter=nsites = 16 - _construct_queries_for_property - failed with error
        ResponseError: Required field `meta->more_data_available` missing from response for structures?filter=nsites = 16.
http://localhost:8000/optimade/jarvisdft/v1/; testing queries for structures->_jarvisdft_optb88vdw_bandgap - _construct_queries_for_property - failed with error
        ResponseError: Found unknown field '_jarvisdft_optb88vdw_bandgap' in `/info/structures` and no provider prefix was provided in `/info`
http://localhost:8000/optimade/jarvisdft/v1/; testing queries for structures->_jarvisdft_mbj_bandgap - _construct_queries_for_property - failed with error
        ResponseError: Found unknown field '_jarvisdft_mbj_bandgap' in `/info/structures` and no provider prefix was provided in `/info`
http://localhost:8000/optimade/jarvisdft/v1/; testing queries for structures->_jarvisdft_formation_energy_peratom - _construct_queries_for_property - failed with error
        ResponseError: Found unknown field '_jarvisdft_formation_energy_peratom' in `/info/structures` and no provider prefix was provided in `/info`
http://localhost:8000/optimade/jarvisdft/v1/; testing queries for structures->_jarvisdft_stability - _construct_queries_for_property - failed with error
        ResponseError: Found unknown field '_jarvisdft_stability' in `/info/structures` and no provider prefix was provided in `/info`
http://localhost:8000/optimade/jarvisdft/v1/; testing queries for structures->_jarvisdft_spacegroup - _construct_queries_for_property - failed with error
        ResponseError: Found unknown field '_jarvisdft_spacegroup' in `/info/structures` and no provider prefix was provided in `/info`
http://localhost:8000/optimade/jarvisdft/v1/structures?filter=_crazyprovider_field=2 - _deserialize_response - failed with error
        ValidationError: 1 validation error for ValidatorStructureResponseMany
        meta
          field required (type=value_error.missing)

OPTIONAL TEST FAILURES
======================

http://localhost:8000/optimade/jarvisdft/versions - _test_versions_headers - failed with error
        ResponseError: Incorrect 'Content-Type' header 'text/plain'.
        Missing at least one expected parameter(s): ['header=present']
http://localhost:8000/optimade/jarvisdft/v1/structures?filter=species_at_sites HAS ANY "Br","Y" - _construct_queries_for_property - failed with error
        ResponseError: Required field `meta->more_data_available` missing from response for structures?filter=species_at_sites HAS ANY "Br","Y".

Passed 17 out of 33 tests.
Additionally passed 2 out of 4 optional tests.
ml-evs commented 2 years ago

Hi @knc6, this is causing some issues again for us as we try to integrate the NIST Materials Resource registry with OPTIMADE. Have the changes above been pushed to production yet? The main thing we are currently running into is the lack of meta and links fields in the structure responses, which means you cannot paginate through the results at all.

knc6 commented 2 years ago

Hi @ml-evs , we just pushed the recent updates to the server.

ml-evs commented 2 years ago

Hi @knc6, looks like some of the problems I mentioned have been fixed (around the static metadata) but pagination does not work (and filtering is not implemented?). Perhaps you could take a look at the PR I made at https://github.com/knc6/jarvis_optimade/pull/1/?

ml-evs commented 2 years ago

Here's what I see with the latest validator:

$ optimade-validator https://jarvis.nist.gov/optimade/jarvisdft/v1
Testing entire implementation at https://jarvis.nist.gov/optimade/jarvisdft/v1
............✖✖✖✖.✖..✖✖✖.✖.✖......

FAILURES
========

https://jarvis.nist.gov/optimade/jarvisdft/v1 - _test_page_limit - failed with error
        ResponseError: Endpoint did not obey page limit: 5 entries vs 4 limit
https://jarvis.nist.gov/optimade/jarvisdft/v1/structures?page_limit=4&response_fields=structure_features,dimension_types,elements_ratios,nelements,assemblies,immutable_id,chemical_formula_descriptive,nsites,chemical_formula_hill,nperiodic_dimensions,last_modified,species,lattice_vectors,species_at_sites,chemical_formula_anonymous,elements,chemical_formula_reduced,cartesian_site_positions - _deserialize_response - failed with error
        ValidationError: 1 validation error for ValidatorStructureResponseMany
        meta -> api_version
          Unable to validate the version string '1.1' as a semantic version (expected <major>.<minor>.<patch>).See https://semver.org/#is-there-a-suggested-regular-expression-regex-to-check-a-semver-string for more information. (type=value_error)
https://jarvis.nist.gov/optimade/jarvisdft/v1/info/structures - _test_must_properties - failed with error
        ResponseError: Some 'MUST' properties were missing from info/structures: {'structure_features'}
https://jarvis.nist.gov/optimade/jarvisdft/v1/structures?response_fields=species,chemical_formula_anonymous,_jarvis_reference&page_limit=1 - _check_response_fields - failed with error
        ResponseError: Response fields not obeyed by 'structures':
        Expected: {'species', 'chemical_formula_anonymous', '_jarvis_reference'}
        Returned: {'_jarvis_source', 'nperiodic_dimensions', 'structure_features', 'dimension_types', 'last_modified', 'species', 'elements_ratios', '_jarvis_reference', 'chemical_formula_descriptive', 'lattice_vectors', 'nelements', 'species_at_sites', 'chemical_formula_anonymous', 'elements', 'chemical_formula_reduced', 'cartesian_site_positions', 'nsites'}
https://jarvis.nist.gov/optimade/jarvisdft/v1/structures?filter=id < "JVASP-6838" - _construct_queries_for_property - failed with error
        ResponseError: Entry JVASP-6838 with value 'id': JVASP-6838 was not excluded by 'id < "JVASP-6838"'
https://jarvis.nist.gov/optimade/jarvisdft/v1/structures?filter=chemical_formula_reduced < "GaTe" - _construct_queries_for_property - failed with error
        ResponseError: Entry JVASP-6838 with value 'chemical_formula_reduced': GaTe was not excluded by 'chemical_formula_reduced < "GaTe"'
https://jarvis.nist.gov/optimade/jarvisdft/v1/structures?filter=chemical_formula_anonymous < "AB" - _construct_queries_for_property - failed with error
        ResponseError: Entry JVASP-6838 with value 'chemical_formula_anonymous': AB was not excluded by 'chemical_formula_anonymous < "AB"'
https://jarvis.nist.gov/optimade/jarvisdft/v1/structures?filter=nelements < 4 - _construct_queries_for_property - failed with error
        ResponseError: Entry JVASP-6838 with value 'nelements': 4 was not excluded by 'nelements < 4'
https://jarvis.nist.gov/optimade/jarvisdft/v1/structures?filter=nsites < 4 - _construct_queries_for_property - failed with error
        ResponseError: Entry JVASP-6838 with value 'nsites': 4 was not excluded by 'nsites < 4'
https://jarvis.nist.gov/optimade/jarvisdft/v1/structures?filter=_crazyprovider_field=2 - _deserialize_response - failed with error
        ValidationError: 1 validation error for ValidatorStructureResponseMany
        meta -> api_version
          Unable to validate the version string '1.1' as a semantic version (expected <major>.<minor>.<patch>).See https://semver.org/#is-there-a-suggested-regular-expression-regex-to-check-a-semver-string for more information. (type=value_error)