ebi-ait / data-catalogue-frontend

0 stars 0 forks source link

upload converted checklists in schema format a store #69

Open amnonkhen opened 2 weeks ago

amnonkhen commented 2 weeks ago

After a discussion with Wei, Amnon & Tony, we decided to keep using the existing schema-store app

one approved, copy all BioSamples converted json schema from checklist-converter/schema to repo. Even at their current state, they do not introduce any new problems that were not there in ena in the XML checklists.

DoD:

Wkt8 commented 1 week ago

Waiting on actions from Isuru

will import our checklist-converter code into this repository. Then all schema can be update with a one > API call. let you know once done that

theisuru commented 5 days ago

GET all_schema http://wp-np2-40:8080/biosamples/schema/store/api/v2/schemas/search

GET schema http://wp-np2-40:8080/biosamples/schema/store/api/v2/schemas/ERC000022/1.0.0

POST http://wp-np2-40:8080/biosamples/schema/store/api/v2/schemas BODY:

PUT http://wp-np2-40:8080/biosamples/schema/store/api/v2/schemas?id= BODY:

JSON_SCHEMA { id: "https://www.ebi.ac.uk/biosamples/schemas/ENA-GSC_MIxS_soil/1.0.0", accession: "ERC000022", name: "ENA-GSC_MIxS_soil", version: "1.0.0", title: "GSC MIxS soil", description: "Genomic.....", domain: "https://www.ebi.ac.uk/biosamples/schemas", metaSchema: "https://schemablocks.org/metaschemas/json-schema-draft-07/1.0.1", schema: , authority: "ENA" }

notes: domain - can be ignored or use existing value schema - generated JSON Schema

theisuru commented 5 days ago

Unfortunatly, there is a mismatch between biovalidator and schema-store running on BioSamples dev environment. This is not trivial as BioSamples is also using biovaliator. So need to check dependencies and update them in biovalidator, schema-store and biosamples. Since this can take some time, only option is to update in the mongodDB. Let me know if you need mongoDB credentials.

Edit: Current version of schema-store expects older version of the biovalidator. If you check its docker-compose file, you can see it is using image: ebispot/elixir-jsonsschema-validator:1.3.0. But currently deployed biovalidator is a newer version which has different input/output format. Therefore if you run schema-store in local machine with included docker-compose file and use POST/PUT endpoint to submit schema, it works. But if you tried to submit to BioSamples dev envrionment, it fails.

Wkt8 commented 5 days ago

I will need mongoDB credentials. I also do not know how to check the dependencies for biovalidator, schema-store and biosamples, but I can investigate.

theisuru commented 5 days ago

@Wkt8 I probably has to check dependency and update the code accordingly. I am guessing BioSamples is using bit older version of biovaidator, which has slightly different input/output formats. I will update code and dependencies.

If you can manually enter documents to Mongo, it will be great.

Wkt8 commented 5 days ago

The way in which schema documents are saved in github (https://github.com/ebi-ait/checklist-converter/blob/main/schema/ERC000011-BSD.json) is different from in MongoDB so we will have to do a lot of manual work or write a script

To-do:

  1. Get the json files from the checklist-converter repo, specifically the BSD json files
  2. Increment the json file version numbers by a minor number (1.0.0 --> 1.1.0)
  3. Wrap the schema into a format similar to MongoDB (strip the $ prefixes, wrap the schema entirely into the "schema" attribute as a string)
  4. Update the databse in MongoDB and watch the changes be propagated in dev (http://wp-np2-40:8080/biosamples/schema/store/schemas) - get MongoDB credentials from isuru
theisuru commented 2 days ago

These are now in dev environment.