Metadata rework - Githubissues

ome / ome2024-ngff-challenge

Project planning and material repository for the 2024 challenge to generate 1 PB of OME-Zarr data

https://pypi.org/project/ome2024-ngff-challenge/

BSD 3-Clause "New" or "Revised" License

11 stars 8 forks source link

Metadata rework #33

Closed joshmoore closed 3 weeks ago

joshmoore commented 3 weeks ago

Proposed changes from the 20240821 morning challenge chat:

[x] Move current code to a resave subcommand to allow future commands (see below)
[x] Make licenses "STRONGLY RECOMMENDED" (via warnings, etc.)
[x] Clearly identify the other fields as recommended (SHOULD) or optional (MAY)
[x] Remove default values everywhere

Follow-up work will include:

Subcommand for looking up organism and modality arguments (e.g., NCBI and FBbi terms)
Subcommand for validating datasets, especially that license is set
Subbcomand for updating metadata in datasets
Possibly add a free-text option for organism and modality

joshmoore commented 3 weeks ago

Thanks, @will-moore. Fixes pushed for most of the above. Another question I have is whether to move from --rocrate- to --metadata- everywhere.

will-moore commented 3 weeks ago

@joshmoore I don't mind rocrate or metadata, but I guess if the args are --rocrate- then maybe we should mention why "rocrate". Actually, the README could do with a short "Metadata" section that simply says "Metadata is written as Ro-Crate to ro-crate-metadata.json", with an example and say "See ome2024-ngff-challenge resave -h for more info (assuming we don't want to duplicate the help in the README).

E.g.

ome2024-ngff-challenge resave in.zarr out.zarr --cc-by --rocrate-organism=NCBI:txid10090 --rocrate-modality=obo:FBbi_00000251

will-moore commented 3 weeks ago

idr0010 plate exported with commit c122913 above: Strangely missing license from:

{
      "@id": "./",
      "@type": "Dataset",
      "resultOf": {
        "@id": "#9c74acf1-aa49-434a-9635-26ef724d9772"
      }
    },

https://deploy-preview-36--ome-ngff-validator.netlify.app/?source=https://uk1s3.embassy.ebi.ac.uk/idr/share/ome2024-ngff-challenge/0.0.5/idr0010/76-45.zarr

converted with:

time ome2024-ngff-challenge resave --input-bucket=bia-integrator-data --input-endpoint=https://uk1s3.embassy.ebi.ac.uk --input-anon S-BIAD885/0046b0d0-f20b-4482-84b1-4b2b154865fd/0046b0d0-f20b-4482-84b1-4b2b154865fd.zarr /data/will/idr0010/76-45.zarr --log debug --rocrate-modality=obo:FBbi_00000246 --rocrate-organism=NCBI:txid9606 --cc-by

But I can't reproduce that - working fine with other images and plates... 👍