Open yarikoptic opened 9 months ago
following advice in https://github.com/ome/ngff/issues/228#issuecomment-1957119902 let's :
.multiscales[].version
and .omero.version
.here is stats across zarrs on S3 - first one for .ome.version, 2nd for .multiscales[].version:
dandi@drogon:~$ sort /tmp/ome-versions.out | uniq -c
20 [0.2,0.2]
4303 ["0.4","0.4"]
572 [null,"0.4"]
where that file was created using for d in *-*-*; do git -C $d annex whereis .zattrs | awk '/versionId=/{print $2;}' | xargs curl --silent | jq -c '[.omero.version, .multiscales[].version]'; done | tee /tmp/ome-versions.out
note that some had it (incorrectly) as floats I think. Might be worth making code robust there and explicitly test for it being a string and otherwise issue validation error
@yarikoptic
well -- ideally cache locally. I thought we already do something similar to dandi schema or used to do for bids at some point.
"nearly-valid" I think most of zarrs in 000108 according to e.g. https://ome.github.io/ome-ngff-validator/?source=https://dandiarchive.s3.amazonaws.com/zarr/e41844a2-dad0-4b1c-9c53-d55883e0553f which errors with
{
"instancePath": "/omero/channels/0/window",
"schemaPath": "#/properties/omero/properties/channels/items/properties/window/required",
"keyword": "required",
"params": {
"missingProperty": "start"
},
"message": "must have required property 'start'"
}
but then otherwise is happy to report 7 Datasets checked ✓
.
I also found one in https://dandiarchive.org/dandiset/000243/draft/files?location=sub-S01%2Fanat&page=1 which fully valid https://ome.github.io/ome-ngff-validator/?source=https://dandiarchive.s3.amazonaws.com/zarr/7723d02f-1f71-4553-a7b0-47bda1ae8b42
also those in 000026 seems to be good: https://dandiarchive.org/dandiset/000026/draft/files?location=sub-I45%2Fses-SPIM%2Fmicr&page=1
A part inspired from
1281
and current problem of presumably having LOTS of non-compliant zarrs in the archive:
For any .zarr we encounter we should
.omero
top level key in .zattrsFor any OME .zarr (be either detected through above or having
.ome.zarr
extension), validate that zarr against the specified version (if no version -- validation error)of schema as provided on https://github.com/ome/ngff under
{schema_version}/schemas/
folder in.schema
json files and issue corresponding validation errors to the users trying to upload non-compliant OME .ngffs.