Closed nsheff closed 6 months ago
This pipestat sachems were updated by @donaldcampbelljr , and I believe they are working and up-to-date
I believe I only updated the pipeline names but did not touch the actual schema fields as they were working during testing with the new pipestat. Pipestat should raise a schema error if the required keys are not present for complex types.
I thought the pipestat schema format changed like this:
I see now. It was not actually updated to be json-schema compatible.
I think we need to change the pipestat schema format to follow json-schema...
The reason I think this is that I think we may be able to use jsonschema functionality for pydantic model creation that will simplify things with bedhost...
See:
@donaldcampbelljr pipestat schemas are now json schemas, right?
Pipestat can accept either the old way or a JSON schema, yes.
However, after our discussion ~ last week, I realized we are still using types such as file and image within the JSON pipestat schema (and later converting them to be objects).
Therefore the schema is still not actually 100% JSON schema: https://json-schema.org/understanding-json-schema/reference/type
I thought we decided on doing:
type: object
object_type: file
that makes it a valid json-schema, but allows our type-specific treatment.
Close, but right now its:
type: file
object_type: file
Pipestat still uses the type
field and if its an image or a file, recursively replaces that with title
and path
and changes the type to object.
Current bbconf example from dev:
pipeline_name: bedfile
samples:
name:
type: string
description: BED file name
genome:
type: object
description: genome assembly of the BED files
bedfile:
type: file
pipestat_type: file
label: bed
description: BED file
bigbedfile:
type: file
pipestat_type: file
label: bigBed
description: bigBed file
that's the old format, not a json schema at all
Yes, pipestat can accept either format currently. However, I'm just showing how the pipestat_type was implemented along with the original type.
We need to update those bbconf schemas to use the new format, and deprecate the old.
Issue raised: https://github.com/databio/bbconf/issues/32
Relevant issues:
Also, eido schemas: https://eido.databio.org/en/latest/writing-a-schema/
Code doing the replacing:
Replacements defined here:
Fixed in 0.4.0 release
There are several schemas here:
https://github.com/databio/bbconf/tree/dev/bbconf/schemas
I believe these are pipestat schemas. But when pipestat was updated, the schema definition changed.
bbconf has I believe been updated to use the new pipestat.
Are these schemas up-to-date?