Sage-Bionetworks / schematic

Package for biomedical data model and metadata ingress management
https://schematicpy.readthedocs.io/en/stable/cli_reference.html
MIT License
22 stars 25 forks source link

Schematic submit endpoint returns "Lookup Error: XX could not be found in the asset view" when schema has error #1074

Closed linglp closed 1 year ago

linglp commented 1 year ago

Describe the bug The command that @andrewelamb ran was: schematic model -c ../schematic/config.yml submit -mp tcga_samples.csv -d syn50864448 -vc Samples (see the full error log here: https://sagebionetworks.slack.com/files/U6NBMENEP/F04JHCS19K8/untitled.txt) But when I investigated further, this seems to be related to schema error and also some logic in validation.

More context for debugging: If I print out the line here in validate_manifest_values function, errorColName returns "wrong schema". Then, the string "Wrong schema" gets fed into function generate_schema_error as attribute_name, and then the string got passed down to get_message_level function and is_node_required function. node_label = self.get_node_label(node_display_name) under is_node_required function returns empty because of the string "wrong schema".

I printed out the error after the following code:

            v = Draft7Validator(jsonSchema)
            for error in sorted(v.iter_errors(annotation), key=exceptions.relevance):

and here's the output of the error:

'Component' is a required property

Failed validating 'required' in schema:
    {'$id': 'http://example.com/Samples_validation',
     '$schema': 'http://json-schema.org/draft-07/schema#',
     'properties': {'Component': {'minLength': 1, 'not': {'type': 'null'}},
                    'sample_name': {'minLength': 1,
                                    'not': {'type': 'null'}}},
     'required': ['Component', 'sample_name'],
     'title': 'Samples_validation',
     'type': 'object'}

On instance:
    {'sample_name': 'TCGA-01-0628'} 

To Reproduce Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior A clear and concise description of what you expected to happen.

Priority (select one)

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (if applicable, please complete the following information):

Additional context Add any other context about the problem here.

GiaJordan commented 1 year ago

Fix is in progress but we were unable to add this item as priority last sprint due to other issues in progress @andrewelamb @MiekoHash

GiaJordan commented 1 year ago

@andrewelamb can you send me the data model you're using?

andrewelamb commented 1 year ago

@GiaJordan This is the one that caused the error at the time: https://sagebionetworks.slack.com/files/U6NBMENEP/F04K2SEL14Z/iatlas_schema.jsonld

andrewelamb commented 1 year ago

The most recent one is here: https://github.com/CRI-iAtlas/iAtlasSchema/blob/develop/iatlas_schema.jsonld

I think this one causes the same error

GiaJordan commented 1 year ago

thank you!

GiaJordan commented 1 year ago

@andrewelamb I was able to complete the validation for the manifest you provided without encountering the error you described while using the dev branch associated with this issue. You can test yourself if you like. To remedy the actual errors you were having, you'll need to provide the patients_id and Component columns, and then fill in the component column