Sage-Bionetworks / schematic

Package for biomedical data model and metadata ingress management
https://schematicpy.readthedocs.io/en/stable/cli_reference.html
MIT License
22 stars 26 forks source link

Improve handling of annotation type changes #401

Closed allaway closed 6 days ago

allaway commented 3 years ago

Describe the bug When updating an annotation, if the new annotation is of a different type than the one currently on Synapse, synapseclient throws an error that causes the app to crash. I am posting this here because I assume it should be addressed at the schematic level, rather than just improved error handling in the shiny app.

I think this could be an issue in scenarios such as:

To Reproduce Steps to reproduce the behavior:

Expected behavior I would expect error handling for this error, or preferably, the opportunity to overwrite the existing annotation on synapse.

Screenshots sample error log on attempted annotation type change where initial value was nan, synapse interprets as double:

Parsed with column specification:
cols(
  .default = col_character(),
  individualID = col_double(),
  genePerturbationType = col_logical(),
  genePerturbationTechnology = col_logical(),
  genePerturbed = col_logical(),
  specimenID = col_double(),
  experimentalTimepoint = col_logical(),
  timePointUnit = col_logical(),
  readDepth = col_logical(),
  readLength = col_logical()
)
See spec(...) for full column specifications.
Warning: Error in py_call_impl: SynapseHTTPError: 400 Client Error: 
Value at [0,22] was not a valid DOUBLE. For input string: "Testing that my changes worked"

Detailed traceback: 
  File "/home/rallaway/ShinyApps/NF_data_curator_staging/schematic/schematic/synapse/store.py", line 461, in associateMetadataWithFiles
    self.syn.store(Table(existingTableId, manifest, etag = existing_results.etag))
  File "/home/rallaway/miniconda3/envs/data_curator_env/lib/python3.7/site-packages/synapseclient/client.py", line 918, in store
    return obj._synapse_store(self)
  File "/home/rallaway/miniconda3/envs/data_curator_env/lib/python3.7/site-packages/synapseclient/table.py", line 1773, in _synapse_store
    return copied_self._update_self(syn)
  File "/home/rallaway/miniconda3/envs/data_curator_env/lib/python3.7/site-packages/synapseclient/table.py", line 1790, in _update_self
    linesToSkip=self.linesToSkip)
  File "/home/rallaway/miniconda3/envs/data_curator_env/lib/python3.7/site-packages/synapsecli [... truncated]
  76: <Anonymous>

Execution halted

Desktop (if applicable, please complete the following information): n/a

Additional context Add any other context about the problem here.

ychae commented 3 years ago

@allaway is this still an issue and how urgent is this fix if it is?

allaway commented 3 years ago

Yes, this is still an issue, but I don't think it's that urgent: Most (all?) of the time I've run into this, it's been caused by #402, so resolution of that would fix this (at least, it would fix where it's obvious, but I think this still might be a good QoL improvement).