search

AllenCellModeling / datasetdatabase

Modeling DB Schema, Creation, and IO
Other
1 stars 0 forks source link

get_dataset crashes for non-existing dataset #25

Closed vianamp closed 6 years ago

vianamp commented 6 years ago

Hi guys.

When I do prod.get_dataset(x) with x corresponding to a non-existing dataset (e.g. x=1), I get this error: 

Reconstructing dataset...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/matheus.viana/anaconda3/envs/qcb/lib/python3.6/site-packages/datasetdatabase/core.py", line 672, in get_dataset
    db=self.db, ds_info=ds_info)
  File "/home/matheus.viana/anaconda3/envs/qcb/lib/python3.6/site-packages/datasetdatabase/introspect/dictionary.py", line 156, in reconstruct
    bar = ProgressBar(len(iota_groups))
  File "/home/matheus.viana/anaconda3/envs/qcb/lib/python3.6/site-packages/datasetdatabase/utils/progressbar.py", line 26, in __init__
    self.update_bar_str()
  File "/home/matheus.viana/anaconda3/envs/qcb/lib/python3.6/site-packages/datasetdatabase/utils/progressbar.py", line 57, in update_bar_str
    complete = self.current / float(self.max)
ZeroDivisionError: float division by zero

Instead of crashing badly like this, is there a way to return something that we can work on? For instance a None value.

Also, is there a function for previewing the size of a dataset before actually downloading it?

evamaxfield commented 6 years ago

Is this a local database or prod?

I ask because there is a known issue with local databases taking two cycles to fully initialize that I am looking it. This would cause this same error.

vianamp commented 6 years ago

It is prod.

evamaxfield commented 6 years ago

Ahhh okay, I will patch through a "No Dataset with that Id" change as well as I just found the fix for the local database issue. This will be in v1.0.10 as the ingest changes also need to go through. Probably by end of day.

evamaxfield commented 6 years ago

Actually, your issue isn't that the dataset doesn't exist. I forgot but I had already added a DATASET_NOT_FOUND error. Your issue was due to the same issue plaguing local which is that you attempted to get_dataset(1). The first dataset to enter any DSDB will always be an empty algorithm parameters dictionary. Because it is empty when it is attempt to reconstruct it fails because division by zero (zero keys means zero items to iterate through).