pesser / edflow

Reduce boilerplate code for your ML projects. TensorFlow and PyTorch.
https://edflow.readthedocs.io/
MIT License
24 stars 13 forks source link

Remove fastnumbers fast_int because of occational error #235

Closed hperrot closed 4 years ago

hperrot commented 4 years ago

Mentioned error looks like:

Exception in thread Thread-6:                                                                                                                                                               | 0/158 [00:00<?, ?it/s]
Traceback (most recent call last):                                                                                                                                                         | 0/158 [00:00<?, ?it/s]
  File "/export/home/hperrot/miniconda3/envs/ffg/lib/python3.6/threading.py", line 916, in _bootstrap_inner                                                                                                        
    self.run()                                                                                                                                                                                                     
  File "/export/home/hperrot/miniconda3/envs/ffg/lib/python3.6/threading.py", line 864, in run                                                                                                                     
    self._target(*self._args, **self._kwargs)                                                                                                                                                                      
  File "/export/home/hperrot/miniconda3/envs/ffg/lib/python3.6/site-packages/chainer/iterators/multiprocess_iterator.py", line 435, in fetch_batch                                                                 
    batch_ret[0] = [self.dataset[idx] for idx in indices]                                                                                                                                                          
  File "/export/home/hperrot/miniconda3/envs/ffg/lib/python3.6/site-packages/chainer/iterators/multiprocess_iterator.py", line 435, in <listcomp>                                                                  
    batch_ret[0] = [self.dataset[idx] for idx in indices]                                                                                                                                                          
  File "/export/home/hperrot/src/FlowFrameGen/src/edflow/edflow/data/dataset_mixin.py", line 172, in __getitem__                                                                                                   
    self._maybe_append_labels(ret_dict, i)                                                                                                                                                                         
  File "/export/home/hperrot/src/FlowFrameGen/src/edflow/edflow/data/dataset_mixin.py", line 185, in _maybe_append_labels                                                                                          
    update(datum, {"labels_": labels})                                                                                                                                                                             
  File "/export/home/hperrot/src/FlowFrameGen/src/edflow/edflow/util.py", line 598, in update                                                                                                                      
    walk(to_update_with, _update, splitval=splitval, pass_key=True)                                                                                                                                                
  File "/export/home/hperrot/src/FlowFrameGen/src/edflow/edflow/util.py", line 164, in walk                                                                                                                        
    result = call(key, val) if pass_key else call(val)                                                                                                                                                             
  File "/export/home/hperrot/src/FlowFrameGen/src/edflow/edflow/util.py", line 149, in call                                                                                                                        
    walk_np_arrays=walk_np_arrays,                                                                                                                                                                                 
  File "/export/home/hperrot/src/FlowFrameGen/src/edflow/edflow/util.py", line 164, in walk                                                                                                                        
    result = call(key, val) if pass_key else call(val)                                                                                                                                                             
  File "/export/home/hperrot/src/FlowFrameGen/src/edflow/edflow/util.py", line 152, in call                                                                                                                        
    return fn(key, value)                                                                                                                                                                                          
  File "/export/home/hperrot/src/FlowFrameGen/src/edflow/edflow/util.py", line 596, in _update                                                                                                                     
    set_value(to_update, key, value, splitval=splitval)                                                                                                                                                            
  File "/export/home/hperrot/src/FlowFrameGen/src/edflow/edflow/util.py", line 534, in set_value
    keys = [fast_int(k) for k in key.split(splitval)]
  File "/export/home/hperrot/src/FlowFrameGen/src/edflow/edflow/util.py", line 534, in <listcomp>
    keys = [fast_int(k) for k in key.split(splitval)]
SystemError: <built-in function fast_int> returned NULL without setting an error