search

TimDettmers / spodernet

NLP
MIT License
14 stars 10 forks source link

ValueError: low >= high while enumerate train_batcher #1

Open gsp2014 opened 7 years ago

gsp2014 commented 7 years ago

I got the following error in the code: "for i, str2var in enumerate(train_batcher):"

Exception in thread Thread-4: Traceback (most recent call last): File "/home/anaconda3/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/home/Pytorch_MLP/src/spodernet/spodernet/preprocessing/batching.py", line 154, in run start = self.rdm.randint(0, n-self.stream_batcher.batch_size+1) File "mtrand.pyx", line 988, in mtrand.RandomState.randint ValueError: low >= high

Can you give me some advice? Thanks a lot!

TimDettmers commented 5 years ago

I did not see this and I believe this is no longer helpful, but it might be helpful for some people in the future: The error appears if your dataset (validation and test set in particular) is smaller than the batch size times the number of thread_loaders, so both reducing the number of threads and the batch size (in particular for evaluation) should help.