search

src-d / datasets

source{d} datasets ("big code") for source code analysis and machine learning on source code
Other
323 stars 82 forks source link

Too many open files #35

Closed zurk closed 6 years ago

zurk commented 6 years ago

When I try to download PGA dataset via

./multitool get-index > index.csv
cat index.csv | grep -oE '[0-9a-f]{40}\.siva' | ./multitool get-dataset -o /media/k/data/PGA/

I get an error 

 90225 / 257391 [=====================>----------------------------------------]
Error: failed to create /media/k/data/PGA/siva/latest/87/870918f74af1095098b84eaf57765bb4e2a2f2d2.siva: open /media/k/data/PGA/siva/latest/87/870918f74af1095098b84eaf57765bb4e2a2f2d2.siva: too many open files

I know that I can change the limit of open files, I do so, but maybe there is a bug.

vmarkovtsev commented 6 years ago

yes this is a known problem, try pga instead