search

HumanCellAtlas / dcp-cli

DEPRECATED - HCA Data Coordination Platform Command Line Interface
https://hca.readthedocs.io/
MIT License
6 stars 8 forks source link

Upload breaks if too many files (100+?) #505

Closed GPelayo closed 4 years ago

GPelayo commented 4 years ago

make bundles

Traceback (most recent call last):
  File "upload_bundles.py", line 56, in <module>
    main()
  File "upload_bundles.py", line 49, in main
    file_uuid_callback=file_uuid_callback)
  File "/home/ubuntu/load-project/.venv/lib/python3.6/site-packages/hca/dss/__init__.py", line 123, in upload
    files_to_upload.append(open(full_file_name, "rb"))
OSError: [Errno 24] Too many open files: '/home/ubuntu/load-project/projects/1a7ccf4f-a500-5aa6-b31a-2fff80cf8f08/bundle/specimen_from_organism_95.json'
Makefile:26: recipe for target 'bundles' failed
make: *** [bundles] Error 1```

Original Ticket: https://github.com/DailyDreaming/load-project/issues/54