Impossible to upload large dataset

codalab / codabench

Codabench is a flexible, easy-to-use and reproducible benchmarking platform. Check our paper at Patterns Cell Press https://hubs.li/Q01fwRWB0

Apache License 2.0

76 stars 28 forks source link

Impossible to upload large dataset #1635

Open Didayolo opened 1 month ago

Didayolo commented 1 month ago

An user reported the upload a 17GB dataset failed in the middle of the uploading, even with an increased quota.

The upload does not work on production MinIO but works on test MinIO. It seems to work with command line but not using the interface.

Possibilities:

Problem with the reverse proxy?
Need to upgrade MinIO?

ObadaS commented 1 month ago

Uploading a large dataset (20 GB) on the codabench test server works. It is using a newer version of MinIO and is behind a different reverse proxy, so this points us towards the potential problem.