search

codalab / codabench

Codabench is a flexible, easy-to-use and reproducible benchmarking platform. Check our paper at Patterns Cell Press https://hubs.li/Q01fwRWB0
Apache License 2.0
76 stars 28 forks source link

Possible Bug: .DS_Store and __MACOSX files in .zip archives #1655

Closed ToastyDom closed 1 day ago

ToastyDom commented 1 week ago

Hello everyone!

I think I found a bug that might affect Mac users. I was trying to upload a dummy scoring_program for a benchmark I’m planning to host, but whenever I uploaded a dummy submission file, nothing happened until the time limit was exceeded.

After some trial and error, I found that uploading the scoring_program from a Codabench example worked—but only if I didn’t unzip and rezip it on my Mac. When I downloaded the template file and uploaded it again without unzipping, it worked as expected. But if I downloaded it, unzipped it, and then rezipped it (without changing anything), I had the same timeout issue as with my dummy program.

I realized it might have to do with hidden files that macOS adds when zipping files (either .DS_Store or __MACOSX). When I used a terminal command to zip it without these extra files, it worked again.

So, I think these hidden files might be causing something to crash internally? I hope this information is helpful!

Best regards, Domenic :)

Didayolo commented 1 week ago

Hi,

Thank you for reporting this problem. I seems surprising to me that the .DS_Store and __MACOSX files could lead to a bug.

Maybe the problem was the way you zipped the files? Did you zip directly the files, or did you include the directory structure? Zipping the folder is known to cause problems.

ToastyDom commented 1 week ago

Hey! Okay interesting... When the error ocurred I just clicked on the folder containing the metadata and the .py and zipped it. To avoid the error I do zip -r -X ../scoring_program_folder.zip * via terminal to exclude these hidden files. Thats why I figured that it must be these files that cause the error... But using the command I also zip the folder and not the files..

By the way, the error only occurs when uploading the scoring program, when uploading submissions it does not care about these hidden files - if that's what's causing the problem.

I can try again by zipping only the files (and leaving in the hidden files) and let you know if anything has changed

ToastyDom commented 1 day ago

Hello! Quick update, I tried again by zipping the collection of files instead of the folder in which they are located in. Like that it works as expected!

ihsaan-ullah commented 1 day ago

It looks like the problem is resolved. Feel free to reopen this issue if you think otherwise.