openai / mle-bench

MLE-bench is a benchmark for measuring how well AI agents perform at machine learning engineering
https://openai.com/index/mle-bench/
Other
529 stars 59 forks source link

Error while preparing `the-icml-2013-whale-challenge-right-whale-redux` #20

Closed yorhaha closed 1 day ago

yorhaha commented 1 day ago

Command: mlebench prepare -c the-icml-2013-whale-challenge-right-whale-redux --data-dir ./data/

the-icml-2013-whale-challenge-right-whale-redux.zip: Skipping, found more recently modified local copy (use --force to force download)
[2024-11-28 17:45:48,981] [data.py:67] Generating checksum for `data/the-icml-2013-whale-challenge-right-whale-redux/the-icml-2013-whale-challenge-right-whale-redux.zip`...
[2024-11-28 17:45:53,454] [data.py:80] Checksum for `data/the-icml-2013-whale-challenge-right-whale-redux/the-icml-2013-whale-challenge-right-whale-redux.zip` matches the expected checksum.
[2024-11-28 17:45:53,455] [data.py:83] Extracting `data/the-icml-2013-whale-challenge-right-whale-redux/the-icml-2013-whale-challenge-right-whale-redux.zip` to `data/the-icml-2013-whale-challenge-right-whale-redux/raw`...
[2024-11-28 17:45:55,400] [data.py:85] Extracted `data/the-icml-2013-whale-challenge-right-whale-redux/the-icml-2013-whale-challenge-right-whale-redux.zip` to `data/the-icml-2013-whale-challenge-right-whale-redux/raw` successfully.
[2024-11-28 17:45:55,402] [data.py:168] Public directory is empty.
[2024-11-28 17:45:55,402] [data.py:96] Preparing the dataset using `prepare` from `mlebench/competitions/the-icml-2013-whale-challenge-right-whale-redux/prepare.py`...
  0%|                                                                                                                                                         | 0/22692 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/.../mle-bench/venv/bin/mlebench", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/.../mle-bench/mlebench/cli.py", line 170, in main
    download_and_prepare_dataset(
  File "/.../mle-bench/mlebench/data.py", line 101, in download_and_prepare_dataset
    competition.prepare_fn(
  File "/.../mle-bench/mlebench/competitions/the-icml-2013-whale-challenge-right-whale-redux/prepare.py", line 46, in prepare
    shutil.copy(sample, public / "train2" / new_sample)
  File "/.../miniconda3/envs/py311/lib/python3.11/shutil.py", line 431, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/.../miniconda3/envs/py311/lib/python3.11/shutil.py", line 258, in copyfile
    with open(dst, 'wb') as fdst:
         ^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'data/the-icml-2013-whale-challenge-right-whale-redux/prepared/public/train2/data/the-icml-2013-whale-challenge-right-whale-redux/prepared/public/train2/20090328_000000_002s3ms_TRAIN0_0.aif'
yorhaha commented 1 day ago

System information:

thesofakillers commented 1 day ago

Hey, thanks for flagging this! We introduced a fix in #21. If you pull the latest from main and try running again it should work. Thanks again