Open oathbound01 opened 3 weeks ago
oathbound01
commented
3 weeks ago I should point out that the issue is very likely not with the Dockerfile or the Docker installation per se, but rather with the install.sh process and the various environment.yml dependencies and/or the channel used for the conda packages
gAldeia
commented
3 weeks ago Hi @oathbound01 !
I am not really sure what may be causing your installation issue.
Are you using the correct branch (docker-compose)?
There is a CI workflow that tries to build and test all containers here https://github.com/cavalab/srbench/blob/docker-compose/.github/workflows/ci-docker.yml. You can also check the logs when trying to build AFP, which seems to be the first algorithm that fails in your local machine https://github.com/cavalab/srbench/actions/runs/11374625817/job/31643734527
gAldeia
commented
3 weeks ago I also noticed that Python 3.13 was released earlier this month, with led to building errors with some algorithms. I had to fix python to 3.12 for one algorithm in order to install it, but the pipeline worked for the others.
oathbound01
commented
3 weeks ago Are you using the correct branch (
docker-compose)?
No, I was not using that branch, since it's not mentioned anywhere, and the user-guide.md simply directs me towards the standard Dockerfile that is present in the main branch.
Having said that, not even that branch seems to work: the standard Dockerfile not only uses broken/dead proxies but also fails to create the mamba environment:
> [ 4/23] RUN mamba env create -f base_environment.yml:
1.034
1.034 CondaValueError: prefix already exists: /opt/conda
1.034
------Commenting that line, the Dockerfile STILL fails since it's looking for install.sh which doesn't exist.
> [ 4/22] RUN bash install.sh afp:
0.264 bash: install.sh: No such file or directoryEven switching to the (intended?) install_algorithm.sh file doesn't work, since it's trying to use micromamba and it doesn't seem to be installed in the Docker image:
0.233 SUBNAME: afp ; SUBFOLDER: ./afp
0.233 vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
0.233 .................... Installing afp ...
0.233 ........................................
0.233 Creating environment
0.233 ........................................
0.234 install_algorithm.sh: line 33: micromamba: command not foundAlso using the docker-compose file created with the appropriate script seems to fail:
=> [eql 5/6] RUN bash install_algorithm.sh eql && micromamba clean 206.1s
=> [itea 5/6] RUN bash install_algorithm.sh itea && micromamba cle 206.0s
=> CANCELED [bingo 5/6] RUN bash install_algorithm.sh bingo && mic 205.1s
=> [ffx 5/6] RUN bash install_algorithm.sh ffx && micromamba clean 205.9s
=> [ps-tree 5/6] RUN bash install_algorithm.sh ps-tree && micromam 205.8s
=> [eplex 5/6] RUN bash install_algorithm.sh eplex && micromamba c 205.8s
=> [afp 5/6] RUN bash install_algorithm.sh afp && micromamba clean 205.7s
=> CANCELED [rils-rols 5/6] RUN bash install_algorithm.sh rils-rols 204.2s
=> [gpgomea 5/6] RUN bash install_algorithm.sh gpgomea && micromam 205.6s
=> ERROR [feat 5/6] RUN bash install_algorithm.sh feat && micromam 202.1s
=> [qlattice 5/6] RUN bash install_algorithm.sh qlattice && microm 205.4s
=> [operon 5/6] RUN bash install_algorithm.sh operon && micromamba 205.3s
=> [gplearn 5/6] RUN bash install_algorithm.sh gplearn && micromam 205.3s
=> [geneticengine 5/6] RUN bash install_algorithm.sh geneticengine 205.2s
=> [e2et 5/6] RUN bash install_algorithm.sh e2et && micromamba cle 205.2s
=> [pysr 5/6] RUN bash install_algorithm.sh pysr && micromamba cle 205.1s
=> [gpzgd 5/6] RUN bash install_algorithm.sh gpzgd && micromamba c 205.1s
=> [tir 3/7] WORKDIR /install 0.5s
=> [tir 4/7] COPY --chown=mambauser:mambauser base_environment.yml insta 0.3s
=> [tir 5/7] COPY --chown=mambauser:mambauser ./algorithms/tir ./tir 0.2s
=> [tir 6/7] RUN bash install_algorithm.sh tir && micromamba clean 108.9s
------
> [feat 5/6] RUN bash install_algorithm.sh feat && micromamba clean --all --yes:
2.196 SUBNAME: feat ; SUBFOLDER: ./feat
2.196 vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
2.196 .................... Installing feat ...
2.196 ........................................
2.196 Creating environment
2.196 ........................................
2.196 using ./feat/environment.yml ...
200.9 error libmamba Could not solve for environment specs
200.9 The following packages are incompatible
200.9 ├─ cmake 3.28.2** does not exist (perhaps a typo or a missing channel);
200.9 ├─ ninja 1.11.1** does not exist (perhaps a typo or a missing channel);
200.9 ├─ pybind11 2.11.1** does not exist (perhaps a typo or a missing channel);
200.9 ├─ pybind11_json 0.2.13** does not exist (perhaps a typo or a missing channel);
200.9 └─ shogun-cpp 6.1.4** does not exist (perhaps a typo or a missing channel).
200.9 critical libmamba Could not solve for environment specs
------
failed to solve: process "/usr/local/bin/_dockerfile_shell.sh bash install_algorithm.sh $ALGORITHM && micromamba clean --all --yes" did not complete successfully: exit code: 1Also, I don't see how Actions has much of anything to do with this, I just want to test out the algorithms inside of a Docker container. At this point, I'm not sure if I'm doing something wrong or if the whole process is just very broken, but in any case a more detailed guide on what I'm REALLY supposed to do wouldn't hurt
folivetti
commented
3 weeks ago Hi @oathbound01 we're sorry for this inconvenience. We're now in the middle of setting things up for a new version of srbench (the docker images is meant to be part of that new version) so things may not yet be working as intended. AFTER we finish setting things up, we will merge the current working branch to main and update the documentation.
Notice that this project is being maintained by very few volunteers with limited time (at this point I would say only @gAldeia is actively working on that) so things may not run as fast as we wish!
We will look into this issue and let you know when we figure it out what's going on. In the meantime I ask you for your patience and, if you figure things out on your end, please let us know and prepare a PR. The success of such projects depend on the whole community working together as a team :-)
gAldeia
commented
3 weeks ago Hey @oathbound01,
If you're looking for a stable version of the master branch, there’s a release that should work: v2.0 release.
If you want to use the Docker images, you'll need to switch to the docker-compose branch. You are right --- there's no mention of this in the main branch, so I will update the README.md to include it. Switching to this branch should resolve your issues, as GitHub Actions can successfully build and test all images, including new algorithms staged for benchmarking.
As @folivetti mentioned, it is still a WIP.
gAldeia
commented
3 weeks ago There is a small guide on how to use the docker here https://github.com/cavalab/srbench/blob/docker-compose/docs/user_guide.md#for-docker-users
oathbound01
commented
3 weeks ago switching to this branch should resolve your issues, as GitHub Actions can successfully build and test all images, including new algorithms staged for benchmarking.
Unfortunately, switching to that branch has not solved my issues, as you can see with the error above. And I don't really know what is causing this, especially when Actions seems to be working fine, especially considering that I've tried following its workflow (which is the same as the guide you linked me) as closely as I could, yet the build just fails.
If you're looking for a stable version of the
masterbranch, there’s a release that should work: v2.0 release.
The stable release is also giving me issues, but to be fair I think it's just conda being conda and refusing to cooperate on my Fedora machine. I might try it on Arch to see if I can get things moving again, although at this point I'm pretty disheartened and might just wait for the Docker image to be done
gAldeia
commented
3 weeks ago I haven't tested it on other operating systems yet. So far, I've managed to get it running in my WSL environment (which can be a bit tricky at times) and on an Ubuntu machine.
A last resource: instead of building it, try to pull it from docherhub.
I know the CI handles DockerHub uploads, but it's tied to William's account, so I don't have the credentials. This means it's not fully up to date, though some algorithms are available: SRBench on DockerHub.
In the meantime, I'll look into the micromamba issue you mentioned. Thanks for bringing it up!
When trying to install the benchmark through Docker, the process fails when trying to install the first algorithm
For some reason it looks like it's trying to pull from a release candidate version of Python, breaking the entire thing in the process.
I've tried running
to no avail. I've also tried manually updating some dependencies, both in the
base_environment.ymland the specificenvironment.yml, but every time I fix something, I end up breaking something else.It would seem that the fix is simpler than I imagine, yet any combination I try doesn't work.