ReproNim / reproman

ReproMan (AKA NICEMAN, AKA ReproNim TRD3)
https://reproman.readthedocs.io
Other
24 stars 14 forks source link

Our tracer is incompatible with latest conda #446

Closed kyleam closed 4 years ago

kyleam commented 5 years ago

As described in gh-445:

[...] our distributions.conda isn't compatible with the latest conda in at least two ways. We use an environment file that doesn't have an extension that conda now expects. That's an easy fix, and there's a patch in gh-443. The bigger issue is that environments seem to no longer have their own bin/conda unless conda is explicitly installed in the environment. (This mirrors the then seemingly hard-to-trigger problem initially described in gh-173.) As a result, our method for identifying an environment's root prefix is broken. Based on a quick scan of envs/NAME/, there may no longer be an easy way to map from an environment to its root prefix.

Re: #443

yarikoptic commented 5 years ago

easy fix, and there's a patch in gh-443. The bigger issue is that environments seem to no longer have their own bin/conda unless conda is explicitly installed in the environment. (This mirrors the then seemingly hard-to-trigger problem initially described in gh-173.) As

I guess we should check if there is anything else in the file tree which could be taken as a sign of it being a conda distribution.

kyleam commented 5 years ago

If we are fine assuming that the environment is in ROOT/envs/ (as we discussed on the call), we can just identify the root by walking up. After doing that and some other fixes, I've managed to push the failures down the line in test_conda_init_install_and_detect, but I still can't get it passing. I'm not sure if the issue is on our end or conda's, but I haven't had any luck yet triggering a similar failure outside of reproman. At any rate, I've pushed the current state to scratch/conda-update-for-upstream for the time being. The main message from the failure is shown in the last commit.

Edit: Here's the link for the Travis run: https://travis-ci.org/ReproNim/reproman/jobs/570012173#L2148

kyleam commented 4 years ago

All right, some progress. The confused state where conda can't resolve things (e.g., fc73bc3bd) seems to be coming from the non-default channels used in the root environment. And I think that's an issue because install_packages calls conda install without specifying channels while the commands that are given an environment yaml do consider the channels (because the channels are included in the yaml). Anyway, I'll need to return to this later to try for a proper fix of specifying the channels, but the two latest pushes to scratch/conda-update-for-upstream are in a passing state at least.