Closed matiasandina closed 3 months ago
adamltyson
commented
4 months ago Thanks for raising the issue @matiasandina. Some of the team are away currently, so we may take a while to get back to you.
I can't reproduce on my (Ubuntu 20.04) machine with:
conda create --name datashuttle-test -c conda-forge datashuttle -y
conda activate datashuttle-test
datashuttle launchCan anyone else @neuroinformatics-unit/neuroinformatics-all?
alessandrofelder
commented
4 months ago I cannot on my macbook, can try on Ubuntu 22 when I am back next week 🤔
IgorTatarnikov
commented
4 months ago I'm able to launch it successfully on Ubuntu 22.04 / Linux 6.5.0.28-generic in a fresh python 3.10 environment.
lochhh
commented
4 months ago Hi, looks like it could be a rich version issue
matiasandina
commented
4 months ago Thanks for the quick checks. Interestingly, this environment works well! :thinking:
conda create --name datashuttle-test -c conda-forge datashuttle -y
conda activate datashuttle-test
datashuttle launchI created the first one as suggested in the docs
conda create -n datashuttle-env python=3.10
conda activate datashuttle-env
conda install -c conda-forge datashuttleMaybe there are some differences in the environment creation ?
rich is the same in both envs
(datashuttle-test) matias@choilab:~$ conda list rich
# packages in environment at /home/matias/miniconda3/envs/datashuttle-test:
#
# Name Version Build Channel
rich 13.7.1 pyhd8ed1ab_0 conda-forge
(datashuttle-test) matias@choilab:~$ conda deactivate
matias@choilab:~$ conda activate datashuttle-env
(datashuttle-env) matias@choilab:~$ conda list rich
# packages in environment at /home/matias/miniconda3/envs/datashuttle-env:
#
# Name Version Build Channel
rich 13.7.1 pyhd8ed1ab_0 conda-forgeThe only difference I can think of is that "my" way, would fetch a different version of Python itself (and from conda-forge).
lochhh
commented
4 months ago Strange 🤔 Followed the docs setup steps on Ubuntu 22.04.4 LTS (WSL2 ) and it launches fine.
matiasandina
commented
4 months ago Indeed
(datashuttle-env) @.***:~$ python Python 3.10.14 (main, May 6 2024, 19:42:50) [GCC 11.2.0] on linux Type "help", "copyright", "credits" or "license" for more information.
quit() (datashuttle-env) @.:~$ conda activate datashuttle-test (datashuttle-test) @.:~$ python Python 3.11.9 | packaged by conda-forge | (main, Apr 19 2024, 18:36:13) [GCC 12.3.0] on linux Type "help", "copyright", "credits" or "license" for more information.
adamltyson
commented
4 months ago Weird, this shouldn't make a difference. Anyway, glad you're up and running @matiasandina, let us know if you have any more problems.
I'll keep this issue open so we can dig into this later.
JoeZiminski
commented
4 months ago Hey @matiasandina thanks for this report!!
This is very interesting case, it seems at runtime the version of rich being used is located at:
/home/matias/.local/lib/python3.10/site-packages/rich/console.py:1330 in │
│ renderrather than the correction version at:
# packages in environment at /home/matias/miniconda3/envs/datashuttle-test:
#
# Name Version Build Channel
rich 13.7.1 pyhd8ed1ab_0 conda-forgeI have no idea why this is happening, though it is a strange thing I've very seen infrequently before with conda when using it on a HPC. I guess the command to install the specific version 3.10 is somehow conflicting with an already existing 3.10 installation? Of course conda should keep these isolated but it may be failing in this case for whatever reason. It seems similar to this report (no answer) and this one that indicates it may be a conflict between conda and pip. Anyways, I think this can be filed under 'conda weirdness' and glad it is working now!
In any case, there is no reason why the docs should specify the install is with python=3.10, in particular if it is causing issues like this.
This can be changed to
conda create -n datashuttle-env
conda activate datashuttle-env
conda install -c conda-forge datashuttleif people agree.
(The reason the multi-line install rather than one-liner is used in the docs is just to make each step clearer to new conda users)
JoeZiminski
commented
4 months ago Coincidentally I had the same problem on another project and it led to discovering some very crazy conda/pip behaviour I did not know about that I think led to this error.
You may have multiple pip installed on your system (which -a pip) and one of these will be used by default when you do a pip command e.g. pip install datashuttle. If you create a conda environment but that environment does not install pip with conda install pip, when you pip install anything when in the conda environment it will use the default pip and install it outside of the environment.
To get it to work properly, you need to conda install pip then deactivate and reactivate the environment. When you reactivate the environment conda will now use the conda-installed pip as default. However it is not always enough.
In some circumstances (I'm not exactly sure which), even if pip is installed by conda during environment setup, the conda env may or may not use the conda installed pip. For example, I am trying it on our cluster after module load miniconda and if I do conda create -n myenv -c conda-forge python==3.8 datashuttle. Pip is installed in the conda env, but nonetheless pip at /nfs/nhome/live/jziminski/.local/lib/python3.8/site-packages is used (pip show pip).* In this case, maybe something similar is happening, Rich is installed by conda in the conda env, but for some reason the systems pip version is used at runtime.
On our HPC, if I request python==3.12 then pip is installed in the conda env and things work okay. pip show pip gives /nfs/nhome/live/jziminski/.conda/envs/myenv/lib/python3.12/site-packages.
So, fundamentally it seems that if you request a python version when installing a conda env and you have a system version of python with pip, conda will not necessarily use the correct version of pip (and possibly any other pip-installed dependency). I guess this problem is in theory is also possible doing conda create -n myenv without specifying a version, but it is much less likely the most recent version of python will be installed on a users system.
I am very surprised by this behaviour, it may be worth a blog post on the 'How to' page. Although I not sure how to reliably deal with it 😅.
* This is also true with python -m pip show pip and even conda run pip show pip or conda run python -m pip show pip.
adamltyson
commented
4 months ago This is very weird. I've had various issues with conda on our HPC, but I've never come across this locally, and I've made about 1-5 conda envs every week for the last 6 years!
JoeZiminski
commented
4 months ago I wonder if this is a clue in itself, do you have any non-conda managed installs of Python on your machine (or maybe, only an old one)? If not, I think you should never see this problem!
adamltyson
commented
4 months ago On some machines I have the system Python, but I've only ever used conda, so it's unlikely I ever have a non-conda version of the Python version I'm trying to use.
JoeZiminski
commented
3 months ago This has been adressed in #386 and properly tested in #394. Thanks @matiasandina please don't hestitate to raise issues / get in contact with any other problems or feedback.
Describe the bug Cannot create a local project Cannot go into the settings
To Reproduce First fresh install using conda-forge in conda env.
Log file
Computer used (please complete the following information):