Closed lucasastro closed 3 months ago
kevin218
commented
4 months ago My initial guess is that Eureka! didn't install correctly and that this may be related to your Python version. I suggest creating a new conda environment using Python 3.9 and watching the output for errors during installation.
taylorbell57
commented
4 months ago Yeah, I agree with Kevin, your setup (including the Python version) indicates that something differed from our installation instructions. First delete your old eureka environment, and then precisely follow the installation instructions at https://eurekadocs.readthedocs.io/en/latest/installation.html#installing-with-a-conda-environment-yml-file.
lucasastro
commented
4 months ago Dear both, Thanks very much for your replies. I will do what you advise (I thought I had done this already but Python appears to have not picked up the right version). Very noob-y question: how do I remove the existing environment, rather than just deactivating it or replacing it?
taylorbell57
commented
4 months ago conda activate base
conda env remove -n eurekaIt still doesn't work.
This output does not bode well:
(base) lucas $ conda create -n eureka python==3.9.7
...
(base) lucas $ conda activate eureka
(eureka) lucas $ python --version
Python 3.12.3Also, if I (because of the M1 chip) try installing with the conda environment.yml file, do I still start by doing step 1 (conda create -n eureka python==3.9.7) i.e. creating 2 conda environments...?
taylorbell57
commented
4 months ago No, the conda environment.yml file should make its own new environment, so no need to manually create an environment first
taylorbell57
commented
4 months ago After you do conda env remove -n eureka, do you get an error message if you try to do conda activate eureka? It's good if that gives you an error message, but if not, then somehow the environment isn't being deleted
lucasastro
commented
4 months ago yes it does give an error, so the removal is working. In reaction to your earlier comment, I am currently installing with a conda environment.yml file
lucasastro
commented
4 months ago Alas, it does not work. I worked through all the steps under "Installing with a conda environment.yml file" including the update, no errors. Then, no matter if I try in the (eureka) environment or in (base), I get:
lucas $ python run_eureka.py
Traceback (most recent call last):
File "~/.../MyFirstEureka/run_eureka.py", line 2, in <module>
import eureka.lib.plots
ModuleNotFoundError: No module named 'eureka'Try using Anaconda (or similar gui) to create your conda environment. If you can solve first problem then the rest might work.
On Tue, Jun 4, 2024, 6:15 PM lucasastro @.***> wrote:
Alas, it does not work. I worked through all the steps under "Installing with a conda environment.yml file" including the update, no errors. Then, no matter if I try in the (eureka) environment or in (base), I get:
lucas $ python run_eureka.py Traceback (most recent call last): File "~/.../MyFirstEureka/run_eureka.py", line 2, in import eureka.lib.plots ModuleNotFoundError: No module named 'eureka'
— Reply to this email directly, view it on GitHub https://github.com/kevin218/Eureka/issues/658#issuecomment-2148500183, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFC2C7LTLLVTAJWOIF74UJ3ZFY36ZAVCNFSM6AAAAABIYLF3OGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBYGUYDAMJYGM . You are receiving this because you commented.Message ID: @.***>
taylorbell57
commented
4 months ago Can you please copy-paste the outputs from the following commands:
which python
python --version
which pip
pip show eureka(eureka) lucas [ ~ ]$ which python
/opt/anaconda3/envs/eureka/bin/python
(eureka) lucas [ ~ ]$ python --version
Python 3.12.3
(eureka) lucas [ ~ ]$ which pip
/opt/anaconda3/envs/eureka/bin/pip
(eureka) lucas [ ~ ]$ pip show eureka
Name: eureka
Version: 0.10
Summary: A data reduction and analysis pipeline intended for time-series observations with JWST.
Home-page: https://github.com/kevin218/Eureka
Author: Eureka! pipeline developers
Author-email: kbstevenson@gmail.com
License: MIT License
Location: /opt/anaconda3/envs/eureka/lib/python3.9/site-packages
Requires: astraeus, astropy, astroquery, batman-package, bokeh, ccdproc, celerite, corner, crds, dynesty, emcee, exotic-ld, george, h5py, lmfit, matplotlib, numpy, pandas, photutils, requests, scipy, setuptools-scm, stdatamodels, svo-filters, tqdm
Required-by: Huh, that is very strange - it seems that Kevin is right, and there is something weird with your Anaconda or Miniconda installation that is somehow forcing the use of Python==3.12.3... I can't really help troubleshoot that, but I recommend fiddling around with your conda installation until you can create an environment with your desired Python version. And perhaps look for people having similar issues on Stack Exchange or websites like that?
lucasastro
commented
4 months ago OK, I will try this and also seek advice. Thanks for the Anaconda suggestion, and for your help so far. Will update this issue when it is resolved, may be useful to the community.
lucasastro
commented
3 months ago It still gives the same error.
I deactivated and removed existing Eureka environment, including the Eureka folder on the disk I reinstalled Python 3.9.7 from python.org, removed mentions of 3.12 from $PATH. Then I reinstalled Eureka following the instructions (through .yml file). Now it does work in the correct Python version but the error persists. Any ideas what may be wrong? I have a feeling that the solution may be simpler than thought before...
I can of course remove and reinstall Anaconda, but it seems to me that the Python version is not the factor causing the problem.
Tomorrow I will retry by more robust reinstalls, but perhaps some nugget of wisdom already comes to mind...
Output:
(eureka) lucas [ ~/Documents/ScienceHD/REVEAL_HD/Analysis/JWST/MyFirstEureka ]$ python run_eureka.py
Traceback (most recent call last):
File "/Users/lucas/Documents/ScienceHD/REVEAL_HD/Analysis/JWST/MyFirstEureka/run_eureka.py", line 2, in <module>
import eureka.lib.plots
ModuleNotFoundError: No module named 'eureka'
(eureka) lucas [ ~/Documents/ScienceHD/REVEAL_HD/Analysis/JWST/MyFirstEureka ]$ which python
/opt/anaconda3/envs/eureka/bin/python
(eureka) lucas [ ~/Documents/ScienceHD/REVEAL_HD/Analysis/JWST/MyFirstEureka ]$ python --version
Python 3.9.7
(eureka) lucas [ ~/Documents/ScienceHD/REVEAL_HD/Analysis/JWST/MyFirstEureka ]$ which pip
/opt/anaconda3/envs/eureka/bin/pip
(eureka) lucas [ ~/Documents/ScienceHD/REVEAL_HD/Analysis/JWST/MyFirstEureka ]$ pip show eureka
Name: eureka
Version: 0.10
Summary: A data reduction and analysis pipeline intended for time-series observations with JWST.
Home-page: https://github.com/kevin218/Eureka
Author: Eureka! pipeline developers
Author-email: kbstevenson@gmail.com
License: MIT License
Location: /opt/anaconda3/envs/eureka/lib/python3.9/site-packages
Requires: astraeus, astropy, astroquery, batman-package, bokeh, ccdproc, celerite, corner, crds, dynesty, emcee, exotic-ld, george, h5py, lmfit, matplotlib, numpy, pandas, photutils, requests, scipy, setuptools-scm, stdatamodels, svo-filters, tqdm
Required-by: Hmm, I'm really at a loss here... Some things come to mind to help me troubleshoot this more.
run_eureka.py file?git clone command, what do you get when you run pytest testsmain branch of Eureka! which has several updates (perhaps one of which will resolve your problem) but has a bit higher a risk for bugs in the code. To use the main branch instead, replace the instance of v0.10 with main when running the git clone commandlucasastro
commented
3 months ago Ok, here goes!
terminal inputs and outputs for when installing the package: see attached Install_Eureka_output.txt Note 1: I added the 2 lines to my bash_profile: export CRDS_PATH=/Users/lucas/crds_cache export CRDS_SERVER_URL=https://jwst-crds.stsci.edu Note 2: I initialized the 'Quickstart' protocol by importing and adapting the wasp39 .ecf files etc.
line 1 of run_eureka.py file: import os (the lines after are import eureka.lib.plots etc. This is what the error I get upon installation refers to, I suppose)
pytest tests generates this output, see attached pytest_output.txt
Installing main branch produced the same result. See attached Install_Eureka_main_output.txt
It really looks like such a trivial error, hope there is a simple solution... your help and advice are appreciated.
taylorbell57
commented
3 months ago I am so deeply confused... Your pip show eureka command works and shows that eureka is installed, and your pytest tests command works (although that actually doesn't use your installed eureka and instead uses the raw code in the downloaded GitHub folder, so at least we know the issue isn't with any of your other dependencies). I initially suspected the issue could be that you're running the update steps right after installing (which you should definitely not do btw), but you didn't do that in the Install_Eureka_main_output.txt file and still got the error so I no longer suspect that's the cause. I see your conda has updates available which could possibly be the issue (could be worth trying their recommended conda update -n base -c defaults conda command first) but I kinda doubt that'll solve anything. The last thing I can think of is that something is wrong with your PATH environment variable - do you mind copy-pasting the outputs of running echo $PATH while your eureka conda environment is active?
lucasastro
commented
3 months ago Indeed, updating conda did not change the outcome.
The output for $PATH while in Eureka:
(eureka) lucas [ ~/Documents/ScienceHD/REVEAL_HD/Analysis/JWST/MyFirstEureka ]$ echo $PATH
/Library/Frameworks/Python.framework/Versions/3.9/bin:/opt/anaconda3/envs/eureka/bin:/Library/Frameworks/Python.framework/Versions/3.9/bin:/Users/lucas/DS9:/Users/lucas/anaconda/bin:/opt/local/bin:/opt/local/sbin:/Library/Frameworks/Python.framework/Versions/Current/bin:/Applications/scisoft/i386/bin:/Applications/scisoft/all/bin:/bin:/usr/bin:/Users/lucas/p2pp-3.3.0/bin:/Users/lucas/10FEBpl1.1/system/unix:/Users/lucas/gcc-4.5.0:/Applications/scisoft/i386/bin:/Applications/scisoft/all/bin:/Applications/rsi/idl:/Applications/rsi/idl/bin:/Users/Lucas/idl/idl_startup.pro:/Users/Lucas/idl/programs:/Applications/itt/idl70/lib:/Applications/itt/idl70/examples:/Users/lucas/gasgano/bin:/Users/lucas/p2pp-3.3.0/bin:/usr/local/bin:/usr/texbin:/Library/TeX/texbin:/Users/lucas/latexdiff:/Users/lucas/cires/bin:/Applications/ImageJ
There's a lot of stuff in there that I never use anymore (idl, gasgano, p2pp...) but it looks harmless. Although I remember that scisoft contained a Python installation.
Perhaps a stupid remark, but in /opt/anaconda3/envs/eureka/bin there is no file called 'eureka'
Note that (see https://github.com/kevin218/Eureka/issues/658#issuecomment-2153139020) I cleaned up $PATH a bit when manually installing Python 3.9 (since it somehow kept refusing to adapt it from the environment). This worked: (eureka) lucas [ ~/Documents/ScienceHD/REVEAL_HD/Analysis/JWST/MyFirstEureka ]$ python --version Python 3.9.7
Note also that I never used a GUI to do the installation, but have run everything from a Terminal. So there is still that option. I have previously installed Anaconda and a (expired) trial version of PyCharm.
Is it a Eureka! moment yet...?
taylorbell57
commented
3 months ago Hmm, I suspect that /Library/Frameworks/Python.framework folder is the root of this issue. I think you likely installed python using brew or some other installer at some point, and because that is at the start of the PATH variable, I think it might be causing issues (see this stackoverflow post where some other folks were having weird issues caused by that python being installed). Can you try either removing that folder from your PATH variable and/or moving it after the /opt/anaconda3/envs/eureka/bin folder?
lucasastro
commented
3 months ago :,-(
I removed the references to Framework etc from $PATH by adapting and sourcing .bash_profile, now it is as follows: (eureka) lucas [ ~/Documents/ScienceHD/REVEAL_HD/Analysis/JWST/MyFirstEureka ]$ echo $PATH /opt/miniconda3/envs/eureka/bin:/Users/lucas/DS9:/Users/lucas/anaconda/bin:/opt/local/bin:/opt/local/sbin:/Applications/scisoft/i386/bin:/Applications/scisoft/all/bin:/bin:/usr/bin:/Users/lucas/p2pp-3.3.0/bin:/Users/lucas/10FEBpl1.1/system/unix:/Users/lucas/gcc-4.5.0:/Applications/scisoft/i386/bin:/Applications/scisoft/all/bin:/Applications/rsi/idl:/Applications/rsi/idl/bin:/Users/Lucas/idl/idl_startup.pro:/Users/Lucas/idl/programs:/Applications/itt/idl70/lib:/Applications/itt/idl70/examples:/Users/lucas/gasgano/bin:/Users/lucas/p2pp-3.3.0/bin:/usr/local/bin:/usr/texbin:/Library/TeX/texbin:/Users/lucas/latexdiff:/Users/lucas/cires/bin:/Applications/ImageJ
Then I removed and reinstalled Eureka through the git option. But now...:
(eureka) lucas [ ~/Documents/ScienceHD/REVEAL_HD/Analysis/JWST/MyFirstEureka ]$ python run_eureka.py
Traceback (most recent call last):
File "/Users/lucas/Documents/ScienceHD/REVEAL_HD/Analysis/JWST/MyFirstEureka/run_eureka.py", line 2, in
Curiously, by the way, I got this error: (base) lucas [ ~/Documents/ScienceHD/REVEAL_HD/Analysis/Eureka ]$ conda env create --file environment.yml --force usage: conda [-h] [-v] [--no-plugins] [-V] COMMAND ... conda: error: unrecognized arguments: --force
This happened before, and like now I typed the command without --force. Earlier, it did accept --force but only after I changed the Python version to 3.9.7. But I guess this is no biggie.
I can get rid of more redundant $PATH variables but it feels a bit desparate. Any ideas?
lucasastro
commented
3 months ago A possible lead: Some creative Googling let me to try the following:
(eureka) lucas [ ~/Documents/ScienceHD/REVEAL_HD/Analysis/JWST/MyFirstEureka ]$ python Python 3.9.7 (v3.9.7:1016ef3790, Aug 30 2021, 16:25:35) [Clang 12.0.5 (clang-1205.0.22.11)] on darwin Type "help", "copyright", "credits" or "license" for more information.
import sys print(sys.path) ['', '/Library/Frameworks/Python.framework/Versions/3.9/lib/python39.zip', '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9', '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload', '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages']
I don't think this path includes the directory where eureka is installed... should it?
taylorbell57
commented
3 months ago Huh that's very strange - yes it seems the python being used is still not the conda python (since that isn't listed anywhere in the sys.path). I have no idea how that could be the case if which python says it'll use the conda python, but it seems clear that it isn't. My sys.path list (on Windows Subsystem for Linux, so the paths are formatted differently but give you an idea of what the sys.path should look like) is:
['/mnt/wsl/ubuntuSSD/home/taylor/miniconda3/envs/eureka_starry/bin',
'/home/taylor/miniconda3/envs/eureka_starry/lib/python39.zip',
'/home/taylor/miniconda3/envs/eureka_starry/lib/python3.9',
'/home/taylor/miniconda3/envs/eureka_starry/lib/python3.9/lib-dynload', '',
'/home/taylor/miniconda3/envs/eureka_starry/lib/python3.9/site-packages', '/home/taylor/Eureka/src']That second-to-last .../python3.9/site-packages folder is where eureka is installed, while eureka's dependencies end up scattered throughout the different folders in the path
lucasastro
commented
3 months ago Bulls eye. Somehow 'python' calls a different (fossilized..?) Python than the one specified by 'which python'. The welcome message looks markedly different. And if I explicitly call the 'conda python' calling Eureka modules works:
(eureka) lucas [ ~/Documents/ScienceHD/REVEAL_HD/Analysis/JWST/MyFirstEureka ]$ which python /opt/miniconda3/envs/eureka/bin/python (eureka) lucas [ ~/Documents/ScienceHD/REVEAL_HD/Analysis/JWST/MyFirstEureka ]$ python Python 3.9.7 (v3.9.7:1016ef3790, Aug 30 2021, 16:25:35) [Clang 12.0.5 (clang-1205.0.22.11)] on darwin Type "help", "copyright", "credits" or "license" for more information.
import eureka.lib.plots Traceback (most recent call last): File "
", line 1, in ModuleNotFoundError: No module named 'eureka' ^D
(eureka) lucas [ ~/Documents/ScienceHD/REVEAL_HD/Analysis/JWST/MyFirstEureka ]$ /opt/miniconda3/envs/eureka/bin/python Python 3.9.7 | packaged by conda-forge | (default, Sep 29 2021, 19:22:19) [Clang 11.1.0 ] on darwin Type "help", "copyright", "credits" or "license" for more information.
help(eureka) Traceback (most recent call last): File "
", line 1, in NameError: name 'eureka' is not defined import eureka.lib.plots Could not import starry and/or pymc3 related packages. Functionality may be limited.
So, this is a 'fix' to the problem, although I would like the alias 'python' to call the conda version when in eureka. And YES indeed, I found the bodged alias. It was in my .bash_profile, stupidly. Removed it now. And, lo and behold:
(eureka) lucas [ ~/Documents/ScienceHD/REVEAL_HD/Analysis/JWST/MyFirstEureka ]$ alias python alias python='/usr/local/bin/python3' (eureka) lucas [ ~/Documents/ScienceHD/REVEAL_HD/Analysis/JWST/MyFirstEureka ]$ unalias python (eureka) lucas [ ~/Documents/ScienceHD/REVEAL_HD/Analysis/JWST/MyFirstEureka ]$ python Python 3.9.7 | packaged by conda-forge | (default, Sep 29 2021, 19:22:19) [Clang 11.1.0 ] on darwin
And now: (eureka) lucas [ ~/Documents/ScienceHD/REVEAL_HD/Analysis/JWST/MyFirstEureka ]$ python run_eureka.py it runs, but shortly complains about inputdirs and ecf files. But that was to be expected, I will set it up properly.
SO! it seems I got it running! Will let you know if indeed the whole thing works tomorrow (now going into CEST night time:)
taylorbell57
commented
3 months ago Whew, it was a challenge but I'm glad we got this sorted out. I'll close this issue now since you've sorted it out and it wasn't something caused on our end, but please do open a new issue if you encounter a bug with Eureka! now that you're actually able to use the code. And this closed issue will still remain archived here on GitHub to help any future people that encounter this issue
lucasastro
commented
3 months ago Understood. Thanks for your time and effort!!
Instrument
No response
What happened?
I have followed the Eureka instructions and am about to run my first Quickstart pipeline.
I have an M1 chip and have thus installed Eureka via the conda environment.yml file. (the first attempt gave an error.)
I have activated the Eureka environment, but on running hte run_eureka.py file from the Quickstart package it seems the eureka module is not installed or activated, see error traceback output.
Could you help me understand what is going on here?
Error traceback output
What operating system are you using?
Mac OSX Sonoma 14.4.1
What version of Python are you running?
Python 3.12.3
What Python packages do you have installed?
Code of Conduct