[Enhancement]: Getting Eureka to work on Windows 10

[arikaegan]

Instrument

Other (any stage)

What is your suggestion?

The supplied installation instructions fail when trying to install on Windows 10 when it reaches pip install -e '.[jwst]', seemingly specifically when it is trying to install the [jwst] collection, but I have figured out how to make it work! There is a different order of operations, one additional package that I needed to install, a need to manually copy the crds_cache folder files from a mac onto my Windows 10 machine, and then a small change in the s3.fitbg function. Here are the steps I took:

I followed the first 4 steps in the Installation Instructions:

conda create -n eureka python==3.9.7
conda activate eureka
git clone -b v0.10 https://github.com/kevin218/Eureka.git
cd Eureka

From here, only install Eureka! without any dependencies: pip install .

Then I tested out installing the [hst] dependency, which was successful: pip install '.[hst]'

From here, I was able to successfully install the ['jwst]' dependency: pip install '.[jwst]'

Some additional steps I had to take in order to successfully complete the NIRSpec example:

The CRDS environmental variables need to be set, as explained in the current Eureka! instructions. On Windows 10, I accomplished this with

setenv CRDS_PATH $HOME/crds_cache
setenv CRDS_SERVER_URL https://jwst-crds.stsci.edu

I verified that by looking at the Environment Variables in the Advanced tab in the System Properties/Advanced system settings.

I needed to install the connection package: pip install connection

I had to copy over the crds_cache files from a mac as they weren't on my computer and weren't cached/downloaded properly when I tried to run the Eureka! NIRSpec example.

The last thing I had to do was make a small change to S3_data_reduction/background.py, in the fitbg function. On lines 208 and 210, near the top of the function, x1 and x2 are being converted into arrays. It only asks if they might be np.int64, but my x1 and x2 were np.int32s. So I added an or condition to these lines:

    ny, nx = np.shape(dataim)
    if isinstance(x1, (int, np.int64)) or isinstance(x1, (int, np.int32)):
        x1 = np.zeros(ny, dtype=int)+x1
    if isinstance(x2, (int, np.int64)) or isinstance(x2, (int, np.int32)):
        x2 = np.zeros(ny, dtype=int)+x2

Following all of those steps, the NIRSpec example was totally successful. I have not tried out any other Eureka! examples and if there are any other functions where Eureka! assumes only int64, there may be additional changes to make. But I think this is a good start!

Error traceback output

No response

What operating system are you using?

Windows 10

What version of Python are you running?

3.9.7

What Python packages do you have installed?

No response

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[taylorbell57]

@arikaegan, can you copy-paste the error message you were getting before when you tried to install Eureka? pip should take caer of all the dependency requirements and the order of installation shouldn't matter, but if it does, then the error message you got might help me understand how I'll need to edit the requirements so that you don't have to do this multi-step process.

Can you also explain how you're going about installing Eureka! - are you using something like the Git Bash program that comes from Anaconda? And can you copy-paste the error message you got which suggested that you'd need the connection package? And finally, can you explain what error message you were getting related to the crds_cache files not being "cached/downloaded properly"?

As for the np.int32 issue, I can make a small little pull request to resolve that issue.

In the meantime, I'll mention that the Windows Subsystem For Linux (WSL) works excellently and allows users to run linux programs in Windows. I've been doing nearly all of my programming work in WSL for the past few months without any significant issue. Fully supporting Windows is not something that is going to be possible for us right now since the starry package that gets installed when using pip install '.[pymc3]' does not support Windows, but my hope is to switch all the PyMC3+starry code over to jax+jaxoplanet sometime which should hopefully work on Windows. But in the meantime, WSL works excellently, I'll resolve that np.int32 issue, and I'll try to add some details to the installation and/or FAQ pages summarizing your findings (once I get those error messages from you and can reproduce your installation methods on my own machine)

[taylorbell57]

@arikaegan are you able to respond to my follow-up questions above? If not, I'll close this issue for now since I don't yet have enough information to reproduce your error or solution, and running the code in WSL is already known to work excellently. That said, I have just made a commit (linked above) that will eventually get merged into the main branch and take care of that np.int32 issue you were having

[arikaegan]

Thank you for the follow-up; my apologies this fell off my radar. When I had first tried to install this on windows, my institution did not have an easy path to installing/using WSL, hence why I was trying to install it on windows. I just tried to recreate the issue in a new conda environment, but was unable to do so as the installation was successful. So perhaps this is no longer an issue.

[taylorbell57]

Alright, I'll close this issue as "can't reproduce" - glad you have it working though!