search

kevin218 / Eureka

Eureka! is a data reduction and analysis pipeline intended for time-series observations with JWST.
https://eurekadocs.readthedocs.io/
MIT License
61 stars 47 forks source link

[Enhancement]: Optimal photometric extraction #491

Open taylorbell57 opened 1 year ago

taylorbell57 commented 1 year ago

Instrument

MIRI (Stages 1-3)

What is your suggestion?

For MIRI imaging (especially with long wavelength filters), the observations are quite background noise limited. Rather than just doing simple aperture photometry, it might be better to do optimal photometric extraction using code very similar to our optimal spectral extraction.

Code of Conduct

hdiamondlowe commented 2 months ago

This would be a very helpful addition for MIRI photometry! We have seen positive results from implementing optimal photometric extraction in other pipelines.

taylorbell57 commented 2 months ago

@hdiamondlowe, thanks for the feedback! We're working hard on getting a stable, thoroughly tested v1.0 release, and this will be somewhat near the top of my to-do list after that

hdiamondlowe commented 2 months ago

Hi @taylorbell57 I've been thinking about this and poking around in @kevin218 's optimal extraction routines in POET, and I wonder if it wouldn't be too terrible to adapt those into Eureka. If this is something that would be potentially helpful to you, I can take a few days to try it out, unless you think is this is a terrible idea. I'm not sure if I'll be successful at any of this, but if I have your blessing I can give it a shot on my forked version of Eureka and let you know how it goes.

kevin218 commented 2 months ago

It would be great to have optimal extraction for photometry. The framework is already in place for spectroscopy, so it would "just" be a matter of pushing the photometry files through the same steps within Eureka!.

@hdiamondlowe we're on the verge of releasing v1.0, which uses Python 3.10 and requires a fresh install with a new Conda environment. Before making any changes, please make sure you're using the latest version. Also, rather than using a forked version of the code, I suggest creating a branch.

taylorbell57 commented 2 months ago

@kevin218, since Hannah isn't a "Contributor" she'll have to work in a fork rather than a branch; doesn't matter too much though, it just means we end up needing to run the test suites locally since the automated testing fails for forks. Please do make a new branch within your forked repo though rather than working in the main branch of your fork - makes some things a bit easier for us

I did implement a functional but messy version of optimal photometric extraction in a Jupyter notebook when working on reducing the data for Greene+2023 TRAPPIST-1b 15 micron eclipse paper, and the performance ended up being similar to but slightly worse than the aperture photometry reduction. Given that was at 15 microns and in a region of the sky with high thermal background, I suspect aperture photometry will likely be the best option for most/all JWST photometric observations, but that's obviously still to be seen and we'll add optimal photometric extraction regardless.

@hdiamondlowe, we're definitely human-resource limited, and if adding this feature would be helpful to you and you're motivated to work on it soon, then by all means please go ahead. Once you have something working, you can submit a Pull Request and we'll work with you to make sure everything is formatted and integrated properly. I tried looking for my old Jupyter notebook to share with you to help get you started, but it seems that I may have accidentally deleted it when purging old files to make room for new datasets. I'll try looking in my Macbook backups

hdiamondlowe commented 2 months ago

Hi @taylorbell57 ,@kevin218: Happy to make a new branch in my forked version as you suggest. I can pull the newest version and build on that, instead of the current stable release. I think this capability will be helpful since we are not using the full array at 15 microns. I'll start taking a look tomorrow or Thursday but then will be on vacation for a bit so not super sure of the timeline. I'll keep in touch though.

taylorbell57 commented 2 months ago

@hdiamondlowe, I just found my old notebook, and edited it a bit to add some comments so that it'd be easier for you to parse. Everything in the notebook is independent from the methods in Eureka! and mostly used photutils; was just quicker and easier for me back then. Hopefully this helps accelerate things for you. I've had to zip the file since GitHub doesn't allow me to upload .ipynb files.

TestingOptimalPhotometricExtraction.ipynb.zip