search

oirlab / liger_iris_pipeline

Data Reduction System (DRS) for the Thirty Meter Telescope IRIS imager/spectrograph
https://oirlab.github.io/iris-pipeline
0 stars 3 forks source link

ROP pipeline prototype #7

Closed arunsurya77 closed 4 years ago

arunsurya77 commented 5 years ago

ROP pipeline prototype implementation. Implements sampling and non-linearity correction. Currently, non-linearity correction coefficients need to be supplied. Sampling code requires drsrop_clib, currently a separate repo. Documentation in drsrop_pipeline.rst (not linked)

zonca commented 5 years ago

thanks @arunsurya77 you can use git submodule https://git-scm.com/book/en/v2/Git-Tools-Submodules and add the Python and Cython code to iris_pipeline and import the C library from iris_readout

zonca commented 5 years ago

Please let me know when this is ready for review

arunsurya77 commented 5 years ago

I have the C library inside the iris_pipeline in my fork now. I am currently working to get the c lib as a separate repo as a submodule. Doing some tweaking for this. i will anyway give a review request.

On Thu, 27 Jun 2019 at 00:52, Andrea Zonca notifications@github.com wrote:

Please let me know when this is ready for review

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/oirlab/iris_pipeline/pull/7?email_source=notifications&email_token=AKB37LU7C3KZX4B2JX3LBQTP4RWTHA5CNFSM4HIYR3FKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYWITZY#issuecomment-506235367, or mute the thread https://github.com/notifications/unsubscribe-auth/AKB37LSJQOSWMXOQ6GC24ELP4RWTHANCNFSM4HIYR3FA .

zonca commented 4 years ago

@arunsurya77 I made some updates to iris_pipeline so there are some conflicts now, also make sure there are some unit tests that automatically run. I currently have some tests which are Jupyter Notebooks, I like them because they are easy to run interactively and to debug anything gone wrong, it also includes some plotting, see for example: https://github.com/oirlab/iris_pipeline/blob/master/iris_pipeline/tests/run_image2_subarray.ipynb

arunsurya77 commented 4 years ago

Hi Andrea, I will check the merge conflicts and try to resolve them and add the tests.

On Mon, 16 Mar 2020 at 10:53, Andrea Zonca notifications@github.com wrote:

@arunsurya77 https://github.com/arunsurya77 I made some updates to iris_pipeline so there are some conflicts now, also make sure there are some unit tests that automatically run. I currently have some tests which are Jupyter Notebooks, I like them because they are easy to run interactively and to debug anything gone wrong, it also includes some plotting, see for example: https://github.com/oirlab/iris_pipeline/blob/master/iris_pipeline/tests/run_image2_subarray.ipynb

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/oirlab/iris_pipeline/pull/7#issuecomment-599677458, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKB37LRIINB37IJIMKY7CBLRHZRRPANCNFSM4HIYR3FA .

zonca commented 4 years ago

@arunsurya77 what is the stage of this? I think this should be high priority, it would help a lot to have a clearer picture of all features of iris_pipeline and improve the documentation.

arunsurya77 commented 4 years ago

Yeah I agree, sorry about the delay. I hadn't had time to work on this. I will try to fix some of the outstanding issues this week and update the rst file of this. There are some problems I am facing with the monkey patching. But I will atleast try to do push all the updates so that you can have a look.

zonca commented 4 years ago

@arunsurya77 do you confirm this code is currently working on your machine?

arunsurya77 commented 4 years ago

High I'm just editing the last bits for the rst documentation. I'll make a pull request tonight. It's just a few more changes but everything is working, I managed to fix the erros.

On Tue, 9 Jun 2020 at 19:24, Andrea Zonca notifications@github.com wrote:

@arunsurya77 https://github.com/arunsurya77 do you confirm this code is currently working on your machine?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/oirlab/iris_pipeline/pull/7#issuecomment-641680666, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKB37LVCOU2CDQFO3VRG56LRV3VGFANCNFSM4HIYR3FA .

arunsurya77 commented 4 years ago

I have added most of the ROP Pipeline Skelton in this pull request. I have verified it working in another ubuntu system also. Just to give you the installation process. I have documented it in the rop_pipeline.rst file also.

requirements -- libcfitsio [ libcfitsio-bin and libcfitsio-dev (sudo apt-get install libcfitsio*)] cython

download 'sample_ramp_new.fits<' https://figshare.com/articles/sample_ramp_new_fits/12462491> to iris_pipeline/tests/data/ and iris_pipeline/readout/tests/ .

The pytest is currently minimal which just runs the ROP pipleine on the data file. I am adding more tests and some frames from the simulator.

zonca commented 4 years ago

let's switch to https://github.com/oirlab/iris_pipeline/pull/27 so I can help edit this.