search

Keck-DataReductionPipelines / MosfireDRP

http://keck-datareductionpipelines.github.io/MosfireDRP
10 stars 13 forks source link

Problems with Wavelength.py #43

Closed Ana-Luisa-GM closed 8 years ago

Ana-Luisa-GM commented 8 years ago

Hi all,

I am a new user of MOSFIRE. The Last January 27, I observed three cosmological fields (UDS,COSMOS and GOODS-N). I have reduced the first two slitmasks using the MOSFIRE SPECTROSCOPY DATA REDUCTION PIPELINE that Luca provided during the observation night. The problem is that when I try to reduce my last slitmask (GOODS-N), the pipeline stops and appears this error in my stream:

2016-02-03 22:20:46,497 - IO. readmosfits - WARNING: /home/analuisa/MOSFIRE/DRP_CODE/DATA/ALL_2016jan28/2016jan28/m160128_0329.fits contains a DATASEC keyword not compatible with the pipeline 2016-02-03 22:20:46,497 - IO. readmosfits - WARNING: The content of the keyword will be erased on the reduced data 2016-02-03 22:20:47,622 - Wavelength. imcombine - INFO: Checking maskname and filter 2016-02-03 22:20:47,623 - Wavelength. imcombine - INFO: /home/analuisa/MOSFIRE/DRP_CODE/DATA/ALL_2016jan28/2016jan28/m160128_0329.fits GOODS-N_field_new4/H Traceback (most recent call last): File "Driver.py", line 23, in Wavelength.imcombine(obsfiles, maskname, band, waveops) File "/home/analuisa/MOSFIRE/DRP_CODE/MOSFIRE/Wavelength.py", line 234, in imcombine electrons = np.median(np.array(ADUs) * Detector.gain, axis=0) MemoryError

I don't know what happened with this slitmask, since the two first slitmasks (UDS and COSMOS) were reduced without any problem. Reviewing the file "Wavelength.py" I found that it could be due to the fact that the slitname is not defined but I am not sure. I remember that when we were starting this slitmask (GOODS-N) in the stream of the control room appeared that the slitname wasn't defined and we put 'GOODS-N' to this. Do you have any solution for my problem?. Thanks in advance.

Ana Luisa

KeckDRP commented 8 years ago

Ana Luisa I am not sure that is the problem either but could you clarify your sentence?

I am not sure I remember that when we were starting this slitmask (GOODS-N) in the stream of the control room appeared that the slitname wasn't defined and we put 'GOODS-N' to this

I am not sure I understand what you mean. Luca

On Feb 5, 2016, at 2:27 PM, Ana-Luisa-GM notifications@github.com wrote:

Hi all,

I am a new user of MOSFIRE The Last January 27, I observed three cosmological fields (UDS,COSMOS and GOODS-N) I have reduced the first two slitmasks using the MOSFIRE SPECTROSCOPY DATA REDUCTION PIPELINE that Luca provided during the observation night The problem is that when I try to reduce my last slitmask (GOODS-N), the pipeline stops and appears this error in my stream:

2016-02-03 22:20:46,497 - IO readmosfits - WARNING: /home/analuisa/MOSFIRE/DRP_CODE/DATA/ALL_2016jan28/2016jan28/m160128_0329fits contains a DATASEC keyword not compatible with the pipeline 2016-02-03 22:20:46,497 - IO readmosfits - WARNING: The content of the keyword will be erased on the reduced data 2016-02-03 22:20:47,622 - Wavelength imcombine - INFO: Checking maskname and filter 2016-02-03 22:20:47,623 - Wavelength imcombine - INFO: /home/analuisa/MOSFIRE/DRP_CODE/DATA/ALL_2016jan28/2016jan28/m160128_0329fits GOODS-N_field_new4/H Traceback (most recent call last): File "Driverpy", line 23, in Wavelengthimcombine(obsfiles, maskname, band, waveops) File "/home/analuisa/MOSFIRE/DRP_CODE/MOSFIRE/Wavelengthpy", line 234, in imcombine electrons = npmedian(nparray(ADUs) * Detectorgain, axis=0) MemoryError

I don't know what happened with this slitmask, since the two first slitmasks (UDS and COSMOS) were reduced without any problem Reviewing the file "Wavelengthpy" I found that it could be due to the fact that the slitname is not defined but I am not sure I remember that when we were starting this slitmask (GOODS-N) in the stream of the control room appeared that the slitname wasn't defined and we put 'GOODS-N' to this Do you have any solution for my problem? Thanks in advance

Ana Luisa

— Reply to this email directly or view it on GitHub https://github.com/Keck-DataReductionPipelines/MosfireDRP/issues/43.

Ana-Luisa-GM commented 8 years ago

Hi Luca, Thanks for your attention. I mean that when we were starting with the slitmask (GOODS-N), in the computer of the control room appeared that the slitname wasn't defined and so we decided to define it as 'GOODS-N'. I don't know if it could have affected.

Cheers, Ana Luisa

KeckDRP commented 8 years ago

HI Ana Luisa, I am downloading your data and I will try to figure out what is going on with the GOODS-N field

Luca

On Feb 5, 2016, at 4:14 PM, Ana-Luisa-GM notifications@github.com wrote:

Hi Luca, Thanks for your attention. I mean that when we were starting with the slitmask (GOODS-N), in the computer of the control room appeared that the slitname wasn't defined and so we decided to define it as 'GOODS-N'. I don't know if it could have affected.

Cheers, Ana Luisa

— Reply to this email directly or view it on GitHub https://github.com/Keck-DataReductionPipelines/MosfireDRP/issues/43#issuecomment-180658979.

Ana-Luisa-GM commented 8 years ago

Hi Luca,

I have chosen the first half of the dataset (from m160128_0244.fits to m160128_0286.fits) and I have reduced without problem this dataset. Therefore, I think that something is wrong in the second part. Anyway I will try reduce this second dataset and see what happens.

Ana Luisa

lucarizzi commented 8 years ago

Ana Luisa:

I have reduced the entire set of GOODS-N images, and it worked just fine. In fact, I can see numerous nice sets of emission lines.

Is there a chance that you might be running out of memory on your computer? It is suspicious that you are able to reduce a subset of the images. Also, this is a very large number of images: during the wavelength imcombine phase, you are trying to combine 92 images.

My suggestion, in your case, is this:

  1. make a copy of the Offset1.5.txt and Offset-1.5.txt files, call them Offset_1.5wave.txt, and Offset-1.5_wave.txt
  2. Remove most of the images from these new files, so that they both contain only about 10 images
  3. In your Driver.py file, replace obsfiles=['Offset1.5.txt','Offset-1.5.txt'] with the new files you have copied
  4. Run mospy Driver.py but only do Flats and the Wavelength blocks (comment out Background and Rectify)
  5. Once it is done, replace the Wavelength_file = 'lambda...." with the file that resulted from the previous step. You will have to find it in your directory.
  6. Just below the Wavelength_file line, add: obsfiles=['Offset1.5.txt','Offset-1.5.txt']
  7. Comment out Flats and Wavelength blocks, and uncomment Background and Rectify.
  8. Run mospy Driver.py

Let me know how it goes.

Ana-Luisa-GM commented 8 years ago

Thank you very much Luca..

Answering the question (Is there a chance that you might be running out of memory on your computer?), I think that yes because I am working with an old computer. In fact, if I omit the last 2 blocks in my observations (from m160128_0323-m160128_0330) taking the 91% of the observations, I am able to reduce without any problem. In fact, I can see the nice set of emission lines that you told me. I think that it does not affect me because I have observed this field more than 3 hours.

Ana Luisa