EoRImaging / eppsilon

eppsilon - error propagated power spectrum with interleaved observed noise
BSD 2-Clause "Simplified" License
5 stars 4 forks source link

Remove Healpix requirement for easy cluster management #143

Open nicholebarry opened 4 years ago

nicholebarry commented 4 years ago

When refreshing the info file, the Healpix cubes are required, even if the uvf_cubes exist. It would be best to extract the information from the Healpix cubes into the uvf_cubes so that the Healpix cubes don't need to be downloaded for aws users.

Example: Mike downloaded the uvf cubes from aws to a local computer. He then tried to refresh_info.

IDL> ps_wrapper, "/Volumes/Faramir/2014_GS/2014_PS/GS_freq_broadcast_dgpb", "Combined_obs_2014_GS_plus_one", /refresh_info, /refresh_beam, /png % PS_FILENAMES: No cube or info files found in folder /Volumes/Faramir/2014_GS/2014_PS/GS_freq_broadcast_dgpb % Execution halted at: PS_FILENAMES 1201 /Users/mikewilensky/Repos/eppsilon/ps_wrappers/ps_filenames.pro % PS_WRAPPER 122

bhazelton commented 4 years ago

So all the required information from the HEALPix files is extracted and saved in the info file. So the solution is to not have to refresh info. Why was that needed?

nicholebarry commented 4 years ago

I copied the uvf cubes and the info file from the cluster onto my local computer, and tried to run eppsilon to see if the refresh_info keyword was needed or not.

IDL> ps_wrapper, '/Users/nabarry/MWA/data/fhd_nb_test/','1095954144',/full_image,/png
datafile = /Users/nabarry/MWA/data/fhd_nb_test/ps/1095954144_059-078_gridded_uvf__even_odd_joint_uvf_info.idlsave
file setup time: 0.010020971
n_vis difference between even & odd cubes: 0
n_vis % difference between even & odd cubes: 0
n_obs:            1           1
getvar_savefile: file /fred/oz048/MWA/CODE/FHD/fhd_nb_gaussbeam_1fits_ultralow2/1095954144_059-078_even_gridded_uvf.sav not found
% PS_IMAGE_TO_UVF: Error getting data out of file: /fred/oz048/MWA/CODE/FHD/fhd_nb_gaussbeam_1fits_ultralow2/1095954144_059-078_even_gridded_uvf.sav If the file name has changed,
                    set "/refresh_info" in the ps_wrapper call.
% Execution halted at: PS_IMAGE_TO_UVF   254 /Users/nabarry/MWA/eppsilon/ps_core/ps_image_to_uvf.pro
%                      PS_KCUBE          163 /Users/nabarry/MWA/eppsilon/ps_core/ps_kcube.pro
%                      PS_POWER           20 /Users/nabarry/MWA/eppsilon/ps_core/ps_power.pro
%                      PS_MAIN_PLOTS     272 /Users/nabarry/MWA/eppsilon/ps_core/ps_main_plots.pro
%                      PS_WRAPPER        304 /Users/nabarry/MWA/eppsilon/ps_wrappers/ps_wrapper.pro
%                      $MAIN$          
IDL> ps_wrapper, '/Users/nabarry/MWA/data/fhd_nb_test/','1095954144',/full_image,/png,/refresh_info
bash: hostname: command not found
% PS_FILENAMES: No cube or info files found in folder /Users/nabarry/MWA/data/fhd_nb_test
% Execution halted at: PS_FILENAMES     1201 /Users/nabarry/MWA/eppsilon/ps_wrappers/ps_filenames.pro
%                      PS_WRAPPER        123 /Users/nabarry/MWA/eppsilon/ps_wrappers/ps_wrapper.pro
%                      PS_IMAGE_TO_UVF   254 /Users/nabarry/MWA/eppsilon/ps_core/ps_image_to_uvf.pro
%                      PS_KCUBE          163 /Users/nabarry/MWA/eppsilon/ps_core/ps_kcube.pro
%                      PS_POWER           20 /Users/nabarry/MWA/eppsilon/ps_core/ps_power.pro
%                      PS_MAIN_PLOTS     272 /Users/nabarry/MWA/eppsilon/ps_core/ps_main_plots.pro
%                      PS_WRAPPER        304 /Users/nabarry/MWA/eppsilon/ps_wrappers/ps_wrapper.pro
%                      $MAIN$          
IDL> 
bhazelton commented 4 years ago

So I just ran into this and I think I understand the confusion better now. The issue (at least in my case) is that in addition to the uvf_cubes, eppsilon also needs the beam_cubes, which are stored in the ps folder under data/beam_cubes. If those are present, the info file should not need to be refreshed and the Healpix cubes should not be needed.