search

PaulHancock / MWA-fast-image-transients

Code and notes for the data reduction and analysis of fast image transients observed with the MWA
Other
5 stars 3 forks source link

dl_obs still crashing - memory issues #13

Closed gemmaanderson closed 6 years ago

gemmaanderson commented 6 years ago

I have tried running the dl_obs on the calibrator observation 1205618352. It seems to get as far as cotter but is reporting download errors too even though the output file reports that the file transfer was successful:

% less dl_1205618352.e2468875 --2018-05-04 14:36:22-- http://mwa-metadata01.pawsey.org.au/metadata/fits?obs_id=1205618352&min_bad_dipoles=2 Resolving mwa-metadata01.pawsey.org.au (mwa-metadata01.pawsey.org.au)... 146.118.87.249 Connecting to mwa-metadata01.pawsey.org.au (mwa-metadata01.pawsey.org.au)|146.118.87.249|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 48960 (48K) [image/fits] Saving to: ‘1205618352_metafits.fits’

 0K .......... .......... .......... .......... .......   100% 45.3M=0.001s

2018-05-04 14:36:22 (45.3 MB/s) - ‘1205618352_metafits.fits’ saved [48960/48960]

ls: cannot access '1205618352*.mwaf': No such file or directory /var/spool/slurm/job2468875/slurm_script: line 63: syntax error near unexpected token fi' /var/spool/slurm/job2468875/slurm_script: line 63:fi' slurmstepd: error: Exceeded step memory limit at some point.

% less cotter_1205618352.e4281046 n unhandled exception occured while running Cotter: Cannot open file 1205618352_metafits_ppds.fits. CFitsio reported an error while accessing fits files. Status = 104: could not open the named file. Error message stack: failed to find or open the following file: (ffopen). 1205618352_metafits_ppds.fits.

PaulHancock commented 6 years ago

This particular issue is actually just another instance of #16, which has been addressed.