Closed ipashchenko closed 8 years ago
Finally i polished dyn_spectra.DynSpectra
API to level when it is the only thing we need to go further (manage pipeline etc.). Now, to proceed, i need M5
(or whatever) instances to create instances of DynSpectra
class (one instance for each chunk of Mark5 file). This part is for you, @akutkin ;) I think it would be easy cause DynSpectra
creating needs only dyn. spectra array + metadata with fixed keys.
resovled with 2866a0f3029c63a49703f3a8a0890b41c268fd73
Could you refactor
raw_data.py
to makeM5
instances to return not onlynumpy
array with chunk of dynamical spectra, but also metadata dictionary with the following keys:n_nu
,n_t
,nu_0
,d_nu
,d_t
,t_0
,exp_code
,antenna
,freq
,band
,pol
? I finally merged metadata with dynamical spectra array in one class. After creatingDynSpectra
instances with array & metadata we don't need anything else for subsequent processing.It could be cool if method that does this will return generator - not to store all arrays in memory but creating them from Mark5 only when they are needed in small chunks (like 5 minutes or even less).
Another option - to put data in
HDF5
format that can save metadata too. Seeutils.save_hdf5
as example.