search

lkilcher / dolfyn

A library for oceanographic doppler instruments such as Acoustic Doppler Profilers (ADPs, ADCPs) and Acoustic Doppler Velocimeters (ADVs).
BSD 3-Clause "New" or "Revised" License
42 stars 25 forks source link

Trouble joining two AD2CP datasets #60

Closed mcfogarty closed 2 years ago

mcfogarty commented 5 years ago

Trying to merge two AD2CP files, one from the first part of the day (.102a.h5) and one from the latter part of the day (.102b.h5).

This merge was attempted in using the following script and works for the main groups (e.g. 'vel', but not the. subgroups (e.g. 'env', 'orient'). This method is failed because of the if statement.

day_wrap0 = dolfyn.load(fname_sig + '.102a.h5')

. 16.80 hours (started: Jul 15, 2017 00:00) . earth-frame . (54 bins, 120993 pings @ 2Hz) day_wrap = dolfyn.load(fname_sig + '.102b.h5') . 7.20 hours (started: Jul 15, 2017 16:48) . earth-frame . (54 bins, 51806 pings @ 2Hz) # Now join the two split datasets (fill NaN between) n_end = len(day_wrap.mpltime) # 51806 n_str = len(day_wrap0.mpltime) # 120993 outd = day_wrap.copy() # copy of latter part of the day, ...102b.h5 for ky in day_wrap.keys(): #day_wrap.keys() # dict_keys(['props', 'alt', 'altraw', 'config', 'env', # 'mpltime', 'orient', 'range', 'range_b5', 'signal', # 'sys', 'vel', 'vel_b5']) try: dend = day_wrap[ky] # dend is the contents of each key, length 51806, latter part of the day dstr = day_wrap0[ky] # dstr is the contents of each key, length 120993, beginning of day except AttributeError: continue if (isinstance(dend, np.ndarray) and dend.shape[-1] == n_end): # isinstance is false if ky = env, because type(dend) is dolfyn.data.base.TimeData # dend.shape[-1] == n_end throws error with ky = env, TimeData' object has no attribute 'shape' # isinstance is true if ky = vel, because type(dend) is numpy.ndarray # dend.shape[-1] == n_end is true if ky = vel shp_out = list(dstr.shape) shp_out[-1] = n_day tmpd = np.empty(shp_out, dtype=dend.dtype) tmpd[..., :n_str] = dstr try: tmpd[..., n_str:-n_end] = np.NaN except ValueError: tmpd[..., n_str:-n_end] = -1 tmpd[..., -n_end:] = dend outd[ky] = tmpd outd.to_hdf5(fname_sig + '.102.h5') This yields . 24.00 hours (started: Jul 15, 2017 00:00) . earth-frame . (54 bins, 172800 pings @ 2Hz) *------------ | mpltime : | range : | range_b5 : | vel : | vel_b5 : But the subgroups have not been merged (e.g. dat.env) : Data Object with Keys: *------------ | c_sound : | press : | temp : I attempted to use the append function from https://lkilcher.github.io/dolfyn/api.html?highlight=append#dolfyn.Velocity.append append(other) Join two data objects together. For example, two data objects d1 and d2 (which must contain the same variables, with the same array dimensions) can be joined together by: >>> dat = d1.append(d2) dat102 = dat1.append(dat2) But nothing happens. If in Visual Source Code I right-click on append, no function is found.
lkilcher commented 2 years ago

This issue is no longer relevant since we are moving to an xarray data-type.