Undesired appending of new data in CDAWeb interface

[jameswilburlewis]

Nick Gross posted on helionauts about a problem he's having with the current version of the pyspedas CDAWeb support:

https://helionauts.org/t/pyspedas-reading-voyager-data/1538

The initial problem was when retrieving more than a month of data, each month would over right the last and I would be left with only the last months data set.

Jim was able to solve the initial problem in the last PySPEDAS release , but now a new problem has arisen. After loading an initial period of data, when. try to load a new period, it appends it to the previous data (which is what seemed to solve the last problem) and I have no way of clearing it. You can see my code block below for loading data. The only other way I have of clearing this is by restarting the “runtime” environment, but this of course is less then ideal.

I can, of course work around this, by using a regular plotting tool rather than plot, but thought I would see if there was an easier approach.

I think the problem might be with using the merge option in cdf_to_tplot (and probably netcdf_to_tplot) to ensure that multiple files from a single request are all loaded into tplot variables.

Both routines support passing multiple filenames in one call. I think the better fix might be to have merge=False the default, and if multiple files are returned by a request, to pass them all in a single call to cdf_to_tplot, and let it handle iterating over and merging the data files, rather than doing the iteration in the cdaweb code with merge=True.

[jameswilburlewis]

It may be necessary to group the data somehow and send it to cdf_to_tplot in batches. But we know cdf_to_tplot can work with having mms1, mms2, mms3, and mms4 data files intermingled, so it's worth trying to send them all in one call, it might just do the right thing!

It also might be necessary to sort the filenames in time order, depending on how CDAWeb, SPDF, or the MAVEN SDC return the file lists.