Cannot merge data

[LewisDrysdale]

I am suddenly unable to merge any seaglider data. This includes data that I have previously successfully merged. I'm not a Pandas expert so I'm finding this quite difficult to debug. I thought it may be an issue with some of the glider files but I've been sub setting the files randomly and it is still an issue. Any help would be much appreciated, or to know if anyone else has experience this issue?

I have gone back to follow the exact instructions from the Glider Tools docs

Screenshot example:

Environment.yml file I am using

name: oceanglider channels:

• conda-forge
• defaults dependencies:
• python=3.9
• numpy=1.23.1
• jupyter
• pandas=1.5
• scikit-learn
• scipy=1.5.3
• glidertools
• cmocean
• pip=23.0
• jupyterlab
• ipympl
• pip:
  • cftime==1.6.2
  • gsw==3.6.16
  • markdown==3.3.4
  • markdown2==2.4.2
  • markdown2pdf==0.1.4
  • mat4py==0.5.0
  • motuclient==1.8.8
  • scp==0.14.4
  • seaborn==0.12.2
  • seawater==3.3.4
  • cf-python==3.14.1 prefix: C:\Users\sa01ld\Miniconda3

[callumrollo]

Hi Lewis. Are you able to run the example in the demo notebooks with the Seaglider files that ship with the package? If not, what error message do you get there?

What are the dimensions of the unmerged dataframes that it returns?

[LewisDrysdale]

Hi Callum, thanks for getting back to me. I'm partly convinced it must be something to do with my data, but I just cant get my head around why,.

The test data run fine - i downloaded a few of the files from GitHub repo

The dimensions are sg_data_point

[LewisDrysdale]

You can find some of the data I am working with here https://thredds.sams.ac.uk/thredds/catalog/sg534/mission_5/catalog.html

[callumrollo]

Ah that makes sense! If the only dimension is sg_point, there's nothing to merge. If you remove 'eng_qsp_PARuV' from the list of variables in the example notebook you'll get the same result.

You should be good to use ds_dict['sg_data_point'] as your merged dataframe for GliderTools functions

[LewisDrysdale]

Ah, OK. So, I already have a merged dataframe - just not on time. I can see that now. I still can't understand why I was able to have merged as my datafram before. I think perhaps because I was using some variable that used GPS time. I need to understand more about Pandas.

Thanks very much, I really appreciate the help.

[callumrollo]

Yep, I think that's the case. If you want to get a time-indexed pandas dataframe from the xarray dataset returned by the load merge function, you can do something like:

merged = ds_dict['sg_data_point']
df = merged.to_pandas()
df.index = df["ctd_time_dt64"]
df = df.drop("ctd_time_dt64", axis=1)