c-proof / pyglider

glider software
https://pyglider.readthedocs.io/
Apache License 2.0
17 stars 24 forks source link

RecursionError: maximum recursion depth exceeded #165

Open mackenziemeier86 opened 4 months ago

mackenziemeier86 commented 4 months ago

Using the example code, I am having trouble with the slocum.raw_to_timeseries. It doesn’t seem to be running correctly and therefore isn’t defining “outname”. I get returned the attached error message-

Error

I am using the example code to try to run this (from example-slocum), so I think it is something with how I am running the code and not my files. Is anyone familiar with this and can provide any advice? I'm not very familiar with python so unsure what this error means or why I am having this issue.

jklymak commented 4 months ago

@mackenziemeier86 can you report the version of pyglider you are using, the version of xarray, and the version of dask? You can usually get these using condo list

It has also been a long time since we did a release, so it's possible that pyglider no longer works properly with the latest versions of its dependencies. We should do a new release "soon", but in the meantime, I'd suggest doing an editable install as https://pyglider.readthedocs.io/en/latest/Install.html#editable-installation and see if that fixes the problem.

If that fails, please post links to a a subset of the breaking data.

Thanks!

mackenziemeier86 commented 4 months ago

pyglider 0.0.4 pyhd8ed1ab_0 conda-forge xarray 2024.3.0 pyhd8ed1ab_0 conda-forge dask-core 2024.4.1 pyhd8ed1ab_0 conda-forge

And this is the data I am using: https://github.com/c-proof/pyglider-example-data/tree/main/example-slocum

Thank you!

jklymak commented 4 months ago

Ah, I see. We definitely should re-release pyglider.

1) In your environment do pip install dbdreader 2) Edit your processing to

# turn *.sdb and *.tbd into timeseries netcdf files
outname = slocum.binary_to_timeseries(
        binarydir, cacdir, l1tsdir, deploymentyaml, search='*.[s|t]bd',
        profile_filt_time=20, profile_min_time=20)
outname = ncprocess.make_gridfiles(outname, griddir, deploymentyaml)

(and remove the raw_to_nc and nc_to_timeseries).

It seems that the old method no longer works with modern xarray, and we very much discourage the old method.

mackenziemeier86 commented 4 months ago

Thanks Jody! How do I get binary_to_timeseries loaded?

jklymak commented 4 months ago

Ah, OK, definitely install as in https://pyglider.readthedocs.io/en/latest/Install.html#editable-installation in your environment. I will work on a proper release in the next couple days

jklymak commented 4 months ago

OK, new release 0.0.5. I think conda install will work soon, but pip install pyglider will install the latest version.

mackenziemeier86 commented 4 months ago

Thanks so much! The only line I am stuck on now is: ncprocess.extract_timeseries_profiles(outname, profiledir, deploymentyaml) is there a new alternative to this too?

jklymak commented 4 months ago

No that should work. What error is that giving?

mackenziemeier86 commented 4 months ago

Working now sorry! Just an order of operations mistake I think.

Thanks so much for your help!

callumrollo commented 4 months ago

I have also been able to reproduce the recursion depth error on a fresh install of pyglider, using the example SeaExplorer data. I believe the problem comes from the interaction of numpy and xarray https://github.com/pydata/xarray/issues/8848

For me it was tiggered by np.fix in utils.nmea2deg which operates on nmea, an xr.DataArray. I fixed it by converting the DataArray to a regular numpy array with nmea = nmea.values

I'm investigating this and will put in a PR when ready

jklymak commented 4 months ago

Weird if it's a xarray issue. Maybe there was a bad version out there