NOAA-PMEL / Ferret

The Ferret program from NOAA/PMEL
https://ferret.pmel.noaa.gov/Ferret/
The Unlicense
55 stars 21 forks source link

Strange memory error after ZAXREPLACE() and cancelling an ENSEMBLE aggregation #1991

Open AndrewWittenberg opened 2 years ago

AndrewWittenberg commented 2 years ago

Create a tiny file containing a z-oriented variable named A.

   NOAA/PMEL TMAP
   FERRET v7.63 (optimized)
   Linux 3.10.0-1160.25.1.el7.x86_64 64-bit - 08/25/21
    4-Feb-22 14:30

yes? let a = zsequence({1,2,3})
yes? save/q/clob/file="a.nc" a
yes? can var a

Define pseudo-variable B, in which A's z-axis is simply replaced with itself, then list the start coordinate. This works fine.

yes? use "a.nc"
yes? let b = zaxreplace(a,a,a)
yes? say "Value of B is: `b,r=zstart`"
 !-> MESSAGE/CONTINUE "Value of B is: 1"
Value of B is: 1

Save out this variable as "C".

yes? let c = b
yes? save/q/clob/file="c1.nc" c
yes? can var c

Duplicate the file.

yes? sp cp -f c1.nc c2.nc

Create an ensemble aggregation, then immediately cancel it. If we comment out either of these lines, then the crash below will disappear!

yes? ensemble ens_agg = {"c1.nc","c2.nc"}
yes? can dat/all

Try to list the start coordinate of pseudo-variable "B" as before. It now fails!

yes? use "a.nc"
yes? say "Value of B is: `b,r=zstart`"
 **ERROR: too many variables to store in memory

This does not fail in the v7.63 compiled on 1/4/21, only the version compiled on 8/25/21. So there must have been some change in early 2021 that caused this. This is causing many of our analysis scripts to fail, so I am marking this as urgent.

eugeneburger commented 2 years ago

@josborne-noaa From the last sentence in Andrews message, can you please confirm if any changes have been committed to the PyFerret release since January 2021?

seasage commented 2 years ago

@AndrewWittenberg I wasn't able to replicate this with the current release. None of the changes made in the last year deal with the memory management portion of the code-base.