MAVEN tests print "unclosed file" warnings for cdflib

[nickssl]

MAVEN tests print three warnings similar to the following:

17-Feb-24 12:55:39: C:\work\github\envs\pyspedas\Lib\site-packages\cdflib\cdfread.py:789: ResourceWarning: unclosed file <_io.BufferedRandom name='...\\AppData\\Local\\Temp\\tmpn5nrmk4e.cdf'> self.temp_file = Path(tempfile.NamedTemporaryFile(suffix='.cdf').name)

This seems to indicate that the "cdflib" library does not properly close some files. If this is the case, it should be fixed.

[jameswilburlewis]

Interesting that I'm not seeing this in the Github test logs! Which tests are you seeing this in, and what Python version? Maybe it only happens on Windows.

[nickssl]

This warning shows up with python 3.11.8 on Windows 11. File pyspedas/maven/tests/tests.py. The result of the tests is "OK" but it prints three similar "unclosed file" warnings.

The following part prints two warnings:

    def test_load_sta_data(self):
        data = maven.sta()
        self.assertTrue(data_exists("hkp_raw_2a-hkp"))

26-Feb-24 10:46:42: C:\work\github\envs\pyspedas\Lib\site-packages\cdflib\cdfread.py:789: ResourceWarning: unclosed file <_io.BufferedRandom name='C:\\Users\\nikos\\AppData\\Local\\Temp\\tmpqhrn6muf.cdf'> self.temp_file = Path(tempfile.NamedTemporaryFile(suffix='.cdf').name)

26-Feb-24 10:46:42: C:\work\github\envs\pyspedas\Lib\site-packages\cdflib\cdfread.py:789: ResourceWarning: unclosed file <_io.BufferedRandom name='C:\\Users\\nikos\\AppData\\Local\\Temp\\tmplfgk61wp.cdf'> self.temp_file = Path(tempfile.NamedTemporaryFile(suffix='.cdf').name)

I now realize that in pyspedas we have the requirement "cdflib<1.0.0". This has to be removed before we look into this, since the current version of cdflib is 1.2.4.

[nickssl]

See also: https://github.com/spedas/pyspedas/issues/504

[jameswilburlewis]

For what it's worth, I don't see this warning on python 3.9 with cdflib versions 1.2.6 or 0.4.9. Can you retest with python 3.11 and cdlflib 1.2.6 and see if the warning is still there? (The latest pyspedas and pytplot-mpl-temp have removed the version requirements on cdflib).

[nickssl]

These are just warnings, not errors, so they might not be important.

I tested the latest cdflib version 1.2.6 on Windows 11. Using python 3.12.2, I do not get the "unclosed file" warnings. But with python 3.11.9 I still get these warnings, using the same cdflib in both cases!

So, this is not a cdflib issue after all, and I think we can close this issue.

[nickssl]

For completion, the MAVEN tests.py file also gives some other warnings:

1. Both for python 3.12.2 and python 3.11.9:

11-Apr-24 11:44:59: C:\work\GitHub\envs\spedas\Lib\site-packages\pytplot\store_data.py:234: UserWarning: Converting non-nanosecond precision datetime values to nanosecond precision. This behavior can eventually be relaxed in xarray, as it is an artifact from pandas which is now beginning to support non-nanosecond precision values. This warning is caused by passing non-nanosecond np.datetime64 or np.timedelta64 values to the DataArray or Variable constructor; it can be silenced by converting the values to nanosecond precision ahead of time. temp = xr.DataArray(values, dims=['time']+dimension_list,

2. Only for python 3.12.2:

11-Apr-24 11:45:03: C:\work\GitHub\envs\spedas\Lib\site-packages\pytplot\store_data.py:161: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC). times = [datetime.datetime.utcfromtimestamp(float(time)) for time in times]

[nickssl]

Complete output files with warnings from maven tests, using python 3.11.9 and python 3.12.2, on windows 11.

maven_test_err_3.11.9.txt maven_test_err_3.12.2.txt

[jameswilburlewis]

I've opened issues for a few of the warnings you noted:

https://github.com/spedas/pyspedas/issues/820 https://github.com/spedas/pyspedas/issues/819

Interesting that the "unclosed file" warnings only show up on python 3.11! I agree we can probably stop worrying about it.