Warning about garbage collection, even with "with", with dask 2024.8

[cmalinmayor]

• nd2 version: 0.10.1
• Python version: 3.11.9
• Operating System: MacOS

Description

I was trying to load an nd2 into a dask array. Calling .to_dask on my ND2File inside the with statement results in a garbage colleciton warning. @tlambert03, can you explain why this is happening? I'm not sure if its expected or not, or if it's an actual problem. It also happens if you just call nd2.imread with dask=True.

What I Did

Minimal example 1:

import nd2
infile = "path_to_nd2.nd2"
with nd2.ND2File(infile) as myfile:
    data = myfile.to_dask()

Minimal Example 2:

import nd2
infile = "path_to_nd2.nd2"
data = nd2.imread(infile, dask=True)

Warning:

...miniforge3/envs/darts/lib/python3.11/site-packages/dask/base.py:1252: 
UserWarning: ND2File file not closed before garbage collection. 
Please use `with ND2File(...):` context or call `.close()`.
  mod.loads(out, buffers=buffers)

[tlambert03]

Do you then later go on to use data outside of the context? Or is the warning seen simply by calling to_dask?

Managing file handles have been tricky with dask and "delayed" reads. You can mostly just ignore it, but I can also try to silence the warning specifically in the dask case

[tlambert03]

Oh, never mind... yeah it wouldn't matter whether you use data there or not. We basically have to leave the handle open for dask, so the warning shows up. so I should just toss that warning in the dask case. Thanks for opening the issue

[tlambert03]

now that I'm back and playing with this, I was unable to reproduce it, but then updated dask and was. So this problem is new to dask 2024.8, (and tests are indeed currently failing accordingly, which is nice)