getting repeated RessourceWarning

[jkobject]

It might be related to me updating my version of lamindb, or running multiple notebooks?

But I get

[asyncio/sslproto.py:320](https://vscode-remote+ssh-002dremote-002bperso.vscode-resource.vscode-cdn.net/home/ml4ig1/miniconda3/envs/scprint/lib/python3.10/asyncio/sslproto.py:320): ResourceWarning: unclosed transport <asyncio.sslproto._SSLProtocolTransport object at 0x7fd20d0b7280>
  _warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback

and

<frozen importlib._bootstrap_external>:672: ResourceWarning: unclosed <ssl.SSLSocket fd=77, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.20.53.10', 59760), raddr=('104.18.38.107', 443)>
ResourceWarning: Enable tracemalloc to get the object allocation traceback

when I call lamindb functions. Not sure why

[Koncopd]

Hi, @jkobject . Does this happen when you run a single notebook with lamindb? Does it happen for an s3 instance? Or for a local instance also? Everything else works correctly when you get this warning?

[jkobject]

Hi,

Sorry for my lack of indications. Similarly to my other issues in here: This is for a personal lamin instance. This happens across all notebooks. I am saving many records from a remote instance into my own local one.

[Koncopd]

Could you please try from a fresh environment (you don't need to relogin or reload the instance)?

[jkobject]

from a fresh conda environment?

[Koncopd]

Yep.

[jkobject]

Yes I still see it, even with lamin 0.63.4.

Even when using a new instance this appears

[Koncopd]

Is it from a fresh environment or the same one?

[jkobject]

fresh environment

[Koncopd]

Thx for checking. What are the versions of aiohttp and aiobotocore?

[jkobject]

aiobotocore 2.8.0 aiohttp 3.9.1

[Koncopd]

And s3fs?

[jkobject]

full list here:

aiobotocore               2.8.0
aiohttp                   3.9.1
aioitertools              0.11.0
aiosignal                 1.3.1
anndata                   0.10.3
antlr4-python3-runtime    4.9.3
anyio                     4.1.0
appdirs                   1.4.4
array-api-compat          1.4
asgiref                   3.7.2
asttokens                 2.4.1
async-timeout             4.0.3
attridict                 0.0.8
attrs                     23.1.0
beautifulsoup4            4.12.2
bio                       1.6.0
biomart                   0.9.2
bionty                    0.35.1
biopython                 1.81
biothings-client          0.3.1
black                     23.11.0
bleach                    6.1.0
botocore                  1.33.1
cellxgene-census          1.8.0
certifi                   2023.11.17
cffi                      1.16.0
chardet                   5.2.0
charset-normalizer        3.3.2
click                     8.1.7
click-log                 0.4.0
colorama                  0.4.6
comm                      0.2.0
contourpy                 1.2.0
coverage                  7.3.2
cryptography              41.0.7
cycler                    0.12.1
debugpy                   1.8.0
decorator                 5.1.1
decoupler                 1.5.0
defusedxml                0.7.1
Deprecated                1.2.14
deprecation               2.1.0
dj-database-url           2.1.0
Django                    4.2.8
docutils                  0.20.1
dotty-dict                1.3.1
exceptiongroup            1.2.0
executing                 2.0.1
fair-esm                  2.0.1
fastjsonschema            2.19.0
fastobo                   0.12.3
filelock                  3.13.1
flake8                    6.1.0
fonttools                 4.46.0
frozenlist                1.4.0
fsspec                    2023.12.1
gget                      0.28.2
ghp-import                2.1.0
gitchangelog              3.0.4
gitdb                     4.0.11
GitPython                 3.1.40
gotrue                    1.3.1
gprofiler-official        1.0.0
graphviz                  0.20.1
gseapy                    1.1.0
h11                       0.14.0
h5py                      3.10.0
httpcore                  0.16.3
httpx                     0.23.3
huggingface-hub           0.19.4
hydra-core                1.3.2
idna                      3.6
igraph                    0.10.8
importlib-metadata        7.0.0
iniconfig                 2.0.0
invoke                    1.7.3
ipykernel                 6.27.1
ipylab                    1.0.0
ipython                   8.18.1
ipywidgets                8.1.1
isort                     5.12.0
jaraco.classes            3.3.0
jedi                      0.19.1
jeepney                   0.8.0
Jinja2                    3.1.2
jmespath                  1.0.1
joblib                    1.3.2
jsonschema                4.20.0
jsonschema-specifications 2023.11.2
jupyter_client            8.6.0
jupyter_core              5.5.0
jupyterlab_pygments       0.3.0
jupyterlab-widgets        3.0.9
keyring                   24.3.0
kiwisolver                1.4.5
lamin_cli                 0.2.4
lamin_utils               0.12.0
lamindb                   0.63.4
lamindb_setup             0.61.1
leidenalg                 0.10.1
lightning                 2.1.2
lightning-utilities       0.10.0
llvmlite                  0.39.1
lnschema_bionty           0.35.2
lnschema_core             0.57.6
lxml                      4.9.3
Markdown                  3.5.1
markdown-it-py            3.0.0
MarkupSafe                2.1.3
matplotlib                3.8.2
matplotlib-inline         0.1.6
mccabe                    0.7.0
mdurl                     0.1.2
mergedeep                 1.3.4
mistune                   3.0.2
mkdocs                    1.5.3
more-itertools            10.1.0
mpmath                    1.3.0
multidict                 6.0.4
mygene                    3.2.2
mypy                      1.7.1
mypy-extensions           1.0.0
mysql-connector-python    8.0.29
natsort                   8.4.0
nbclient                  0.9.0
nbconvert                 7.12.0
nbformat                  5.9.2
nbproject                 0.9.2
nbstripout                0.6.1
nest-asyncio              1.5.8
networkx                  3.2.1
nh3                       0.2.15
numba                     0.56.4
numpy                     1.23.5
nvidia-cublas-cu12        12.1.3.1
nvidia-cuda-cupti-cu12    12.1.105
nvidia-cuda-nvrtc-cu12    12.1.105
nvidia-cuda-runtime-cu12  12.1.105
nvidia-cudnn-cu12         8.9.2.26
nvidia-cufft-cu12         11.0.2.54
nvidia-curand-cu12        10.3.2.106
nvidia-cusolver-cu12      11.4.5.107
nvidia-cusparse-cu12      12.1.0.106
nvidia-nccl-cu12          2.18.1
nvidia-nvjitlink-cu12     12.3.101
nvidia-nvtx-cu12          12.1.105
omegaconf                 2.3.0
orjson                    3.9.10
owlready2                 0.45
packaging                 23.2
pandas                    2.1.3
pandocfilters             1.5.0
parso                     0.8.3
pathspec                  0.11.2
patsy                     0.5.4
pexpect                   4.9.0
Pillow                    10.1.0
pip                       23.3.1
pkginfo                   1.9.6
platformdirs              4.1.0
pluggy                    1.3.0
pooch                     1.8.0
postgrest                 0.10.6
prompt-toolkit            3.0.41
pronto                    2.5.5
protobuf                  4.25.1
psutil                    5.9.6
psycopg2-binary           2.9.9
ptyprocess                0.7.0
pure-eval                 0.2.2
pyarrow                   14.0.1
pycodestyle               2.11.1
pycparser                 2.21
pydantic                  1.10.13
pydot                     1.4.2
pyflakes                  3.1.0
Pygments                  2.17.2
pynndescent               0.5.11
pyparsing                 3.1.1
pytest                    7.4.3
pytest-cov                4.1.0
python-dateutil           2.8.2
python-dotenv             1.0.0
python-gitlab             3.15.0
python-semantic-release   7.33.2
pytorch-lightning         2.1.2
pytz                      2023.3.post1
PyYAML                    6.0.1
pyyaml_env_tag            0.1
pyzmq                     25.1.2
rapidfuzz                 3.5.2
readme-renderer           42.0
realtime                  1.0.2
referencing               0.32.0
regex                     2023.10.3
requests                  2.31.0
requests-toolbelt         1.0.0
rfc3986                   1.5.0
rich                      13.7.0
rpds-py                   0.13.2
s3fs                      2023.12.1
safetensors               0.4.1
scanpy                    1.9.5
scib                      1.1.3
scikit-learn              1.3.2
scikit-misc               0.3.0
scipy                     1.11.4
scprint                   0.1.0        /home/ml4ig1/Documents code/scPRINT
seaborn                   0.13.0
SecretStorage             3.3.3
semver                    2.13.0
session-info              1.0.0
setuptools                69.0.2
six                       1.16.0
smmap                     5.0.1
sniffio                   1.3.0
somacore                  1.0.4
soupsieve                 2.5
sqlparse                  0.4.4
stack-data                0.6.3
statsmodels               0.14.0
stdlib-list               0.10.0
storage3                  0.5.3
StrEnum                   0.4.15
supabase                  1.0.3
supafunc                  0.2.2
sympy                     1.12
texttable                 1.7.0
threadpoolctl             3.2.0
tiledb                    0.23.5
tiledbsoma                1.5.1
tinycss2                  1.2.1
tokenizers                0.15.0
tomli                     2.0.1
tomlkit                   0.12.3
torch                     2.1.1
torchdata                 0.7.1
torchmetrics              1.2.1
torchtext                 0.16.1
torchvision               0.16.1
tornado                   6.4
tqdm                      4.66.1
traitlets                 5.14.0
transformers              4.35.2
triton                    2.1.0
twine                     3.8.0
typing_extensions         4.8.0
tzdata                    2023.3
umap-learn                0.5.5
universal-pathlib         0.1.4
urllib3                   2.0.7
watchdog                  3.0.0
wcwidth                   0.2.12
webencodings              0.5.1
websockets                11.0.3
wheel                     0.42.0
widgetsnbextension        4.0.9
wrapt                     1.16.0
yarl                      1.9.4
zipp                      3.17.0

[Koncopd]

Interesting, we have pins on an older version of s3fs and the related requirements. https://github.com/laminlabs/lamindb/blob/6f497b853fd1be16a10815c1db9f7cdada579e84/pyproject.toml#L41 We pin s3fs to 2023.9.0. You didn't install lamindb via lamindb[aws], right? Downgrading via pip install lamindb[aws] might help. So if pip install lamindb[aws] helps, then the problem is actually in newer versions of these packages. I will check and get back.

[jkobject]

No I didn't do lamindb[aws].

doing it gets me:

  Attempting uninstall: urllib3
    Found existing installation: urllib3 2.0.7
    Uninstalling urllib3-2.0.7:
      Successfully uninstalled urllib3-2.0.7
  Attempting uninstall: fsspec
    Found existing installation: fsspec 2023.12.1
    Uninstalling fsspec-2023.12.1:
      Successfully uninstalled fsspec-2023.12.1
  Attempting uninstall: botocore
    Found existing installation: botocore 1.33.1
    Uninstalling botocore-1.33.1:
      Successfully uninstalled botocore-1.33.1
  Attempting uninstall: aiobotocore
    Found existing installation: aiobotocore 2.8.0
    Uninstalling aiobotocore-2.8.0:
      Successfully uninstalled aiobotocore-2.8.0
  Attempting uninstall: s3fs
    Found existing installation: s3fs 2023.12.1
    Uninstalling s3fs-2023.12.1:
      Successfully uninstalled s3fs-2023.12.1
Successfully installed aiobotocore-2.5.4 boto3-1.28.17 botocore-1.31.17 fsspec-2023.9.0 s3fs-2023.9.0 s3transfer-0.6.2 urllib3-1.26.18

and I still get the ResourceWarning afterward

[Koncopd]

import error?

[jkobject]

my bad, the ResourceWarning

[jkobject]

I have switched back to my previous instance and now getting:

when doing ln.File()

It seems to be because of some files that I might have remove or something?

[Koncopd]

It looks like you just provided a File without any arguments.

[jkobject]

oh right, I forgot that to get all the files one need to do ln.File.filter()

My bad! I am getting too confident with the library I guess. 😅

[Zethson]

I'm closing this issue now but if there's something that we can help you with, please feel free to open new issues or ping us!