search

simonw / sqlite-utils

Python CLI utility and library for manipulating SQLite databases
https://sqlite-utils.datasette.io
Apache License 2.0
1.67k stars 111 forks source link

ResourceWarning: unclosed file #534

Closed djhenderson closed 1 year ago

djhenderson commented 1 year ago

Issuing either

py -Wdefault -m sqlite_utils insert dogs.db dogs dogs0.csv --csv
  [#############-----------------------]   36%
  [####################################]  100%C:\Users\Doug\AppData\Local\Programs\Python\Python311\Lib\site-packages\sqlite_utils\cli.py:1187: ResourceWarning: unclosed file <_io.TextIOWrapper name='dogs0.csv' encoding='utf-8-sig'>
  insert_upsert_implementation(
ResourceWarning: Enable tracemalloc to get the object allocation traceback

or

set pythonwarnings=default
sqlite-utils insert dogs.db dogs dogs0.csv --csv
  [#############-----------------------]   36%
  [####################################]  100%C:\Users\Doug\AppData\Local\Programs\Python\Python311\Lib\site-packages\sqlite_utils\cli.py:1187: ResourceWarning: unclosed file <_io.TextIOWrapper name='dogs0.csv' encoding='utf-8-sig'>
  insert_upsert_implementation(
ResourceWarning: Enable tracemalloc to get the object allocation traceback

exhibits a ResourceWarning indicating that the CSV file being loaded is not closed.

sqlite-utils --version sqlite-utils, version 3.30 py --version Python 3.11.2 Windows Version 10.0.19045 Build 19045 SQLite version 3.41.0 2023-02-21 18:09:37

simonw commented 1 year ago

On macOS this shows the same warning:

% python -Wdefault $(which sqlite-utils) insert dogs.db dogs dogs.csv --csv
  [############------------------------]   35%
  [####################################]  100%/Users/simon/Dropbox/Development/sqlite-utils/sqlite_utils/cli.py:1187: ResourceWarning: unclosed file <_io.TextIOWrapper name='dogs.csv' encoding='utf-8-sig'>
  insert_upsert_implementation(
ResourceWarning: Enable tracemalloc to get the object allocation traceback

The file itself is a click.File which is automatically closed - https://click.palletsprojects.com/en/8.1.x/api/#click.File - but it looks like it's the _io.TextIOWrapper which is not being closed:

https://github.com/simonw/sqlite-utils/blob/2376c452a56b0c3e75e7ca698273434e32945304/sqlite_utils/cli.py#L949-L956