simonw / sqlite-utils

Python CLI utility and library for manipulating SQLite databases
https://sqlite-utils.datasette.io
Apache License 2.0
1.64k stars 111 forks source link

Fix test to pass `pytest -Werror` #623

Open lbellomo opened 6 months ago

lbellomo commented 6 months ago

Refs https://github.com/simonw/sqlite-utils/issues/541

Still TODO:


📚 Documentation preview 📚: https://sqlite-utils--623.org.readthedocs.build/en/623/

lbellomo commented 6 months ago

The first case is not the test you mark, but the previous test_upsert_pk_required. When you run the upsert "click" opens the file first but as it then gives the error "Error: Missing option '--pk" it doesn't close it correctly.

https://github.com/simonw/sqlite-utils/blob/1d050dcdc7c8099108abd2987932a3d8d7bf55aa/sqlite_utils/cli.py#L868

The solution is to pass lazy=True to the click.File so that it does not open it until it is necessary.

lbellomo commented 6 months ago

It seems that in test_memory_csv only for when use_stdin=False the io.TextIOWrapper on line 297 is not closed correctly.

https://github.com/simonw/sqlite-utils/blob/1d050dcdc7c8099108abd2987932a3d8d7bf55aa/sqlite_utils/utils.py#L295-L302

The only way I can think of is to pass the buffer to _extra_key_strategy and close it once the generator is finished, but surely there must be a nicer way.

lbellomo commented 6 months ago

Similar to the first case, it is not test_memory_dump but test_memory_csv_encoding the previous test that is the culprit, and also only when use_stdin=False.

It seems that when the enconding is wrong it raises a UnicodeDecodeError error when it tries to iterate over the generator from rows_from_file and that's why it doesn't reach the end to close the fp file.

https://github.com/simonw/sqlite-utils/blob/1d050dcdc7c8099108abd2987932a3d8d7bf55aa/sqlite_utils/cli.py#L1949-L1968

The only thing that I can think of is to catch the error, close the file and raise it again.

lbellomo commented 6 months ago

test_insert_files.py is a deprecation warning. datetime.utcfromtimestamp is replaced by datetime.fromtimestamp in py3.12, I pass the timezone to make sure it is utc (otherwise it grabs the local one) and stripe the timezone to be the same as it was before.