Closed simonw closed 4 months ago
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 95.69%. Comparing base (
17eb818
) to head (ddf5ce1
).
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Tests failed attempting to install numpy
I think:
Message:
CPU Optimization Options
baseline:
Requested : min
Enabled : SSE SSE2 SSE3
dispatch:
Requested : max -xop -fma4
Enabled : SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2 AVX512F AVX512CD AVX512_KNL AVX512_KNM AVX512_SKX AVX512_CLX AVX512_CNL AVX512_ICL
Library m found: YES
Run-time dependency scipy-openblas found: NO (tried pkgconfig)
Run-time dependency mkl found: NO (tried pkgconfig and system)
Run-time dependency mkl found: NO (tried pkgconfig and system)
Run-time dependency accelerate found: NO (tried system)
Found CMake: /usr/local/bin/cmake (3.28.3)
WARNING: CMake Toolchain: Failed to determine CMake compilers state
../numpy/meson.build:124:13: ERROR: Unhandled python exception
This is a Meson bug and should be reported!
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
https://github.com/simonw/sqlite-utils/actions/runs/8290227555/job/22687937164
A bunch of new warnings:
tests/test_insert_files.py::test_insert_files[True]
/home/runner/work/sqlite-utils/sqlite-utils/sqlite_utils/cli.py:3195: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
"mtime_iso": lambda p: datetime.utcfromtimestamp(p.stat().st_mtime).isoformat(),
tests/test_insert_files.py::test_insert_files[False]
tests/test_insert_files.py::test_insert_files[False]
tests/test_insert_files.py::test_insert_files[False]
tests/test_insert_files.py::test_insert_files[True]
tests/test_insert_files.py::test_insert_files[True]
tests/test_insert_files.py::test_insert_files[True]
/home/runner/work/sqlite-utils/sqlite-utils/sqlite_utils/cli.py:3196: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
"ctime_iso": lambda p: datetime.utcfromtimestamp(p.stat().st_ctime).isoformat(),
tests/test_recipes.py::test_dateparse_errors[None-parsedate]
/opt/hostedtoolcache/Python/3.13.0-alpha.5/x64/lib/python3.13/site-packages/_pytest/unraisableexception.py:80: PytestUnraisableExceptionWarning: Exception ignored in: <function Table.convert.<locals>.convert_value at 0x7a02ebf3ab60>
Traceback (most recent call last):
File "/home/runner/work/sqlite-utils/sqlite-utils/sqlite_utils/db.py", line 2841, in convert_value
return jsonify_if_needed(fn(v))
~~^^^
File "/home/runner/work/sqlite-utils/sqlite-utils/tests/test_recipes.py", line 77, in <lambda>
fresh_db["example"].convert("dt", lambda value: getattr(recipes, fn)(value))
~~~~~~~~~~~~~~~~~~~~^^^^^^^
File "/home/runner/work/sqlite-utils/sqlite-utils/sqlite_utils/recipes.py", line 19, in parsedate
parser.parse(value, dayfirst=dayfirst, yearfirst=yearfirst)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.13.0-alpha.5/x64/lib/python3.13/site-packages/dateutil/parser/_parser.py", line 1368, in parse
return DEFAULTPARSER.parse(timestr, **kwargs)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.13.0-alpha.5/x64/lib/python3.13/site-packages/dateutil/parser/_parser.py", line 643, in parse
raise ParserError("Unknown string format: %s", timestr)
dateutil.parser._parser.ParserError: Unknown string format: invalid
warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
tests/test_recipes.py::test_dateparse_errors[None-parsedatetime]
/opt/hostedtoolcache/Python/3.13.0-alpha.5/x64/lib/python3.13/site-packages/_pytest/unraisableexception.py:80: PytestUnraisableExceptionWarning: Exception ignored in: <function Table.convert.<locals>.convert_value at 0x7a02ebeb8a40>
Traceback (most recent call last):
File "/home/runner/work/sqlite-utils/sqlite-utils/sqlite_utils/db.py", line 2841, in convert_value
return jsonify_if_needed(fn(v))
~~^^^
File "/home/runner/work/sqlite-utils/sqlite-utils/tests/test_recipes.py", line 77, in <lambda>
fresh_db["example"].convert("dt", lambda value: getattr(recipes, fn)(value))
~~~~~~~~~~~~~~~~~~~~^^^^^^^
File "/home/runner/work/sqlite-utils/sqlite-utils/sqlite_utils/recipes.py", line 42, in parsedatetime
return parser.parse(value, dayfirst=dayfirst, yearfirst=yearfirst).isoformat()
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.13.0-alpha.5/x64/lib/python3.13/site-packages/dateutil/parser/_parser.py", line 1368, in parse
return DEFAULTPARSER.parse(timestr, **kwargs)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.13.0-alpha.5/x64/lib/python3.13/site-packages/dateutil/parser/_parser.py", line 643, in parse
raise ParserError("Unknown string format: %s", timestr)
dateutil.parser._parser.ParserError: Unknown string format: invalid
warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
Weirdly the tests failed on cog
:
Checking README.md
Checking docs/changelog.rst
Checking docs/cli-reference.rst (changed)
I installed Python 3.13 locally like this:
pyenv install 3.13-dev
cd /tmp
mkdir hello
cd hello
pyenv local 3.13-dev
cat .python-version
eval "$(pyenv init -)"
git clone git@github.com:simonw/sqlite-utils
cd sqlite-utils
python -m venv venv
source venv/bin/activate
pip install -e '.[docs]'
cog -r docs/*.rst
Now if I run cog -r docs/*.rst
it produces this diff:
diff --git a/docs/cli-reference.rst b/docs/cli-reference.rst
index 55517c9..04c4a59 100644
--- a/docs/cli-reference.rst
+++ b/docs/cli-reference.rst
@@ -615,25 +615,25 @@ See :ref:`cli_convert`.
r.jsonsplit(value, delimiter=',', type=<class 'str'>)
- Convert a string like a,b,c into a JSON array ["a", "b", "c"]
+ Convert a string like a,b,c into a JSON array ["a", "b", "c"]
r.parsedate(value, dayfirst=False, yearfirst=False, errors=None)
- Parse a date and convert it to ISO date format: yyyy-mm-dd
-
- - dayfirst=True: treat xx as the day in xx/yy/zz
- - yearfirst=True: treat xx as the year in xx/yy/zz
- - errors=r.IGNORE to ignore values that cannot be parsed
- - errors=r.SET_NULL to set values that cannot be parsed to null
+ Parse a date and convert it to ISO date format: yyyy-mm-dd
+
+ - dayfirst=True: treat xx as the day in xx/yy/zz
+ - yearfirst=True: treat xx as the year in xx/yy/zz
+ - errors=r.IGNORE to ignore values that cannot be parsed
+ - errors=r.SET_NULL to set values that cannot be parsed to null
r.parsedatetime(value, dayfirst=False, yearfirst=False, errors=None)
- Parse a datetime and convert it to ISO datetime format: yyyy-mm-ddTHH:MM:SS
-
- - dayfirst=True: treat xx as the day in xx/yy/zz
- - yearfirst=True: treat xx as the year in xx/yy/zz
- - errors=r.IGNORE to ignore values that cannot be parsed
- - errors=r.SET_NULL to set values that cannot be parsed to null
+ Parse a datetime and convert it to ISO datetime format: yyyy-mm-ddTHH:MM:SS
+
+ - dayfirst=True: treat xx as the day in xx/yy/zz
+ - yearfirst=True: treat xx as the year in xx/yy/zz
+ - errors=r.IGNORE to ignore values that cannot be parsed
+ - errors=r.SET_NULL to set values that cannot be parsed to null
You can use these recipes like so:
Yeah there's a difference. In this screenshot the left is Python 3.10 and the right is Python 3.13-dev:
Weird, there is now a visible character in that README when it is displayed in the help:
It doesn't show if I just run sqlite-utils convert --help
on the terminal:
But it DOES if I run that through sqlite-utils convert --help | pbcopy
:
Usage: sqlite-utils convert [OPTIONS] DB_PATH TABLE COLUMNS... CODE
Convert columns using Python code you supply. For example:
sqlite-utils convert my.db mytable mycolumn \
'"\n".join(textwrap.wrap(value, 10))' \
--import=textwrap
"value" is a variable with the column value to be converted.
Use "-" for CODE to read Python code from standard input.
The following common operations are available as recipe functions:
r.jsonsplit(value, delimiter=',', type=<class 'str'>)
Convert a string like a,b,c into a JSON array ["a", "b", "c"]
r.parsedate(value, dayfirst=False, yearfirst=False, errors=None)
Parse a date and convert it to ISO date format: yyyy-mm-dd
- dayfirst=True: treat xx as the day in xx/yy/zz
- yearfirst=True: treat xx as the year in xx/yy/zz
- errors=r.IGNORE to ignore values that cannot be parsed
- errors=r.SET_NULL to set values that cannot be parsed to null
r.parsedatetime(value, dayfirst=False, yearfirst=False, errors=None)
Parse a datetime and convert it to ISO datetime format: yyyy-mm-ddTHH:MM:SS
- dayfirst=True: treat xx as the day in xx/yy/zz
- yearfirst=True: treat xx as the year in xx/yy/zz
- errors=r.IGNORE to ignore values that cannot be parsed
- errors=r.SET_NULL to set values that cannot be parsed to null
You can use these recipes like so:
sqlite-utils convert my.db mytable mycolumn \
'r.jsonsplit(value, delimiter=":")'
Options:
--import TEXT Python modules to import
--dry-run Show results of running this against first
10 rows
--multi Populate columns for keys in returned
dictionary
--where TEXT Optional where clause
-p, --param <TEXT TEXT>... Named :parameters for where clause
--output TEXT Optional separate column to populate with
the output
--output-type [integer|float|blob|text]
Column type to use for the output column
--drop Drop original column afterwards
--no-skip-false Don't skip falsey values
-s, --silent Don't show a progress bar
--pdb Open pdb debugger on first error
-h, --help Show this message and exit.
That weird visible character is a \x08
which is the same thing as a \b
:
'\n \x08\n - dayfirst=True: treat xx
>>> '\x08' == '\b'
True
https://github.com/simonw/sqlite-utils/blob/fd803d7043121f8618932c6c45888da60282ff5b/docs/cli-reference.rst#convert displays without visible glitch characters now.
I learned about
allow-prerelease: true
from https://github.com/dateutil/dateutil/pull/1337/files