Can't build 3.13b2 on Ubuntu: test test_sqlite3 failed

[RobH123]

Bug report

Bug description:

After successfully running ./configure --enable-optimizations on Ubuntu 24.04 (with these lines perhaps relevant):

...
checking for decimal libmpdec machine... uint128
checking for sqlite3 >= 3.15.2... yes
checking for sqlite3.h... yes
checking for sqlite3_bind_double in -lsqlite3... yes
checking for sqlite3_column_decltype in -lsqlite3... yes
checking for sqlite3_column_double in -lsqlite3... yes
checking for sqlite3_complete in -lsqlite3... yes
checking for sqlite3_progress_handler in -lsqlite3... yes
checking for sqlite3_result_double in -lsqlite3... yes
checking for sqlite3_set_authorizer in -lsqlite3... yes
checking for sqlite3_trace_v2 in -lsqlite3... yes
checking for sqlite3_value_double in -lsqlite3... yes
checking for sqlite3_load_extension in -lsqlite3... yes
checking for sqlite3_serialize in -lsqlite3... yes
checking for --enable-loadable-sqlite-extensions... no
checking for tcl >= 8.5.12 tk >= 8.5.12... yes
...
checking for stdlib extension module readline... yes
checking for stdlib extension module _sqlite3... yes
checking for stdlib extension module _tkinter... yes
...
config.status: creating pyconfig.h
configure: creating Modules/Setup.local
configure: creating Makefile

Then make -s -j4

...
0:00:45 load avg: 8.53 [36/44] test_set
0:00:50 load avg: 8.41 [37/44] test_sqlite3
test test_sqlite3 failed
0:00:50 load avg: 8.41 [38/44] test_statistics -- test_sqlite3 failed (4 errors)
0:00:57 load avg: 7.89 [39/44] test_str
0:00:58 load avg: 7.89 [40/44] test_struct
0:00:59 load avg: 7.89 [41/44] test_tabnanny
0:01:00 load avg: 7.89 [42/44] test_time
0:01:02 load avg: 7.74 [43/44] test_xml_etree
0:01:03 load avg: 7.74 [44/44] test_xml_etree_c

Total duration: 1 min 4 sec
Total tests: run=9,203 skipped=177
Total test files: run=44/44 failed=1
Result: FAILURE
make: *** [Makefile:892: profile-run-stamp] Error 2

dpkg -l | grep sqlite3

ii  libaprutil1-dbd-sqlite3:amd64                               1.6.3-1.1ubuntu7                            amd64        Apache Portable Runtime Utility Library - SQLite3 Driver
ii  libsqlite3-0:amd64                                          3.45.1-1ubuntu2                             amd64        SQLite 3 shared library
ii  libsqlite3-dev:amd64                                        3.45.1-1ubuntu2                             amd64        SQLite 3 development files
ii  sqlite3                                                     3.45.1-1ubuntu2                             amd64        Command line interface for SQLite 3

Not sure where to look next? (Can't see anything relevant on the issue tracker.)

(I had the same problem last month with beta1, but hoped beta2 might help me get 3.13 running for testing.)

CPython versions tested on:

3.13

Operating systems tested on:

Linux

[mcepl]

On Fri Jun 7, 2024 at 5:29 AM CEST, Robert wrote:

0:00:50 load avg: 8.41 [38/44] test_statistics -- test_sqlite3 failed (4 errors)

What happens when you run

./python -mtest -v test_sqlite3 ?

[RobH123]

Thanks for looking at this @mcepl.

...
test_win_sum_int (test.test_sqlite3.test_userfunctions.WindowFunctionTests.test_win_sum_int) ... ok

======================================================================
ERROR: test_open_undecodable_uri (test.test_sqlite3.test_dbapi.OpenTests.test_open_undecodable_uri)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/robert/Downloads/Python-3.13.0b2/Lib/test/test_sqlite3/test_dbapi.py", line 729, in test_open_undecodable_uri
    with contextlib.closing(sqlite.connect(uri, uri=True)) as cx:
                            ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
sqlite3.OperationalError: unable to open database file

======================================================================
ERROR: test_open_undecodable_uri (test.test_sqlite3.test_dbapi.OpenTests.test_open_undecodable_uri)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/robert/Downloads/Python-3.13.0b2/Lib/test/support/os_helper.py", line 345, in unlink
    _unlink(filename)
    ~~~~~~~^^^^^^^^^^
OSError: [Errno 84] Invalid or incomplete multibyte or wide character: b'@test_195094_tmp\xe7w\xf0'

======================================================================
ERROR: test_open_with_undecodable_path (test.test_sqlite3.test_dbapi.OpenTests.test_open_with_undecodable_path)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/robert/Downloads/Python-3.13.0b2/Lib/test/test_sqlite3/test_dbapi.py", line 682, in test_open_with_undecodable_path
    with contextlib.closing(sqlite.connect(path)) as cx:
                            ~~~~~~~~~~~~~~^^^^^^
sqlite3.OperationalError: unable to open database file

======================================================================
ERROR: test_open_with_undecodable_path (test.test_sqlite3.test_dbapi.OpenTests.test_open_with_undecodable_path)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/robert/Downloads/Python-3.13.0b2/Lib/test/support/os_helper.py", line 345, in unlink
    _unlink(filename)
    ~~~~~~~^^^^^^^^^^
OSError: [Errno 84] Invalid or incomplete multibyte or wide character: b'@test_195094_tmp\xe7w\xf0'

----------------------------------------------------------------------
Ran 481 tests in 0.517s

FAILED (errors=4, skipped=4)
test test_sqlite3 failed
test_sqlite3 failed (4 errors)

== Tests result: FAILURE ==

1 test failed:
    test_sqlite3

Total duration: 689 ms
Total tests: run=481 skipped=4
Total test files: run=1/1 failed=1
Result: FAILURE
Sat 08:25:~/Downloads/Python-3.13.0b2$ ll Lib/test/test_sqlite3/
...
-rw-r--r--  1 robert robert 74629 Jun  6 02:46 test_dbapi.py
...```

[mcepl]

Probably completely wrong question. What did you get when running this?

$ ./python -c 'import sys; print(sys.maxunicode > 0xffff)'

[RobH123]

True (and if I print sys.maxunicode it's 1114111 or 0x10FFFF).

Oh, and dumb me hadn't even realised at first that I'm actually running python3.13 for these commands. (I'm guessing that it's the unoptimised version???)

[Zheaoli]

----------------------------------------------------------------------
Ran 480 tests in 0.625s

OK (skipped=4)
Warning -- files was modified by test_sqlite3
Warning --   Before: []
Warning --   After:  ['gmon.out']
test_sqlite3 failed (env changed)

== Tests result: SUCCESS ==

1 test altered the execution environment (env changed):
    test_sqlite3

Total duration: 1.2 sec
Total tests: run=480 skipped=4
Total test files: run=1/1 env_changed=1
Result: SUCCESS

╭─   manjusaka@manjusaka-garuda   ~/Documents/projects/cpython                                                                                                                                                                                                        main ?9   21:29:47 
╰─ sqlite3 --version
3.46.0 2024-05-23 13:25:27 96c92aba00c8375bc32fafcdf12429c58bd8aabfcadab6683e35bbb9cdebalt1 (64-bit)

Could not reproduce the issue on main branch

[RobH123]

Oh. Do you install sqlite3 from a ppa? (I have 3.45.1 on Ubuntu Noble 24.04, you have 3.46). My Ubuntu has been upgraded every six months for almost five years, so possibly some compatibility thing makes my system not identical to a new install??? Any ideas what else I could possibly look for @Zheaoli? I guess I could try to figure out how to build 3.13 in Docker or something??? Otherwise I'll just have to wait for the eventual release to reach Ubuntu 24.10. (Never had problems building betas before.)

[vstinner]

Can you run the following commands and copy/paste the output?

$ ./python
>>> from test.support import os_helper
>>> os_helper.TESTFN_UNDECODABLE
b'@test_950535_tmp\xe7w\xf0'

>>> import sys
>>> sys.getfilesystemencoding()
'utf-8'
>>> sys.getfilesystemencodeerrors()
'surrogateescape'

>>> import locale
>>> locale.getencoding()
'UTF-8'

>>> import sqlite3
>>> sqlite3.sqlite_version
'3.45.1'
>>> sqlite3.sqlite_version_info
(3, 45, 1)

[RobH123]

Python 3.13.0b2 (main, Jun 7 2024, 14:59:58) [GCC 13.2.0] on linux

b'@test_4030379_tmp\xe7w\xf0'

'utf-8' 'surrogateescape'

'UTF-8'

'3.45.1' (3, 45, 1)

Thanks @vstinner. (Without the optimisations, 3.13 is about half the speed of 3.12 in my applications.)