simonw / sqlite-utils

Python CLI utility and library for manipulating SQLite databases
https://sqlite-utils.datasette.io
Apache License 2.0
1.58k stars 106 forks source link

Add spatialite arm64 linux path #600

Closed MikeCoats closed 8 months ago

MikeCoats commented 8 months ago

According to both Debian and Ubuntu, the correct “target triple” for arm64 is aarch64-linux-gnu, so we should be looking in /usr/lib/aarch64-linux-gnu for mod_spatialite.so.

I can confirm that on both of my Debian arm64 SBCs, libsqlite3-mod-spatialite installs to that path.

$ ls -l /usr/lib/*/*spatial*
lrwxrwxrwx 1 root root      23 Dec  1  2022 /usr/lib/aarch64-linux-gnu/mod_spatialite.so -> mod_spatialite.so.7.1.0
lrwxrwxrwx 1 root root      23 Dec  1  2022 /usr/lib/aarch64-linux-gnu/mod_spatialite.so.7 -> mod_spatialite.so.7.1.0
-rw-r--r-- 1 root root 7348584 Dec  1  2022 /usr/lib/aarch64-linux-gnu/mod_spatialite.so.7.1.0

This is a set of before and after snippets of pytest’s output for this PR.

Before

$ pytest
tests/test_get.py ......                                                 [ 73%]
tests/test_gis.py ssssssssssss                                           [ 75%]
tests/test_hypothesis.py ....                                            [ 75%]

After

$ pytest
tests/test_get.py ......                                                 [ 73%]
tests/test_gis.py ............                                           [ 75%]
tests/test_hypothesis.py ....                                            [ 75%]

Issue: #599


:books: Documentation preview :books:: https://sqlite-utils--600.org.readthedocs.build/en/600/

simonw commented 8 months ago

Thanks for this!

codecov[bot] commented 8 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (622c3a5) 95.77% compared to head (b1a6076) 95.77%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #600 +/- ## ======================================= Coverage 95.77% 95.77% ======================================= Files 8 8 Lines 2840 2840 ======================================= Hits 2720 2720 Misses 120 120 ``` | [Files](https://app.codecov.io/gh/simonw/sqlite-utils/pull/600?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/db.py](https://app.codecov.io/gh/simonw/sqlite-utils/pull/600?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.22% <ø> (ø)` | | | [sqlite\_utils/utils.py](https://app.codecov.io/gh/simonw/sqlite-utils/pull/600?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL3V0aWxzLnB5) | `94.56% <ø> (ø)` | |

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

simonw commented 8 months ago

The tests failed because they found a spelling mistake in a completely unrelated area of the code - not sure why that had not been caught before.

simonw commented 8 months ago

Testing this manually on macOS using Docker Desk top like this:

docker run -it --rm arm64v8/ubuntu /bin/bash

Then inside the container:

uname -m

Outputs: aarch64

Then:

apt install spatialite-bin libsqlite3-mod-spatialite git python3 python3-venv -y
cd /tmp
git clone https://github.com/simonw/sqlite-utils
cd sqlite-utils
python3 -m venv venv
source venv/bin/activate
pip install -e '.[test]'
sqlite-utils memory "select spatialite_version()" --load-extension=spatialite

Which output:

Traceback (most recent call last):
  File "/tmp/sqlite-utils/venv/bin/sqlite-utils", line 33, in <module>
    sys.exit(load_entry_point('sqlite-utils', 'console_scripts', 'sqlite-utils')())
  File "/tmp/sqlite-utils/venv/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/tmp/sqlite-utils/venv/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/tmp/sqlite-utils/venv/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/tmp/sqlite-utils/venv/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmp/sqlite-utils/venv/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/tmp/sqlite-utils/sqlite_utils/cli.py", line 1959, in memory
    _load_extensions(db, load_extension)
  File "/tmp/sqlite-utils/sqlite_utils/cli.py", line 3232, in _load_extensions
    if ":" in ext:
TypeError: argument of type 'NoneType' is not iterable

Then I ran this:

git checkout -b MikeCoats-spatialite-paths-linux-arm main
git pull https://github.com/MikeCoats/sqlite-utils.git spatialite-paths-linux-arm

And now:

sqlite-utils memory "select spatialite_version()" --load-extension=spatialite

Outputs:

[{"spatialite_version()": "5.0.1"}]
simonw commented 8 months ago

The GIS tests now pass in that container too:

pytest tests/test_gis.py
======================== test session starts =========================
platform linux -- Python 3.10.12, pytest-7.4.3, pluggy-1.3.0
rootdir: /tmp/sqlite-utils
plugins: hypothesis-6.88.1
collected 12 items                                                   

tests/test_gis.py ............                                 [100%]

========================= 12 passed in 0.48s =========================