ERROR 1: PROJ: SQLite3 version is 3.7.17, whereas at least 3.11 should be used

[fphilip]

Hi hello, i am going around in circle with the error ERROR 1: PROJ: SQLite3 version is 3.7.17, whereas at least 3.11 should be used

if I run sqlite /opt/bin/sqlite3 -version 3.33.0 2020-08-14 13:23:32 fca8dc8b578f215a969cd899336378966156154710873e68b3d9ac5881b0ff3f

I'm lost.. any idea?

[fphilip]

I test it using export LD_LIBRARY_PATH="/opt/lib:$LD_LIBRARY_PATH" python -c "import sqlite3; print(sqlite3.sqlite_version)"

and the result is 3.7.17

[vincentsarago]

Hi @fphilip can you tell how you are getting this issue? what are you running?

would help if I could reproduce the issue

[Kirill888]

@vincentsarago @fphilip I'm getting the same issue, I think what happens is that there is already /lib64/libsqlite3 that gets picked up by python before pyproj can load the version it was linked against. I have been able to "solve" this issue by running LD_PRELOAD=/opt/lib/libsqlite3.so python -m pyproj -v.

Would it possible to compile against a system libsqlite3 instead of downloading a custom version, or are there extra feature that happened between 3.7.17 and 3.11?

[Kirill888]

Would it possible to compile against a system libsqlite3 instead of downloading a custom version, or are there extra feature that happened between 3.7.17 and 3.11?

I'll answer my own question: NO. PROJ==9.2.0 requires libsqlite3>3.11.

What is surprising to me is that despite LD_LIBRARY_PATH pointing to the directory with a more recent libsqlite3, python ends up with an older libsqlite3 from elsewhere on the system. LD_PRELOAD works fine when testing inside the docker, I'm yet to try it on an actual lambda though.

[tatornator12]

Running into the same issue which seems to have even larger issues doing transformations. A bit new to this, but @Kirill888 where are you running the LD_PRELOAD?

[vincentsarago]

I'll answer my own question: NO. PROJ==9.2.0 requires libsqlite3>3.11.

👀

we use rpath so proj should always use the custom libsqlite installed in /opt/lib!

Oh no I forgot to add rpath in the latest build https://github.com/lambgeo/docker-lambda/blob/master/dockerfiles/Dockerfile.gdal3.6#LL201C8-L201C42 😭

[tatornator12]

I set the LD_PRELOAD as an environment variable in my lambda function and it fixed that error. A sufficient workaround for now.

[Kirill888]

where are you running the LD_PRELOAD?

I have only tried inside the docker,

I set the LD_PRELOAD as an environment variable in my lambda function and it fixed that error. A sufficient workaround for now.

but good to know this works fine on actual lambda also.

Oh no I forgot to add rpath in the latest build

I guess we can test with patchelf --set-rpath '$ORIGIN' /opt/lib/libproj.so, my concern is that it might not be enough if you happen to import sqlite3 BEFORE importing anything that loads libproj.so, because by then wrong libsqlite3 is in memory already.