fix student and glovebox sqlite versions

lewagon / data-setup

Setup instructions for Le Wagon's students on their first day of Data Science Bootcamp

https://www.lewagon.com/data

1.81k stars 507 forks source link

fix student and glovebox sqlite versions #256

Open gmanchon opened 1 year ago

gmanchon commented 1 year ago

sqlite versions on student env and in glovebox can differ since sqlite is not installed by python (python only provides a wrapper)

the slack thread issue was a student using a right join in their challenge when the sqlite version in glovebox does not support it

the solution is to fix sqlite version in student setup and glovebox so they match

https://formulae.brew.sh/formula/sqlite https://doc.ubuntu-fr.org/sqlite

julesvanrie commented 6 months ago

For future reference based on my setups:

macOS sqlite version 3.43.2
my linux setup on Ubuntu 23.10 version 3.42
RIGHT and FULL JOIN introduced in version 3.39
WSL using Ubuntu 22.04 version 3.37.2
python:3.10.6-slim-buster version 3.27.2

So:

macOS and recent Ubuntu is not aligned with glovebox (or the other way around).
Windows is aligned with glovebox. For the time being, because Ubuntu 24.04 is about to come out and will probably become default version in WSL at some point.

julesvanrie commented 4 months ago

Experimented a bit with the data-runner and tried out some things, all unsuccesful:

apt update libsqlite: sticks to the same version because the package is not available for buster (needs a more recent glibc
building from source 🤓 : doesn't build because the necessary files to compile are missing (duh)
looking at other python-3.10 base images: would need debian bookworm to come with a new enough sqlite version, but bookworm is not available

Conclusion: this will have to wait until we switch to a newer python (something I want to do, currently experimenting myself with a new setup using 3.12, new pandas, sklearn, tensorflow 2.16 and keras>3 - which is a big change).