UKGovernmentBEIS / inspect_ai

Inspect: A framework for large language model evaluations
https://inspect.ai-safety-institute.org.uk/
MIT License
588 stars 107 forks source link

[bug] Inspect fails to run in the vs code python 3 devcontainer #665

Open domdomegg opened 2 weeks ago

domdomegg commented 2 weeks ago

When starting up, Inspect sets up a folder to help store temporary files. It uses the https://github.com/tox-dev/platformdirs library (specifically the user_runtime_path function) to find an appropriate place to store these files.

On Linux systems without $XDG_RUNTIME_DIR set, this returns a path in /run/user. Unfortunately in VS Code dev containers, there is no /run/user folder in the default Python container.

Upstream bug: https://github.com/tox-dev/platformdirs/issues/315

Proposed next steps

jjallaire-aisi commented 2 weeks ago

Another thing we could do is change to user_state_dir() ? (which defaults to ~/.local/state). Our use of this is actually much more in the category of "state" than what is classically associated w/ the runtime dir. If you could confirm that this resolves the issue for you I'll make that change