Open TheZoq2 opened 11 months ago
I've done some further investigation into this. Naturally, when I create a venv with my system python, that venv will use the system python too which seems to cause conflicts.
I still want a venv because I would like to install some additional dependencies per project, so I figured I could create a venv using the python version that oss-cad-suite provides, i.e.
> tabbypy3 -m venv tabbyvenv
Error: Command '['/tmp/cocotb_test/test/bin/tabbypy3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 127.
However, that seems to fail half way through the creation process. Running that raw command gives
/tmp/cocotb_test/tabbyvenv/bin/tabbypy3: line 10: /tmp/cocotb_test/tabbyvenv/lib/ld-linux-x86-64.so.2: No such file or directory
And another update: Not being able to create venvs with the cad suite seems to be because release_bindir
does not resolve symlinks. So when python3 -m venv venv
tries to run python, it runs python3
which is a symlink in venv/bin/python3
which points to tabbypy3
However, tabbypy3
relies on the location of the script as it was invoked without resolving symlinks, it therefore thinks that oss-cad-suite is installed invenv
and the last exec
call looks for the cad suite libraries in the venv.
I attempted to fix this by following the symlink as release_bindir="$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")"
But that updates PYTHONEXECUTABLE
to the tabbycad install location, overriding the venv (maybe?, I still don't understand how python figures out which venv is active)
Ok, I think I have figured out a fix that works for me. I don't know if it is worth upstreaming it since I don't really know what i'm doing, my use case might be strange etc. But I figure it is worth posting this for others who run into similar issues
Python seems to determine venvs partially based on where the python executable being run is, before resolving symlinks so that needs to be set correctly. oss-cad-suite on the other hand relies on the executable location to be where the cad suite is installed, not following symlinks.
Modifying tabbypy3
to this. i.e. resolving the symlink for release_bindir
and setting PYTHONEXECUTABLE
to whatever invoked tabbypy3, rather than the resolved exe. PYTHONHOME
seems to not do much, not sure if it is needed for something else
#!/usr/bin/env bash
release_bindir="$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")"
release_bindir_abs="$(readlink -f "$release_bindir")"
release_topdir_abs="$(readlink -f "$release_bindir/..")"
export PATH="$release_bindir_abs:$PATH"
export PYTHONEXECUTABLE="${BASH_SOURCE[0]}"
# export PYTHONEXECUTABLE="$release_topdir_abs/bin/tabbypy3"
# export PYTHONHOME="$release_topdir_abs"
export PYTHONNOUSERSITE=1
export SSL_CERT_FILE="$release_topdir_abs"/etc/cacert.pem
exec "$release_topdir_abs"/lib/ld-linux-x86-64.so.2 --inhibit-cache --inhibit-rpath "" --library-path "$release_topdir_abs"/lib "$release_topdir_abs"/libexec/python3.8 "$@"
Seems to resolve my issues, namely, oss-cad-sutie works to the point where cocotb works, both with and without virtual environments
To use it in a venv, you can make a new venv with tabbypy3 -m venv venv --system-site-packages
. The flag is required in order for pip
to work, though crucially, you need to run pip
as python3 -m pip
, not just pip
as that uses the system version
As discussed in discord, this workflow breaks with the ubuntu bump, so libpthread will need to be added to the bundle
File is added now in release, and I have rebuilt releases, so please do try if it fixes complete flow now. Thanks
Thanks for the quick fix! It does seem to work.
Unfortunately, now i'm running into some problems which I think are due to cocotb being bumped to 2.0-dev. It feels fixable by installing cocotb from pip, but I can't seem to get that to work
I'm not sure this is a supported use case, it seems like having oss-cad-suite in path at the same time as having a venv-installed cocotb version results in a glibc linker error on arch linux
Steps to reproduce
Create a cocotb "project" as follows
test_my_design.py
my_design.sv
Makefile
Then create a venv, activate it, and install cocotb with
pip
Running
make
now results inNot cativating the venv, or not having oss-cad suite in PATH results in the expected output
This is on an up to date Arch system with the latest oss-cad-suite