FFY00
commented
4 years ago This seems like a broken pip installation.
python -m ensurepip
python -m pip install --upgrade pipClosed whitequark closed 3 years ago
FFY00
commented
4 years ago This seems like a broken pip installation.
python -m ensurepip
python -m pip install --upgrade pip
whitequark
commented
4 years ago That's a standard Debian pip that works perfectly well for everything else. If I run python -m pip install --upgrade pip then it whines about the script not matching the package each time I run it, so not going to do that.
What's the point of using all the awful conda stuff if you're still going to stick with system pip, anyway?
FFY00
commented
4 years ago I don't use conda, I manage the binary dependencies with the system package manager (I use arch, most stable stuff is in the repos and we have https://github.com/FFY00/symbiflow-arch-pkgs for master).
AFAIK conda works in virtual environments. If python -m pip install --upgrade pip runs the system's pip, then something is wrong, or conda does not do full isolation, only semi. Even if it was only doing semi-isolation and pip was installed in the venv, running pip should work.
Btw, the whole point of running python -m pip instead of pip is to make sure you run pip from the environment if PATH isn't properly set up. If it runs system pip, it is definitely not isolated.
I can install everything fine here:
$ pip install git+https://github.com/symbiflow/fasm git+https://github.com/antmicro/quicklogic-fasm git+https://github.com/antmicro/quicklogic-fasm-utils
Collecting git+https://github.com/symbiflow/fasm
Cloning https://github.com/symbiflow/fasm to /tmp/pip-req-build-p_5w7owv
Running command git clone -q https://github.com/symbiflow/fasm /tmp/pip-req-build-p_5w7owv
Collecting git+https://github.com/antmicro/quicklogic-fasm
Cloning https://github.com/antmicro/quicklogic-fasm to /tmp/pip-req-build-pox_bj_o
Running command git clone -q https://github.com/antmicro/quicklogic-fasm /tmp/pip-req-build-pox_bj_o
Running command git submodule update --init --recursive -q
Collecting git+https://github.com/antmicro/quicklogic-fasm-utils
Cloning https://github.com/antmicro/quicklogic-fasm-utils to /tmp/pip-req-build-0_vqrj6y
Running command git clone -q https://github.com/antmicro/quicklogic-fasm-utils /tmp/pip-req-build-0_vqrj6y
Collecting textx
Downloading textX-2.1.0-py2.py3-none-any.whl (62 kB)
|████████████████████████████████| 62 kB 364 kB/s
Collecting click==7.0
Using cached Click-7.0-py2.py3-none-any.whl (81 kB)
Collecting Arpeggio>=1.9.0
Downloading Arpeggio-1.9.2-py2.py3-none-any.whl (57 kB)
|████████████████████████████████| 57 kB 1.8 MB/s
Building wheels for collected packages: fasm, quicklogic-fasm, fasm-utils
Building wheel for fasm (setup.py) ... done
Created wheel for fasm: filename=fasm-0.0.1-py2.py3-none-any.whl size=8231 sha256=83b036312cd378bc51487ff7b8b328afbb3832fb72d794f34d0a429ab701047b
Stored in directory: /tmp/pip-ephem-wheel-cache-ds9ljd20/wheels/99/13/30/83ebd1452c46e01b4533e202afb18931341abc56685f541936
Building wheel for quicklogic-fasm (setup.py) ... done
Created wheel for quicklogic-fasm: filename=quicklogic_fasm-0.0.1-py3-none-any.whl size=2794082 sha256=36aadfff813571c22fead435bfebbb47cfcd4d20c7707ec7dd5b52a90c609584
Stored in directory: /tmp/pip-ephem-wheel-cache-ds9ljd20/wheels/30/b2/87/b7f0e44e08b4ad44ffd414c64cabcf0ea402d89d0e321fe08f
Building wheel for fasm-utils (setup.py) ... done
Created wheel for fasm-utils: filename=fasm_utils-0.0.1-py3-none-any.whl size=9269 sha256=5150c34d1967daf2c83cb403909ba9e2d68362e2a0d1ccabc2fc69349f9a098a
Stored in directory: /tmp/pip-ephem-wheel-cache-ds9ljd20/wheels/6a/c7/66/731a98a77088671e1d550fad5c15e6e4fd38e859179c5b3f4a
Successfully built fasm quicklogic-fasm fasm-utils
Installing collected packages: click, Arpeggio, textx, fasm, quicklogic-fasm, fasm-utils
Successfully installed Arpeggio-1.9.2 click-7.0 fasm-0.0.1 fasm-utils-0.0.1 quicklogic-fasm-0.0.1 textx-2.1.0$ fasm --help [15:17:49]
usage: FASM tool [-h] [--canonical] file
positional arguments:
file Filename to process
optional arguments:
-h, --help show this help message and exit
--canonical Return canonical form of FASM.
$ qlfasm --help
usage: qlfasm [-h] [--db-root DB_ROOT] [-d] [-v] infile outfile
Converts FASM file to the bitstream or the other way around
positional arguments:
infile The input file (FASM, or bitstream when disassembling)
outfile The output file (bitstream, or FASM when disassembling)
optional arguments:
-h, --help show this help message and exit
--db-root DB_ROOT Path to the fasm database (def. '/home/anubis/.virtualenvs/quicklogic/lib/python3.8/site-packages/quicklogic_fasm/ql732b')
-d, --disassemble Disasseble bitstream
-v, --verbose Adds some verbose messages during bitstream productionI just created a virtual environment and run pip, everything seems to be working as expected.
To me, it seems like you have a bad conda environment set up, or conda is just even worse than I thought :stuck_out_tongue:.
whitequark
commented
4 years ago To me, it seems like you have a bad conda environment set up
I didn't set up a conda environment. I would never use conda voluntarily. I downloaded the installer provided by QuickLogic, which uses conda internally, and evidently that installer did not set a good conda environment, which is exactly what I'm reporting!
whitequark
commented
4 years ago I can install everything fine here:
Oh, and fasm is not the problem--Yosys and VPR are, since those can't be simply installed from PyPI but have to be built from sources.
FFY00
commented
4 years ago I didn't set up a conda environment. I would never use conda voluntarily. I downloaded the installer provided by QuickLogic, which uses conda internally, and evidently that installer did not set a good conda environment, which is exactly what I'm reporting!
Ah, right. Sorry.
Oh, and fasm is not the problem--Yosys and VPR are, since those can't be simply installed from PyPI but have to be built from sources.
I install all that from source. Even if I need to package it myself, it is significantly easier than dealing with conda. I can add quicklogic yosys & vtr to my arch nightly repo, should be fairly easy. But you are running debian so it probably wouldn't be very helful to you.
whitequark
commented
4 years ago But you are running debian so it probably wouldn't be very helful to you.
Indeed. I'd like to eventually provide builds as a part of YoWASP but that requires the Yosys bits to get upstream first.
kkumar23
commented
4 years ago @whitequark : Most of the posts shows by uninstalling pip and re-installing would fix this. Below link has few suggestions, hope it helps. https://stackoverflow.com/questions/49478573/pip3-install-not-working-no-module-named-pip-vendor-pkg-resources
whitequark
commented
4 years ago @kkumar23 Why does the conda installer care about the system pip? What if I don't have a system pip at all?
mithro
commented
4 years ago Something very weird is going on there -- you are correct that conda shouldn't be looking at the system pip.
Could you give me the output of any Python related environment variables (like PYTHONPATH) and similar?
whitequark
commented
4 years ago I don't have any:
$ env|grep PYTHON
$
kgugala
commented
4 years ago @whitequark did you manage to get any more debug info?
whitequark
commented
4 years ago Sort of. I discovered that although I had no conda environments still present, I had a ~/.condarc and ~/.conda which are actually read and modified by the QuickLogic installer. This is clearly a bug (as the installation must be self-contained and not modify any files outside of INSTALL_DIR, otherwise it will conflict with other conda-using software) but deleting them did not fix the pip problem.
mithro
commented
4 years ago @whitequark -- Frustratingly, at the moment there is no way to prevent conda from reading and modifying of ~/.condarc and ~/.conda (except to directly patch conda).
See https://github.com/conda/conda/issues/8599 and https://github.com/conda/conda/issues/8804
whitequark
commented
3 years ago Okay, I figured out what the problem is. The problem is that the Symbiflow conda installation is intended to be an isolated environment, but it is not, and as a result, running pip from the conda environment loads random modules from pythonX.Y/site-packages in both my $HOME/.local and /usr. This, of course, leads to breakage.
The proper fix is to apply this patch to the conda_build_install_package.sh file:
--- conda_build_install_package.sh.orig 2020-11-07 16:14:29.613111775 +0000
+++ conda_build_install_package.sh 2020-11-07 16:08:52.019547039 +0000
@@ -19,6 +19,7 @@
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O conda_installer.sh
bash conda_installer.sh -b -p $INSTALL_DIR/conda && rm conda_installer.sh
source "$INSTALL_DIR/conda/etc/profile.d/conda.sh"
+echo "include-system-site-packages=false" >> $INSTALL_DIR/conda/pyvenv.cfg
conda config --set always_yes yes --set changeps1 no
conda config --add channels conda-forge
conda config --add channels quicklogic-corp/label/qlI couldn't find where this file is generated or I would have sent a PR.
The workaround for the time being is to export PYTHONNOUSERSITE=1 before running the installer, and then run echo "include-system-site-packages=false" >> $INSTALL_DIR/conda/pyvenv.cfg manually.
kkumar23
commented
3 years ago @whitequark : I have updated with "+echo "include-system-site-packages=false" >> $INSTALL_DIR/conda/pyvenv.cfg" . Please verify with v1.3.0
whitequark
commented
3 years ago I believe this issue is fixed.
I'm installing the v1.0.0 release on Debian 10 buster with Python 3.7.3 and pip 18.1. This does not work because conda pip is trying to load something from system pip: