I ran into mostly the same errors when setting up Capreolus on CC (Cedar) with setup-cc.md as this issue here. Thus, I followed the setup described in this comment within that issue.
Some of the new issues I noticed were that scipy-stack now uses python 3.8, so the python version changes to 3.8.10 when running module load scipy-stack. I also noticed that the TensorFlow version installed was 2.8.0, despite the requirements.txt listing it to be between 2.3 and 2.5.
Additionally, when running pip install tensorflow-ranking==0.4.2 --no-cache-dir I ran into the error:
ERROR: Could not build wheels for pyarrow which use PEP 517 and cannot be installed directly
This was solved by manually importing pyarrow with the commands below:
Despite this, I was able to verify my installation as per these commands. However, when running
python -m capreolus.run rerank.train with file=docs/reproduction/config_msmarco.txt
to fine-tune a mini MS MARCO, I run into this set of warnings/errors:
2022-04-12 17:37:07.215423: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-04-12 17:37:09.466046: F tensorflow/core/platform/default/env.cc:73] Check failed: ret == 0 (11 vs. 0)Thread tf_numa_-1_Eigen creation via pthread_create() failed.
Aborted
Am a bit stuck here (perhaps it is a mismatching TensorFlow version issue?), so any help would be much appreciated!
I ran into mostly the same errors when setting up Capreolus on CC (Cedar) with setup-cc.md as this issue here. Thus, I followed the setup described in this comment within that issue.
Some of the new issues I noticed were that scipy-stack now uses python 3.8, so the python version changes to 3.8.10 when running
module load scipy-stack. I also noticed that the TensorFlow version installed was 2.8.0, despite the requirements.txt listing it to be between 2.3 and 2.5.Additionally, when running
pip install tensorflow-ranking==0.4.2 --no-cache-dirI ran into the error:ERROR: Could not build wheels for pyarrow which use PEP 517 and cannot be installed directlyThis was solved by manually importing pyarrow with the commands below:
Despite this, I was able to verify my installation as per these commands. However, when running
to fine-tune a mini MS MARCO, I run into this set of warnings/errors:
Am a bit stuck here (perhaps it is a mismatching TensorFlow version issue?), so any help would be much appreciated!