Open jameskim0987 opened 2 years ago
You will need to install doit manually, you should be able to install everything you need by running
pip install -r buildchainrequirements.txt
I just checked and yes apparently the requirements section for build did not get updated with the latest changes.
Installing doit
should be done by the ./doit.sh
script:
# Install/reinstall the virtual environment only if it either doesn't exist or
# the requirements have changed since its creation.
if [ "${REQ_TSTAMP}" -gt "${WIT_TSTAMP}" ]
then
"${PYTHON_SYS}" -m venv --clear "${BUILDENV}"
"${BUILDENV}/bin/pip" install -r "${REQUIREMENTS}"
touch "${WITNESS_FILE}"
fi
I'd suggest recreating the venv from scratch (rm -rf buildchain/.venv
), and if the issue still occurs, please share the installation logs @jameskim0987.
@TeddyAndrieux It seems like the pip install -r is called from ./doit.sh
@gdemonet I previously did not have pip3
not installed so I installed pip3 and aliased to pip.
After rm -rf buildchain/.venv
and running ./doit
, I get the following log:
Ignoring macfsevents: markers 'sys_platform == "darwin"' don't match your environment
Collecting certifi==2021.10.8 (from -r buildchain/requirements.txt (line 7))
Using cached https://files.pythonhosted.org/packages/37/45/946c02767aabb873146011e665728b680884cd8fe70dde973c640e45b775/certifi-2021.10.8-py2.py3-none-any.whl
Collecting charset-normalizer==2.0.10 (from -r buildchain/requirements.txt (line 8))
Using cached https://files.pythonhosted.org/packages/84/3e/1037abe6498e65d645ce7a22d3402605d49a3b2c7f20c3abb027760da4f0/charset_normalizer-2.0.10-py3-none-any.whl
Collecting cloudpickle==2.0.0 (from -r buildchain/requirements.txt (line 9))
Using cached https://files.pythonhosted.org/packages/07/3c/bf72ebd3e78eb1ef773f4f0650ecdc29c6454aeafe9c08f6da3f227dd2bc/cloudpickle-2.0.0-py3-none-any.whl
Collecting docker==4.1.0 (from -r buildchain/requirements.txt (line 10))
Using cached https://files.pythonhosted.org/packages/cc/ca/699d4754a932787ef353a157ada74efd1ceb6d1fc0bfb7989ae1e7b33111/docker-4.1.0-py2.py3-none-any.whl
Collecting doit==0.34.0 (from -r buildchain/requirements.txt (line 11))
Using cached https://files.pythonhosted.org/packages/98/af/569733b2f7ef5e4c865a47685f8ee7a168d00e53dd0b96f8bd027d4cdb4c/doit-0.34.0-py3-none-any.whl
Collecting idna==3.3 (from -r buildchain/requirements.txt (line 12))
Using cached https://files.pythonhosted.org/packages/04/a2/d918dcd22354d8958fe113e1a3630137e0fc8b44859ade3063982eacd2a4/idna-3.3-py3-none-any.whl
Collecting pyyaml==5.3.1 (from -r buildchain/requirements.txt (line 13))
Using cached https://files.pythonhosted.org/packages/64/c2/b80047c7ac2478f9501676c988a5411ed5572f35d1beff9cae07d321512c/PyYAML-5.3.1.tar.gz
Collecting requests==2.27.1 (from -r buildchain/requirements.txt (line 14))
Using cached https://files.pythonhosted.org/packages/2d/61/08076519c80041bc0ffa1a8af0cbd3bf3e2b62af10435d269a9d0f40564d/requests-2.27.1-py2.py3-none-any.whl
Collecting six==1.16.0 (from -r buildchain/requirements.txt (line 15))
Using cached https://files.pythonhosted.org/packages/d9/5a/e7c31adbe875f2abbb91bd84cf2dc52d792b5a01506781dbcf25c91daf11/six-1.16.0-py2.py3-none-any.whl
Collecting urllib3==1.26.8 (from -r buildchain/requirements.txt (line 16))
Using cached https://files.pythonhosted.org/packages/4e/b8/f5a25b22e803f0578e668daa33ba3701bb37858ec80e08a150bd7d2cf1b1/urllib3-1.26.8-py2.py3-none-any.whl
Collecting websocket-client==1.2.3 (from -r buildchain/requirements.txt (line 17))
Using cached https://files.pythonhosted.org/packages/88/03/cd5aa2c97356b7eee484a2b63fe850076ecb82fdeda82979a234b62a9a37/websocket_client-1.2.3-py3-none-any.whl
Collecting pyinotify==0.9.6 (from -r buildchain/requirements.txt (line 20))
Building wheels for collected packages: pyyaml
Running setup.py bdist_wheel for pyyaml ... error
Complete output from command /root/workspace/metalk8s-2.11.2/buildchain/.venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-7q_m9v5j/pyyaml/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpuwwwe8bapip-wheel- --python-tag cp36:
usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: -c --help [cmd1 cmd2 ...]
or: -c --help-commands
or: -c cmd --help
error: invalid command 'bdist_wheel'
----------------------------------------
Failed building wheel for pyyaml
Running setup.py clean for pyyaml
Failed to build pyyaml
Installing collected packages: certifi, charset-normalizer, cloudpickle, idna, urllib3, requests, websocket-client, six, docker, pyinotify, doit, pyyaml
Running setup.py install for pyyaml ... done
Successfully installed certifi-2021.10.8 charset-normalizer-2.0.10 cloudpickle-2.0.0 docker-4.1.0 doit-0.34.0 idna-3.3 pyinotify-0.9.6 pyyaml-5.3.1 requests-2.27.1 six-1.16.0 urllib3-1.26.8 websocket-client-1.2.3
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
Traceback (most recent call last):
File "/root/workspace/metalk8s-2.11.2/buildchain/.venv/lib/python3.6/site-packages/doit/doit_cmd.py", line 299, in run
return command.parse_execute(args)
File "/root/workspace/metalk8s-2.11.2/buildchain/.venv/lib/python3.6/site-packages/doit/cmd_base.py", line 151, in parse_execute
return self.execute(params, args)
File "/root/workspace/metalk8s-2.11.2/buildchain/.venv/lib/python3.6/site-packages/doit/cmd_base.py", line 591, in execute
self.task_list = self.loader.load_tasks(cmd=self, pos_args=args)
File "/root/workspace/metalk8s-2.11.2/buildchain/.venv/lib/python3.6/site-packages/doit/cmd_base.py", line 391, in load_tasks
args=pos_args, config=self.config)
File "/root/workspace/metalk8s-2.11.2/buildchain/.venv/lib/python3.6/site-packages/doit/loader.py", line 213, in load_tasks
_process_gen(ref, creator_kwargs)
File "/root/workspace/metalk8s-2.11.2/buildchain/.venv/lib/python3.6/site-packages/doit/loader.py", line 166, in _process_gen
gen_tasks = generate_tasks(name, ref(**creator_kwargs), ref.__doc__)
File "/root/workspace/metalk8s-2.11.2/buildchain/.venv/lib/python3.6/site-packages/doit/loader.py", line 384, in generate_tasks
for task_dict, x_doc in flat_generator(gen_result, gen_doc):
File "/root/workspace/metalk8s-2.11.2/buildchain/.venv/lib/python3.6/site-packages/doit/loader.py", line 27, in flat_generator
for item in gen:
File "/root/workspace/metalk8s-2.11.2/buildchain/buildchain/format.py", line 21, in task_format
yield create_format_task()
File "/root/workspace/metalk8s-2.11.2/buildchain/buildchain/format.py", line 51, in format_python
filepath for filepath in utils.git_ls() if ".py" in filepath.suffixes
File "/root/workspace/metalk8s-2.11.2/buildchain/buildchain/utils.py", line 131, in git_ls
["git", "ls-files", "-z", root], encoding="utf-8"
File "/usr/lib/python3.6/subprocess.py", line 356, in check_output
**kwargs).stdout
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['git', 'ls-files', '-z', PosixPath('/root/workspace/metalk8s-2.11.2')]' returned non-zero exit status 128.
The errors that stand out to me are
error: invalid command 'bdist_wheel'
, Failed building wheel for pyyaml
fatal: not a git repository (or any of the parent directories): .git
The git one and the python callback error log is something that concerns me the most. How can I resolve those errors?
@jameskim0987 thanks for reporting back.
The first errors mentioned are not problematic, and quite common when using pip
without having the wheel
package installed. As can be seen, there's most often a fallback to installation using setuptools
(the Running setup.py install for pyyaml ... done
line confirms this), which succeeded in your case.
This means the buildchain environment was correctly setup, and the rest is about using the buildchain itself.
The last error explains it: your directory /root/workspace/metalk8s-2.11.2
is not a Git repository, which it needs to be (the buildchain uses Git metadata to know which files are tracked and updated). This directory should be obtained from git clone git@github.com:scality/metalk8s
or similar.
Hope this helps!
WhenI run
./doit.sh
for building, I get the following errorInitially, I thought the problem was from python not pointing to python3 but the error persists. How can I fix this issue?