Open rsdoiel opened 2 weeks ago
Updated the Python install to set the default python to 3.9 to ensure that calling invenio-rdm container start --lock --build --setup
wasn't actually getting confused. Now the error message is slightly difference (gets further into the process).
Starting InvenioRDM instance...
Locking python dependencies...
Creating a virtualenv for this project...
Pipfile: /Sites/vanilla/Pipfile
Using /usr/bin/python3 (3.9.19) to create virtualenv...
⠼ Creating virtual environment...created virtual environment CPython3.9.19.final.0-64 in 309ms
creator CPython3Posix(dest=/home/ubuntu/.local/share/virtualenvs/vanilla-FOfWfWzA, clear=False, no_vcs_ignore=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/ubuntu/.local/share/virtualenv)
added seed packages: pip==24.0, setuptools==69.5.1, wheel==0.43.0
activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
✔ Successfully created virtual environment!
Virtualenv location: /home/ubuntu/.local/share/virtualenvs/vanilla-FOfWfWzA
Traceback (most recent call last):
File "/usr/local/bin/pipenv", line 8, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.9/dist-packages/pipenv/vendor/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.9/dist-packages/pipenv/cli/options.py", line 58, in main
return super().main(*args, **kwargs, windows_expand_args=False)
File "/usr/local/lib/python3.9/dist-packages/pipenv/vendor/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.9/dist-packages/pipenv/vendor/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.9/dist-packages/pipenv/vendor/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.9/dist-packages/pipenv/vendor/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.9/dist-packages/pipenv/vendor/click/decorators.py", line 92, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "/usr/local/lib/python3.9/dist-packages/pipenv/vendor/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.9/dist-packages/pipenv/vendor/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.9/dist-packages/pipenv/cli/command.py", line 344, in lock
do_lock(
File "/usr/local/lib/python3.9/dist-packages/pipenv/routines/lock.py", line 34, in do_lock
lockfile = project.lockfile(categories=lockfile_categories)
File "/usr/local/lib/python3.9/dist-packages/pipenv/project.py", line 775, in lockfile
plette.Pipfile.load(pf), categories=categories
File "/usr/local/lib/python3.9/dist-packages/pipenv/vendor/plette/pipfiles.py", line 65, in load
return cls(data)
File "/usr/local/lib/python3.9/dist-packages/pipenv/vendor/plette/models/base.py", line 8, in __init__
self.validate(data)
File "/usr/local/lib/python3.9/dist-packages/pipenv/vendor/plette/pipfiles.py", line 44, in validate
klass.validate(data[key])
File "/usr/local/lib/python3.9/dist-packages/pipenv/vendor/plette/models/base.py", line 118, in validate
cls.item_class.validate(d)
File "/usr/local/lib/python3.9/dist-packages/pipenv/vendor/plette/models/packages.py", line 35, in validate
PackageSpecfiers.validate(data)
File "/usr/local/lib/python3.9/dist-packages/pipenv/vendor/plette/models/base.py", line 49, in validate
raise DataValidationError(f"Invalid type for field {k}: {type(data[k])}")
pipenv.vendor.plette.models.base.DataValidationError: Invalid type for field editable: <class 'pipenv.vendor.tomlkit.items.String'>
Failed to start containerized instance.
failed to complete: invenio-cli containers start --lock --build --setup
ubuntu@invenio-rdm:~$
This isn't a system setup problem, it's this bug in the Pipfile https://github.com/inveniosoftware/cookiecutter-invenio-rdm/pull/283 and the new pipenv version. I emailed about it last week.
You can either manually make the change in the Pipfile or delete your local setup and re-run invenio-cli init -c master
.
I also would use the local install commands invenio-cli install
and invenio-cli services setup
instead of the container commands, since the fully containerized version is hard to work with.
NOTE: to self, per Tom, on line 11 editable="True",
should be editable=true
in Pipefile. I can use sed to fix that.
Now I need to understand the build processing as /bin/sh -c pipenv install --deploy --system
is failing.
Building images...
[+] Building 1.1s (14/41) docker:default
=> [worker internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.09kB 0.0s
=> [worker internal] load .dockerignore 0.0s
=> => transferring context: 86B 0.0s
=> [web-api internal] load metadata for registry.cern.ch/inveniosoftware/almalinux:1 0.6s
=> [web-api internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.09kB 0.0s
=> [web-api internal] load .dockerignore 0.0s
=> => transferring context: 86B 0.0s
=> [web-ui internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.09kB 0.0s
=> [web-ui internal] load .dockerignore 0.0s
=> => transferring context: 86B 0.0s
=> [worker 1/11] FROM registry.cern.ch/inveniosoftware/almalinux:1@sha256:66cfd9d08441708169397c0243c83097c3585638932c24c66ce93bbd053dc0d1 0.0s
=> [web-api internal] load build context 0.2s
=> => transferring context: 19.92MB 0.2s
=> [web-ui internal] load build context 0.2s
=> => transferring context: 19.92MB 0.2s
=> [worker internal] load build context 0.2s
=> => transferring context: 19.92MB 0.2s
=> CACHED [web-api 2/11] COPY site ./site 0.0s
=> [web-api 3/11] COPY Pipfile Pipfile.lock ./ 0.2s
=> ERROR [web-api 4/11] RUN pipenv install --deploy --system 0.1s
------
> [web-api 4/11] RUN pipenv install --deploy --system:
0.131 exec /bin/sh: exec format error
------
failed to solve: process "/bin/sh -c pipenv install --deploy --system" did not complete successfully: exit code: 1
Failed to start containerized instance.
failed to complete: invenio-cli containers start --lock --build --setup
Another note to self. The "container" option is broken. It really means "all container" option. When you choose a "local" install it still uses containers for the parts (E.g. Postgres, Redis, etc).
Martin Fenner has a good setup for the full containerized option see https://starter.front-matter.io/
OK, now I can install a local instance in the Multipass VM. The invenio-cli run
command seems to work. The command invenio-cli services start
invenio-cli services stop` work.
Now I need to wrap up the install setting things up to work like our production system and with the related steps.
Note to self: I have invenio "local" install working reliably. I need to finish adding additional files like those to configure systemd management, web server configuration, etc. These are partially implemented with today's code commit - 6f09cfd
Still tracking down the some NginX vars that needed escaping in the Bash scripts. Commit hash - 68da3d8
The vanilla invenio-rdm machine's cloud init is still not quire right. I'm breaking on the following step when logged in as ubuntu user in
/Sites/vanilla
I think this is failing due to a Node/npm "package.json" file not existing but the error message from pipenv seems to point to a TOML configuration issue. If the problem is TOML related I am not sure which file it is in.