search

YunoHost-Apps / bonfire_ynh

Bonfire federated social network platform
https://bonfirenetworks.org/
GNU Affero General Public License v3.0
1 stars 2 forks source link

Make sure build process works [was: Install fails on erlang dependency] #7

Open mayel opened 3 months ago

mayel commented 3 months ago 
 Installing bonfire…
Info: Provisioning sources...
Info: Provisioning ports...
Info: Provisioning system_user...
Info: Provisioning install_dir...
Info: Provisioning permissions...
Info: Provisioning apt...
Warning:   en_us
Warning: ls: cannot access '/etc/yunohost/hooks.d': No such file or directory
Warning: ls: cannot access '/opt/yunohost': No such file or directory
Warning: ls: cannot access '/etc/yunohost/hooks.d': No such file or directory
Warning: ls: cannot access '/opt/yunohost': No such file or directory
Warning: ls: cannot access '/etc/yunohost/hooks.d': No such file or directory
Warning: ls: cannot access '/opt/yunohost': No such file or directory
Warning: E: Packages need to be removed but remove is disabled.
Warning: The following packages have unmet dependencies:
Warning:  elixir : Depends: erlang-base (>= 1:25.0.3) but 1:23.2.6+dfsg-1+deb11u1 is to be installed or
Warning:                    erlang-base-hipe (>= 1:25.0.3) or
Warning:                    esl-erlang (>= 1:25.0.3) but it is not going to be installed
Warning:           Depends: erlang-crypto (>= 1:25.0.3) or
Warning:                    esl-erlang (>= 1:25.0.3) but it is not going to be installed
Warning:           Depends: erlang-inets (>= 1:25.0.3) or
Warning:                    esl-erlang (>= 1:25.0.3) but it is not going to be installed
Warning:           Depends: erlang-ssl (>= 1:25.0.3) or
Warning:                    esl-erlang (>= 1:25.0.3) but it is not going to be installed
Warning:           Depends: erlang-syntax-tools (>= 1:25.0.3) or
Warning:                    esl-erlang (>= 1:25.0.3) but it is not going to be installed
Warning: Unable to install dependencies
Error: provision_or_update failed for apt : An error occured inside the script snippet
Info: The operation 'Install the 'bonfire' app' could not be completed. Please share the full log of this operation using the command 'yunohost log share 20240413-095032-app_install-bonfire' to get help
Warning: Here's an extract of the logs before the crash. It might help debugging the error:
Info: DEBUG - + ynh_wait_dpkg_free
Info: DEBUG - + grep -v 'fix-broken\|Reading state info'
Info: DEBUG - + return 0
Info: DEBUG - + LC_ALL=C
Info: DEBUG - + DEBIAN_FRONTEND=noninteractive
Info: DEBUG - + apt-get --assume-yes --quiet -o=Acquire::Retries=3 -o=Dpkg::Use-Pty=0 --no-remove --option Dpkg::Options::=--force-confdef --option Dpkg::Options::=--force-confold install elixir --dry-run
Info: WARNING - The following packages have unmet dependencies:
Info: WARNING -  elixir : Depends: erlang-base (>= 1:25.0.3) but 1:23.2.6+dfsg-1+deb11u1 is to be installed or
Info: WARNING -                    erlang-base-hipe (>= 1:25.0.3) or
Info: WARNING -                    esl-erlang (>= 1:25.0.3) but it is not going to be installed
Info: WARNING -           Depends: erlang-crypto (>= 1:25.0.3) or
Info: WARNING -                    esl-erlang (>= 1:25.0.3) but it is not going to be installed
Info: WARNING -           Depends: erlang-inets (>= 1:25.0.3) or
Info: WARNING -                    esl-erlang (>= 1:25.0.3) but it is not going to be installed
Info: WARNING -           Depends: erlang-ssl (>= 1:25.0.3) or
Info: WARNING -                    esl-erlang (>= 1:25.0.3) but it is not going to be installed
Info: WARNING -           Depends: erlang-syntax-tools (>= 1:25.0.3) or
Info: WARNING -                    esl-erlang (>= 1:25.0.3) but it is not going to be installed
Info: DEBUG - + ynh_die '--message=Unable to install dependencies'
Info: DEBUG - + ynh_exit_properly
Warning: Failed to provision apt : The operation 'Install the 'bonfire' app' could not be completed. Please share the full log of this operation using the command 'yunohost log share 20240413-095032-app_install-bonfire' to get help
Info: Deprovisioning sources...
Info: Deprovisioning ports...
Info: Deprovisioning system_user...
Info: Deprovisioning install_dir...
Info: Deprovisioning permissions...
Info: Deprovisioning apt...
Error: Provisioning, deprovisioning, or updating resources for bonfire failed: The operation 'Install the 'bonfire' app' could not be completed. Please share the full log of this operation using the command 'yunohost log share 20240413-095032-app_install-bonfire' to get help
Error: None
lapineige commented 3 months ago

Warning: ls: cannot access '/etc/yunohost/hooks.d': No such file or directory Warning: ls: cannot access '/opt/yunohost': No such file or directory Warning: ls: cannot access '/etc/yunohost/hooks.d': No such file or directory Warning: ls: cannot access '/opt/yunohost': No such file or directory Warning: ls: cannot access '/etc/yunohost/hooks.d': No such file or directory Warning: ls: cannot access '/opt/yunohost': No such file or directory

Wait what ? :thinking:

Did you run Yunohost post-install ?

mayel commented 3 months ago

yes, using the browser wizard, I can try re-running on cli

mayel commented 3 months ago

yeah: Error: YunoHost is already installed

mayel commented 3 months ago

but seems to fail on the erlang version?

lapineige commented 3 months ago

Warning: ls: cannot access '/etc/yunohost/hooks.d': No such file or directory Warning: ls: cannot access '/opt/yunohost': No such file or directory Warning: ls: cannot access '/etc/yunohost/hooks.d': No such file or directory Warning: ls: cannot access '/opt/yunohost': No such file or directory Warning: ls: cannot access '/etc/yunohost/hooks.d': No such file or directory Warning: ls: cannot access '/opt/yunohost': No such file or directory

Wait what ? 🤔

Did you run Yunohost post-install ?

Uh, same in CI : https://ci-apps-dev.yunohost.org/ci/job/15018

lapineige commented 3 months ago

Ok that's strange…

Debian 11 CI works well : https://ci-apps-dev.yunohost.org/ci/job/15018 Debian 12 CI fails with the same issue as you had : https://ci-apps-bookworm-dev.yunohost.org/ci/job/501

mayel commented 3 months ago

hmm, I'm running Debian GNU/Linux 11 (bullseye), started with a Debian 11.5 VM image and updated all packages...

mayel commented 3 months ago

ah maybe because I'm on arm64?

lapineige commented 3 months ago

Ah yes, I didn't set it up for ARM right now, I started simple :)

lapineige commented 3 months ago

I mean there is only one "common" architecture for now https://github.com/YunoHost-Apps/bonfire_ynh/blob/master/manifest.toml#L92, and I didn't restrict it outside of x86 https://github.com/YunoHost-Apps/bonfire_ynh/blob/master/manifest.toml#L25 because I didn't have the time to test if that would work. But that might be an issue with the dependencies on custom repositories, not supporting ARM I guess.

Sorry for the inconvenience.

mayel commented 3 months ago

Yeah I'll try again later on x86

lapineige commented 3 months ago

For your information, latest CI job logs : https://ci-apps-dev.yunohost.org/ci/logs/15018.log

1015195 DEBUG 10:30:12.191 [info] Running Bonfire.Web.Endpoint with Bandit 1.1.2 at 0.0.0.0:36535 (http)

1015195 DEBUG 10:30:12.192 [error] Could not warm up static assets: could not find static manifest at "/var/www/bonfire/_build/prod/rel/bonfire/lib/bonfire-0.1.0-beta.3/priv/static/cache_manifest.json". Run "mix phx.digest" after building your static files or remove the "cache_static_manifest" configuration from your config files.

1015195 DEBUG 10:30:12.192 [info] Access Bonfire.Web.Endpoint at http://sub.domain.tld:443

1015196 DEBUG 10:30:12.194 [info] Tz is fetching the latest IANA time zone data version at https://data.iana.org/time-zones/tzdb/version

1015196 DEBUG 10:30:12.584 [info] Latest version of the IANA time zone data is 2024a

1015196 DEBUG 10:30:12.584 [info] Tz is downloading the IANA time zone data version 2024a at https://data.iana.org/time-zones/releases/tzdata2024a.tar.gz

1015196 DEBUG 10:30:13.868 [info] Tz download done

1015196 DEBUG [os_mon] memory supervisor port (memsup): Erlang has closed

1015196 DEBUG error: Recipe `cmd` was terminated on line 733 by signal 31'
mayel commented 3 months ago

I also see this:

1026321 INFO DEBUG - Light that fire! bonfire with classic flavour in prod - docker:no - 0.9.8-beta.26 - unknown - flavours/classic - unix/linux on x86_64
1026321 INFO DEBUG - ** (TokenMissingError) nofile:1:50: missing terminator: ) (for "(" starting at line 1)
1026321 INFO DEBUG -     |
1026321 INFO DEBUG -   1 | Bonfire.Me.make_account_and_user(package_checker,
1026321 INFO DEBUG -     |                                                  ^
1026321 INFO DEBUG -     (elixir 1.13.4) lib/code.ex:403: Code.validated_eval_string/3
1026321 INFO DEBUG - error: Recipe `cmd` failed on line 733 with exit code 1

which may be because the " needs to be escaped in this line? ynh_exec_warn_less ynh_exec_as $app -s $SHELL -lc "WITH_DOCKER=no just cmd /var/www/bonfire/_build/prod/rel/bonfire/bin/bonfire rpc 'Bonfire.Me.make_account_and_user("$admin", "$(ynh_user_get_info --username="$admin" --key=mail)", "$password")'"

mayel commented 3 months ago

@lapineige I've now tried building on an x86 (emulated) VM but would need a way to either:

  1. build a custom recipe (from local folder or fork)
  2. use CI instead while being able to both make changes to the recipe and inspect the artifacts
lapineige commented 3 months ago

Thanks for trying this and your feedback <3

build a custom recipe (from local folder or fork)

I don't really understand what that implies. Can you explain to me the basic of such a process on a common use case ?

use CI instead while being able to both make changes to the recipe and inspect the artifacts

For the changed, I guess it can be made in a PR, and then the CI is triggered. For the artifacts, I don't know what they are, but if it's files from the CI output, you can't have a command line access to them. But in the CI you could include some commands that show you the content of some file/folder for instance (that has to be done before the installation fails, otherwise they won't be executed).

lapineige commented 3 months ago

I also see this:

1026321 �[36m�[1mINFO �[mDEBUG - Light that fire! bonfire with classic flavour in prod - docker:no - 0.9.8-beta.26 - unknown - flavours/classic - unix/linux on x86_64
1026321 �[36m�[1mINFO �[mDEBUG - ** (TokenMissingError) nofile:1:50: missing terminator: ) (for "(" starting at line 1)
1026321 �[36m�[1mINFO �[mDEBUG -     |
1026321 �[36m�[1mINFO �[mDEBUG -   1 | Bonfire.Me.make_account_and_user(package_checker,
1026321 �[36m�[1mINFO �[mDEBUG -     |                                                  ^
1026321 �[36m�[1mINFO �[mDEBUG -     (elixir 1.13.4) lib/code.ex:403: Code.validated_eval_string/3
1026321 �[36m�[1mINFO �[mDEBUG - error: Recipe `cmd` failed on line 733 with exit code 1

which may be because the " needs to be escaped in this line? ynh_exec_warn_less ynh_exec_as $app -s $SHELL -lc "WITH_DOCKER=no just cmd /var/www/bonfire/_build/prod/rel/bonfire/bin/bonfire rpc 'Bonfire.Me.make_account_and_user("$admin", "$(ynh_user_get_info --username="$admin" --key=mail)", "$password")'"

Good catch !

mayel commented 3 months ago

I don't really understand what that implies

I just mean instead of using the main branch of this repo can I make changes locally (optionally pushing them to a fork) and then tell yunohost to install it from that source?

mayel commented 3 months ago

Oh looks like I can just do sudo yunohost app install a/local/path 👍

lapineige commented 3 months ago

then tell yunohost to install it from that source?

yunohost app install URL is the way to go :)

mayel commented 3 months ago

@lapineige after banging on our CI with a hammer for a while it is now building OTP realeases using Debian 11, and I've started adapting this recipe to use that (using the same deps-debian.sh script from our repo than CI to make sure it uses the same toolset and we can maintain that it one place), though I haven't time to debug tye systemd service yet and make the app start.

I'm thinking this should either be a separate app or we should preserve your work for a build-on-install approach in a branch as an extra option for those who may want that...

lapineige commented 3 months ago

after banging on our CI with a hammer for a while it is now building OTP realeases using Debian 11

Oh wow… that's awesome ! :tada: Thanks you very much for that change, it will be a lot more comfortable for us to manage the packaging with precompiled binaries :astonished: And it will be a lot faster/lighter to install for every user. :rocket:

though I haven't time to debug tye systemd service yet and make the app start.

I can take a look at this :)

I'm thinking this should either be a separate app or we should preserve your work for a build-on-install approach in a branch as an extra option for those who may want that...

In which case could we want it ? That could be kept as a branch (I made one : https://github.com/YunoHost-Apps/bonfire_ynh/tree/install_with_build_archivedBranch) or in the commit history if we had to revive it someday.

mayel commented 3 months ago

Sounds great, thanks! It may be wanted for people who want to use a forked extension, or one they wrote themselves which hasn't (yet) been added to the app build.

mayel commented 3 months ago

Ah one possible issue with systemd may be missing the mise shims in the PATH

lapineige commented 3 months ago

Sounds great, thanks! It may be wanted for people who want to use a forked extension, or one they wrote themselves which hasn't (yet) been added to the app build.

Ok, well it will be worked on in the future then, it's low priority, we haven't a working package yet :smile:

Ah one possible issue with systemd may be missing the mise shims in the PATH

I don't know what this is

mayel commented 3 months ago

The same as here, which makes the things installed using mise (notably Erlang/elixir) available in the path: https://github.com/bonfire-networks/bonfire-yunohost/blob/88c74d6ed32d97f9378cfdb2838d421c6f1e7cac/scripts/install#L73