Failing Builds (Spark2014; e3-core 22.5.0; NixOS)

atalii commented 2 months ago

Hi,

I'm trying to get build Spark2014 on NixOS at the moment, but I'm running into some issues.

The anod files don't seem to be compatible with e3-core version 22.5.0. There were a few things I needed to fix to get it working with that:

Referencing ReturnValue.force_skip raised an attribute error. Seemingly, that enum variant is no longer around. I simply removed that offending reference.
SandBox was missing the required root directory positional argument. I moved the assignment in the subsequent line to the constructor, which seemed to fix the issue.
EmptyJob's constructor call here was missing a required positional parameter. I quickly hacked in a line change to fix that.

Now, I've got a new and exciting runtime error that I don't know how to fix on my own. I've copied the command output below:

The log's quite long :)

``` INFO downloading /home/atalii/Documents/GNAT-FSF-Builds/sbx/tmp/cache/spark2014-13.2.0.tar.gz size=0 244it [00:03, 65.80it/s] Exception in thread download.spark2014-13.2.0.tar.gz: Traceback (most recent call last): File "/nix/store/gd3shnza1i50zn8zs04fa729ribr88m9-python3-3.11.8/lib/python3.11/threading.py", line 1045, in _bootstrap_inner self.run() INFO [default success 5s] download source spark2014-13.2.0.tar.gz File "/nix/store/gd3shnza1i50zn8zs04fa729ribr88m9-python3-3.11.8/lib/python3.11/threading.py", line 982, in run INFO [default unknown 0s] get source spark2014-13.2.0.tar.gz self._target(*self._args, **self._kwargs) File "/home/atalii/Documents/e3/ld-distro/src/e3/job/__init__.py", line 133, in task_function self.run() File "/home/atalii/Documents/GNAT-FSF-Builds/lib/anod/build.py", line 161, in run f.write(hash.sha1(os.path.join(cache_dir, builder.filename))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/atalii/Documents/e3/ld-distro/src/e3/hash.py", line 53, in sha1 return __compute_hash(path, "sha1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/atalii/Documents/e3/ld-distro/src/e3/hash.py", line 22, in __compute_hash raise HashError(kind, f"cannot find {path}") e3.hash.HashError: cannot find /home/atalii/Documents/GNAT-FSF-Builds/sbx/tmp/cache/fsf-13: sha1 ERROR alt-ergo build fails _ Traceback (most recent call last): _ File "/home/atalii/Documents/e3/ld-distro/src/e3/anod/spec.py", line 517, in primitive_func _ result = f(self, *args, **kwargs) _ ^^^^^^^^^^^^^^^^^^^^^^^^ _ File "/home/atalii/Documents/GNAT-FSF-Builds/specs/alt-ergo.anod", line 14, in build _ sync_tree(os.path.join(self.spec_dir, "..", "alt-ergo_artifact"), _ File "/home/atalii/Documents/e3/ld-distro/src/e3/fs.py", line 985, in sync_tree _ raise FSError(origin="sync_tree", message=f"{source} does not exist") _ e3.fs.FSError: sync_tree: /home/atalii/Documents/GNAT-FSF-Builds/specs/../alt-ergo_artifact does not exist _ ERROR got exception while building _ Traceback (most recent call last): _ File "/home/atalii/Documents/e3/ld-distro/src/e3/anod/spec.py", line 517, in primitive_func _ result = f(self, *args, **kwargs) _ ^^^^^^^^^^^^^^^^^^^^^^^^ _ File "/home/atalii/Documents/GNAT-FSF-Builds/specs/alt-ergo.anod", line 14, in build _ sync_tree(os.path.join(self.spec_dir, "..", "alt-ergo_artifact"), _ File "/home/atalii/Documents/e3/ld-distro/src/e3/fs.py", line 985, in sync_tree _ raise FSError(origin="sync_tree", message=f"{source} does not exist") _ e3.fs.FSError: sync_tree: /home/atalii/Documents/GNAT-FSF-Builds/specs/../alt-ergo_artifact does not exist _ _ _ The above exception was the direct cause of the following exception: _ _ Traceback (most recent call last): _ File "/home/atalii/Documents/GNAT-FSF-Builds/lib/anod/build.py", line 90, in run _ self.data.anod_instance.build() _ File "/home/atalii/Documents/e3/ld-distro/src/e3/anod/spec.py", line 525, in primitive_func _ raise AnodError(error_msg) from err _ e3.anod.error.AnodError: alt-ergo build fails INFO [default failure 0s] build alt-ergo for native x86_64-linux INFO downloading /home/atalii/Documents/GNAT-FSF-Builds/sbx/x86_64-linux/base_gcc/build/gnat-x86_64-linux-13.2.0-2.tar.gz size=333967041 319it [00:07, 44.23it/s] INFO [default success 14s] build base_gcc for native x86_64-linux INFO [default success 0s] download source gprbuild-no-static.patch INFO [default unknown 0s] get source gprbuild-no-static.patch INFO downloading /home/atalii/Documents/GNAT-FSF-Builds/sbx/tmp/cache/gmp-6.2.1.tar.bz2 size=2493916 3it [00:01, 1.55it/s] INFO [default success 2s] download source gmp-6.2.1.tar.bz2 INFO [default unknown 0s] get source gmp-6.2.1.tar.bz2 INFO downloading /home/atalii/Documents/GNAT-FSF-Builds/sbx/tmp/cache/libgpr-24.0.0.tar.gz size=0 57it [00:00, 162.64it/s] Exception in thread download.libgpr-24.0.0.tar.gz: Traceback (most recent call last): File "/nix/store/gd3shnza1i50zn8zs04fa729ribr88m9-python3-3.11.8/lib/python3.11/threading.py", line 1045, in _bootstrap_inner self.run() File "/nix/store/gd3shnza1i50zn8zs04fa729ribr88m9-python3-3.11.8/lib/python3.11/threading.py", line 982, in run self._target(*self._args, **self._kwargs) INFO [default success 1s] download source libgpr-24.0.0.tar.gz File "/home/atalii/Documents/e3/ld-distro/src/e3/job/__init__.py", line 133, in task_function INFO [default unknown 0s] get source libgpr-24.0.0.tar.gz self.run() File "/home/atalii/Documents/GNAT-FSF-Builds/lib/anod/build.py", line 161, in run f.write(hash.sha1(os.path.join(cache_dir, builder.filename))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/atalii/Documents/e3/ld-distro/src/e3/hash.py", line 53, in sha1 return __compute_hash(path, "sha1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/atalii/Documents/e3/ld-distro/src/e3/hash.py", line 22, in __compute_hash raise HashError(kind, f"cannot find {path}") e3.hash.HashError: cannot find /home/atalii/Documents/GNAT-FSF-Builds/sbx/tmp/cache/v24.0.0.tar.gz: sha1 ERROR why3 build fails _ Traceback (most recent call last): _ File "/home/atalii/Documents/e3/ld-distro/src/e3/anod/spec.py", line 517, in primitive_func _ result = f(self, *args, **kwargs) _ ^^^^^^^^^^^^^^^^^^^^^^^^ _ File "/home/atalii/Documents/GNAT-FSF-Builds/specs/why3.anod", line 14, in build _ sync_tree(os.path.join(self.spec_dir, "..", "why3_artifact"), _ File "/home/atalii/Documents/e3/ld-distro/src/e3/fs.py", line 985, in sync_tree _ raise FSError(origin="sync_tree", message=f"{source} does not exist") _ e3.fs.FSError: sync_tree: /home/atalii/Documents/GNAT-FSF-Builds/specs/../why3_artifact does not exist _ ERROR got exception while building _ Traceback (most recent call last): _ File "/home/atalii/Documents/e3/ld-distro/src/e3/anod/spec.py", line 517, in primitive_func _ result = f(self, *args, **kwargs) _ ^^^^^^^^^^^^^^^^^^^^^^^^ _ File "/home/atalii/Documents/GNAT-FSF-Builds/specs/why3.anod", line 14, in build _ sync_tree(os.path.join(self.spec_dir, "..", "why3_artifact"), _ File "/home/atalii/Documents/e3/ld-distro/src/e3/fs.py", line 985, in sync_tree _ raise FSError(origin="sync_tree", message=f"{source} does not exist") _ e3.fs.FSError: sync_tree: /home/atalii/Documents/GNAT-FSF-Builds/specs/../why3_artifact does not exist _ _ _ The above exception was the direct cause of the following exception: _ _ Traceback (most recent call last): _ File "/home/atalii/Documents/GNAT-FSF-Builds/lib/anod/build.py", line 90, in run _ self.data.anod_instance.build() _ File "/home/atalii/Documents/e3/ld-distro/src/e3/anod/spec.py", line 525, in primitive_func _ raise AnodError(error_msg) from err _ e3.anod.error.AnodError: why3 build fails INFO [default failure 0s] build why3 for native x86_64-linux INFO downloading /home/atalii/Documents/GNAT-FSF-Builds/sbx/tmp/cache/gcc-13.2.0.tar.gz size=153347847 147it [00:05, 27.38it/s] INFO [default success 5s] download source gcc-13.2.0.tar.gz INFO [default unknown 0s] get source gcc-13.2.0.tar.gz INFO downloading /home/atalii/Documents/GNAT-FSF-Builds/sbx/tmp/cache/z3-4.12.2-x64-glibc-2.31.zip size=47043310 45it [00:01, 26.13it/s] INFO [default success 2s] download source z3-4.12.2-x64-glibc-2.31.zip INFO [default unknown 0s] get source z3-4.12.2-x64-glibc-2.31.zip INFO downloading /home/atalii/Documents/GNAT-FSF-Builds/sbx/tmp/cache/cvc5-linux size=27486200 27it [00:01, 26.47it/s] INFO [default success 2s] download source cvc5-linux INFO [default unknown 0s] get source cvc5-linux INFO downloading /home/atalii/Documents/GNAT-FSF-Builds/sbx/tmp/cache/gprconfig_kb-23.0.0.tar.gz size=0 2it [00:00, 17.84it/s] INFO [default success 0s] download source gprconfig_kb-23.0.0.tar.gz INFO [default unknown 0s] get source gprconfig_kb-23.0.0.tar.gz INFO downloading /home/atalii/Documents/GNAT-FSF-Builds/sbx/tmp/cache/xmlada-24.0.0.tar.gz size=0 46it [00:00, 126.96it/s] INFO [default success 1s] download source xmlada-24.0.0.tar.gz INFO [default unknown 0s] get source xmlada-24.0.0.tar.gz INFO downloading /home/atalii/Documents/GNAT-FSF-Builds/sbx/tmp/cache/gprbuild-24.0.0.tar.gz size=0 56it [00:00, 133.98it/s] INFO [default success 1s] download source gprbuild-24.0.0.tar.gz INFO [default unknown 0s] get source gprbuild-24.0.0.tar.gz INFO downloading /home/atalii/Documents/GNAT-FSF-Builds/sbx/tmp/cache/gnatcoll-core-24.0.0.tar.gz size=0 68it [00:00, 153.94it/s] Exception in thread download.gnatcoll-core-24.0.0.tar.gz: Traceback (most recent call last): File "/nix/store/gd3shnza1i50zn8zs04fa729ribr88m9-python3-3.11.8/lib/python3.11/threading.py", line 1045, in _bootstrap_inner INFO [default success 1s] download source gnatcoll-core-24.0.0.tar.gz self.run() INFO [default unknown 0s] get source gnatcoll-core-24.0.0.tar.gz File "/nix/store/gd3shnza1i50zn8zs04fa729ribr88m9-python3-3.11.8/lib/python3.11/threading.py", line 982, in run self._target(*self._args, **self._kwargs) File "/home/atalii/Documents/e3/ld-distro/src/e3/job/__init__.py", line 133, in task_function self.run() File "/home/atalii/Documents/GNAT-FSF-Builds/lib/anod/build.py", line 161, in run f.write(hash.sha1(os.path.join(cache_dir, builder.filename))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/atalii/Documents/e3/ld-distro/src/e3/hash.py", line 53, in sha1 return __compute_hash(path, "sha1") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/atalii/Documents/e3/ld-distro/src/e3/hash.py", line 22, in __compute_hash raise HashError(kind, f"cannot find {path}") e3.hash.HashError: cannot find /home/atalii/Documents/GNAT-FSF-Builds/sbx/tmp/cache/v24.0.0.tar.gz: sha1 ```

What do I need to make the build environment work, and/or am I missing any documentation? Thanks so much!

Fabien-Chouteau commented 2 months ago

Hello @atalii ,

This repos is indeed still using e3-core version 22.1.0: https://github.com/alire-project/GNAT-FSF-builds/blob/57ff8786baca38bf005ee5c03dabe95a6750d67e/.github/workflows/linux.yml#L25

If you are able to use this version that would be the best solution so far.

Fabien-Chouteau commented 2 months ago

I did a quick test with 22.5.0 and after fixing the issues you identified (patch below) I got a successful build...

I do see the error messages but it seems like the passes anyway.

diff --git a/lib/anod/build.py b/lib/anod/build.py
index 5201922..e45d615 100644
--- a/lib/anod/build.py
+++ b/lib/anod/build.py
@@ -60,7 +60,7 @@ def add_anod_files_to_fingerprint(

 class FSFBuildEmptyJob(EmptyJob):
     def __init__(self, uid, data, notify_end, sandbox):
-        super(FSFBuildEmptyJob, self).__init__(uid, data, notify_end)
+        super(FSFBuildEmptyJob, self).__init__(uid, data, notify_end, 0)

 class FSFBuildJob(Job):
diff --git a/lib/anod/util.py b/lib/anod/util.py
index d55990f..22f5036 100644
--- a/lib/anod/util.py
+++ b/lib/anod/util.py
@@ -35,7 +35,7 @@ def create_anod_context(spec_dir: str) -> AnodContext:

 def create_anod_sandbox(sbx_dir: str, spec_dir: str) -> SandBox:
-    sbx = SandBox()
+    sbx = SandBox(sbx_dir)
     sbx.root_dir = sbx_dir
     sbx.specs_dir = spec_dir

diff --git a/lib/anod_build.py b/lib/anod_build.py
index 56ac5f2..1c1c9ee 100644
--- a/lib/anod_build.py
+++ b/lib/anod_build.py
@@ -22,7 +22,6 @@ os.environ["FSFBUILD_ROOT_DIR"] = FSFBUILD_ROOT_DIR
 # Define what we mean by a successful build.
 BUILD_SUCCESS = [
     ReturnValue.success,
-    ReturnValue.force_skip,
     ReturnValue.skip,
     ReturnValue.unchanged,
 ]

atalii commented 2 months ago

The build still fails on my machine with both with e3-core==22.1.0 and under 22.5.0 with that patch applied. I'll try to replicate the environment in the workflow and try again. Thanks for the quick and helpful response!

atalii commented 2 months ago

I was able to replicate the build environment in the workflow and I believe I've got things working. Thanks again for the quick response; sorry to bother you!

Fabien-Chouteau commented 2 months ago

No problem, do you plan to contribute SPARK builds to Nixpkgs?

atalii commented 2 months ago

We'll see - Eventually I want to set something up to make Alire crates build easily (likely in a similar manner to the existing buildGoModule), and that may or may not entail SPARK builds from this repository. I've not got anything planned at the moment, though. This is all just something I'd eventually like to get done in my free time :)

alire-project / GNAT-FSF-builds

Failing Builds (Spark2014; e3-core 22.5.0; NixOS) #64