[Bug] - Fails to build: Could not resolve dependencies for project

[yurivict]

K Version

5.3.178

Description

[INFO] This project has been banned from the build due to previous failures.
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for K Framework Tool Parent 1.0-SNAPSHOT:
[INFO] 
[INFO] K Framework Tool Parent ............................ SUCCESS [  3.545 s]
[INFO] K Framework KORE ................................... SUCCESS [  2.481 s]
[INFO] K Framework Tool Kernel ............................ SUCCESS [  1.956 s]
[INFO] K Framework KTree .................................. SUCCESS [  0.683 s]
[INFO] K Framework Java Backend ........................... FAILURE [  0.081 s]
[INFO] K Framework Haskell Backend ........................ FAILURE [  0.038 s]
[INFO] K Framework LLVM Backend Pattern Matching .......... SUCCESS [  7.650 s]
[INFO] K Framework LLVM Backend ........................... FAILURE [  0.024 s]
[INFO] K Framework Tool Distribution ...................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  17.634 s
[INFO] Finished at: 2022-08-29T16:13:33-07:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project java-backend: Could not resolve dependencies for project com.runtimeverification.k:java-backend:jar:1.0-SNAPSHOT: com.runtimeverification.k:kernel:jar:tests:1.0-SNAPSHOT was not found in https://s3.us-east-2.amazonaws.com/runtimeverificationmaven/snapshots during a previous attempt. This failure was cached in the local repository and resolution is not reattempted until the update interval of runtime.verification.snapshots has elapsed or updates are forced -> [Help 1]
[ERROR] Failed to execute goal on project haskell-backend: Could not resolve dependencies for project com.runtimeverification.k:haskell-backend:jar:1.0-SNAPSHOT: com.runtimeverification.k:kernel:jar:tests:1.0-SNAPSHOT was not found in https://s3.us-east-2.amazonaws.com/runtimeverificationmaven/snapshots during a previous attempt. This failure was cached in the local repository and resolution is not reattempted until the update interval of runtime.verification.snapshots has elapsed or updates are forced -> [Help 1]
[ERROR] Failed to execute goal on project llvm-backend: Could not resolve dependencies for project com.runtimeverification.k:llvm-backend:jar:1.0-SNAPSHOT: com.runtimeverification.k:kernel:jar:tests:1.0-SNAPSHOT was not found in https://s3.us-east-2.amazonaws.com/runtimeverificationmaven/snapshots during a previous attempt. This failure was cached in the local repository and resolution is not reattempted until the update interval of runtime.verification.snapshots has elapsed or updates are forced -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :java-backend
*** Error code 1

Java 17 maven-3.8.6 FreeBSD 13.1

[SchmErik]

@yurivict as far as I know, I don't think we officially support freebsd... Have you followed the steps here to install K framework in the README? https://github.com/runtimeverification/k#prerequisite-install-guide

I think you're missing the submodule initialization command.. Try git submodule update --init --recursive and then do mvn package again...

[yurivict]

After fixing git submodules it now fails like this:

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for K Framework Tool Parent 1.0-SNAPSHOT:
[INFO] 
[INFO] K Framework Tool Parent ............................ SUCCESS [  0.446 s]
[INFO] K Framework KORE ................................... SUCCESS [  1.602 s]
[INFO] K Framework Tool Kernel ............................ SUCCESS [  6.877 s]
[INFO] K Framework KTree .................................. SUCCESS [  0.057 s]
[INFO] K Framework Java Backend ........................... SUCCESS [  0.758 s]
[INFO] K Framework Haskell Backend ........................ FAILURE [01:19 min]
[INFO] K Framework LLVM Backend Pattern Matching .......... SKIPPED
[INFO] K Framework LLVM Backend ........................... SKIPPED
[INFO] K Framework Tool Distribution ...................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:30 min
[INFO] Finished at: 2022-08-29T23:27:09-07:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (build-haskell) on project haskell-backend: An Ant BuildException has occured: exec returned: 1
[ERROR] around Ant part ...<exec failonerror="true" dir="/disk-samsung/freebsd-ports/lang/k/work/k-5.3.178/haskell-backend/src/main/native/haskell-backend" executable="stack">... @ 5:151 in /disk-samsung/freebsd-ports/lang/k/work/k-5.3.178/haskell-backend/target/antrun/build-main.xml
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :haskell-backend

There's no specific error in the error message. Adding -e and -X flags only add maven stack printouts, but still don't show the original error message.

[dwightguth]

The specific error would be earlier in the log when it's actually trying to build that module. We can't really help without that fragment of the log.

[yurivict]

The log doesn't seem to contain any error messages before the final failure: k-build-failure.log

[SchmErik]

I'm assuming you did the curl -sSL https://get.haskellstack.org/ | sh command to install stack. Maybe try mvn clean and rebuild?

[dwightguth]

This appears to be all that shows up in the log:

[exec] <stderr>: commitAndReleaseBuffer: invalid argument (invalid character)

I'm not entirely sure why this would happen, unless maybe you downloaded k to a directory with some unusual characters in its path and some part of stack or ghc choked on that file path. If doing a completely clean build doesn't work, try reinstalling stack and/or sanitizing the directory you downloaded k to if needed? Beyond that, I have no idea.

[dwightguth]

It's possible it's just a bug in ghc or stack that only happens on freebsd. We don't officially support that platform, so there's not a lot we can try to do to fix it, but if you figure out what is wrong, feel free to share for future reference.

[yurivict]

It builds using a tarball and not a git repository. Does it by any chance run 'git' on it?

[dwightguth]

Hmmm. I believe that stack uses git to download a couple dependencies that aren't on stackage. I wouldn't have expected that to lead to an error like the one I see here, but if it makes sense to you, sure.

[yurivict]

The need to download something during build makes software not packageable, because most port systems require that all source files are known before build for security reasons.

[yurivict]

Yes, it runs git clone https://github.com/ttuegel/pipes-ghc-events.git /tmp/with-repo86726/cloned which breaks.

This isn't a FreeBSD problem. This is a general porting/packaging problem. I just happened to be the first person who attempted to package K, and I am on FreeBSD. Debian/RPM systems will all have the same problem.

[yurivict]

Is there a way to pre-download all prerequisites so that the build wouldn't attempt to download anything?

[dwightguth]

I'm not sure. There might be such an option for stack. I know there is for maven (which also extensively downloads things during the build process), but that feature of maven is quite buggy in my experience. I have not had much luck in the past building k from source without an internet connection because of that bug, which is part of why more resources haven't been invested in that, but I certainly am not opposed to effort being put into it.

There are two dependencies in question: one is a fork of a package that is already in hackage with minor changes because the package appears to be abandoned. If you can't get what you need from stack, the way forward here would probably be to try and see if maintenance of the package in hackage could be transferred to someone else (maybe us?), and then upstream the changes. I will talk with the team that maintains our haskell code to see what we can do here, if you would like to pursue this approach.

The second package is one that, as far as I can tell, was developed specifically for use by our codebase by a former employee, and we probably can just migrate its code directly into the main repo. We also may be able to transfer the repo into our organization and then work on getting it into hackage ourselves. Again, I'll talk to the team.

This is not my area of expertise, for what it's worth. I'm not super familiar with the haskell ecosystem. I will bring this to the attention of the engineers who are, however, and may be able to get an answer for you.

[jberthold]

This appears to be all that shows up in the log:

[exec] <stderr>: commitAndReleaseBuffer: invalid argument (invalid character)

This particular error is caused by 1) the haskell source code containing unicode, like φ ∈ Σ in comments 2) the machine that runs the built using a locale that does not support unicode.

[jberthold]

The discussion on this ticket seems to be two-fold, about the build failure (explained above) and about non-standard package dependencies causing grief when trying to package for a linux distro (for which we have work planned under #3226 ).

[ana-pantilie]

Blocked on #3226