Error 1 on "make dist" on linux VM

mmartida commented 3 years ago

Hi, we are trying to build and compile the Pipeline2 project.

Following the advice of running it on a Unix environment we installed a Linux VM in order to run it, but we encountered some issues on it.

The build crashes after "make dist" command executed. Any help would be appreciated.

Environment

Operating system: VM Kali Linux (Debian) }
DAISY Pipeline 2 version: 1.10.3
Interface: super-project
Java: 11.0.10

Logs

bash logs: $ make dist Failed to compute Maven dependencies. See maven.log for more info. make: *** [.make/main.mk:134: .make/target/effective-pom.xml] Error 1
maven.log: [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must be unique: org.daisy.pipeline.modules.braille:html-to-pef:jar -> duplicate declaration of version (?) @ line 43, column 17 [WARNING] 'build.plugins.plugin[org.apache.maven.plugins:maven-antrun-plugin].dependencies.dependency.systemPath' for com.sun:tools:jar refers to a non-existing file /usr/lib/jvm/java-11-openjdk-amd64/../lib/tools.jar. Please verify that you run Maven using a JDK and not just a JRE. @ org.daisy:daisy:4, /home/kali/Downloads/pipeline-master/.make/target/poms/org/daisy/daisy/4/daisy-4.pom, line 115, column 27

Then we downgraded Java version to 8 to avoid the "tools.jar" error but got another unavoidable error.

We dont know how could there be an error on a generated pom.

Java: 1.8.0_281

Logs:

bash logs: $ make dist Failed to compute Maven dependencies. See maven.log for more info. make: *** [.make/main.mk:134: .make/target/effective-pom.xml] Error 1
maven.log: [ERROR] The project (/home/kali/Downloads/pipeline-master/website/target/maven/pom.xml) has 1 error [ERROR] Non-parseable POM /home/kali/Downloads/pipeline-master/website/target/maven/pom.xml: TEXT must be immediately followed by END_TAG and not START_TAG (position: START_TAG seen ...>org.daisy.pipe... @1:308) @ line 1, column 308 -> [Help 2]

bertfrees commented 3 years ago

Hi @mmartida. I have updated the master branch. Could you please try again?

mmartida commented 3 years ago

Yes, thanks, that solved the problem on maven. Maven now build up correctly.

But now another error jump on scene: (root💀kali)-[/home/kali/Downloads/pipeline-master] └─# make dist 2 ⨯ Error in build script. Contact maintainer. make: *** [.make/main.mk:275: .group-eval] Error 1

and this on the main.mk:

Please answer.

bertfrees commented 3 years ago

I've tried with a clean checkout. Can't reproduce your error. What is your version of make? What does the file .make/target/commands contain?

mmartida commented 3 years ago

The make command is at 4.3 version and about the contents of that file:

Contents of .make/target/commands


make[1]: Entering directory '/home/kali/Downloads/pipeline-master'
: xxx ./assembly-make.sh dmg
: xxx 'test -e' /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT.dmg
: xxx 'touch' /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT.dmg
: xxx cp /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT.dmg pipeline2-1.14.3-SNAPSHOT_mac.dmg
: xxx ./assembly-make.sh exe
: xxx 'test -e' /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT.exe
: xxx 'touch' /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT.exe
: xxx cp /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT.exe pipeline2-1.14.3-SNAPSHOT_windows.exe
: xxx ./assembly-make.sh zip-linux
: xxx 'test -e' /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT-linux.zip
: xxx 'touch' /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT-linux.zip
: xxx cp /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT-linux.zip pipeline2-1.14.3-SNAPSHOT_linux.zip
: xxx ./assembly-make.sh zip-minimal
: xxx 'test -e' /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT-minimal.zip
: xxx 'touch' /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT-minimal.zip
: xxx cp /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT-minimal.zip pipeline2-1.14.3-SNAPSHOT_minimal.zip
: xxx ./assembly-make.sh zip-win
: xxx 'test -e' /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT-win.zip
: xxx 'touch' /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT-win.zip
: xxx cp /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT-win.zip pipeline2-1.14.3-SNAPSHOT_windows.zip
: xxx ./assembly-make.sh zip-mac
: xxx 'test -e' /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT-mac.zip
: xxx 'touch' /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT-mac.zip
: xxx cp /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT-mac.zip pipeline2-1.14.3-SNAPSHOT_mac.zip
: xxx ./assembly-make.sh deb
: xxx 'test -e' /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT.deb
: xxx 'touch' /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT.deb
: xxx cp /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT.deb pipeline2-1.14.3-SNAPSHOT_debian.deb
: xxx ./assembly-make.sh rpm
: xxx 'test -e' /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT-linux.rpm
: xxx 'touch' /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT-linux.rpm
: xxx cp /home/kali/Downloads/pipeline-master/.maven-workspace/org/daisy/pipeline/assembly/1.14.3-SNAPSHOT/assembly-1.14.3-SNAPSHOT-linux.rpm pipeline2-1.14.3-SNAPSHOT_redhat.rpm
# see webui README for instructions on how to make a signed package for distribution
cd webui && \
./activator -Dmvn.settings.localRepository="file:/home/kali/Downloads/pipeline-master/.maven-workspace" clean debian:packageBin | cat>>/home/kali/Downloads/pipeline-master/maven.log
mv webui/target/*deb .
# see webui README for instructions on how to make a signed package for distribution
cd webui && \
./activator -Dmvn.settings.localRepository="file:/home/kali/Downloads/pipeline-master/.maven-workspace" clean rpm:packageBin
mv webui/target/rpm/RPMS/noarch/*.rpm .
make[1]: Leaving directory '/home/kali/Downloads/pipeline-master'

This is all. Any other thing you'll need?

bertfrees commented 3 years ago

I assume this is GNU make?

Hmm, I just updated to 4.3 and it broke my script :-(. With 4.2.1 things still worked fine.

bertfrees commented 3 years ago

I managed to find a solution. Can you try the latest? Thanks.

ghost commented 3 years ago

still not working for me .. yes GNU make 4.3 (Fedora). I don't see a file .make/target/commands 8-(

mmartida commented 3 years ago

Trying on another way i installed a CentOS VM image and configured it to try build up the project. The make dist command crash on: ts logged to /usr/local/share/gems/gems/racc-1.5.2/ext/racc/cparse/gem_make.out 1 gem installed [root@centos7 pipeline-master]# make dist .make/main.mk:50: .make/target/maven.mk: No existe el fichero o el directorio bash: bundle: no se encontró la orden make[1]: [gems] Error 127 make: [website/target/maven/pom.xml] Error 2

and when trying to install nokogiri and commaparty only managed to install nokogiri. The package commaparty refuse to install no matter what you do or put on command line. Any idea?

bertfrees commented 3 years ago

You don't have to install any Ruby packages manually. Bundler will take care of it. But you need to have Bundler installed: https://bundler.io/.

But before you continue, let me first do a change to the build script. Ruby is only needed to build the website. You probably don't want to build the website. So I'm going to make the Ruby dependency optional.

bertfrees commented 3 years ago

Please try again.

VK2BEA commented 3 years ago

no change

[industrynumbers Downloads] $ cd pipeline/ [industrynumbers pipeline] $ make dist-zip-linux /bin/bash: -c: line 4: unexpected EOF while looking for matching `'' /bin/bash: -c: line 5: syntax error: unexpected end of file make: *** [.make/main.mk:275: .group-eval] Error 1 [industrynumbers pipeline] $ ls .make/target/ effective-pom.xml effective-settings.xml gradle-pom.xml maven-aggregators maven.mk maven-modules mk poms properties [industrynumbers pipeline] $ make -v GNU Make 4.3 Built for x86_64-redhat-linux-gnu Copyright (C) 1988-2020 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. [industrynumbers pipeline] $

bertfrees commented 3 years ago

@VK2BEA Your issue is not related to this one.

mmartida commented 3 years ago

Hi @bertfrees I tried the suggestion of VK2BEA and put "make dist-exe" to try generate an exe for windows, my main OS. But now i received new error from the build part:

feb 25, 2021 5:14:59 PM org.apache.http.impl.client.DefaultRequestDirector tryExecute INFORMACIÓN: I/O exception (org.apache.http.NoHttpResponseException) caught when processing request: The target server failed to respond feb 25, 2021 5:14:59 PM org.apache.http.impl.client.DefaultRequestDirector tryExecute INFORMACIÓN: Retrying request feb 25, 2021 5:15:12 PM org.apache.http.impl.client.DefaultRequestDirector tryExecute INFORMACIÓN: I/O exception (org.apache.http.NoHttpResponseException) caught when processing request: The target server failed to respond feb 25, 2021 5:15:12 PM org.apache.http.impl.client.DefaultRequestDirector tryExecute INFORMACIÓN: Retrying request make[1]: *** [mvn] Error 1

command exited with value 2 make: *** [.group-eval] Error 2 [root@centos7 pipeline-master]#

Idk whats going on now, Any idea? I supposed that with both gem, ruby27 and bundler installed we would be able to compile and build the system without any problem. Any help will be appreciated, thanks.

bertfrees commented 3 years ago

There are no Ruby dependencies anymore.

This error doesn't ring a bell. Where are the messages coming from? Can you give some more context please?

Note that there is sometimes more information about your error in the file maven.log.

mmartida commented 3 years ago

Well... We tried with "mvn deploy" thinking that maybe would necessary compile maven and this came out at the bottom of the execution:

[INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 50.287s [INFO] Finished at: Thu Feb 25 17:46:10 CET 2021 [INFO] Final Memory: 38M/91M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.8.2:deploy (default-deploy) on project daisy: Failed to deploy artifacts: Could not transfer artifact org.daisy:daisy:pom:6-20210225.164610-2 from/to sonatype-nexus-snapshots (https://oss.sonatype.org/content/repositories/snapshots/): Failed to transfer file: https://oss.sonatype.org/content/repositories/snapshots/org/daisy/daisy/6-SNAPSHOT/daisy-6-20210225.164610-2.pom. Return code is: 401, ReasonPhrase: Unauthorized. -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

And from the maven.log: maven-log.txt

That is all. Thanks

bertfrees commented 3 years ago

~~Where were the "org.apache.http.impl.client.DefaultRequestDirector" messages coming from? Can you send me the complete log?~~ (never mind)

bertfrees commented 3 years ago

OK I can see in the maven.log that you sent me that you are missing the program "makensis". This is indeed a requirement if you want to build the EXE. I guess we did not mention that in the documentation. I'm not sure how you install it on Linux. (I'm on MacOS myself.) I'm sure you can find it on the internet. Let me know if you need help.

mmartida commented 3 years ago

When we executed the command "mvn deploy" there was an error about a snapshot recently updated that gives me error of access: 401 Unauthorized. Is this snapshot from your team?

Any help, as always, will be apreciated thanks

bertfrees commented 3 years ago

Forget about running mvn deploy in the super project. That doesn't work.

bertfrees commented 3 years ago

@mmartida I have tried to install NSIS on CentOS 7 and Debian Jessie.

I had no luck with CentOS 7: the only package that I could find was a third party one with an old version of NSIS, 2.46, which did not work. For reference, this was the command for installing it:

rpm -Uvh http://www6.atomicorp.com/channels/atomic/centos/7/x86_64/RPMS/mingw32-nsis-2.46-12.el7.art.x86_64.rpm

With Debian I had more luck. NSIS is an official package in Debian. I tried to install it on Debian Jessie (which is old but happens to be the version of Debian that I have). I downloaded nsis-common_3.04-1_all.deb and nsis_3.04-1_amd64.deb from http://ftp.br.debian.org/debian/pool/main/n/nsis/ and installed them as follows:

dpkg -i nsis-common_3.04-1_all.deb
dpkg -i nsis_3.04-1_amd64.deb

This worked. It seems that Debian Buster has version 3.04 of NSIS, so in that case you can simply do:

apt-get install nsis

Debian Bullseye has a newer version, 3.06. I haven't tried yet if that one works too. It did not work on MacOS so don't' have good hopes.

bertfrees commented 3 years ago

Thanks to Marcos, here are instructions for building NSIS on CentOS: http://blog.shahada.abubakar.net/post/build-nsis-for-centos.

daisy / pipeline