Giant binary file generated upon installation?

ekernf01 commented 6 years ago

Hi Audiveris, I'm not sure if the issues page is the best place to raise this concern, but how much space do I need in order to build Audiveris? I keep getting a 10+GB file in build/test-results/test/binary.

Thanks! Really excited to get this up and running!

maximumspatium commented 6 years ago

Hi Eric,

I keep getting a 10+GB file in build/test-results/test/binary.

Hmm, such a huge file is definitely not okay, so you're in the right place to ask for troubleshooting.

Just for comparison, the content of the same directory after grade clean build is 340 KB big on my Mac OS X 11.6.

It looks like an issue with Gradle. What OS are you running on?

Would you kindly execute gradle -v in your terminal and post the result here?

Cheers Max

ekernf01 commented 6 years ago

Here it is:

------------------------------------------------------------
Gradle 4.5.1
------------------------------------------------------------

Build time:   2018-02-05 13:22:49 UTC
Revision:     37007e1c012001ff09973e0bd095139239ecd3b3

Groovy:       2.4.12
Ant:          Apache Ant(TM) version 1.9.9 compiled on February 2 2017
JVM:          1.7.0_151 (Oracle Corporation 24.151-b01)
OS:           Linux 4.4.0-43-Microsoft amd64

As for the operating system, I am not entirely sure. I am using Windows 10, but I call gradle clean build from within the Windows subsystem for Linux. When I do cat /etc/issue, I get Ubuntu 14.04.4 LTS. I think this is as if Gradle is running on Linux.

maximumspatium commented 6 years ago

I think this is as if Gradle is running on Linux.

Why don't you simple try Gradle under Windows?

ekernf01 commented 6 years ago

That worked. Serves me right for trying to fuck around with bash on ubuntu on windows. Thanks!

mustefan commented 6 years ago

I have the same issue on Fedora 28 x64. At 81th test started to slow down and I left it overnight, in the morning the output.bin file had 83Gb and I had no space left on device. Since nobody is pointing a priori to this, I suspect that trying with a bigger disk will not help. Yet, might be that not everyone is falling into the issue, otherwise it would have been documented. Any ideas?

maximumspatium commented 6 years ago

First of all, let's reopen the issue.

Due to the fact that I cannot reproduce the problem, any help from people experiencing that is greatly appreciated. In other words, you'd need to do some investigation to get the issue solved/fixed.

Please double-check first that you're working with the latest version of the development branch. If it isn't the case, re-clone Audiveris and change to the development branch with git checkout development terminal command.

Then run these terminal commands:

java -version
javac -version
gradle --version (or ./gradlew --version if you're using the Gradle Wrapper)

The tools above should be as recent as possible (Java >= 1.8.0_162 and Gradle >= 4.6). If it isn't the case, you'd need to update the tools first.

If all of the above mentioned requirements are satisfied and all checks passed, try to run audiveris with

gradle clean run

Did it work?

mustefan commented 6 years ago

Please disregard below unless you are really curious. After I installed tesseract (!!!) everything went swell w/o any issues.

I knew I had it installed but I forgot that I removed it; really sorry for the burden.

maxim, I cloned again, switched to dev branch, updated my gradle and ... I stumbled into the same thing. See the details below:

[mustefan@lacktop binary]$ pwd /home/mustefan/soft/audiveris/build/test-results/test/binary [mustefan@lacktop binary]$ du output.bin 21880580 output.bin

[mustefan@lacktop binary]$ /opt/gradle-4.8.1/bin/gradle --version

'------------------------------------------------------------ Gradle 4.8.1 '------------------------------------------------------------

Build time: 2018-06-21 07:53:06 UTC Revision: 0abdea078047b12df42e7750ccba34d69b516a22

Groovy: 2.4.12 Ant: Apache Ant(TM) version 1.9.11 compiled on March 23 2018 JVM: 1.8.0_172 (Oracle Corporation 25.172-b11) OS: Linux 4.17.2-200.fc28.x86_64 amd64

[mustefan@lacktop binary]$ javac -version javac 1.8.0_172

[mustefan@lacktop binary]$ java -version openjdk version "1.8.0_172" OpenJDK Runtime Environment (build 1.8.0_172-b11) OpenJDK 64-Bit Server VM (build 25.172-b11, mixed mode)

-=-=-=-=-=-=-=-=-= meanwhile on a separate console =-=-=-=-=-=-=-=-=-=-=-

[mustefan@lacktop audiveris]$ /opt/gradle-4.8.1/bin/gradle build

Welcome to Gradle 4.8.1!

Here are the highlights of this release:

Dependency locking
Maven Publish and Ivy Publish plugins improved and marked stable
Incremental annotation processing enhancements
APIs to configure tasks at creation time

For more details see https://docs.gradle.org/4.8.1/release-notes.html

Starting a Gradle Daemon (subsequent builds will be faster)

Configure project : targetOS=linux-x86_64 Download https://jcenter.bintray.com/org/jgrapht/jgrapht-core/1.0.1/jgrapht-core-1.0.1.pom Download https://jcenter.bintray.com/org/jgrapht/jgrapht/1.0.1/jgrapht-1.0.1.pom Download https://jcenter.bintray.com/org/projectlombok/lombok/1.16.20/lombok-1.16.20.pom Download https://jcenter.bintray.com/org/jgrapht/jgrapht-core/1.0.1/jgrapht-core-1.0.1.jar Download https://jcenter.bintray.com/org/projectlombok/lombok/1.16.20/lombok-1.16.20.jar

Task :compileJava warning: [options] bootstrap class path not set in conjunction with -source 1.7 Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. Note: Some input files use unchecked or unsafe operations. Note: Recompile with -Xlint:unchecked for details. Download https://jcenter.bintray.com/org/jgrapht/jgrapht-ext/1.0.1/jgrapht-ext-1.0.1.pom Download https://jcenter.bintray.com/org/jgrapht/jgrapht-ext/1.0.1/jgrapht-ext-1.0.1.jar

Task :compileTestJava warning: [options] bootstrap class path not set in conjunction with -source 1.7

Task :test

org.audiveris.omr.image.ChamferMatchingTest > testMatch FAILED java.lang.ExceptionInInitializerError at ChamferMatchingTest.java:104 Caused by: org.audiveris.omr.WellKnowns$InstallationException at ChamferMatchingTest.java:104 <==========---> 81% EXECUTING [19m 52s]

:test > 24 tests completed, 1 failed :test > Executing test org.audiveris.omr.util.BrokenLineTest

at this point I stopped, since I didn't want to clutter my device space again if you have any suggestion how I can help, what data I can collect, please let me know.

Thanks

maximumspatium commented 6 years ago

if you have any suggestion how I can help, what data I can collect, please let me know.

First thing is to switch to the Gradle Wrapper - that is, just invoke the wrapper script supplied in the development branch instead of system-wide gradlecommand:

./gradlew clean run

Does audiveris build and start this way?

maximumspatium commented 6 years ago

@mustefan Has the Gradle Wrapper solved your issue?

maximumspatium commented 6 years ago

@mustefan I assume you was able to fix the problem. I'll therefore close the issue now. Feel free to reopen it if the problem still persists.

eilivhaugland commented 6 years ago

Hi Audiveris I am in the same situation as mustefan with nearly identical error messages. I have switched to the development branch, git checkout development and recloned. I have tried both the gradle clean build and the same with the wrapper gradlew clean build all resulting in the same 'Executing test org.audiveris.omr.util.BrokenLineTest'

Can I test or provide better information i order for you to help me out here?

gradle build

Configure project : targetOS=linux-x86_64

Task :compileJava warning: [options] bootstrap class path not set in conjunction with -source 1.7 Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. Note: Some input files use unchecked or unsafe operations. Note: Recompile with -Xlint:unchecked for details.

Task :compileTestJava warning: [options] bootstrap class path not set in conjunction with -source 1.7 <==========---> 81% EXECUTING [21s] :test > 2 tests completed :test > Executing test org.audiveris.omr.util.BrokenLineTest

I have the following versions installed:

gradle -v

Gradle 4.9

Build time: 2018-07-16 08:14:03 UTC Revision: efcf8c1cf533b03c70f394f270f46a174c738efc

Kotlin DSL: 0.18.4 Kotlin: 1.2.41 Groovy: 2.4.12 Ant: Apache Ant(TM) version 1.9.11 compiled on March 23 2018 JVM: 1.8.0_181 (Oracle Corporation 25.181-b13) OS: Linux 4.15.0-30-generic amd64

javac -version javac 1.8.0_181

java -version java version "1.8.0_181" Java(TM) SE Runtime Environment (build 1.8.0_181-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

tesseract -v tesseract 4.0.0-beta.1 leptonica-1.75.3 libgif 5.1.4 : libjpeg 8d (libjpeg-turbo 1.5.2) : libpng 1.6.34 : libtiff 4.0.9 : zlib 1.2.11 : libwebp 0.6.1 : libopenjp2 2.3.0

maximumspatium commented 6 years ago

Hello Eiliv,

try to delete Gradle cache with rm -f ~/.gradle and then to rebuild Audiveris with

./gradlew clean run

Please note that ALL dependencies must be downloaded anew, so you'll need a working internet connection.

Does it work?

eilivhaugland commented 6 years ago

Hi Audiveris I am in the same situation as mustefan with nearly identical error messages. I have switched to the development branch, git checkout development and recloned. I have tried both the gradle clean build and the same with the wrapper gradlew clean build all resulting in the same 'Executing test org.audiveris.omr.util.BrokenLineTest'

Can I test or provide better information i order for you to help me out here?

gradle build

Configure project : targetOS=linux-x86_64

Task :compileJava warning: [options] bootstrap class path not set in conjunction with -source 1.7 Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. Note: Some input files use unchecked or unsafe operations. Note: Recompile with -Xlint:unchecked for details.

Task :compileTestJava warning: [options] bootstrap class path not set in conjunction with -source 1.7 <==========---> 81% EXECUTING [21s] :test > 2 tests completed :test > Executing test org.audiveris.omr.util.BrokenLineTest

I have the following versions installed:

gradle -v

Gradle 4.9

Build time: 2018-07-16 08:14:03 UTC Revision: efcf8c1cf533b03c70f394f270f46a174c738efc

Kotlin DSL: 0.18.4 Kotlin: 1.2.41 Groovy: 2.4.12 Ant: Apache Ant(TM) version 1.9.11 compiled on March 23 2018 JVM: 1.8.0_181 (Oracle Corporation 25.181-b13) OS: Linux 4.15.0-30-generic amd64

javac -version javac 1.8.0_181

java -version java version "1.8.0_181" Java(TM) SE Runtime Environment (build 1.8.0_181-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

tesseract -v tesseract 4.0.0-beta.1 leptonica-1.75.3 libgif 5.1.4 : libjpeg 8d (libjpeg-turbo 1.5.2) : libpng 1.6.34 : libtiff 4.0.9 : zlib 1.2.11 : libwebp 0.6.1 : libopenjp2 2.3.0

eilivhaugland commented 6 years ago

Hi Unfortunately this did not make any change. Removing the rm -rf ~/.gradle and trying to rebuild everything again ./gradlew clean run results in the same error Please see below for a complete log (this is rather long - sorry)

root@eiliv-System-Product-Name:/home/eiliv/audiveris# ./gradlew clean run Starting a Gradle Daemon (subsequent builds will be faster)

Configure project : targetOS=linux-x86_64

Task :compileJava warning: [options] bootstrap class path not set in conjunction with -source 1.7 Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. Note: Some input files use unchecked or unsafe operations. Note: Recompile with -Xlint:unchecked for details.

Download https://jcenter.bintray.com/org/bytedeco/javacpp-presets/tesseract/3.04.01-1.3/tesseract-3.04.01-1.3-linux-x86_64.jar Download https://jcenter.bintray.com/org/bytedeco/javacpp-presets/leptonica/1.73-1.3/leptonica-1.73-1.3-linux-x86_64.jar

Task :run FAILED Exception in thread "main" java.lang.ExceptionInInitializerError at org.audiveris.omr.Main.(Main.java:64) at Audiveris.main(Audiveris.java:49) Caused by: org.audiveris.omr.WellKnowns$InstallationException: Tesseract data could not be found. Try setting the TESSDATA_PREFIX environment variable to the parent folder of "tessdata". at org.audiveris.omr.WellKnowns.scanOcrLocations(WellKnowns.java:536) at org.audiveris.omr.WellKnowns.getOcrFolder(WellKnowns.java:488) at org.audiveris.omr.WellKnowns.(WellKnowns.java:139) ... 2 more

FAILURE: Build failed with an exception.

What went wrong: Execution failed for task ':run'.

Process 'command '/opt/java/jdk1.8.0_181/bin/java'' finished with non-zero exit value 1
Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
Get more help at https://help.gradle.org

BUILD FAILED in 48s 6 actionable tasks: 5 executed, 1 up-to-date root@eiliv-System-Product-Name:/home/eiliv/audiveris# rm -rf /root/.gradle root@eiliv-System-Product-Name:/home/eiliv/audiveris# touch testlog root@eiliv-System-Product-Name:/home/eiliv/audiveris# cd /home/eiliv/audiveris/ root@eiliv-System-Product-Name:/home/eiliv/audiveris# ./gradlew clean run > testlog warning: [options] bootstrap class path not set in conjunction with -source 1.7 Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. Note: Some input files use unchecked or unsafe operations. Note: Recompile with -Xlint:unchecked for details. Exception in thread "main" java.lang.ExceptionInInitializerError at org.audiveris.omr.Main.(Main.java:64) at Audiveris.main(Audiveris.java:49) Caused by: org.audiveris.omr.WellKnowns$InstallationException: Tesseract data could not be found. Try setting the TESSDATA_PREFIX environment variable to the parent folder of "tessdata". at org.audiveris.omr.WellKnowns.scanOcrLocations(WellKnowns.java:536) at org.audiveris.omr.WellKnowns.getOcrFolder(WellKnowns.java:488) at org.audiveris.omr.WellKnowns.(WellKnowns.java:139) ... 2 more

FAILURE: Build failed with an exception.

What went wrong: Execution failed for task ':run'.

Process 'command '/opt/java/jdk1.8.0_181/bin/java'' finished with non-zero exit value 1
Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
Get more help at https://help.gradle.org

BUILD FAILED in 38s root@eiliv-System-Product-Name:/home/eiliv/audiveris# nano testlog root@eiliv-System-Product-Name:/home/eiliv/audiveris# root@eiliv-System-Product-Name:/home/eiliv/audiveris# ./gradlew clean run

Configure project : targetOS=linux-x86_64

Task :compileJava warning: [options] bootstrap class path not set in conjunction with -source 1.7 Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. Note: Some input files use unchecked or unsafe operations. Note: Recompile with -Xlint:unchecked for details.

Task :run FAILED Exception in thread "main" java.lang.ExceptionInInitializerError at org.audiveris.omr.Main.(Main.java:64) at Audiveris.main(Audiveris.java:49) Caused by: org.audiveris.omr.WellKnowns$InstallationException: Tesseract data could not be found. Try setting the TESSDATA_PREFIX environment variable to the parent folder of "tessdata". at org.audiveris.omr.WellKnowns.scanOcrLocations(WellKnowns.java:536) at org.audiveris.omr.WellKnowns.getOcrFolder(WellKnowns.java:488) at org.audiveris.omr.WellKnowns.(WellKnowns.java:139) ... 2 more

FAILURE: Build failed with an exception.

What went wrong: Execution failed for task ':run'.

Process 'command '/opt/java/jdk1.8.0_181/bin/java'' finished with non-zero exit value 1
Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
Get more help at https://help.gradle.org

BUILD FAILED in 14s 6 actionable tasks: 6 executed root@eiliv-System-Product-Name:/home/eiliv/audiveris# ^C root@eiliv-System-Product-Name:/home/eiliv/audiveris#

maximumspatium commented 6 years ago

Eiliv,

your log tells you the exact failure reason:

Tesseract data could not be found. Try setting the TESSDATA_PREFIX environment variable to the parent folder of "tessdata".

I couldn't recognize that earlier because you didn't provide the full log.

Please go ahead and install Tesseract data as described here. For the start, it could be enough to install the english language via

dnf install tesseract-langpack-eng

and then to verify that TESSDATA_PREFIX is properly set.

Moreover, I noticed that you're using Tesseract 4.0 which is still beta. We didn't ever tried Audiveris with Tesseract 4 because there are no Java bindings currently available. I suspect it won't work so you'll need to downgrade to Tesseract 3.x. Porting to Tesseract 4 which uses a completely different recognition technology as well as general improvement of the text recognition in Audiveris is our next big goal (and a great challenge)...

eilivhaugland commented 6 years ago

Hi Downgrading tesseract to 3.5.1 and setting the correct Prefix did the trick. I now have a running Audiveris Be advised that there are no APT source ready for downgrading tesseract. You will have to download and unpack to the selected spot manually. I did not find any language packs with the same version as the tesseract but the language ver. 4.x seem to work correct Thanks for you help

Audiveris / audiveris

Giant binary file generated upon installation? #102

Gradle 4.9

Gradle 4.9