Switch to OpenJDK and Java 11 / 64bit

[berndmoos]

This will be necessary because (of):

• Oracle's licence policy
• MAC OS no longer supports 32bit applications (starting with Catalina=MAC OS 10.15)
• There are no 32bit JREs/JDKs for Java 11

There are lots of issues involved:

• Netbeans 8.x won't work with Java 9+, need to switch to Apache Netbeans 11 (partly done)
• JSmooth does not support 64bit, need to switch to Launch4j
• In Java 11, support is missing for com.apple.eawt
• In Java 11, support is missing for com.sun.awt.AWTUtilities
• JavaFX isn't part of Java 11, libraries need to be integrated separately
• JRE/JDK should be bundled with application on MAC, appbundler can do this
• ... (tbc)

And then there is the question of players:

• JMF player will probably have to go (no 64bit support in native libs)
• JAVAFX player should be made available on Win
  • partly done, works pretty good for MP4 videos
• BASPlayer does not work with Java 11 / 64bit (reason as yet unknown)
  • but works fine on Java 8 / 64 bit (Oracles JDK on Windows)
  • and not with Java 13 / 64bit (OpenJDK on Windows)
  • plays okay with Java 11 (on Windows) when old ipsk.jar is replaced with (1) ips.audiotools.jar and (2) ips.commons.jar from BAS Speech Recorder Windows Installation. But: audio cursor disappears some half second before the end of the selection (we've had that before...)
  • same on Mac with Java 11
• JDSPlayer needs to switch to 64bit
  • works okay when 64bit version of DSPlayer.dll is in library path
• AVFoundation player must be integrated on MAC
  • no success yet -- need to find out where to put native libraries

[Herrner]

What do the BAS people do about their player?

[berndmoos]

There are now experimental MAC OS versions of FOLKER and Partitur-Editor on the previews page with:

• OpenJDK 11 embedded
• JAVA FX player somehow working

[wuddel]

When I use the JAVA FX player on Windows 7, the "play selection" on a regular audio-wav file does not stop at the right crosshair. In the debug info it says:

INFORMATION: JFXMediaPlayer URL: file:/[....wav] (1b - alpha) - Media file is now set for player of PartiturTableWithActions Player says that media file has length 0.0 (1c) PartiturTableWithActions: handing the media file to the audio/video panel (3a) AudioPanel: openSoundFile Dez 03, 2019 1:22:34 PM mpi.eudico.client.annotator.player.JFXMediaPlayer printMediaInfo INFORMATION: Media Information: Track: 0 audio/x-raw-int name: audio/x-raw-int encoding: PCM enabled: false

(3b) AudioPanel: taking care of the visual component JavaFXPlayer says the movie BETV_E_00001_SE_01_A_01_DF_01.wav has width 0and height 0 Preferred size set to: 0/0

When I add a mp4 video-file, the "play selection" action performs as expected.

[berndmoos]

No luck with MAC version of FOLKER on somebody's machine:

Ich konnte die jetzt zwei Male heruntergeladene App zuerst nicht geöffnet kriegen. Mein Mac sagte mir bloß, dass das Programm nicht von einem verifizierten Entwickler stammt. Dann habe ich mehrfach versucht, nach den Anweisungen auf der Hilfe-Seite die Sperre manuell zu umgehen und genau nach Anweisung dennoch das Programm zu öffnen, woraufhin auch nur immer wieder die Meldung kam: Die App kann nicht geöffnet werden, da das Programm nicht reagiert.

Dann habe ich es doch irgendwie geöffnet gekriegt. In der Statuszeile ganz oben auf dem Bildschirm erschien FOLKER 1.2.2 und ich freute mich. Aber ich konnte und kann keinen zugehörigen Startbildschirm öffnen. Es geht nur das eine Untermenü Folker 1.2.2 und auch ein Klicken auf „Alle einblenden“ bringt gar nichts. Es bleibt alles leer.

[berndmoos]

Minimal rescue plan (?):

• make FOLKER and Partitur-Editor work with JavaFX player and embedded JDK on MAC: need somebody to test this on MAC OS Catalina
• build new distribution for Windows with launch4j

[wuddel]

Test of Partitur-Editor on Catalina:

• The Audio/Video panel [JavaFX] says: "Audio/Video file ...mp4 opened successfully", however the video image does not appear on the player window.
• The waveform (extracted form the video?) appears in the appropriate place, however it does not play-back on the loud speakers. The crosshair runs from left to right though.
• Strange behaviour of "play selection": Crosshair starts at beginning of recording (00:00) and stops at the endpoint of the selection (as said before: no audio is coming out of the speakers).
• When wav-file used on top of the media-list: strange behaviour: every 2nd or every 3rd time, "play selection" stops playback at the end of the selection. All other times, the playback runs through the end-point. (However, each time it starts correctly at the start of the selection).

Debug info: Debug-Info_TestPECatalina.txt

[berndmoos]

Error message in log:

Exception in thread "AWT-EventQueue-0" java.lang.NoClassDefFoundError: javax/media/NoPlayerException at org.exmaralda.folker.timeview.AbstractTimeProportionalViewer.setSoundFile(AbstractTimeProportionalViewer.java:97) at org.exmaralda.partitureditor.partiture.PartiturEditor.setupMedia(PartiturEditor.java:726) at org.exmaralda.partitureditor.partiture.PartitureTableWithActions.setupMedia(PartitureTableWithActions.java:2047) at org.exmaralda.partitureditor.partiture.transcriptionActions.EditRecordingsAction.editRecordings(EditRecordingsAction.java:41)

... is saying that it misses JMF after all, need to get rid of it in this place.

[berndmoos]

After all, it is starting on Catalina and with JRE 11 (but why is Oracle the vendor?)

Operating system : Mac OS X Operating system architecture : x86_64 OS version : 10.15.1 JRE version : 11.0.5 JRE vendor : Oracle Corporation JVM version (specification/implementation) : 11 / 11.0.5+10-LTS

@wuddel : Kannst Du auch mal auf den BAS-Player umschalten, neu starten und dann noch mal schauen?

[wuddel]

Wait... my Mac just says: "An update is available for your Mac"

• Pro Video Formats 2.1.1.

Therein:

• Apple ProRes ...
• Apple Intermediate Codec
• AVC-Intra 50 / 100/ 200 4:4:4 /LT
• AVC-LongG
• Canon XF-HEVC
• XAVC
• XF-AVC
• DVCPRO HD
• HDV
• XDCAM EX / HD / HD422
• MPEG IMX
• Uncompressed 4:2:2

The Pro Video Formats package also includes the following MXF support:

• Plax MFX files in Quicktime Player and other supported macOS applications
• MXF plug-in and presents for use in Compressor 4.3 and earlier

Learn more about working with MXF; https;//support.apple.com/en-us/HT204322

First I do the update and test again. Later I also swithch to BAS and test

[wuddel]

• After the update, the video plays well in the FX-Player ... and the audio, too!
• While the audio and video plays now, the strange behaviour of the start- and end-points still remains.
• I realise that the audio did not come out of the speakers before, because the video that I chose at random did not have an audio track on it. Sorry!
• BAS player still plays the wav-file.
• Crosshair of the BAS-Player stops moving ca. 500ms before end of selection; Audio plays correctly up to the end of the selection.
• After switching from BAS to FX or from FX to BAS and re-starting the program to take effect of the change, the list of referenced media-files is empty. Despite explicit "save as", before the switch, the list is empty after the restart. I think the reason is that when I double click on the "Testmp4fxplayer.exb" (juat saved) or choose "open with -> PE", the file that opens is marked as "untitled.exb", i.e. a new file.

[wuddel]

After downloading and installing the FOLKER Disk Image for test on Catalina, the Program seems to have difficulties at the start. An Exclamation-Mark-Error says: "/-psn_0_499834 (No such file of directory)". I can then click on OK and the program seems to run fine.

[berndmoos]

The uploaded versions of Partitur-Editor and FOLKER can run on MAC OS Catalina, video playback of MPEG4 is possible with JavaFX Player, audio playback with BAS Audio Player. Some glitches remain such as a weird error message on startup (the app seems to think it was handed a file to open as a parameter). Can't do much more than this now.

[berndmoos]

Main things to do (somebody, eventually):

• Make a 64 bit / Java 11 distribution for Windows (using launch4j -- current problem is missing or unsuitable glass.dll for JavaFX)
• Make the AVFoundation player work for the MAC (current problem is UnsatisfiedLinkError for some native library)

[berndmoos]

About the MAC: I am still on Yosemite (10.10), ELAN says that the AVFoundation player is for "High Sierra" (10.13). Maybe the UnsatisfiedLink Error will not occur when I upgrade. Will do, some time...

[Herrner]

I have no updateable Mac anymore, btw.

[berndmoos]

I am now on Catalina: the "Unsatisfied Link" error is still there :-(

[Herrner]

I'm getting a catalina-capable Mac soon, too.

[berndmoos]

Documentation from Han: Packaging is done with jpackage, so maybe no need to bother with Launch4J and AppBundler. Will try, if no one else does...

N.B.: jpackage is not included in Java 11, 12 or 13. It is part of an early release of Java 14, no guarantee it will stay there.

[berndmoos]

Some screenshots when downloading the current version of FOLKER for Catalina:

Won't start with double click.

Will start after right click and "Open..." from the context menu.

Error message after opening (does no harm).

... and then it's there. Audio plays okay with BAS player.

[berndmoos]

One more new insight: instead of using the ELAN players from mpi.eudico.client.annotator.player, it is probably a good idea to use the "independent" versions from nl.mpi,*.player. However, this means a lot of additional work, e.g. setting a controller. Oh Lord.

[berndmoos]

Then again: the Unsatisfied LinkError goes away when I make a new project in NetBeans. Maybe a lot of this has to do with buggy NetBeans.

Yes -- confirmed.

[berndmoos]

Another new insight: The new elan.jar does not have CocoaQTPlayer in it, so this has to go, too.

[berndmoos]

Revised rescue plan:

• Cleanup code, add new player(s), remove old player(s)
• Setup new app structure
• Build distributions with jpackage

[berndmoos]

This one does a part of the trick:

C:\Program Files\Java\jdk-14\bin>jpackage.exe --type app-image --input D:\EXMARaLDA_GIT\exmaralda\EXMARaLDA_NetBeans_Project\dist --dest D:\EXMARaLDA_NEW_BUILD_TEST --main-jar EXMARaLDA_NetBeans_Project.jar --icon D:\EXMARaLDA_GIT\exmaralda\EXMARaLDA_NetBeans_Project\templates\windows\icons\partitureditor.ico Creates an app-image with an PartiturEditor.exe, this will run and playback with BAS Audio Player Need yet to figure out:

• how to include native libraries
• how to set the icon
• how to generate multiple exes (for the other tools)

[berndmoos]

A batch job:

cd "C:\Program Files\Java\jdk-14\bin"

rmdir D:\EXMARaLDA_NEW_BUILD_TEST\PartiturEditor /s
rmdir D:\EXMARaLDA_NEW_BUILD_TEST\FOLKER /s

jpackage.exe --type app-image --input D:\EXMARaLDA_GIT\exmaralda\EXMARaLDA_NetBeans_Project\dist --dest D:\EXMARaLDA_NEW_BUILD_TEST --main-jar EXMARaLDA_NetBeans_Project.jar --app-version 1.6.2 --icon D:\EXMARaLDA_GIT\exmaralda\src\org\exmaralda\partitureditor\partiture\Icons\partitureditor.ico --java-options -Djava.library.path=$ROOTDIR\app\

jpackage.exe --type app-image --input D:\EXMARaLDA_GIT\exmaralda\EXMARaLDA_NetBeans_Project\dist --dest D:\EXMARaLDA_NEW_BUILD_TEST --main-jar EXMARaLDA_NetBeans_Project.jar --main-class org.exmaralda.folker.application.ApplicationFrame --app-version 1.2.2 --icon D:\EXMARaLDA_GIT\exmaralda\EXMARaLDA_NetBeans_Project\templates\windows\icons\folker.ico --java-options -Djava.library.path=$ROOTDIR\app\ --name FOLKER 

copy "D:\EXMARaLDA_LIBS_2020\native" "D:\EXMARaLDA_NEW_BUILD_TEST\PartiturEditor\app"
copy "D:\EXMARaLDA_LIBS_2020\native" "D:\EXMARaLDA_NEW_BUILD_TEST\FOLKER\app"

• icons are set correctly
• native libraries are used (i.e. JDSPlayer etc. work)

Seems like tools need to be distributed (or at least installed) separately

[berndmoos]

Not too bad:

• exes for all tools
• inside a single structure
• players run
• JRE embedded

[berndmoos]

Forgot TEI Drop...

[berndmoos]

Next on:

• transfer the Batch Job to an Ant Job
• adapt Inno Setup

[berndmoos]

https://www.exmaralda.org/files/prevDL/exmaralda_setup_13_64.exe

Oh, it's Java 14. Probably because I used that for packaging. Not sure if that's a good idea.

Any testers?

[berndmoos]

Next on: Make the AVF Player work on the MAC. Not now.

[Herrner]

Any testers?

Installing on Windows 10 worked (no previous EXMARaLDA-install), running PE from the setup worked. Playing AnneWill.avi (JDS), AnneWill.wav (JDS) works, webm doesn't. Desktop and start menu shortcuts don't find their targets (COMA does, tho).

[Herrner]

==> Wrong file assocications in innosetup.iss, I can fix this

[berndmoos]

Any testers?

Installing on Windows 10 worked (no previous EXMARaLDA-install), running PE from the setup worked. Playing AnneWill.avi (JDS), AnneWill.wav (JDS) works, webm doesn't. Desktop and start menu shortcuts don't find their targets (COMA does, tho).

==> if webm is mpeg4, it might work with JavaFX player

[berndmoos]

Uh! Ah!

[berndmoos]

There are now also disk images for all tools on MAC OS, but they get corrupted in upload somehow. The build, packaging, distribution process for Windows and MAC is automated again (and somehwhat simpler than before). Remaining known issues:

• Uploaded DMGs are corrupt
• All of this is based on non-official versions of OpenJDK 14. It would be better to use a stable JDK release.
• The embedded JDK is huge, size could be reduced via jlink
• the AVF-Player behaves badly in FOLKER when it plays for the first time (see #204)
• FOLKER and OrthoNormal still have "ApplicationFrame" as name in the Application Menu

A temporary download page for everything is here:

https://exmaralda.org/files/prevDL/experimental.html

Testers are badly needed. Please join this issue!

[berndmoos]

When I use the JAVA FX player on Windows 7, the "play selection" on a regular audio-wav file does not stop at the right crosshair.

Yes, same here: the JavaFX-Player is no good for WAV files. It seems okay for Mpeg4 video, and that's more important

[berndmoos]

About the corrpution: maybe due to signing:

https://stackoverflow.com/questions/22458027/mac-dmg-oddity-signing-and-damaged-applications https://apple.stackexchange.com/questions/175634/all-downloaded-files-cannot-be-opened-as-they-are-damaged-and-should-be-moved-t https://bugs.openjdk.java.net/browse/JDK-8230629 http://kothar.net/macos_catalina_java_11 http://www.zarkonnen.com/signing_notarizing_catalina

[berndmoos]

This is absurd. I have perfect disk images locally. When I upload them (via ant ftp) and download them again, MAC is saying "the application is damaged". It seems to be related to Gatekeeper and signing/notarization, but there is no way to find out what exactly the problem is.

This is intended to make the experience safer for end-users and prevent the spread of malicious programs, but it does also make things more complicated, especially for people who aren’t primarily Mac developers but would like to support or keep supporting Macs. [http://www.zarkonnen.com/signing_notarizing_catalina]

[berndmoos]

codesign -v {x.app}
code has no resources but signature indicates they must be present

codesign --remove-signature /Users/thomasschmidt/Desktop/PartiturEditor.app

And then it will run ?!?!?!?

[berndmoos]

Tada.

• use package with type=app-image
• run code sign with "--remove-signature" on the resulting .app
• pack the disk image
• upload it

Gatekeeper will then only ask if the "application downloaded from the internet" should really be opened. This is for the case that settings are as follows:

If it's not "überall", the app can still be started via the right-click menu and open.

[berndmoos]

Next on:

• let @Herrner test the MAC distributions from https://exmaralda.org/files/prevDL/experimental.html
• let the user in question check if they can now work with FOLKER in their project
• announce the experimental.html page officially
• (consider signing and notarizing the MAC applications)
• make real previews for all tools (reduce embedded Java via JLink first?)
• prepare an official release
• take a two-year holiday

[berndmoos]

Windows and MAC OS distributions are now on the "regular" preview page. Five users have tested on Catalina successfully.

[berndmoos]

Switched to offical JDK 14 release on Windows. No problems afaics

[berndmoos]

Let's just say this is done -- at least, there's no way of rolling back. and it seems to be largely okay on Windows and MAC OS. Time for that:

[berndmoos]

Next round:

• Another attempt for M1
• jpackage type 'dmg' instead of 'app-image'?
• sign MAC OS distributions?
• adapt Linux distribution (different JDK? #339)?
• other distribution forms for Linux?

@Herrner, @wuddel: Please join the party: https://exmaralda.org/files/prevDL/experimental.html

[berndmoos]

MAC OS apps built with type=dmg won't open:

"PartiturEditor.app" ist beschädigt und kann nicht geöffnet werden.

This is probably due to faulty signing.

[berndmoos]

I am messing up issues. This story is continued in two other issues:

• 205

• 256

[berndmoos]

This is done now. Soon: Java 22 / 128bit?