DamRsn / NeuralNote

Audio Plugin for Audio to MIDI transcription using deep learning.
Apache License 2.0
1.17k stars 61 forks source link

Linux support #33

Open dromer opened 1 year ago

dromer commented 1 year ago

Duh.

DamRsn commented 1 year ago

Contributions are welcome 🙂

The building process should be very similar to OSx.

gnac commented 1 year ago

I've made some steps to build this (and the /libonnxruntime-neuralnote dependency) in Linux. At the moment, I've got NeuralNote compiling but it fails in the linker stage. I'll push what I have to my NeuralNote and onnxruntime lib forks. But in short, I'm currently stuck on the following error:

[100%] Built target NeuralNote_Standalone
/usr/lib64/gcc/x86_64-suse-linux/12/../../../../x86_64-suse-linux/bin/ld: libBasicPitchCNN.a(BasicPitchCNN.cpp.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/usr/lib64/gcc/x86_64-suse-linux/12/../../../../x86_64-suse-linux/bin/ld: failed to set dynamic section sizes: bad value
collect2: error: ld returned 1 exit status
gnac commented 1 year ago

I've created a (pull request](https://github.com/DamRsn/NeuralNote/pull/44) that is dependent on a prebuilt onnxruntime library. It isn't building (linking errors) yet, but I figured its a place to start a discussion.
FWIW, I also have a pending pull request on the libonnx project which adds support for building the linux archive. You will probably need to pull this project to build an archive to test the linux build of this pull request.

RustoMCSpit commented 1 year ago

just tried running this on wine with yabridge on bitwig, it kinda works but the gui has a weird rendering issues which causes huge delay / latency

dromer commented 1 year ago

@RustoMCSpit but we have the source-code, so no need for wine.

Thnx for the work @gnac not sure if I'll be able to get your stuff tested, but looking forward to see it working!

RustoMCSpit commented 1 year ago

@RustoMCSpit but we have the source-code, so no need for wine.

Thnx for the work @gnac not sure if I'll be able to get your stuff tested, but looking forward to see it working!

no need for wine until it actually is supported you mean right?

dromer commented 1 year ago

@RustoMCSpit sure, if it works for you and you can play with the software that's great.

RustoMCSpit commented 1 year ago

@RustoMCSpit sure, if it works for you and you can play with the software that's great.

ehhh, 'works' but barely, very bugged

RustoMCSpit commented 12 months ago

yeah after a few days the app is pretty unbareable to work with through WINE, if the devs want to see it through they could collect crash reports they get and see whats going on

dromer commented 12 months ago

@RustoMCSpit if you want to go that route please open a separate ticket so we don't mix up topics.

I think it would make more sense to build the program for Linux then to try and debug wine issues.

thelabcat commented 11 months ago

Lemme know if this goes anywhere! RN it is only usable in a VM for me ;-(

RustoMCSpit commented 10 months ago

just to update, program still really weird on WINE after new update

dromer commented 10 months ago

@RustoMCSpit please open a separate topic on WINE, this request is about actual Linux builds and NOT any WINE work around.

Lets not dilute either topic by cramming it into the same ticket.

polygon commented 9 months ago

@gnac Did you get any further with this? I've also managed to build my own version of onnxruntime and actually got NeuralNote to compile. But right now it segfaults when starting. I've so far traced this back to this line of code (juce_XmlElement.cpp):

static const String juce_xmltextContentAttributeName ("text");

For some reason, this static variable is not initialized correctly and the pointer inside the string (to the actual text data) is a Null pointer so it crashes when it's being used (in this case, to parse the Linux fontconfig). Might just be a small issue with some JUCE settings required on Linux, I'm sifting through the things, but it's a pretty strange error.

polygon commented 9 months ago

To me, this looks like a case of "static initialization fiasco". In NeuralNote/Lib/Components/UIDefines.h there are pointers to fonts declared like so:

// Fonts
const juce::Typeface::Ptr MONTSERRAT_BOLD =
    juce::Typeface::createSystemTypefaceFor(BinaryData::MontserratBold_ttf, BinaryData::MontserratBold_ttfSize);

const juce::Typeface::Ptr MONTSERRAT_SEMIBOLD =
    juce::Typeface::createSystemTypefaceFor(BinaryData::MontserratSemiBold_ttf, BinaryData::MontserratSemiBold_ttfSize);

const juce::Typeface::Ptr MONTSERRAT_REGULAR =
    juce::Typeface::createSystemTypefaceFor(BinaryData::MontserratRegular_ttf, BinaryData::MontserratRegular_ttfSize);

Under Linux, this triggers (unless JUCE_FONT_PATH is set) a font lookup that involves parsing /etc/fonts/fonts.conf using the XML parser. The XML parser has the static string initialization mentioned above. In my compiled variant, this static string has not been initialized when the createSystemTypefaceFor calls are being made. I guess this works for Windows and Mac because loading fonts there does not involve the XML parser. Still, this could be considered unsound (at least on Linux). I will try to work around this.

Setting JUCE_FONT_PATH seems to solve the problem and makes the program start. Seems to work well so far. I'm not sure why JUCE insists on parsing the fontconfig since this plugin only loads fonts from its local resources. But it's probably a simple patch to fix this (or just set JUCE_FONT_PATH environment to literally anything).

So in summary, it's working in Linux for me, note detection etc. is working as expected. Unfortunately, I cannot seem to get MIDI out of the plugin. I can start the dragging, but neither dropping it into my DAW (Bitwig), nor into the file browser seems to be doing anything.

gnac commented 9 months ago

@polygon you've gotten further than I have. How did you resolve the linker issue?

polygon commented 9 months ago

I don't remember running into this linker issue to be honest. I've uploaded my current state here: https://github.com/polygon/NeuralNote/tree/linux

I re-factored the UIDefines.h code into a singleton provider to work around the initialization order issues. Setting JUCE_FONT_PATH would solve the issue for running the program standalone but not when used as a plugin. With the refactoring, I also got things working in Bitwig.

I'm not currently using build.sh but just run CMake directly like so:

cmake -B build -DCMAKE_BUILD_TYPE="Release"
cmake --build build -j

This expects to have a Linux built libonnxruntime-neuralnote in place as well, similarly unpacked as described in the build.sh. I've also got this done, but it's still quite hacky and I'm in the process of writing scripts for all of this. So you can poke around if you want, but it's still all a bit hacky and I'm working on consolidating everything.

CodesoundR commented 4 months ago

I don't remember running into this linker issue to be honest. I've uploaded my current state here: https://github.com/polygon/NeuralNote/tree/linux

I re-factored the UIDefines.h code into a singleton provider to work around the initialization order issues. Setting JUCE_FONT_PATH would solve the issue for running the program standalone but not when used as a plugin. With the refactoring, I also got things working in Bitwig.

I'm not currently using build.sh but just run CMake directly like so:

cmake -B build -DCMAKE_BUILD_TYPE="Release"
cmake --build build -j

This expects to have a Linux built libonnxruntime-neuralnote in place as well, similarly unpacked as described in the build.sh. I've also got this done, but it's still quite hacky and I'm in the process of writing scripts for all of this. So you can poke around if you want, but it's still all a bit hacky and I'm working on consolidating everything.

Hi, and thanks so much for porting this beautiful project in Linux (I'm under Mint). I ask you if you have any news on the script you proposed. I'm having a hard time figuring out how to install this softare... thanks again!!!

Ken-Andre commented 4 months ago

I don't remember running into this linker issue to be honest. I've uploaded my current state here: https://github.com/polygon/NeuralNote/tree/linux I re-factored the UIDefines.h code into a singleton provider to work around the initialization order issues. Setting JUCE_FONT_PATH would solve the issue for running the program standalone but not when used as a plugin. With the refactoring, I also got things working in Bitwig. I'm not currently using build.sh but just run CMake directly like so:

cmake -B build -DCMAKE_BUILD_TYPE="Release"
cmake --build build -j

This expects to have a Linux built libonnxruntime-neuralnote in place as well, similarly unpacked as described in the build.sh. I've also got this done, but it's still quite hacky and I'm in the process of writing scripts for all of this. So you can poke around if you want, but it's still all a bit hacky and I'm working on consolidating everything.

Hi, and thanks so much for porting this beautiful project in Linux (I'm under Mint). I ask you if you have any news on the script you proposed. I'm having a hard time figuring out how to install this softare... thanks again!!!

Yes it will be really great. Think that they had already started working on it.

CodesoundR commented 4 months ago

Yes it will be really great. Think that they had already started working on it.

Thanks ..... I'm waiting.... ;-)

polygon commented 4 months ago

I drove this a bit further and opened Pull Requests in both project components:

https://github.com/DamRsn/NeuralNote/pull/85 https://github.com/tiborvass/libonnxruntime-neuralnote/pull/2

However, communication quickly died down (or didn't happen), so I assumed that the maintainers were not interested in this. I finished the packaging for the Linux distribution of my choice (NixOS) and carried on. I'll give a short rundown on how to build this yourself, though. You will need to use my forks of libonnxruntime-neuralnote and NeuralNote. For both, you want the "Linux" branch.

I will link my Nix build recipes, since they are also a good indicator on the dependencies that you will need. You can ignore a lot of the CMake download links replacement, that is a Nix/NixOS peculiarity (no internet access during builds).

libonnxruntime-neuralnote (Nix-recipe)

Once you installed everything unter buildInputs, you just run the instructions under buildPhase . This should yield you a libonnxruntime-neuralnote.tar.gz which you will need in the next step.

NeuralNote (Nix-recipe)

Again, make sure you have all the tools and dependencies installed. Now, run all the steps under postPatch in the root-folder of the repo (in line 83, reference the tar.gz file from the build above). This also patches a Drag&Drop bug in JUCE. You then just run CMake like above and this should give you the final binary / plugin.

Hopefully this gets you started. Let me know if you get stuck anywhere.

thelabcat commented 4 months ago

Thanks for doing this.

CodesoundR commented 4 months ago

I drove this a bit further and opened Pull Requests in both project components:

85 tiborvass/libonnxruntime-neuralnote#2

However, communication quickly died down (or didn't happen), so I assumed that the maintainers were not interested in this. I finished the packaging for the Linux distribution of my choice (NixOS) and carried on. I'll give a short rundown on how to build this yourself, though. You will need to use my forks of libonnxruntime-neuralnote and NeuralNote. For both, you want the "Linux" branch.

I will link my Nix build recipes, since they are also a good indicator on the dependencies that you will need. You can ignore a lot of the CMake download links replacement, that is a Nix/NixOS peculiarity (no internet access during builds).

libonnxruntime-neuralnote (Nix-recipe)

Once you installed everything unter buildInputs, you just run the instructions under buildPhase . This should yield you a libonnxruntime-neuralnote.tar.gz which you will need in the next step.

NeuralNote (Nix-recipe)

Again, make sure you have all the tools and dependencies installed. Now, run all the steps under postPatch in the root-folder of the repo (in line 83, reference the tar.gz file from the build above). This also patches a Drag&Drop bug in JUCE. You then just run CMake like above and this should give you the final binary / plugin.

Hopefully this gets you started. Let me know if you get stuck anywhere.

hello and thank you for your interest. But I don't understand how to install it on my Mint system....

Ken-Andre commented 4 months ago

Ok, thanks for this contribution.

On Wed, 20 Mar 2024 at 17:57 CodesoundR @.***> wrote:

I drove this a bit further and opened Pull Requests in both project components:

85 https://github.com/DamRsn/NeuralNote/pull/85

tiborvass/libonnxruntime-neuralnote#2 https://github.com/tiborvass/libonnxruntime-neuralnote/pull/2

However, communication quickly died down (or didn't happen), so I assumed that the maintainers were not interested in this. I finished the packaging for the Linux distribution of my choice (NixOS) and carried on. I'll give a short rundown on how to build this yourself, though. You will need to use my forks of libonnxruntime-neuralnote https://github.com/polygon/libonnxruntime-neuralnote and NeuralNote https://github.com/polygon/NeuralNote. For both, you want the "Linux" branch.

I will link my Nix build recipes, since they are also a good indicator on the dependencies that you will need. You can ignore a lot of the CMake download links replacement, that is a Nix/NixOS peculiarity (no internet access during builds). libonnxruntime-neuralnote (Nix-recipe https://github.com/polygon/audio.nix/blob/master/vst/neuralnote/libonnxruntime-neuralnote.nix )

Once you installed everything unter buildInputs https://github.com/polygon/audio.nix/blob/1217ff4ed70c562b5eaf9832151967b3a568c65c/vst/neuralnote/libonnxruntime-neuralnote.nix#L32-L41, you just run the instructions under buildPhase https://github.com/polygon/audio.nix/blob/1217ff4ed70c562b5eaf9832151967b3a568c65c/vst/neuralnote/libonnxruntime-neuralnote.nix#L52-L55. This should yield you a libonnxruntime-neuralnote.tar.gz which you will need in the next step. NeuralNote (Nix-recipe https://github.com/polygon/audio.nix/blob/master/vst/neuralnote/neuralnote.nix )

Again, make sure you have all the tools and dependencies https://github.com/polygon/audio.nix/blob/1217ff4ed70c562b5eaf9832151967b3a568c65c/vst/neuralnote/neuralnote.nix#L42-L58 installed. Now, run all the steps under postPatch https://github.com/polygon/audio.nix/blob/1217ff4ed70c562b5eaf9832151967b3a568c65c/vst/neuralnote/neuralnote.nix#L77-L86 in the root-folder of the repo (in line 83, reference the tar.gz file from the build above). This also patches a Drag&Drop bug in JUCE. You then just run CMake like above https://github.com/DamRsn/NeuralNote/issues/33#issuecomment-1784055326 and this should give you the final binary / plugin.

Hopefully this gets you started. Let me know if you get stuck anywhere.

hello and thank you for your interest. But I don't understand how to install it on my Mint system....

— Reply to this email directly, view it on GitHub https://github.com/DamRsn/NeuralNote/issues/33#issuecomment-2010055747, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZIUF3B3NWRW67RNGPVVABTYZG5WDAVCNFSM6AAAAAAYBK7CWSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJQGA2TKNZUG4 . You are receiving this because you commented.Message ID: @.***>

polygon commented 4 months ago

hello and thank you for your interest. But I don't understand how to install it on my Mint system....

These instructions are to build the project, distribution for specific Linux systems comes later. I'm very sorry, but I don't have the time to build and maintain packages for a Linux distribution I am not using myself. If you find someone who knows how to package for Ubuntu/Mint and is interested, I'm happy to assist getting things started.

CodesoundR commented 4 months ago

hello and thank you for your interest. But I don't understand how to install it on my Mint system....

These instructions are to build the project, distribution for specific Linux systems comes later. I'm very sorry, but I don't have the time to build and maintain packages for a Linux distribution I am not using myself. If you find someone who knows how to package for Ubuntu/Mint and is interested, I'm happy to assist getting things started.

Hi, I wanted to thank you anyway for what you did. I try to see in the various formums if there is someone who can help us for this porting.... bye