DamRsn / NeuralNote

Audio Plugin for Audio to MIDI transcription using deep learning.
Apache License 2.0
1.12k stars 61 forks source link

Linux support #85

Open polygon opened 8 months ago

polygon commented 8 months ago

I've made NeuralNote build under Linux and was wondering if there is interest to upstream the changes. This PR would be the first step and contains the required changes to the code and the build system. The changes I was doing are:

This change also requires an accompanying change to libonnxruntime-neuralnote to get a Linux build there which I have also available. But first wanted to start here to gauge interest in something like this.

There is another issue with external Drag&Drop not working properly in Linux. This seems to be a JUCE issue and I have a workaround available. But since this requires patching the external JUCE dependency I think that would probably be more suitable to put into the build scripts. Also, it's not strictly required for a Linux build.

Edit: Link to patched libonnxruntime-neuralnote: https://github.com/polygon/libonnxruntime-neuralnote/tree/linux

DamRsn commented 8 months ago

Hey, thanks for the contribution!

This looks great, I definitely want to merge this. However I don't have a lot of bandwidth at the moment, so it might take a bit of time to review/test everything on my side.

Is the issue with exporting the MIDI fixed, or if you have a fix what is it?

Few notes after a super quick look at the changes:

Also you can open a PR on libonnxruntime-neuralnote so we can review it as well.

polygon commented 8 months ago

Hey, thanks for the response. I'm also a bit short on time so before doing some of the refactors, I'd like your opinion if that would be fine:

Regarding the drag and drop issue, JUCE has this piece of code that manages communication between the Drag&Drop processes:

    void handleExternalDragAndDropStatus (const XClientMessageEvent& clientMsg)
    {
        if (expectingStatus)
        {
            expectingStatus = false;
            canDrop         = false;
            silentRect      = {};

            const auto& atoms = getAtoms();

            if ((clientMsg.data.l[1] & 1) != 0
                && ((Atom) clientMsg.data.l[4] == atoms.XdndActionCopy
                    || (Atom) clientMsg.data.l[4] == atoms.XdndActionPrivate))
            {
                if ((clientMsg.data.l[1] & 2) == 0) // target requests silent rectangle
                    silentRect.setBounds ((int) clientMsg.data.l[2] >> 16, (int) clientMsg.data.l[2] & 0xffff,
                                          (int) clientMsg.data.l[3] >> 16, (int) clientMsg.data.l[3] & 0xffff);
                canDrop = true;
            }
        }
    }

I am not fully understanding of the details of Drag&Drop in Linux/X11 but I found that these if-conditions are no longer satisfied once the dragged data is moved outside of the plugin window. Hence, canDrop becomes false and in the following code which handles what happens after the mouse button is released to end the drop:

    void handleExternalDragButtonReleaseEvent()
    {
        if (dragging)
            X11Symbols::getInstance()->xUngrabPointer (getDisplay(), CurrentTime);

        if (canDrop)
        {
            sendExternalDragAndDropDrop();
        }
        else
        {
            sendExternalDragAndDropLeave();
            externalResetDragAndDrop();
        }
    }

the drop operation would be canceled. This had the funny side effect that while dragging, I could even see the MIDI notes in Bitwig, but once I released the mouse button, they were gone again. My (probably) dirty fix involved replacing if (canDrop) with if (1) to always force the drop to complete. This makes dropping files from NeuralNote to file managers and also to Bitwig Studio work without issues. I've not yet encountered any negative side-effects of always completing the drop.

I'll open a Pull Request for libonnxruntime-neuralnote later today once I am off work.

KottV commented 6 months ago

Hey! This is exciting, I compiled and it works. And more over this drag-and-drop "fix" looks like a key for a long standing JUCE DnD Linux bug:)

I only had to add set(CMAKE_POSITION_INDEPENDENT_CODE TRUE) do make the linker happy.

raphaelbastide commented 1 month ago

@polygon If any linux build is available somewhere, I would be very happy to try it.

buscon commented 3 weeks ago

@polygon If any linux build is available somewhere, I would be very happy to try it.

same here, I am a Linux user and I am happy to try it and give feedback.