Replace "Plugin API" with "Extensions"

cbjeukendrup commented 3 years ago

Task description The Plugin API may need actualization to fit well into MU4, regarding both the new UI and the new technical architecture.

Tantacrul commented 2 years ago

@cbjeukendrup - can you flesh this out / assign it / target it, etc?

If it's no longer relevant, then close it instead.

cbjeukendrup commented 2 years ago

We will definitely have to do work on the plugins API. Especially functions like creating a new score will not work, because of the new architecture with multiple instances etc.

But, it is a question how much work we need to do. We will have to investigate which functionality is currently broken and to decide whether we will fix it, replace it, or remove it.

Tantacrul commented 2 years ago

OK, well I've assigned it to you / targeted it at Beta then.

cbjeukendrup commented 2 years ago

@Eism Assigning you too, since you've done some work on plugins before, so you might have an opinion about this.

gregsadetsky commented 2 years ago

Hey, I have a few questions regarding plugin support in MU4 if you don't mind:

Are there plans to let plugins create GUIs in MU4 i.e. add UI elements to a panel?

I see that all of the example plugins in the repository don't create UIs, and the previous MU3 plugins that did were removed in this commit.

Could you let me know where to find the output of console.log() calls in MU4? I tried running mscore -d in the command line, and to find a window in the app with the output, to no avail.
Regarding the comment above that "creating new score will not work": are there plans to add that functionality back i.e. let plugins create new scores in MU4?

Thank you!

Tantacrul commented 2 years ago

Hi Greg, as with many things in MS4, we've encountered technical issues that we'd prefer to solve 'properly' in later releases of 4.x.

One of those is a more permissive and robust system for allowing people to create third party plugins. This is why we didn't port over the plugin creator from MS3 because it needs significant change in order to work. This is just a non-technical overview of our decision-making. We want people to be able to create more powerful plugins in future but won't get around to it until we've released a few (smaller) versions of 4.x.

Perhaps others here can be more technically useful to you!

Tantacrul commented 2 years ago

We plan to build a system to make plugin creation much more powerful in a later release. It is one of our highest priorities.

For the moment, only the plugins packaged with MuseScore 4.0 will work for users.

I'm moving this to 4.x and closing. We'll reopen once 4.0 is released. Thanks!

lgvr123 commented 2 years ago

Hi, You said >For the moment, only the plugins packaged with MuseScore 4.0 will work for users. Shouldn't we block all non-native plugins in the Plugin panel rather than give the impression that we could run them while in fact they don't work, or even crash MS4?

gregsadetsky commented 1 year ago

Hey @Tantacrul would it be possible to re-open this since MU4 came out? Thank you!

lgvr123 commented 1 year ago

How to take part to this ? I've listed over the last months some issues with the 3.6 API that could be addressed during this redesign phase:

Need/Issue	Discussion	Issue ref
Clef element details are not available	Bossa Nova setting/plugin?
Sibling elements in other parts/excerpts are not exposed	Get cursor to score from parts
Fret diagrams dots and bars are not available	Retrieving Fret diagrams details
Getting the "parent" score for curScore when curScore is an excerpt of this "parent" score	How to get a "full score" object when curScore is one of the score excerpts ?
Dots not exposed for rests	Plugin: Accessing rest dots
Key signature details not available	How to add a KeySignature ?	#100501
Changing bar to a StartRepeat bar doesn't work correclty	Changing a barline type via code
Impossible to access text frames	How can I access other text on the page (besides lyrics)?, …	#337263
Expose All Staff Properties	Any way to retrieve staff characteristics?	#311093
More methods for QProcess class	#275294
Creation of folders within the FileIO objects	#334952
Creation of folders with the PluginAPI.writeScore method	#334953

(from the API improvments forum thread)

tormyvancool commented 1 year ago

Without the Macro functionality we had in 3.x version, even the integration with Reaper is not any longer possible. This is blocking issue. We're in december 2022 now and I hope an update of Musescore to a 4.1 will be released soon containing this feature. Meanwhile I have to reinstall the 3.6 and get rid of the 4.0. It's pity, but that feature is more important than the great voicing (for my use).

daniellumertz commented 1 year ago

Without the Macro functionality we had in 3.x version, even the integration with Reaper is not any longer possible. This is blocking issue. We're in december 2022 now and I hope an update of Musescore to a 4.1 will be released soon containing this feature. Meanwhile I have to reinstall the 3.6 and get rid of the 4.0. It's pity, but that feature is more important than the great voicing (for my use).

well the "integration with reaper"* you are refering to was just a plugin to export midi, you can do that manually at musescore 4, so you still can use musescore 4 with MIDI Transfer.

as I am here, Thanks for remaking the plugins system!

= the "integration with reaper" shouldnt be discussed here, as is not the place, it is a REAPER script I made (MIDI Transfer), and I am sorry musescore devs this have leaked here.

tormyvancool commented 1 year ago

Without the Macro functionality we had in 3.x version, even the integration with Reaper is not any longer possible. This is blocking issue. We're in december 2022 now and I hope an update of Musescore to a 4.1 will be released soon containing this feature. Meanwhile I have to reinstall the 3.6 and get rid of the 4.0. It's pity, but that feature is more important than the great voicing (for my use).

well the "integration with reaper"* you are refering to was just a plugin to export midi, you can do that manually at musescore 4, so you still can use musescore 4 with MIDI Transfer.

as I am here, Thanks for remaking the plugins system!

= the "integration with reaper" shouldnt be discussed here, as is not the place, it is a REAPER script I made (MIDI Transfer), and I am sorry musescore devs this have leaked here.

The goal is to automate as much as possible the export rather than to perform a lot of "clicks". just one click and you get what you need. I stil prefer the 3.0 for that.

It's NOT a Reaper's script. It's a script was running on Musescore. And as user of both, I need this feature. Hence I'm here since it was removed from Musescore. Not an evolution but and involution, for the time being. I do hope the future will bring something more interesting about this feature.

cbjeukendrup commented 1 year ago

@tormyvancool In principle, plugins mostly still work in MS4, it's only not "supported" at the moment. So (if you haven't done so already), you could just try it out, by putting the plugin in the folder specified in Preferences > Folders > Plugins. You might find that it just works.

However, the future of plugins is bright: I've heard of plans to implement a completely new, way more powerful plugin system. We'll see...

tormyvancool commented 1 year ago

@tormyvancool In principle, plugins mostly still work in MS4, it's only not "supported" at the moment. So (if you haven't done so already), you could just try it out, by putting the plugin in the folder specified in Preferences > Folders > Plugins. You might find that it just works.

However, the future of plugins is bright: I've heard of plans to implement a completely new, way more powerful plugin system. We'll see...

To implement it I did use the macro and saved it. I will try to find everything and to try to move it for the time being. I can't wait for the new system of course <3

lgvr123 commented 1 year ago

@cbjeukendrup I'm far to share your optimism about the compatibility of the 3.6 plugins with MU4.0. I've tried to migrate most of my plugins to MU4.0 and few are working (see my guidelines). Actually, all the plugins using readScore, writeScore are not working, The ones that are "dock"-type plugins are not working either, or using some deep score manipulation methods, ...

cbjeukendrup commented 1 year ago

Hm... you may be right. Manipulation within the current score should work mostly fine, but more advanced things like opening other files, or writing files, and indeed dock plugins, is broken at the moment. Anyway, as said, the plugin API was one of those things that were just too big to tackle with MS 4.0.0, but it will surely get the love it deserves at a later moment.

tormyvancool commented 1 year ago

Hm... you may be right. Manipulation within the current score should work mostly fine, but more advanced things like opening other files, or writing files, and indeed dock plugins, is broken at the moment. Anyway, as said, the plugin API was one of those things that were just too big to tackle with MS 4.0.0, but it will surely get the love it deserves at a later moment.

All what I need is, for the time being, to write this code in order to make it work under the 4.0 https://youtu.be/Bb4wldM0rmo?t=362 ... hoping that in the 4.1 there will be an easier way :-)

because however, creating a plugin following the examples of the other ones already present, it doesn't show up. nor into the indicated folder, nor in the one under Musescore main folder

I did create a folder called "export_midi"

Inside it 2 files: export_midi.qml midi_export.png

the code

` import QtQuick 2.2 import MuseScore 3.0

MuseScore { version: "2.0" title: "MIDI Exporter" description: "Export MIDI for REAPER" categoryCode: "composing-arranging-tools" thumbnailName: "midi_export.png"

  MessageDialog {
        id: versionError
        visible: false
        title: "Unsupported MuseScore Version"
        text: "This plugin needs MuseScore v3.0.5 or higher"
        onAccepted: { Qt.quit() }
  }

onRun{
    var s = curScore;
    var p = curScore path;
        p = p.slice(0, -5);
        p = p.replace(/\//gi,"\\");
        writeScore(s, p, 'mid');

        curScore.endCmd();

        quit();
}

} `

lgvr123 commented 1 year ago

All what I need is, for the time being, to write this code in order to make it work under the 4.0 https://youtu.be/Bb4wldM0rmo?t=362 ... hoping that in the 4.1 there will be an easier way :-)

Little chance that you can have it working with MU4.0 as writeScore is broken under MU4.0

tormyvancool commented 1 year ago

All what I need is, for the time being, to write this code in order to make it work under the 4.0 https://youtu.be/Bb4wldM0rmo?t=362 ... hoping that in the 4.1 there will be an easier way :-)

Little chance that you can have it working with MU4.0 as writeScore is broken under MU4.0

Even by omitting that line (I tried it), the plugin doesn't pop-up

lgvr123 commented 1 year ago

@tormyvancool let's maybe open a separated question MuseScore plugin's forum for this.

tormyvancool commented 1 year ago

All what I need is, for the time being, to write this code in order to make it work under the 4.0 https://youtu.be/Bb4wldM0rmo?t=362 ... hoping that in the 4.1 there will be an easier way :-)

Little chance that you can have it working with MU4.0 as writeScore is broken under MU4.0

Perfect I opened it :-) https://musescore.org/en/node/340725

schef commented 1 year ago

Hey guys..did you think of making support for python plugins? Something like blender has implemented. The nice thing about python is that it has an interpreter where you can test your code live and it has a lot of libraries (even AI) not to mention that it is very popular. Also C/C++ and python are very friendly because python is written in C. If this is out of scope or not possible at the moment would you consider making mscz/mscx as a documented standard so external manipulation would be done more easily? Thanks

ephemer commented 1 year ago

Hi @schef,

The plugin functionality is provided by the 3rd party framework Musescore is built on, called QT, which supports a version of JavaScript called qml for its plugins. It's unlikely that Python or other languages will be supported because they're not supported by QT.

I am not associated with the musescore project in any way, so don't take this as official word on the direction of the project by any means, but since no-one else has responded to you I think it's worth setting your expectations that this probably isn't going to happen, and why.

schef commented 1 year ago

Hi.. I am aware of the current QT/QML GUI architecture as I am a programmer myself (Python/C++). I'm following this project for some years now and i use it myself as I also have Master's Degree in Music. This is why I'm so interested in seeing this happen. :) I would also like to contribute if this is something the DEV team would consider going into.

On Thu, Jan 19, 2023 at 1:24 PM Geordie J @.***> wrote:

Hi @schef https://github.com/schef,

The plugin functionality is provided by the 3rd party framework Musescore is built on, called QT, which supports a version of JavaScript called qml for its plugins. It's unlikely that Python or other languages will be supported because they're not supported by QT.

I am not associated with the musescore project in any way, so don't take this as official word on the direction of the project by any means, but since no-one else has responded to you I think it's worth setting your expectations that this probably isn't going to happen, and why.

— Reply to this email directly, view it on GitHub https://github.com/musescore/MuseScore/issues/8859#issuecomment-1396899438, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAF5KTTGMPAP4OJ2BWGJTYDWTEXBPANCNFSM5CGJIY4A . You are receiving this because you were mentioned.Message ID: @.***>

Tantacrul commented 1 year ago

Hey. Right now, the dev team are completely swamped. We do have a planning session coming very soon, where we'll discuss how we want to enable plugin creation but it won't be something we can immediately hop on. Apologies for this. Just a lot of work in front of us right now. We'll be opening up to the community about our internal plans once we get them all agreed on.

lgvr123 commented 1 year ago

Thanks for the feedback.

how we want to enable plugin creation

Plugin creation is a key feature of MuseScore. I hope your plans will not go to reduced plugin creation capabilities.

cbjeukendrup commented 1 year ago

It has already been said that our plans will go to completely re-built but 100x more powerful plugin creation capabilities.

MNeill73 commented 1 year ago

Just a breadcrumb for eventually revisiting the Python/QT discussion...

https://doc.qt.io/qtforpython/

cbjeukendrup commented 1 year ago

Yes, you can use Qt in Python. But what we need (beside that) is the opposite: running Python plugins in a C++ app. That requires building in a full Python interpreter, which is not realistic and annoying for most of users because it makes MuseScore use more disk space while most users won't ever use plugins.

kwertyops commented 1 year ago

For most users with a modern-ish computer, hopefully a handful of extra MB wouldn't be too annoying, but even if that's the case (and I understand not everyone does have a modern computer) maybe a Python plugin system could be opt-in (like Muse Sounds).

As far as realistic, here are two options for which this (integrated Python plugins for a C++/Qt application) seems to be the exact application, and which seem realistically simple to implement:

Shiboken (Hybrid Application @ 3:15) - https://www.youtube.com/watch?v=6u6IifkCK8g&t=315

pybind11 - https://pybind11.readthedocs.io/en/stable/advanced/embedding.html

Don't get me wrong, I'm fine with the current plugin system (especially if more functionality is eventually exposed), but as someone who has written a good number of Musescore plugins, being able to do so in Python is too beautiful of a dream to let die.

Also maybe worth pointing out MicroPython, which is a full interpreter that is intended to fit on systems with as little as 256kiB flash + 16kiB RAM.

https://github.com/micropython/micropython

Does MicroPython work with pybind11? Maybe this person knows:

https://forum.micropython.org/viewtopic.php?t=11862

MNeill73 commented 1 year ago

running Python plugins in a C++ app

That's exactly what the library above does. And as kwertyops said, the library could be loaded optionally (via a toggle or an import command), if it ends up being particularly bulky.

ephemer commented 1 year ago

We do have a planning session coming very soon, where we'll discuss how we want to enable plugin creation but it won't be something we can immediately hop on.

@Tantacrul do I understand correctly that the current blocker for the plug-in (re)implementation is organisational rather than technical?

Just wondering what the result of the discussions you mentioned was and if there's anything the community can do to help? Does the dev team have enough resources (people, time, money) and is there scope for the community to support with that?

ecstrema commented 1 year ago

I'll leave my thoughts here about the requirements of the redesigned plugin system:

Make musescore's native UI elements available. That means sticking with QML for plugins.
Keep the simple API: here's a proposal:
- Score class:
  - Score.open: Open a score from filepath or Score object. The currently opened score can be accessed with document.
  - Score.create: Create an empty score. Can be opened with Score.open(Score.create(...))
  - Score.walk: This is what most plugins will use. Recursively walk through children to modify them.

Example of using Score.walk to make all sharp notes red.

document.walk({ // do not use abbreviations like curScore in the API
    note: (note) => { 
        // see https://musescore.github.io/MuseScore_PluginAPI_Docs/plugins/html/tpc.html
        if (note.getAttribute("tonalPitch") > 20) note.setAttribute("color", "red");
    }
}

This is how a lot of parsers work, and it works exceptionally well with DOM-like structures, such as MS's element tree.

Element class. Create, append and modify elements.
- Element.create(type): Creates the specified element and returns it.
- Element.appendChild(child): add an element to the element element list.
- Element.remove: removes this element from the DOM.
- Element.parent: get the parent element. there should also be Element.children
- Element.setAttribute(targetAttribute, value): since having a separate class for all element types is probably too much to keep the plugin API maintainable, extract the specific attributes into Element.getAttribute and Element.setAttribute. This allows making virtual calls to the underlying class, and can be easily abstracted. It should throw if the element doesn't have the attribute.

With that already, you get a nice plugin api that can do most of what could be done before:

Examples

See all examples

### color notes: ```qml Plugin { name: "color notes" onRun: { if (!document) throw new Error("A score is required to run this plugin"); const colors = ["red", "green", "blue", "etc"] // etc. for the 12 notes document.walk({ note: (note) => { note.setAttribute("color", colors[note.getAttribute("midiPitch") % 7]) } } } } ``` ### Note names ```qml Plugin { name: "note names" onRun: { if (!document) throw new Error("A score is required to run this plugin"); const names = ["C", "D", "E", "F", "G", "A", "B"] // for the seven notes, see https://musescore.github.io/MuseScore_PluginAPI_Docs/plugins/html/tpc.html document.walk({ note: (note) => { const text = document.createElement("text"); text.setAttribute("value", names[note.getAttribute("tonalPitch") % 7]) note.appendChild(text); } } } } ``` ### Mirror intervals ```qml Plugin { name: "mirror intervals" property var noteOptions: [ {text: "C-1", value: "0"}, {text: "D-1", value: "2"}, {text: "E-1", value: "4"}, {text: "F-1", value: "5"}, // ... etc for all 7 * 70 midi tones ] property var referenceNote: 60 // middle C Window { Select { title: "Mirror around what note?" options: noteOptions value: options[reference] } Button { text: "Run" onClick: { if (!document) throw new Error("A score is required to run this plugin"); if (!document.selection) throw new Error("This plugin operates on the selection. please make one"); Score.walk(selection, { note: (note) => { const midiPitch = note.getAttribute("midiPitch"); const newMidiPitch = 2 * noteOptions(referenceNote) - midiPitch; note.setAttribute(reversedMidiPitch); } } } } } } ``` ### modal tuning ```qml Plugin { name: "mirror intervals" property var tunings: [0, 0, 0, 0, 0,] // for all twelve notes Window { VerticalLayout { Repeater { NumberSelect { required property index value: tunings[index] onValueChange: (value) => tunings[index] = value } } Button { text: "run" onClick: () => { if (!document) throw new Error("A score is required to run this plugin"); const target = document.selection || document Score.walk(target, { note: (note) => { note.setAttribute("tuning", tunings[note.getAttribute("midiPitch") % 12]) } } } } } } } ```

But the very important part is: keep it simple. Most plugins' functionality fits in a single file. Don't throw in separate configuration files, required bindings, etc. A single file is nice

lgvr123 commented 1 year ago

I'll leave my thoughts here about the requirements of the redesigned plugin system:

Nice suggestion to have a DOM-like approach. Nevertheless, I see some issues in this approach. Maybe have you already take this into account: Contrary to a HTML document which has tree structure, a score is a kind of 3D model: axis X: the instruments, axis Y: the measures, axis Z the parts/excerpts :

You cannot add a measure/remove/enlarge/reduce in instrument A without a measure being added/removed/... at the same place in all the other instruments.
If you add a note on instrument A on part X (e.g. the conductor), the same note will be added for the instrument A on all the other parts, but not with the same properties, at least not all. E.g. placement/size/...

But the very important part is: keep it simple. Most plugins' functionality fits in a single file. Don't throw in separate configuration files, required bindings, etc. A single file is nice.

I don't agree with this. A single file might be difficult to maintain. I prefer splitting my plugin into a series of files, one main QML plugin file, and optionnaly a few QML component files, and some JS files. This allows (a.o) for creating reusable pieces of code and makes the plugin development faster to develop and easier to maintain.

cbjeukendrup commented 1 year ago

The DOM model is also broken by "spanners": slurs, crescendos etc. These don't have a clear parent. Especially not when they have to be broken into segments, because they span a system break.

Daniel63656 commented 9 months ago

Is there a rough estimate when a stable plugin API for MuseScore 4 will be released?

cbjeukendrup commented 6 months ago

Update about this: it's happening! See https://github.com/musescore/MuseScore/pulls?q=is%3Apr+extensions+module for a list of related PRs. Since these PRs are going to the current master branch, you'll see the (first) results of this in MuseScore 4.4.

I will close this issue, as it doesn't seem the best place for further discussion. If anyone does want further discussion, please open a discussion at https://github.com/musescore/MuseScore/discussions.

musescore / MuseScore

Replace "Plugin API" with "Extensions" #8859

Examples