Open delvh opened 2 years ago
I think https://github.com/hashicorp/go-plugin should also be taking in consideration, afaik the "normal" plugin system has a lot of shortcomings (eg crashing the whole process because of issues with the plugin), that the Hashicorp one does not suffer from.
See also:
Yeah, I've also seen their implementation when looking for a plugin system.
However, to me, it appears as if they don't support executing dynamic functions as I think I read the sentence sending functions is not possible
anywhere, and then I stopped reading and didn't investigate further as that is something we most likely need.
After having read through their docs for a little, it seems I was a bit too fast with that.
Regarding the unexpected crashes: I intended to call every plugin by using something like
func PluginCall(function func(plugin *Plugin)err) {
for _, plugin := range plugins {
executePluginCall(plugin)
}
}
func executePluginCall(plugin *Plugin, function func(plugin *Plugin)err) {
defer func(){
if err := recover(); err != nil { // Error handling
}
}()
function(plugin)
}
and thus mitigating the potential for those errors.
So, the question is now, should we use the RPC version which might offer a few benefits and higher latency, or do we stay with the Go native approach that we can completely customize to our liking?
However, to me, it appears as if they don't support executing dynamic functions as I think I read the sentence
sending functions is not possible
anywhere, and then I stopped reading and didn't investigate further as that is something we most likely need.
Do you want to compile/execute Go code on-the-fly?
Not really, but we certainly need to handle the custom routes which can best be achieved dynamically, and making this interface-based might be possible, but (a little bit) more work.
Wish you many luck with this advanced logic.
I prefer the plugin system based on javascript or nodejs because that will allow it include frontend code.
Possbile library be used. https://github.com/dop251/goja, there is also a plugin support nodejs .
Interesting how many different views there are regarding this feature. Now we have three completely different options and no agreement on what to use...
Interesting how many different views there are regarding this feature. Now we have three completely different options and no agreement on what to use...
Use javascript based plugin system, we can implement a plugin market and one click installation. That's impossible for Go based plugin system.
The problem I can see is: We do not even have the same definition of what we want in the end: What you want seems to be a per-user plugin mechanism, what I thought of was a per-instance plugin mechanism...
The problem I can see is: We do not even have the same definition of what we want in the end: What you want seems to be a per-user plugin mechanism, what I thought of was a per-instance plugin mechanism...
I don't think so. I think I'm also talking about per-instance plugin. If I made you confusing, I'm sorry.
-1 for the go plugin
package.
It can't be removed per compatibility promise, but it is absolutely not stable, and there are far too many caveats to get it working properly.
I think we can begin from a theme-kind plugin, which is simple and useful. A plugin could be a zip/tar file with a main.js
or index.js
and other files like theme.sass
, theme.css
and images
and etc.
I do not think the Go's plugin
package is the future. Is there any success story of using it?
Most large Go projects uses gPRC for inter-process commutation, including plugins, but it's also heavy.
Ok, seems as if the plugin
package is generally disliked, so I'll exclude it from the possible options.
I'm still not a fan of a purely frontend-based plugin mechanism as that will have many shortcomings of its own, and I don't know if
plugin market and one click installation.
should be the deciding factor for how to implement the plugin system... Server admins should be sure of what they want for their instance, and not install it with a simple misplaced click. Also, even with the Go-based approach, a plugin "market" would be possible as a third-party tool that simply stores the executables and perhaps some other information about the plugin...
So, as I can see that now, the two possible options are
go-plugin
)Isn't this library lunny suggested meant for a backend-based plugin mechanism? But instead of writing go you can write js? Or would that be to difficult to build a bridge between the Gitea go code and js code?
Yes, it could expose structs, methods to frontend. i.e. If we expose getdb method, then we can operate database with javascript.
I think the best way to go would be using grpc so that plugins could be implemented using any language. Plugin could return asset files on init (JavaScript etc) that would need to be injected/used in specific places
I would also like to throw yaegi into the mix. I think gRPC is probably the only way to go (unfortunately), but figured yaegi is worth considering as well.
Honestly, I would like to see the native go plugin implementation. Yes it has drawbacks, but it would be fast and admins had full controll over what to use.
But since thats off the table for now, I would rather go with the gRPC approach, since its more open (i.e. you can choose more languages) and imo more secure then javascript.
The problems I see with js plugins is, that it lead to a huge security risk, since js can be so minified or encrypted that you not even need any letters (only parentheses) to write correct js. We then would need an very strict system so no js can accidentally just execute "DROP ALL TABLES;" (or similar) on startup.
Also an interpreter (of any kind) in gitea would bring back the risk of one plugin crashing the whole instance, or to eat up so much performance that gitea stops responding (i.e. a infinite loop in a script somewhere). To mitigate this we then would need to sandbox the plugins into own processes, which would nearly the same as the gRPC approach but with less freedom.
To be fair, you can also DROP ALL TABLES;
in Go or consume all available memory.
But yeah, I can see your point.
Then you have to think twice about an official plug-in market like lunny suggested. Official also indicates that they are save for usage and have no malicious intentions. An easy one click install could potentially break servers from not experienced users who don't understand what the plugin is doing and just pressing install.
To prevent this, this would at least requires some reviewing and analysis of newly published plugins, which is time expensive.
We could probably get some ideas on how mattermost have implemented their plugin system.
We could probably get some ideas on how mattermost have implemented their plugin system.
For posterity: mattermost plugin package It looks like they use go-plugin
Another inspiration could be woodpecker proposal around extensions https://github.com/woodpecker-ci/woodpecker/pull/915
Since nobody has mentioned it yet, projects like Adobe Lightroom, MPV, or Wireshark use Lua for this. You'd read and run Lua files from a directory where they would register events with callbacks or maybe return properties and functions. It's very light. simple, and fast but would make it very easy to make plugins and shouldn't be any harder to implement
Made up possible example:
-- Greet users on their first repo
gitea.on("newRepo", function(repo)
if #repo.owner.repos == 1 then
local msg = ("Welcome to our Gitea, %s!"):format(repo.owner.name)
repo.owner:sendNotifacation(msg)
end
end)
The Lua integration could be plugin too which forwards callbacks to Lua.
Are there any updates on this one? ^^
For most backend tasks, Gitea Actions can be considered as somewhat plugins.
Yeah for my usecase the webhooks are sufficient. I will just have to host another small webserver alongside Gitea. But it would have been nice to just write a plugin and call it a day :)
Then there are 4 point of views could be considered for plugins
Webhooks
and I will add the 5th
What should it be?
Architecture
Plugins are called using(see discussion below, was generally disliked)Go
splugin
package.go-plugin
Communication: _Gitea :arrow_right: plugin :arrowright: Gitea
modules/
:map[string]?
, where possible to avoid breaking changesCommunication: _plugin :arrow_right: Gitea :arrowright: plugin
Plugin Lifecycle (What the plugins can set/ "use")
Application is completely initialized and running
- HookOR
/AND
relations (depending on the chosen algorithm)?What data is needed by every/ some plugin(s)?
io.gitea.example_plugin
)Example Plugin V1
)Why this long text rambling?