[scripting] Provide a RPC server so third party programs can interact with maptool

[bobifle]

The reasoning of this proposal is based on soem statements. These statements are no absolute truth, they only reflect my humble personal opinion, but I think they can make sense for some MT users.

Statement 1: the current MT scripting language is not particularly easy to learn and has no use outside MT, it's a hard and long investement. Statement 2: MT2 new scripting language will fall into the same pit, as it is still a new thing to learn Statement 3: writing a lexer/parser is hard and without the proper expertise, it yields to limitations due to the complexity

One way of fixing this, is for MT to provide an API so macro calls can be done by external programs/scripting languages.

Possible techno: 1/ REST API, coudl work, but not realy designed for this usage, writing end points could be really tricky 2/ Remote Procedure Call, the name of the techno speaks for itself. It's designed to remotely call APIs

gRPC is an open source RPC lib developed by google. It's a strong alternative to popular REST APIs, it supports a lot of languages, and server/clients may used different language. For instance, MT may run a gRPC server in java. Any client may connect to it and call macro, it supports bi-directional calls.

From there, MT can delegate the scripting language to the user's choise, ruby, go, python, javascript, java, c, java, c#, you name it... MT could provide a default embeded client in MT language choosing. I know some ppl have already talked about javascript support.

MT could still supports the legacy scripting language, for those who are happy with it and keep compatibilty with all the amazing stuff that have been developped with the current scripting language.

[Azhrei]

I agree with the goal of providing external access to MapTool game session information. But I'm not too concerned about the implementation.

REST does have a couple benefits: it's stateless, and it's based on HTTP. This means implementing some kind of client-side viewer in a web browser is possible and much easier than it would be for RPC. Otherwise, I don't see that either one has a particularly strong argument over the other. Although I expect folks with more experience than I will speak up and prove me wrong. 😉

[bobifle]

I found this article https://code.tutsplus.com/tutorials/rest-vs-grpc-battle-of-the-apis--cms-30711 Also yo ucan generate a REST API along with the RPC protocol : https://github.com/grpc-ecosystem/grpc-gateway

Though I may be diving too much into impl details, maybe we shoudl first agree on the high principles :)

[cwisniew]

The reasoning of this proposal is based on soem statements. These statements are no absolute truth, they only reflect my humble personal opinion, but I think they can make sense for some MT users.

Statement 1: the current MT scripting language is not particularly easy to learn and has no use outside MT, it's a hard and long investement. Statement 2: MT2 new scripting language will fall into the same pit, as it is still a new thing to learn Statement 3: writing a lexer/parser is hard and without the proper expertise, it yields to limitations due to the complexity

Firstly I agree something like this would be nice to have, I would never see it as a valid replacement for a proper internal scripting environment.

From the above statements I think you are seeing this as a valid replacement for an internal scripting language. The problem is that it would not be, at the moment frameworks can be included in a campaign and there is no need for GMs to perform any other set up of their environment. This does not offer that same functionality, and as such, these add ons would also not be included in the framework repository list that we are looking at building for this exact reason.

I also don't believe your statement 2 is true, if you already know one or two programming languages another is not too hard, and for those that don't already know programming a well thought out (which I agree current one isn't) tailored scripting language is a significantly easier concept to deal with than asynchronous remote procedure calls or REST API.

This request covers a different use case and is not a replacement for fixing other things, I myself would put a much higher priority on the new internal scripting.

MT could still supports the legacy scripting language, for those who are happy with it and keep compatibilty with all the amazing stuff that have been developped with the current scripting language.

But this would never replace a proper internal scripting language.

[JamzTheMan]

This ^^^

I think it you did a poll of every user that uses scripting in MapTool then asked each of them if they were really a developer/programmer I suspect most are not.

There is a use case for an external API to MapTool for lots of reasons but not because of the short comings of the scripting language.

[bobifle]

• MT can include its own language into the game: write a custom language, build a client out of it, make API calls.
• The client can be fully integrated into MT, the user does not need to know there is a client server architecture.

On the MT2 script side: I read the document, it's def better than the current one and will make the life of ppl easier, but clearly not as much as any existing fully fledge language (IMO). It will still be difficult for non computer scientists.

On MT dev side, the amount of work required is humongous:

• principles
• syntax
• grammar
• parser
• interpreter
• reference
• documentation
• stackoverflow ;)

MT2 will not be able to compete with open source languages that have 10 years of existence with a world wide community improving it. Some of them have been designed to be easy to learn, read and write, some of them are adopted widely outside the computer scientist community because of that.

But I would stress the main point:

There's no need to restrict one client over another, so MT can still implements its own client version. This issue is not about dropping MT2 scripting, it's about the internal architecture that define how the scripting language interacts with the "java MT". And in this proposal, it's through language agnostic RPC APIs.

[cwisniew]

I read the document, it's def better than the current one and will make the life of ppl easier, but clearly not as much as any existing fully fledge language (IMO). It will still be difficult for non computer scientists.

Still vastly easier than RPC for non programmers.

On MT dev side, the amount of work required is humongous:

Its large for both both the internal scripting and the client/server and they wills share much of the same code for dealing with the API, and while both are good to have internal scripting is the more important so will come first.

But I would stress the main point:

There's no need to restrict one client over another, so MT can still implements its own client version. This issue is not about dropping MT2 scripting, it's about the internal architecture that define how the scripting language interacts with the "java MT". And in this proposal, it's through language agnostic RPC APIs.

No one said it shouldn't be done, nor am I suggesting that it won't be done, just setting the expectation that it's not likely to be looked at by most before internal scripting is completed. If you want to submit a design and are willing to work on it then by all means please do so and we can discuss it.

[bobifle]

Still vastly easier than RPC for non programmers.

Let me clarify this point, users don't write RPC calls, they write go, java, ruby or MT2 script calls.

import maptool as mt
mt.connect('localhost:12345')
tokens = mt.getSelectedTokens()

[bobifle]

I'm willing to give a prototype a go however my knowledge in the java ecosystem is very limited, especially regarding all the gradle stuff

Would someone be willing to help and integrate the gRPC server on MT part in a given branch ? As soon as import io.grpc works in MT I think I can manage the subsequent java code.

TBH I don't even know if I'm asking for a 30 minutes or 30 day work.

[bobifle]

I'm moving my content to another VTT, so I won't personally do anything regarding this issue. You may want to close it, that would be fair. (I still think the suggestion makes sense)