Embedded Javascript/Lua/Python scripting

[k0d]

Description

Implement some form of scripting inside of CasparCG Server to enable rapid development of modules/plugins etc.

Solution suggestion

--- ### Additional information Am adding this issue just so it doesn't get forgotten. Also I wanted an issue ID so I can create a branch for the investigation.

[Julusian]

I'm a bit confused as to the purpose of this scripting.

Can you give some examples of modules/plugins this would be used for?

[k0d]

RIght now it's an early-idea, but I could think that it'd be much easier to implement a new protocol/interface/module etc in javascript (for example) than to have to develop it in C/C++. Even some of the current features/modules could work nicer if written in a scripting language.

I'm thinking that exposing certain hooks/commands could be a good start.

The core code should stay in C/C++, for performance/threading etc reasons. But I believe we can add some missing features much quicker if we can do that in some scripting language. Maybe connections to other devices such as ATEM, audio mixers etc.

[jesperstarkar]

What would the difference between these hooks and AMCP/OSC be? Anything these hooks can do that we can't add to the existing protocols - or new more modern and better protocols for the future...

[k0d]

My view is that AMCP/OSC are protocols that would be very suitable to be developed in the scripting system. Internally we have an API with hooks/signals/slots etc etc, and we can just build protocols on top of that...it'd be much easier to create new/custom protocols such as AMCPv3....

I'm building some prototypes right now....give me some hours/days and we'll see what I can come up with.

[Julusian]

Maybe connections to other devices such as ATEM, audio mixers etc.

That is something I have considered in the past, but it was concluded that anything like that should be handled by the client instead.

I'm not sure I see the value in being able to define custom protocols in the server. For other things maybe, but I can't think of anything that I think would be appropriate to use it for right now.

[k0d]

For me, the biggest benefit for scripting is the ability to very quickly add new features, even if they are for a one-time project. Also, it would open up a whole new pool of developers if they can develop things in javascript. Just to be clear, I don’t suggest moving everything into javascript, but things lile network services, parsing of data and talking to devices over usb/serial are all perfect use cases for embedded scripting.

[jesperstarkar]

It all boils down to how the proposed hooks will differ from things exposed over AMCP/OSC. Looking forward to seeing examples!

[olzzon]

Could GraphQL be considered as protocol, as it can cover both the ACMP (via mutations) protocol and the OSC protocol (via subscriptions) this way it would be easy to build a protocol that can give the actual state of everything AND realtime updates for countdown, vol etc. There´s standard libraries for pretty much everything, and it´s a great way to hook up things via web sockets. I´m using it on a CasparCG-state-scanner. https://github.com/olzzon/casparcg-state-scanner-example

It works great and I very stable. Still need to test it with lots of realtime updates, but it should work for that too.c Right now it gives realtime countdown for all channels.

[ronag]

I think @k0d is trying to achieve something similar to GStreamer bindings.

[k0d]

Kind of...maybe its best that I just produce my prototype first :)

[k0d]

@olzzon Of course, we could have any protocol. That's why I'd like to create some common hooks/api inside of the server, that we can create protocols/services on top of that. It's not to create bindings (AMCP <-> Javacript or AMCP <-> Python), but to enable CasparCG to be scripted internally with a higher level language than C++. 'CasparCG SDK maybe?'

[olzzon]

@k0d ok, now I got it. :)

[jesperstarkar]

The contributions to the CEF producer from @hreinnbeck https://github.com/nrkno/tv-automation-casparcg-server/pull/14 will allow for html templates to act as controllers, hence provide a JS scripting layer.

[k0d]

It's not the same thing that I'm suggesting, mostly due to the fact that it requires CEF. I took a break from coding the past week or so, I nearly have my prototype scripting project finished, and when I commit it somewhere, you'll see what I hope to achieve with it.

[mint-dewit]

The contributions to the CEF producer from @hreinnbeck nrkno#14 will allow for html templates to act as controllers, hence provide a JS scripting layer.

But they don't provide an implementation close to the code. If we have to maintain that as well it just creates more abstraction layers on top of old code. This issue has the potential to replace AMCP/OSC/DIAG etc. Basically anything that doesn't have to do with the actual images CasparCG produces/consumes.

This is great because it would allow more developers to work on non-core issues without needing proficient knowledge about a low level language as C++. It could give a huge boost to contributor numbers and free up resources from the core developers for non trivial tasks.

[blockspacer]

cling based C++ as scripting language / hot code reload Why? Able to run C++ script in runtime or compile it for max speed ( as in example https://github.com/derofim/cling-cmake ) Also: May be used to improve modules compile time

HOT code reload possible approaches:

store app state fix cling undo for files https://root-forum.cern.ch/t/loading-unloading-class-as-interpreted-macro-in-cling-multiple-times/32976/2

execute cling code to change callbacks & variables nested cling::Interpreter with multiple cling::MetaProcessor IDK how to do it, but you can create child cling::Interpreter