Hot Swapping tunnels during execution

[FrostTusk]

It would be nice to add the ability to swap in and out new tunnels for specific tricks/obstacles/ginger during execution. This can be implemented at the core level.

[iPwnPancakes]

In order to have hot swapping be a thing, we should first understand what is required when a hot swap is performed. I will be using tunnels as a concrete example but this could extend to how obstacles, etc would work as well.

It would probably go in this order:

• Some kind of event needs to be emitted in order for a hotswap to begin, possibly passing in some metadata about the file name, path, os, command information, etc. Usually this is a file changed event from some sort of file watcher.
• Ginger must be aware that a tunnel can go down at any point in time and needs to know how to handle it. Therefore, ginger will shutdown any other tunnels, obstacles, moods, processes, etc that depend on this tunnel.
• The tunnel's state needs to be able to have its state saved/cached to some sort of temporary file or in-memory. It is vitally important that all information required in order to create a new tunnel needs to be stored. Possibly connection information, authentication, and metadata for re-creating another tunnel.
• Once the state has been cached, then we can proceed to start the process of shutting down that tunnel. Usually this can look like some sort of socket being closed or some kind of SIGTERM being sent via some arbitrary medium
• Now that the process has been closed and the information to create a new one has been cached. A new tunnel must be created using the cached information, including any authentication information.
• From then on, everything that was shut down previously can now be spun back up as well

[iPwnPancakes]

I would say a couple of things would need to be in place for hot swapping to work. Again, I use tunnels as a concrete example but this could extend to how other modules would be hot swapped as well

Separation from ginger initialization <-> tunnel initialization

Right now, tunnels are initialized when we initialize ginger. Thus, we have no way to spin up or shut down a new tunnel. Some sort of interface for how tunnels should be started up/shut down can provide an API for core to work with. Some sort of tunnel lifecycle state would also need to be implemented within some sort of tunnel class as well. Ginger would also need to be aware of were a tunnel is in it's lifecycle.

Filewatcher

A little self-explanatory but we would need a filewatcher to monitor any ginger files for modification. When a file is modified, it can let ginger know to hot swap a file

Tunnel Serializer/Deserializer

We need a way to represent the entirety of what state the tunnel was in when we want to hot swap the tunnel. AKA a serializer. That way it wont matter where the state will be stored (In-Memory vs File on disk). It will be serialized/deserialized the same way

[FrostTusk]

Looks good! Although I do want to stress the separation of concrete implementation again. The first step wouldn't make much sense -- hot swapping would be defined in code (for example in callbacks or attention scripts). Also, where you can reuse -- reuse. Why not make caching an obstacle or something?

[FrostTusk]

"Right now, tunnels are initialized when we initialize ginger. Thus, we have no way to spin up or shut down a new tunnel." <-- This isn't really right though, tunnels are only created through the createTunnel interface, Ginger can easily manage that.

It's not the files that get hot swapped, it's the tunnels in for example an hdmi trick. While the trick runs, it should be able to change tunnels.

[iPwnPancakes]

Hmmm. Maybe we are misunderstanding eachother's definition of hot swapping.

Pulling the definition from wiki: Hot swapping is the replacement or addition of components to a computer system without stopping, shutting down, or rebooting the system

While having that definition in mind, I'm not sure if my own understanding of hot swapping would be run by a script directly inside of its code. When I think hotswapping, I think of the tunnel being shutdown/recreated using a newer version of itself for the purposes of fast development.

For example writing this:

// Some_trick.js
const ginger = {}; // Create ginger

const tunnel_1 = new BinaryInputTunnel();
const tunnel_2 = new OtherInputTunnel();

const some_trick = ginger.createSomeTrick(tunnel_1);
some_trick.swapBinaryTunnel(tunnel_2);

Could really really be simplified to something like this:

// Some_trick.js
const ginger = {}; // Create ginger

const tunnel_1 = new BinaryInputTunnel();
const tunnel_2 = new OtherInputTunnel();

const some_trick = ginger.createSomeTrick(tunnel_2); // If you know ahead of time that you will hotswap, why not just swap it in the first place?

[iPwnPancakes]

This sort of leads into another question though:

Is ginger more of a framework, or a library?

In my mind, it feels like users will primarily be writing scripts. Scripts are written beforehand and ideally tested by the user to make sure they get whatever output they want from whatever tricks/tunnels they want. Again, thats written in advance.

The only reason I could see swapping tunnels being used would be if someone wanted to control Ginger externally via some kind of REST api or something. But wouldnt we want people to use ginger AS the api server?

[FrostTusk]

The term hot swapping was indeed perhaps not ideal. But, very simple use case: HTTPtunnel goes down because your wifi broke, now you want to move over and dynamically load in another tunnel that could work locally.

Behavior is dependent on changing environment, that is exactly why Ginger is being developed. So I would also say that Ginger is definitely more a framework than library.