Define a serverless protocol

[sameo]

A protocol for the host (Lumper) to send commands to the guest (serverless agent #6) and receive replies from it.

• Define commands and responses
• Define packet format

[mmoreiradj]

Commands

Configure

Request

This command would configure the agent with fields such as:

• language
• variables/inputs
• code
• log level

Note: it could be just one command with all the needed configuration.

Response

On success, return OK (or a similar message).

On error, return NOK (or similar), with a reason.

Run

This command can be called once the configuration was successful.

Request

Starts the code

Response

This command would start a build (if needed) and the run code.

It would return nothing on successful reception of the command, but it would log the steps it takes.

Logs

Request

A simple log from the guest to the host.

They include:

• Error: runtime errors, build error, configuration errors
• Info: code result, maybe compiler warnings
• Debug: steps taken, building time, starting time, etc.

Response

Interruptions / Stop

Request

Just a signal to stop the guest

Response

Nothing.

Subjects to consider:

• Do we need an interruption command ?
• Metrics ? Can't the VMM get those without the agent communicating them ?
• Timeouts ? Retries ?
• Multiple Files ? Since we've separated the configuration and runtime, this could be a possibility.

[mmoreiradj]

Communication

Slip Based

As for the communication, for now, I thought we could either use a SLIP Based protocol over the serial line. It would look like this:

Information	Length (in bits)
Message Type	16
Checksum	16
Data	...

The message type could be log, config, etc.

Protobuf

We could also define .proto file and serialize / deserialize the messages with it.

Note that this could be reused if we manage to set up virtio-net and use gRPC further down the line.

[lapsus-ord]

@virt-do/vmm

[charley04310]

Hello @mmoreiradj and thanks for your job !

Every thing looks good to me !

Regarding the interruption command, I agree that it would be beneficial to have such a feature. It would allow for the proper termination of code execution, which can be more efficient in terms of resource management compared to letting a code run that will inevitably need to be restarted.

As for metrics, it indeed depends on the available time, but having feedback such as execution time and memory usage could be valuable. These metrics could provide insights into the performance and efficiency of the code, which can be useful for optimization purposes.

Concerning multiple files, it still depends on the available time, one potential use case could be when a user's code is divided into several modules or when the codebase is large ?? But is this really useful for the moment ?

Regarding the communication part, I would suggest using Protobuf for its efficient serialization and deserialization capabilities, as well as its strict and maintainable nature and also as you said, it could be reused if we manage to get virtio up and running-net and use gRPC further.

[charley04310]

Also, Muriel, would suggest that the "run" command should return the container ID, which should be used for all other commands (such as metrics, shutdown, etc.).

[lapsus-ord]

Also, Muriel, would suggest that the "run" command should return the container ID, which should be used for all other commands (such as metrics, shutdown, etc.).

We (agent) needs to run only one process, so I don't think it's useful to return an id for this.

[mmoreiradj]

After experimenting with communication over serial ports, here's my two cents:

Reminder:

• One vmm = One Agent
• Neither the VMM nor the Agent handle orchestration
• For now we only manage build and execution of code but we should prepare to run http servers for example in the future

Communication

The VMM and agent will exchange messages over the serial port until Virt-IO is ready to be implemented. To make sure each message is received, it should be request response based.

Messages Types

pub enum MessageType {
    Start = 0,
    Exit = 1,
    Interrupt = 2,
    Ok = 3,
    Log = 4,
}

/// Expects OkMessage
/// Sent when the Agent starts
pub struct StartMessage {
    content: String,
}

/// Expects OkMessage
/// Sent when the agent exits
pub struct ExitMessage {
    code: i32,
    content: String,
}

/// Expects OkMessage
/// Sent when the VMM wants to stop the agent
pub struct InterruptMessage {
    signal: i32,
}

/// Mostly used to answer other messages
pub struct OkMessage {}

/// Expects OkMessage
pub struct LogMessage {
    kind: String,
    content: String,
}

ATM I'm unsure of whether the LogMessage is useful since the program could simply send them to stdout, I'd like to have your input.

Message Format

Information	Length (in bits)
Message Type	16
Checksum	16
Data	...

We should go for this format, which would be easier to implement. It's the easiest format to make evolve.

The Message Type would identify what to serialize the data into:

pub enum MessageType {
    A = 0,
    B = 1,
    ...
}

For example, if the receiver gets a message type of 1, we would serialize the message in the struct B {}. It will also be used to determine what to answer with.

The checksum is simply here to make sure we didn't miss any data. We just need to determine what algorithm to use and the size of the checksum.

Please give me your input and whether you think we could implement that.

@virt-do/vmm @virt-do/guest-agent

[mmoreiradj]

Also, I forgot to mention that we should have a terminator to determine when the message is done.

Here's an idea:

Hex Value	Dec Value	Oct Value	Abbreviation	Description
0xC0	192	300	END	Frame End
0xDB	219	333	ESC	Frame Escape
0xDC	220	334	ESC_END	Transposed Frame End
0xDD	221	335	ESC_ESC	Transposed Frame Escape

[lapsus-ord]

I have questions, only for the definition, not code related:

1. What is the purpose of the content field in the StartMessage and ExitMessage? Is it to send the program content, only in StartMessage, why for ExitMessage?
2. Does the structure you send refers only to the request messages? Do you have an idea on the format for the responses?
   (If they're only for requests, we should refer to them as StartRequest, ExitRequest, etc)
3. Do you think we should send these messages over IP? (with the SLIP protocol, for a smooth transition with virtio-net in the future)
4. What does the OkMessage is for? To respond to requests with an Ok/Err?

I also think we should consider the protobuf de/serialization and/or grpc protocol over the slip, maybe it could be easier, but I wonder if it adds too much overhead (since it's using http) and if it's possible over serial

[mmoreiradj]

1. You can leave them empty, it's just if we need to send data, config or anything else
2. They are for request and responses
3. If you manage to implement it and provide a small proof of concept, I'd love to.
4. It's to make sure the request was received.

Again, if you manage to make a proof of concept, I'd love to do it.

For information, here's a POC of what I described: https://github.com/mmoreiradj/serial-port-demo