Definition AppendEntries.1.0.dsdl

[maksimdrachov]

For AppendEntries, what type to use for the entries variable?

From paper:

I'm thinking something like this:

# This service is used for the AppendEntries RPC

uint64 term
uint64 leaderID
uint64 prevLogIndex
uint64 prevLogTerm
LogEntry.1.0[<64] entries
uint64 leaderCommit

@sealed

---

uint64 term
bool success

@sealed

# This type is used to define a LogEntry for AppendEntries RPC
uint64 term
uint8[<=256] name
uint64 value
@sealed

[pavel-kirienko]

A log entry is essentially a row in our distributed database. Since we store mappings between topic names and their ID (or names of arbitrary numerical resources), we need two fields: name and value, as you described.

I would decouple your term value from the entry structure, if possible, to avoid leaky abstraction. Maybe you need two types for this: the pure Entry, and a LogEntry that contains the former entry plus the term. There may be better names for these entities.

In the interest of minimizing the memory footprint we may want to reduce the capacity of the entries array, as 256*64=16384 bytes is a lot of bytes. Maybe just one would be enough?

Also, please replace uint8[<=256] name with uavcan.primitive.String.1.0 name.

[maksimdrachov]

Ok, so:

AppendEntries.1.0.dsdl:

# This service is used for the AppendEntries RPC

uint64 term
uint64 leader_id
uint64 prev_log_index
uint64 prev_log_term
LogEntry.1.0 log_entry
uint64 leader_commit

@sealed

---

uint64 term
bool success

@sealed

LogEntry.1.0.dsdl:

# This type is used to define a LogEntry for AppendEntries RPC

uint64 term
Entry.1.0 entry
@sealed

Entry.1.0.dsdl:

# This type is used to define an Entry in the Raft database

uavcan.primitive.String.1.0 topic
uint64 topic_id
@sealed

Note: term is passed twice (in AppendEntries and in LogEntry). This is required if a list of log entries can be passed at once, however in our case unnecessary, correct?

[pavel-kirienko]

AppendEntries

• The leader-ID is not necessary because it is the same as the Cyphal node-ID. It is provided by PyCyphal to your service call callback as metadata.
• I would recommend moving the leader commit before the log_entry in the interest of following best DSDL practices --- fixed-size fields are best grouped together in the beginning of the definition for faster serialization. In this case performance will hardly matter but we should encourage best design whenever possible.

Entry

What if we keep name and value instead of topic and topic-ID to keep things generic and reusable for distributed storage of any other named numbers on the network? Like service names or configuration parameters. These may come useful later.

This is required if a list of log entries can be passed at once, however in our case unnecessary, correct?

You are more equipped to answer this question as you read the Raft spec just recently, but IIRC these are different terms. The Entry term signifies when the entry was created, and the top-level term is the latest term seen by the node sending the AppendEntries request. No?

[maksimdrachov]

AppendEntries.1.0.dsdl:

# This service is used for the AppendEntries RPC

uint64 term
uint64 prev_log_index
uint64 prev_log_term
uint64 leader_commit
LogEntry.1.0 log_entry

@sealed
---

uint64 term
bool success

@sealed

LogEntry.1.0.dsdl:

# This type is used to define a LogEntry for AppendEntries RPC

uint64 term
Entry.1.0 entry
@sealed

Entry.1.0.dsdl:

# This type is used to define an Entry in the raft database

uavcan.primitive.String.1.0 name
uint64 value
@sealed

You are more equipped to answer this question as you read the Raft spec just recently...

Yeah, I think you're right. :|

Thumb up/down on the above?

[maksimdrachov]

What if we keep name and value instead of topic and topic-ID to keep things generic and reusable for distributed storage of any other named numbers on the network? Like service names or configuration parameters. These may come useful later.

I was thinking along the same lines initially, used name and id, however id become id_ after compilation. I guess that's a protected variable name?

[pavel-kirienko]

I guess that's a protected variable name?

Yes, it's a built-in name in Python:

If the source definition contains identifiers, type names, namespace components, or other entities whose names are listed in nunavut.lang.py.PYTHON_RESERVEDIDENTIFIERS, the compiler applies stropping by suffixing such entities with an underscore . A small subset of applications may require access to a generated entity without knowing in advance whether its name is a reserved identifier or not (i.e., whether it’s stropped or not). To simplify usage, this submodule provides helper functions pycyphal.dsdl.get_attribute() and pycyphal.dsdl.set_attribute() that provide access to generated class/object attributes using their original names before stropping. Likewise, the function pycyphal.dsdl.get_model() can find a generated type even if any of its name components are stropped; e.g., a DSDL type str.Type.1.0 would be imported as str_.Type_1_0