meshtastic / network-management-client

A Meshtastic desktop client, allowing simple, offline deployment and administration of an ad-hoc mesh communication network. Built in Rust and TypeScript.
GNU General Public License v3.0
189 stars 15 forks source link

Prototype and Data Models #18

Closed uhuruhashimoto closed 2 years ago

barkincavdaroglu commented 2 years ago

Data Types

On admin’s side, we want to to save data in json format represented below:

AdminDB

{
    "nodes": list[int],
    "nodes_data": list[NodeData]
}

NodeData

{
    "node_id": int,
    "gps_history": list[Position],
    "speed_history": list[float],
    "signal_quality": map[node_id, list[Signal]],
    "firmware_version": string,
    "device_model": string,
    "device_capabilities": list[string],
    "macid": string
}

Position is Meshtastic’s protobuf. NodeData is almost exactly the same as NodeInfo protobuf of Meshtastic but instead of containing point-in-time data it will hold the history of data.

Signal

{
    "signal_quality": float,
    "source_of_signal": node_id,
    "time_of_signal": int,
}

We will use Meshtastic’s protobufs to represent other data. We will especially make use of the highlighted cells.

Admin Message

Field   Type Description
oneof variant.set_owner User Set the owner for this node
oneof variant.set_channel Channel Set channels (using the new API). A special channel is the "primary channel". The other records are secondary channels. Note: only one channel can be marked as primary. If the client sets a particular channel to be primary, the previous channel will be set to SECONDARY automatically.
oneof variant.get_channel_request uint32 Send the specified channel in the response to this message NOTE: This field is sent with the channel index + 1 (to ensure we never try to send 'zero' - which protobufs treats as not present)
oneof variant.get_channel_response Channel
oneof variant.get_owner_request bool Send the current owner data in the response to this message.
oneof variant.get_owner_response User
oneof variant.get_config_request AdminMessage.ConfigType Ask for the following config data to be sent
oneof variant.get_config_response Config Send the current Config in the response to this message.
oneof variant.set_config Config Set the current Config
oneof variant.confirm_set_config bool Sent immediatly after a config change has been sent to ensure comms, if this is not recieved, the config will be reverted after 10 mins
oneof variant.get_module_config_request AdminMessage.ModuleConfigType Ask for the following config data to be sent
oneof variant.get_module_config_response ModuleConfig Send the current Config in the response to this message.
oneof variant.set_module_config ModuleConfig Set the current Config
oneof variant.confirm_set_module_config bool Sent immediatly after a config change has been sent to ensure comms, if this is not recieved, the config will be reverted after 10 mins
oneof variant.get_all_channel_request bool Send all channels in the response to this message
oneof variant.confirm_set_channel bool Setting channels/radio config remotely carries the risk that you might send an invalid config and the radio never talks to your mesh again. Therefore if setting either of these properties remotely, you must send a confirm_xxx message within 10 minutes. If you fail to do so, the radio will assume loss of comms and revert your changes. These messages are optional when changing the local node.
oneof variant.confirm_set_radio bool
oneof variant.exit_simulator bool This message is only supported for the simulator porduino build. If received the simulator will exit successfully.
oneof variant.reboot_seconds int32 Tell the node to reboot in this many seconds (or <0 to cancel reboot)
oneof variant.get_canned_message_module_messages_request bool Get the Canned Message Module messages in the response to this message.
oneof variant.get_canned_message_module_messages_response string Get the Canned Message Module messages in the response to this message.
oneof variant.set_canned_message_module_messages string Set the Canned Message Module messages text.
oneof variant.shutdown_seconds int32 Tell the node to shutdown in this many seconds (or <0 to cancel shutdown)
oneof variant.get_device_metadata_request uint32 Request the node to send device metadata (firmware, protobuf version, etc)
oneof variant.get_device_metadata_response DeviceMetadata Device metadata response

Position Config

Field Type Description
position_broadcast_secs uint32 We should send our position this often (but only if it has changed significantly) Defaults to 15 minutes
position_broadcast_smart_disabled bool Disable adaptive position braoadcast, which is now the default.
fixed_position bool If set, this node is at a fixed position. We will generate GPS position updates at the regular interval, but use whatever the last lat/lon/alt we have for the node. The lat/lon/alt can be set by an internal GPS or with the help of the app.
gps_disabled bool Should the GPS be disabled for this node?
gps_update_interval uint32 How often should we try to get GPS position (in seconds) or zero for the default of once every 30 seconds or a very large value (maxint) to update only once at boot.
gps_attempt_time uint32 How long should we try to get our position during each gps_update_interval attempt? (in seconds) Or if zero, use the default of 30 seconds. If we don't get a new gps fix in that time, the gps will be put into sleep until the next gps_update_rate window.
position_flags uint32 Bit field of boolean configuration options for POSITION messages (bitwise OR of PositionFlags)

Config.PositionConfig.PositionFlags

Name Number Description
POS_UNDEFINED 0 Required for compilation
POS_ALTITUDE 1 Include an altitude value (if available)
POS_ALT_MSL 2 Altitude value is MSL
POS_GEO_SEP 4 Include geoidal separation
POS_DOP 8 Include the DOP value ; PDOP used by default, see below
POS_HVDOP 16 If POS_DOP set, send separate HDOP / VDOP values instead of PDOP
POS_SATINVIEW 32 Include number of "satellites in view"
POS_SEQ_NOS 64 Include a sequence number incremented per packet
POS_TIMESTAMP 128 Include positional timestamp (from GPS solution)
POS_HEADING 256 Include positional heading Intended for use with vehicle not walking speeds walking speeds are likely to be error prone like the compass
POS_SPEED 512 Include positional speed Intended for use with vehicle not walking speeds walking speeds are likely to be error prone like the compass

Device State

Field Type Description
my_node MyNodeInfo Read only settings/info about this node
owner User My owner info
node_db NodeInfo
receive_queue MeshPacket Received packets saved for delivery to the phone
version uint32 A version integer used to invalidate old save files when we make incompatible changes This integer is set at build time and is private to NodeDB.cpp in the device code.
rx_text_message MeshPacket We keep the last received text message (only) stored in the device flash, so we can show it on the screen. Might be null
no_save bool Used only during development. Indicates developer is testing and changes should never be saved to flash.
did_gps_reset bool Some GPSes seem to have bogus settings from the factory, so we always do one factory reset.

Local Config

Field Type Description
device Config.DeviceConfig The part of the config that is specific to the Device
position Config.PositionConfig The part of the config that is specific to the GPS Position
power Config.PowerConfig The part of the config that is specific to the Power settings
wifi Config.WiFiConfig The part of the config that is specific to the Wifi Settings
display Config.DisplayConfig The part of the config that is specific to the Display
lora Config.LoRaConfig The part of the config that is specific to the Lora Radio
bluetooth Config.BluetoothConfig The part of the config that is specific to the Bluetooth settings
version uint32 A version integer used to invalidate old save files when we make incompatible changes This integer is set at build time and is private to NodeDB.cpp in the device code.

Data

Field Type Description
portnum PortNum Formerly named typ and of type Type
payload bytes
want_response bool Not normally used, but for testing a sender can request that recipient responds in kind (i.e. if it received a position, it should unicast back it's position). Note: that if you set this on a broadcast you will receive many replies.
dest fixed32 The address of the destination node. This field is is filled in by the mesh radio device software, application layer software should never need it. RouteDiscovery messages must populate this. Other message types might need to if they are doing multihop routing.
source fixed32 The address of the original sender for this message. This field should only be populated for reliable multihop packets (to keep packets small).
request_id fixed32 Only used in routing or response messages. Indicates the original message ID that this message is reporting failure on. (formerly called original_id)
reply_id fixed32 If set, this message is intened to be a reply to a previously sent message with the defined id.
emoji fixed32 Defaults to false. If true, then what is in the payload should be treated as an emoji like giving a message a heart or poop emoji.

Mesh Packet

Field Type Description
from fixed32 The sending node number. Note: Our crypto implementation uses this field as well. See crypto for details. FIXME - really should be fixed32 instead, this encoding only hurts the ble link though.
to fixed32 The (immediatSee Priority description for more details.y should be fixed32 instead, this encoding only hurts the ble link though.
channel uint32 (Usually) If set, this indicates the index in the secondary_channels table that this packet was sent/received on. If unset, packet was on the primary channel. A particular node might know only a subset of channels in use on the mesh. Therefore channel_index is inherently a local concept and meaningless to send between nodes. Very briefly, while sending and receiving deep inside the device Router code, this field instead contains the 'channel hash' instead of the index. This 'trick' is only used while the payloadVariant is an 'encrypted'.
oneof payloadVariant.decoded Data
oneof payloadVariant.encrypted bytes
id fixed32 A unique ID for this packet. Always 0 for no-ack packets or non broadcast packets (and therefore take zero bytes of space). Otherwise a unique ID for this packet, useful for flooding algorithms. ID only needs to be unique on a per sender basis, and it only needs to be unique for a few minutes (long enough to last for the length of any ACK or the completion of a mesh broadcast flood). Note: Our crypto implementation uses this id as well. See crypto for details. FIXME - really should be fixed32 instead, this encoding only hurts the ble link though.
rx_time fixed32 The time this message was received by the esp32 (secs since 1970). Note: this field is never sent on the radio link itself (to save space) Times are typically not sent over the mesh, but they will be added to any Packet (chain of SubPacket) sent to the phone (so the phone can know exact time of reception)
rx_snr float Never* sent over the radio links. Set during reception to indicate the SNR of this packet. Used to collect statistics on current link quality.
hop_limit uint32 If unset treated as zero (no forwarding, send to adjacent nodes only) if 1, allow hopping through one node, etc... For our usecase real world topologies probably have a max of about 3. This field is normally placed into a few of bits in the header.
want_ack bool This packet is being sent as a reliable message, we would prefer it to arrive at the destination. We would like to receive a ack packet in response. Broadcasts messages treat this flag specially: Since acks for broadcasts would rapidly flood the channel, the normal ack behavior is suppressed. Instead, the original sender listens to see if at least one node is rebroadcasting this packet (because naive flooding algorithm). If it hears that the odds (given typical LoRa topologies) the odds are very high that every node should eventually receive the message. So FloodingRouter.cpp generates an implicit ack which is delivered to the original sender. If after some time we don't hear anyone rebroadcast our packet, we will timeout and retransmit, using the regular resend logic. Note: This flag is normally sent in a flag bit in the header when sent over the wire
priority MeshPacket.Priority The priority of this message for sending. See MeshPacket.Priority description for more details.
rx_rssi int32 rssi of received packet. Only sent to phone for dispay purposes.
delayed MeshPacket.Delayed Describe if this message is delayed

NodeInfo

The Bluetooth to device link.

Field Type Description
num uint32 The node number
user User The user info for this node
position Position This position data. Note: before 1.2.14 we would also store the last time we've heard from this node in position.time, that is no longer true. Position.time now indicates the last time we received a POSITION from that node.
snr float Returns the Signal-to-noise ratio (SNR) of the last received message, as measured by the receiver. Return SNR of the last received message in dB
last_heard fixed32 Set to indicate the last time we received a packet from this node
device_metrics DeviceMetrics The latest device metrics for the node.

MyNodeInfo

Field Type Description
my_node_num uint32 Tells the phone what our node number is, default starting value is lowbyte of macaddr, but it will be fixed if that is already in use
has_gps bool Note: This flag merely means we detected a hardware GPS in our node. Not the same as UserPreferences.location_sharing
max_channels uint32 The maximum number of 'software' channels that can be set on this node.
firmware_version string 0.0.5 etc...
error_code CriticalErrorCode An error message we'd like to report back to the mothership through analytics. It indicates a serious bug occurred on the device, the device coped with it, but we still want to tell the devs about the bug. This field will be cleared after the phone reads MyNodeInfo (i.e. it will only be reported once) a numeric error code to go with error message, zero means no error
error_address uint32 A numeric error address (nonzero if available)
error_count uint32 The total number of errors this node has ever encountered (well - since the last time we discarded preferences)
reboot_count uint32 The total number of reboots this node has ever encountered (well - since the last time we discarded preferences)
bitrate float Calculated bitrate of the current channel (in Bytes Per Second)
message_timeout_msec uint32 How long before we consider a message abandoned and we can clear our caches of any messages in flight Normally quite large to handle the worst case message delivery time, 5 minutes. Formerly called FLOOD_EXPIRE_TIME in the device code
min_app_version uint32 The minimum app version that can talk to this device. Phone/PC apps should compare this to their build number and if too low tell the user they must update their app
air_period_tx uint32 24 time windows of 1hr each with the airtime transmitted out of the device per hour.
air_period_rx uint32 24 time windows of 1hr each with the airtime of valid packets for your mesh.
has_wifi bool Is the device wifi capable?
channel_utilization float Utilization for the current channel, including well formed TX, RX and malformed RX (aka noise).
air_util_tx float Percent of airtime for transmission used within the last hour.

Position

Field Type Description
latitude_i sfixed32 The new preferred location encoding, divide by 1e-7 to get degrees in floating point
longitude_i sfixed32
altitude int32 In meters above MSL (but see issue #359)
time fixed32 This is usually not sent over the mesh (to save space), but it is sent from the phone so that the local device can set its RTC If it is sent over the mesh (because there are devices on the mesh without GPS), it will only be sent by devices which has a hardware GPS clock. seconds since 1970
location_source Position.LocSource
altitude_source Position.AltSource
pos_timestamp fixed32 Positional timestamp (actual timestamp of GPS solution) in integer epoch seconds
pos_time_millis int32 Pos. timestamp milliseconds adjustment (rarely available or required)
altitude_hae sint32 HAE altitude in meters - can be used instead of MSL altitude
alt_geoid_sep sint32 Geoidal separation in meters
PDOP uint32 Horizontal, Vertical and Position Dilution of Precision, in 1/100 units - PDOP is sufficient for most cases - for higher precision scenarios, HDOP and VDOP can be used instead, in which case PDOP becomes redundant (PDOP=sqrt(HDOP^2 + VDOP^2)) TODO: REMOVE/INTEGRATE
HDOP uint32
VDOP uint32
gps_accuracy uint32 GPS accuracy (a hardware specific constant) in mm multiplied with DOP to calculate positional accuracy Default: "'bout three meters-ish" :)
ground_speed uint32 Ground speed in m/s and True North TRACK in 1/100 degrees Clarification of terms: - "track" is the direction of motion (measured in horizontal plane) - "heading" is where the fuselage points (measured in horizontal plane) - "yaw" indicates a relative rotation about the vertical axis TODO: REMOVE/INTEGRATE
ground_track uint32
fix_quality uint32 GPS fix quality (from NMEA GxGGA statement or similar)
fix_type uint32 GPS fix type 2D/3D (from NMEA GxGSA statement)
sats_in_view uint32 GPS "Satellites in View" number
sensor_id uint32 Sensor ID - in case multiple positioning sensors are being used
pos_next_update uint32 Estimated/expected time (in seconds) until next update: - if we update at fixed intervals of X seconds, use X - if we update at dynamic intervals (based on relative movement etc), but "AT LEAST every Y seconds", use Y
pos_seq_number uint32 A sequence number, incremented with each Position message to help detect lost updates if needed

Mesh Packet Priority

Name Number Description
UNSET 0 Treated as Priority.DEFAULT
MIN 1
BACKGROUND 10 Background position updates are sent with very low priority - if the link is super congested they might not go out at all
DEFAULT 64 This priority is used for most messages that don't have a priority set
RELIABLE 70 If priority is unset but the message is marked as want_ack, assume it is important and use a slightly higher priority
ACK 120 Ack/naks are sent with very high priority to ensure that retransmission stops as soon as possible
MAX 127

PortNum

Name Number Description
UNKNOWN_APP 0 Deprecated: do not use in new code (formerly called OPAQUE) A message sent from a device outside of the mesh, in a form the mesh does not understand NOTE: This must be 0, because it is documented in IMeshService.aidl to be so
TEXT_MESSAGE_APP 1 A simple UTF-8 text message, which even the little micros in the mesh can understand and show on their screen eventually in some circumstances even signal might send messages in this form (see below)
REMOTE_HARDWARE_APP 2 Reserved for built-in GPIO/example app. See remote_hardware.proto/HardwareMessage for details on the message sent/received to this port number
POSITION_APP 3 The built-in position messaging app. Payload is a Position message
NODEINFO_APP 4 The built-in user info app. Payload is a User message
ROUTING_APP 5 Protocol control packets for mesh protocol use. Payload is a Routing message
ADMIN_APP 6 Admin control packets. Payload is a AdminMessage message
TEXT_MESSAGE_COMPRESSED_APP 7 Compressed TEXT_MESSAGE payloads.
WAYPOINT_APP 8 Waypoint payloads. Payload is a Waypoint message
REPLY_APP 32 Provides a 'ping' service that replies to any packet it receives. Also serves as a small example module.
IP_TUNNEL_APP 33 Used for the python IP tunnel feature
SERIAL_APP 64 Provides a hardware serial interface to send and receive from the Meshtastic network. Connect to the RX/TX pins of a device with 38400 8N1. Packets received from the Meshtastic network is forwarded to the RX pin while sending a packet to TX will go out to the Mesh network. Maximum packet size of 240 bytes. Module is disabled by default can be turned on by setting SERIAL_MODULE_ENABLED = 1 in SerialPlugh.cpp.
STORE_FORWARD_APP 65 STORE_FORWARD_APP (Work in Progress) Maintained by Jm Casler (MC Hamster) : jm@casler.org
RANGE_TEST_APP 66 Optional port for messages for the range test module.
TELEMETRY_APP 67 Provides a format to send and receive telemetry data from the Meshtastic network. Maintained by Charles Crossan (crossan007) : crossan007@gmail.com
ZPS_APP 68 Experimental tools for estimating node position without a GPS Maintained by Github user a-f-G-U-C (a Meshtastic contributor) Project files at https://github.com/a-f-G-U-C/Meshtastic-ZPS
PRIVATE_APP 256 Private applications should use portnums >= 256. To simplify initial development and testing you can use "PRIVATE_APP" in your code without needing to rebuild protobuf files (via regen-protos.sh)
ATAK_FORWARDER 257 ATAK Forwarder Module https://github.com/paulmandal/atak-forwarder
MAX 511 Currently we limit port nums to no higher than this value

Source: https://meshtastic.org/docs/developers/protobufs/api