search

firmata / protocol

Documentation of the Firmata protocol.
996 stars 151 forks source link

Support to query and signal whether arduino has been resettet / reinitialized #4

Open ntruchsess opened 10 years ago

ntruchsess commented 10 years ago

Szenario to be addressed: Arduino is connected by other means than serial and connection drops and reconnects. There's no way to know whether firmata did reset while being disconnected. To ensure synchronized state between firmata and host the host has to reinitialize all pins and features.

What is needed is some kind of session-token that can be both used to query whether firmata has been resetted (which would invalidate the session-token on firmata-side) and to signal firmata became uninitialized (either by physical reset or other means like watchdog).

This is related to: https://github.com/firmata/protocol/issues/3 https://github.com/firmata/arduino/issues/102

soundanalogous commented 10 years ago

Do you have any specific implementation ideas in mind?

soundanalogous commented 8 years ago

This is clearly needed now that a few different transports are available in Firmata. The first step is deciding what is needed. It doesn't need to be fully comprehensive but should be scalable to add new functionality as needed.

To start we'd need at least the following:

The difference between 'connected' and 'reconnected' is that on 'connected' the user can assume the state of board is newly initialized, but upon receipt of 'reconnected' the user assumes that board already has state, which should be queried.

I'm not exactly sure what this should look like yet. I'd ideally like it to be covered by a single command (but can have multiple sub-commands - one for each feature / event) but don't have any clever ideas for what to call this (FIRMATA_CONNECTION?) nor have I thought much about how to structure it yet.

jnsbyr commented 8 years ago

Maybe it should be called CONNECTION_STATUS with the event/notification type as 1st parameter:

  1. Initial connect after boot
  2. Reconnect, optionally with connection counter [2 byte]?
  3. Alive/Watchdog + period [seconds, byte], optionally with sequence counter [byte]?

For the Alive/Watchdog one has to determine if it is should be an independent message for both sides (this can ensure that both sides will sooner or later notice when a connection is down regardless of their transmit behaviour), or a host initiated message that the Firmata device can rely on to be received and that it replies to with an echo.

soundanalogous commented 8 years ago

what is the sequence counter for the alive/watchdog?

Having the alive/watchdog be independent for both sides could be useful since either side could disconnect (the user restarting the host or the board losing connection or crashing and the host is still running).

jnsbyr commented 8 years ago

what is the sequence counter for the alive/watchdog?

A sequence counter typically starts with zero and is incremented by one with each transmitted alive/watchdog message until it overflows. It is especially useful on connectionless links like raw serial and UDP. If a received sequence number differs by more than one data has already been lost and depending on your application you may decide to resync. Another way to use it is a master/slave mode where the master increments the sequence counter and the slave replies with the sequence counter it receives from the master. A side effect of a sequence counter is a constantly varying value if you look at logging that is similar to a timestamp.

If you intend to check for a watchdog timeout (e.g. double agreed interval) on the receiver side you don't really need a sequence counter.

jnsbyr commented 8 years ago

All variants of the connection message have event character and are sent without query.

(A) The reporting sequence after a connect/reconnect should be fixed, e.g.

  1. connect (connection oriented host link) or transmitter ready (connectionless host link)
  2. transmit of protocol version from Firmata device to host, required
  3. transmit of the connection message from Firmata device to host, required
  4. transmit of current state or missed changes from Firmata device to host, optional, only on reconnect

A configuration option is needed to disable the reconnect message for using new Firmata releases with existing host applications that cannot tolerate this new message.

(B) For the alive/watchdog message the transmit interval must be configurable on both sides. A value higher than zero enables sending the message and expecting the message. As long as no message has been received it is assumed that the other side sends with the same interval. With receiving the first message the interval value from the message is used to check the receive timeout. The first interval starts when the receive is initialized if using a connectionless host links and with the connect if using a connection oriented host links. The first transmit is initiated at the end of the first interval. A receive timeout occurs after e.g. 10 times the interval has passed without receiving a message. With a connection oriented host link the connection is explicitly disconnected after a receive timeout and if using a client connection type a new connection is initiated.

Another option is to use the watchdog to establish a logical connect with connectionless host links.Then the alive/watchdog has precedence over (A) and the first transmit should occur when the transmitter is ready.

jnsbyr commented 8 years ago

Here is my proposal for a watchdog message:

To monitor the state of the host connection a watchdog message is sent independently and periodically from both sides. If a watchdog message is pending for more than double the period the connection should be treated as disconnected.

This message is especially useful for connectionless links (e.g. serial) where a disconnect cannot (easily) be detected on the link layer, but it also improves connection oriented links (e.g. TCP) by speeding up a link layer disconnect/reconnect. Additionally the watchdog message monitoring can detect application level delays/hangs of the peer even if the link layer is not disconnected.

The watchdog message has a sequence counter that is incremented each period after the connection is initially established on application level. Any change of a received sequence counter by more than one (apart from the wrap around) should be interpreted as a disconnect/reconnect. Receiving a sequence value of 0 should be interpreted as a reboot/restart of the other side.

The watchdog message transmission and monitoring is disabled by default for backward compatibility and can be enabled by application configuration. It can also be enabled and disabled by a watchdog message. The watchdog period received from the host overrides the application default.

The first watchdog message should be send after 

0  START_SYSEX     (0xF0)
1  WATCHDOG        (0x59)
2  sequence LSB    counter, 0-16283, starts with 0 at boot, incremented by one, wraps around to 1
3  sequence MSB     
4  period   LSB    period of transmission, 1-16283 seconds, 0 disables watchdog transmission/monitoring
5  period   MSB
6  END_SYSEX       (0xF7)

With connectionless links the watchdog monitor of the Firmata device should call the hostConnectionCallback directly to notify connect and disconnect events while with connection oriented links (e.g. TCP) it should only close the connection.

soundanalogous commented 8 years ago

I'm not sure if we need to send the period with every transmission, but if we eliminated it we'd only save one byte since we'd need an extra subcommand to identify config vs ping so one extra byte per transmission is not so bad I guess.

I'm not sure we'd need to send the message when a Firmata device reports its protocol version since this is a new feature and any implementation would also implement the connect / reconnect message. Unless the idea is that with the watchdog there is no need for connect/reconnect because it can be derived from the watchdog broadcast.

jnsbyr commented 8 years ago

I'm not sure we'd need to send the message when a Firmata device reports its protocol version ...

Sending a periodic message has to start somewhere. My idea was that it should not be before the protocol version message because the host needs to know if the Firmata device supports this feature or not. For this I also assumed that the host supports devices running older protocol versions providing full backward compatibility. Another way to do it would be a watchdog capability query or a watchdog enable request but both variant require more implementation overhead and additional communication round trips.

Considering the reason of this thread ("... whether arduino has been resettet / reinitialized") it is still hard to figure out if a Firmata device was reset based on the watchog message I proposed, because connection monitoring is not the same as init state reporting. What is still missing is a flag or a status byte that reports if the Firmata device was initialized by the host. This flag could be set if the Firmata device has received at least one configuration command from the host. The host could set this flag after it has configured the Firmata device. This way both sides can determine if the other side was booted.

The flag could be one bit in the counter value or the period value if telegram size is the priority. Spending another byte seems oversized, especially on a periodic message, but leaves room for future use.

There is another aspect: With "only" an init flag there is no way for both sides to be sure they are talking to the same peer after a reconnect. This would require an explicit session token. From my point of view this enhancement will not be required for most implementations but it should be taken into account and decided.

dmag commented 8 years ago

A related issue: If a device gets disconnected, is there a way to automatically turn off lights, motors, etc? This seems like a basic safety requirement for anything you want people to build real robots on top of. But I don't see it discussed anywhere. Am I missing something?

jnsbyr commented 8 years ago

The WiFi variants of Firmata already have the new hostConnectedCallback feature that may be used to do just that. A simple example can be found here. You would have to add your custom shutdown code to the HOST_CONNECTION_DISCONNECTED case. This is necessary because the host cannot do it while disconnected and it depends on your project what IOs to modify.

The reasons for this protocol enhancement request is to make sure that a host disconnect is detected as soon as possible (and the host connection callback is triggered) regardless of the host connection type and that both device and host can tell the difference between reboot and reconnect.