Endless wait if timing on REPORT_VERSION is missed

[ajfisher]

I've been using firmata between a Raspberry Pi and an Arduino Uno and over USB it works absolutely fine (brilliantly in fact).

Where I'm now having a problem is that if I use a direct UART serial connection between the GPIO on the RPi and the Arduino it doesn't cause a hardware reset (which DOES happen if you connect via USB - which is because the USB->Serial chip is tied to the hardware reset pin on the arduino). The implication of this is that node firmata is waiting for the REPORT VERSION header to come through before it continues and does anything else - of course if the arduino doesn't reset then that never eventuates.

@soundanalogous talks about this problem with Firmata clients in this blog post http://www.jeffhoefs.com/2013/06/wireless-firmata-using-xbees/ then proceeds to hack some xbees to make them work (this was the only reference I could find to this problem).

At this point I'm not sure a hardware hack is entirely viable - not one that is any more appropriate than just manually resetting the arduino when you start the node application anyway.

Resetting the arduino fixes the issue correctly every time however I'm wondering if there's an approach that's viable in software. My understanding of the firmata protocol is pretty weak but I have managed to trace it down to this bit of code:

https://github.com/jgautier/firmata/blob/master/lib/firmata.js#L339

this.sp.on('data', function(data) {
    var byt, cmd;

    if (!board.versionReceived && data[0] !== REPORT_VERSION) {
        return;
    } else {
        board.versionReceived = true;
    }

Where the return on failing having the version received and the data not being the version info is what seems to be causing the problem.

For comparison (not suggesting this is a better approach just a different one that happens to work in this specific case). The python firmata client doesn't wait for the header message, it just waits for a period of time then assumes you're connected. There's definite flaws in this approach and you can see the TODOs here showing there's problems to be resolved.

https://github.com/tino/pyFirmata/blob/master/pyfirmata/pyfirmata.py#L74

def __init__(self, port, layout, baudrate=57600, name=None):
    self.sp = serial.Serial(port, baudrate)
    # Allow 5 secs for Arduino's auto-reset to happen
    # Alas, Firmata blinks it's version before printing it to serial
    # For 2.3, even 5 seconds might not be enough.
    # TODO Find a more reliable way to wait until the board is ready
    self.pass_time(BOARD_SETUP_WAIT_TIME)
    self.name = name
    if not self.name:
        self.name = port
    self.setup_layout(layout)

I'm more than happy to keep digging with this and prepared to try and build a fix but if there's a view on what might be an appropriate approach for resolving this then that may get me to a patch faster.

[soundanalogous]

One solution is to add a listener for the open event from the serial port sp.on("open", function (e) {})

At that time start a timer and if upon timeout board.versionReceived is still false, call the reportVersion method. That way you can continue to support auto reset hardware with a fallback to hardware that does not support auto reset (such as using the UART on a raspberry pi as you describe).

You can also reduce the startup time in StandardFirmata by passing a reference to the Serial stream to the Firmata.begin function. Rather than calling Firmata.begin(57600) in the setup() method of StandardFirmata do this (this is also the same technique you'd use to use Ethernet, WiFi, another Hardware Serial port such as on Arduino Mega, etc):

Serial.begin(57600);
Firmata.begin(Serial);

This will bypass the Firmata startup blink routine (but not the bootloader blink routine), shaving 1.6 seconds off the start time. You'll need the latest stable version of Firmata (2.3.6) installed in your Arduino application (be sure to note the new library directory structure if you're using Arduino 1.5.2) for this to work. You can also update the bootloader (advanced technique) to a faster one without the annoying blink sequence. I saw instructions a while ago... I think on adafruit or sparkfun. That could potentially shave startup time down to a few milliseconds rather than the 5 seconds the python library author is trying to deal with.

[rwaldron]

Does this address the issue? https://github.com/jgautier/firmata/pull/27

[soundanalogous]

When the reportVersion method is called immediately upon opening the serial port there is still the risk that you will not get a response because Firmata is running the blink routine at that time, calling a sequence of blocking delay functions. The Arduino uart will still process incoming serial data even when in the middle of a delay but it can't send a response (at least I don't think it can and likely not reliably). The input stream is buffered though so the firmware may still be able to respond appropriately after the blink sequence completes. I recommend testing #27 to be sure that the reportVersion response is received 100% of the time. If it is not, a delay may be necessary.

[rwaldron]

@soundanalogous thanks!

[ajfisher]

@rwldrn thanks for pointing that PR out - I've grabbed it down and it doesn't seem to be working in this instance. The port is opening but I think it's proving susceptible to the timing issue that @soundanalogous has pointed out and the fact that the arduino may be receiving it but can't respond.

I'll put some more notes over at #27 as this seems to be trying to fix this issue and will close this thread.