Reading from GPIO takes ~80ms

[mxbi]

When running the following code:

r = Robot()
gpio = r.servo_board.gpios[3]
while True:
     gpio.read()

The gpio.read() operation consistently takes ~80ms to run, meaning GPIO can only read signals at about 10Hz. Looking through the source code, it appears the read() operation is encoding a message as JSON, sending it over to the serial to the ardiuno (where the arduino supposedly has to decode it, do a trivial GPIO read, encode another message), and then sent back to the pi where it's then decoded again.

https://github.com/sourcebots/robot-api/blob/master/robot/board.py#L112-L129

Is there a faster way of doing this? Seems like quite a lot of overhead to just read a pin. We're trying to use rotary encoders but we'd need to read a 250Hz square wave (2 or 3 milliseconds for reading the GPIO) - this would be easy on the Arduino itself or using the Pi's GPIO where a read takes just a few microseconds.

Are there any other solutions that would speed this operation up? Are we allowed to just use the Pi's GPIO and read from that without the expensive overhead?

Thanks,
Mikel

[kierdavis]

You're right that there is a lot of overhead here. I don't think we've benchmarked it but 80ms round-trip time wouldn't surprise me.

We don't allow access to the Pi's GPIO pins since there's too much potential for the Pi to be damaged this way, and replacing a Pi is somewhat more expensive than replacing an Arduino (and would also prevent you from doing any development on your robot until it gets replaced).

Your other suggestion, however, is spot on - the recommended way to solve this would be to extend the Arduino's software to handle everything that needs to be done at high frequency, and use the Pi to poll the Arduino only when you actually need the information.

While the software on the Arduino can be changed in a matter of minutes (it's just a standard Arduino sketch), it's difficult to extend the API since it requires changing things in the Pi's SD card image (which is something we don't support).

Currently, a workaround could be "hijack" an existing part of the protocol. For example, it's unlikely you're using all 16 servos, so you could send a value to the Arduino by writing it to a particular servo channel. The Arduino code could then be set up so that when it receives a request to set this servo channel to some value, it calls your code rather than actually moving the corresponding servo. It's not ideal, but it should work.

In theory you could also modify the relevant parts of the Pi's SD card image (robot-api and robotd). However we haven't yet come to a decision on whether this is allowable within the rules; since the Pis have WiFi and Bluetooth connectivity, allowing custom Pi images would lower the barrier to cheating.

Making it easier to extend the Arduino's functionality is near the top of our to-do list, and might be making its way into your kit via an update in a few weeks time. There are also larger improvements planned that should improve the performance of gpio.read() and other functions, but these probably won't arrive until after the competition.

Kudos for going ahead and digging through our code - that's the kind of behaviour we'd like to encourage here!

[mxbi]

Thanks for the idea of just extending the code running on the Arduino, I hadn't thought of that :) That may be a viable solution for us. Maybe exposing some way to just arbitrary send information between the pi and arduino would be helpful.

Or what about having the ardiuno copy its input pins to its output pins and then connecting the arduino to the Pi's GPIO? It would solve the latency issue and I guess it would still protect the pi from getting fried (as you're only connecting it to the arduino)

As for allowing custom pi images, I personally think it would be really helpful (eg. with things like installing additional packages). Right now on our Pi I've installed a few extra packages like matplotlib to create plots of what the camera sees - obviously this isn't necessary for the actual competition but it's a little restrictive to say we can't modify anything on the pi. (Don't worry, I imaged the SD card before changing anything in case I screwed something up)

If someone was really intent on cheating they could just run dchpcd & ssh at the start of their script to bring wifi up on a fresh pi image (yes, it's a bit extreme, but so is trying to secretly remote control it with a custom image). If there was some way to prevent wifi/bluetooth connectivity despite letting people run what they want on the pi, it would be ideal.

Personally, in our team we have used Pis a lot, so we would feel comfortable modifying the software or interfacing directly with the GPIO (using our own Pis if you are worried about the sourcebots ones getting fried) - as long as it's not against the rules.

[kierdavis]

Maybe exposing some way to just arbitrary send information between the pi and arduino would be helpful.

Yeah, this is more or less what I'm thinking in terms of making the kit more friendly to extending the arduino software.

Or what about having the ardiuno copy its input pins to its output pins and then connecting the arduino to the Pi's GPIO?

I like your ingenuity, but this doesn't solve the problem of someone shorting out the Pi's power supply while (mis)wiring it up, which is likely a more common situation.

As for allowing custom pi images, I personally think it would be really helpful (eg. with things like installing additional packages).

I agree with your intent here - there are situations were it would be useful to have software installed or features present that aren't in the standard image. I think approach would be to provide another interface to do these than changing the Pi image (which is a rather inconvenient process anyway) - e.g. allowing the USB stick to contain a folder of extra Python packages to install.

(Don't worry, I imaged the SD card before changing anything in case I screwed something up)

If/when you want to restore the Pi to the state it was in when you received it, the SD card image that was shipped at Kickstart can be downloaded here.

If someone was really intent on cheating they could just run dchpcd & ssh at the start of their script to bring wifi up on a fresh pi image

For what it's worth, this wouldn't actually work :) This is because user code is run in an unprivileged container with no networking. If you're interested in the gory details, see the code that starts the container and the manual for systemd-nspawn.

This is related to one of the reasons against allowing people to change the Pi image, which is that this protection would be bypassed (i.e. the containerisation is worthless if users have access to the host system anyway).

[mxbi]

Thanks for the info :)

We'll look into doing the GPIO-heavy work directly on the Arduino (although since I guess anything we do on it will be blocking, we can't run extended operations on another thread like in python) - maybe using the extra servo channels like you suggested.

If, say, we needed to send some data back from the arduino to the pi, would it be possible to use something like:

r.servo_boards[0].send_and_receive(payload)

and then modify the CommandHandler in the arduino firmware to add another function? This seems like the cleanest way to do it, but maybe you have some tips?

[Adimote]

Hey there,

Looking into this, send_and_receive probably doesn't do what you expect to. The very fact you can see that function is an error in our part.

Also using the servos is a bad idea if you want to actually get information, because the Arduino actually doesn't tell the Pi anything about the servos. The Pi just remembers what it last told the Arduino.

The way I'd recommend you do this is either through ultrasound or the Analogue pins (both of those are actually asking for a decimal number from the arduino.).

Assuming you use the Analogue method, I would do the following:

1. call: r.servo_board.read_analogue()["encoder_1"]

2. Modify the analogue_read function in the arduino firmware to add the line:

   serialWrite(commandId, '>', "encoder_1"+ " " + String(wheelEncoderSpeed));

   where wheelEncoderSpeed is the wheel encoder speed as a double value.

If you need more help with how to modify the Arduino firmware, just ask!

[mxbi]

So I tried using the method I suggested above, defining a function and adding it to the CommandHandler

static CommandError test_func(int commandId, String argument) {
  String arg1 = pop_option(argument);
  serialWrite(commandId, '>', arg1); // Copied from gpio-read, not sure how this works
  return OK;
}

static const CommandHandler commands[] = {
  ...,
  CommandHandler("test", &count_rotary_encoder, "please work"),
};

However, when calling it manually from the robot with:

r.servo_board.send_and_receive({'test': ['arg1']})

it doesn't seem to actually do anything on the arduino, and the output is the same as when sending a bunch of gibberish.

I noticed that the GPIO read operation is called gpio-read in the Arduino firmware, but the Pi actually sends read-pin to robotd. It looks like robotd is doing some translation somewhere which means I'm not sure how I can actually call this function I've defined on the arduino - I guess it would additionally need to be defined in robotd. (https://github.com/sourcebots/robotd/blob/cc1e692de7c9f028127a9daf4105494295b5eebc/robotd/devices.py#L362-L364). Since we're not allowed to modify robotd (and I think the only way to interact with it is through the socket, so no manually calling functions), this seems like a bit of a dead end.

I could override the servos function you provide, but it's unclear how to send information to and from the arduino - in an ideal scenario: 1) I'd be able to send multiple arguments to the arduino function (or a string where I could encode multiple arguments into one) 2) The arduino would be capable of sending back data to the Pi a few times per second (without exiting). No Pi -> arduino communication needed during this time 3) The function loop needs to stay running on the arduino during this time (or at least not get blocked for more than a couple ms) or signals will be missed and the calculations will be off.

We would be really restricted without this ability, because we want to control the motor board based on realtime sensor inputs to the arduino - the workaround I was trying to do is to do the signal processing on the arduino and then send some computed results back to the pi 5-10 times a second or so, which the pi can then use to do motion planning.

Do you have any pointers about how we might achieve this? Of course, it could all be avoided by simply using the Pi's gpio, so it's a little annoying that this isn't allowed, even if we used our own non-sourcebots pi :)

[Adimote]

lol simultaneous comments.

[mxbi]

:P

[Adimote]

I think I've pre-emptively answered most of your questions

[mxbi]

Specifically, what we're trying to do is this: 1) Pi tells arduino to watch the input and tell pi when N pulses have been recorded 2) Pi starts the motors (needs to be done in this order because of the ~40ms latency of starting the arduino function - ardiuno needs to be watching the input before we turn on the motors) 3) Pi blocks and waits for response from arduino 4) Arduino counts pulses, predicts when N pulses will be reached, and returns ~40ms early - so that the Pi receives the response at the exact right time (compensating for latency). 5) Pi turns off motors at the right time.

Now this should mostly be possible with the solution you're suggesting except that I'm not sure how to pass N to my overridden read_analogue() function on the Arduino.

Simply calling read_analogue() in a loop to get the waveform won't work because you can't read a ~250Hz square wave if your read function takes 100ms, so at the very least some of the processing would have to be done on the Arduino.

And that's just for the most basic functionality. The arduino's able to figure out how much the robot's drifting from the correct course from the GPIO signals. So if the pi had access to this GPIO data, we'd be able to correct the robot's motion by adjusting motor speeds in realtime - or by alternatively calculating this drift on the Arduino and sending that back to the Pi in a packet a few times a second. It's just basic arithmetic if we have direct access to the GPIO, but this becomes infinitely more complicated if we try to use the GPIO over a 100ms RTT, 10Hz polling rate connection.

Is this just not possible with the current setup? (so without using Pi GPIO or modifying robotd)

Looking into this, send_and_receive probably doesn't do what you expect to. The very fact you can see that function is an error in our part.

FYI I only see this because I looked through the source code. It's not in the docs or anything.

[kierdavis]

It looks like robotd is doing some translation somewhere which means I'm not sure how I can actually call this function I've defined on the arduino

Correct, it is - this is why we have to do this horrific protocol hijacking stuff, because robotd is that part you can't modify.

The function loop needs to stay running on the arduino during this time (or at least not get blocked for more than a couple ms) or signals will be missed and the calculations will be off.

I'd recommend researching the use of interrupts, which are a common way to do more than one thing at once on an embedded system. Reading a rotary encoder can be done without a loop that's constantly running, but "in the background" by making use of interrupts that trigger only when the rotary encoder's pins change. You can either have a go at implementing this yourself (e.g. by using a state machine to keep track of where you are in the rotary encoder's cycle, and a counter to keep track of the total number of cycles), or grab a library that does it for you. You can look at how we import the Adafruit_PWMServoDriver library in our firmware for info on how to add another one.

Now this should mostly be possible with the solution you're suggesting except that I'm not sure how to pass N to my overridden read_analogue() function on the Arduino.

You'll likely want N to be stored in some global variable on the Arduino - then read_analogue can just grab its value and use it instead of doing an analogue measurement.

I'd recommend using ultrasound as your channel rather than analogue, since read_analogue does some slightly strange business around reading all the analogue pins and sending them in one go. Taking an analogue measurement takes a significant amount of time, so trying to use read_analogue in a time-critical scenario probably isn't a good idea.

P.S. if you want really precise timing on your motor control, you might be pleased to know that the motor board can be controlled directly from the Arduino over a serial/UART connection (you can't use the motor board's USB connection at the same time however). This is not something we can provide support for though.

[mxbi]

Thanks for the suggestion about interrupts, I'll look into it. If that works (assuming other operations such as the analogue read don't somehow block it and cause it to not fire) then it would be a decent workaround.

[mxbi]

So I've been thinking about which function to override:

• Analogue read isn't ideal because the pi can't pass arguments to the arduino (so for example, it couldn't be a conditional override, the whole analogue read functionality would need to be replaced).
• Ultrasound is bad because it sends the command to the arduino three times and takes the median of the response. This would probably take like 300ms to run. (Shouldn't the median be calculated on the arduino to save a ton of time?)
• GPIO read appears to be the only one that doesn't do any type-checking or fancy logic to the output. It looks like I could just send an additional pin ID (eg. 99), then check for that on the arduino and redirect to another function. It looks like it would also let me send arbitray information back from the arduino (eg a string), and is the shortest function name which would save a few bytes. :)

Would this work?

[Adimote]

you'll be glad to see https://github.com/sourcebots/robot-api/pull/55 adds a command to the API for calling custom commands to the arduino. We'll see if we can fit it into the update we intend send out very soon.

[kierdavis]

GPIO read is probably a good shout - it doesn't look like robotd validates the pin number (though it really should do), so you should be fine with that.

[kierdavis]

Support for custom servo board commands has just been merged into robot-api and robotd. We'll ship this in an official update soon, but if you're eager you can grab the updated SD card image or the update bundle in advance. Once the build finishes I'll post download links for these.

[kierdavis]

SD card image can be downloaded here.

Alternatively, you can grab an update archive from here - put this on an otherwise empty USB stick, plug it in to the Pi and wait a few minutes to install the updates.

[mxbi]

Thanks! @kierdavis We'll try updating the pi tonight

[mxbi]

So we've just tried updating the Pi through the USB stick method (dpkg and apt fired up on the pi so it definitely did something), and rebooted the pi.. However r.servo_board.direct_command() just gives me AttributeError: 'ServoBoard' object has no attribute 'direct_command'. It looks like maybe it hasn't updated?

Is there a version number I can check to see if the update actually ran?

EDIT: So import robot seems to be importing from /usr/lib/python3/dist-packages - looking in that folder, the code seems to be a very old version.

EDIT 2: Here's the update.log from the SD card:

INFO:root:Starting updates from /media/usb0/update.tar.xz
DEBUG:root:Found 7 potential packages
DEBUG:root:Adding usbmount_0.0.24_all.deb to internal repo
DEBUG:root:Adding robotd_0_armhf.deb to internal repo
DEBUG:root:Adding robotd-dbgsym_0_armhf.deb to internal repo
DEBUG:root:Adding sb-vision_0_armhf.deb to internal repo
DEBUG:root:Adding robot-api_0_armhf.deb to internal repo
DEBUG:root:Adding sb-vision-dbgsym_0_armhf.deb to internal repo
DEBUG:root:Adding runusb_0_all.deb to internal repo
DEBUG:root:Rebuilding apt repo
DEBUG:root:Updating Repos
DEBUG:root:b'Listing...\nbase-files/stable 9.9+rpi1+deb9u3 armhf [upgradable from: 9.9+rpi1+deb9u1]\nbind9-host/stable 1:9.10.3.dfsg.P4-12.3+deb9u4 armhf [upgradable from: 1:9.10.3.dfsg.P4-12.3+deb9u3]\nbluez-firmware/stable 1.2-3+rpt2 all [upgradable from: 1.2-3+rpt1]\ncurl/stable 7.52.1-5+deb9u4 armhf [upgradable from: 7.52.1-5+deb9u3]\ngir1.2-gdkpixbuf-2.0/stable 2.36.5-2+deb9u2 armhf [upgradable from: 2.36.5-2+deb9u1]\nlibavcodec-dev/stable 7:3.2.10-1~deb9u1+rpt1 armhf [upgradable from: 7:3.2.9-1~deb9u1]\nlibavcodec57/stable 7:3.2.10-1~deb9u1+rpt1 armhf [upgradable from: 7:3.2.9-1~deb9u1]\nlibavformat-dev/stable 7:3.2.10-1~deb9u1+rpt1 armhf [upgradable from: 7:3.2.9-1~deb9u1]\nlibavformat57/stable 7:3.2.10-1~deb9u1+rpt1 armhf [upgradable from: 7:3.2.9-1~deb9u1]\nlibavutil-dev/stable 7:3.2.10-1~deb9u1+rpt1 armhf [upgradable from: 7:3.2.9-1~deb9u1]\nlibavutil55/stable 7:3.2.10-1~deb9u1+rpt1 armhf [upgradable from: 7:3.2.9-1~deb9u1]\nlibbind9-140/stable 1:9.10.3.dfsg.P4-12.3+deb9u4 armhf [upgradable from: 1:9.10.3.dfsg.P4-12.3+deb9u3]\nlibcurl3/stable 7.52.1-5+deb9u4 armhf [upgradable from: 7.52.1-5+deb9u3]\nlibcurl3-gnutls/stable 7.52.1-5+deb9u4 armhf [upgradable from: 7.52.1-5+deb9u3]\nlibdns-export162/stable 1:9.10.3.dfsg.P4-12.3+deb9u4 armhf [upgradable from: 1:9.10.3.dfsg.P4-12.3+deb9u3]\nlibdns162/stable 1:9.10.3.dfsg.P4-12.3+deb9u4 armhf [upgradable from: 1:9.10.3.dfsg.P4-12.3+deb9u3]\nlibgdk-pixbuf2.0-0/stable 2.36.5-2+deb9u2 armhf [upgradable from: 2.36.5-2+deb9u1]\nlibgdk-pixbuf2.0-common/stable 2.36.5-2+deb9u2 all [upgradable from: 2.36.5-2+deb9u1]\nlibgdk-pixbuf2.0-dev/stable 2.36.5-2+deb9u2 armhf [upgradable from: 2.36.5-2+deb9u1]\nlibisc-export160/stable 1:9.10.3.dfsg.P4-12.3+deb9u4 armhf [upgradable from: 1:9.10.3.dfsg.P4-12.3+deb9u3]\nlibisc160/stable 1:9.10.3.dfsg.P4-12.3+deb9u4 armhf [upgradable from: 1:9.10.3.dfsg.P4-12.3+deb9u3]\nlibisccc140/stable 1:9.10.3.dfsg.P4-12.3+deb9u4 armhf [upgradable from: 1:9.10.3.dfsg.P4-12.3+deb9u3]\nlibisccfg140/stable 1:9.10.3.dfsg.P4-12.3+deb9u4 armhf [upgradable from: 1:9.10.3.dfsg.P4-12.3+deb9u3]\nliblwres141/stable 1:9.10.3.dfsg.P4-12.3+deb9u4 armhf [upgradable from: 1:9.10.3.dfsg.P4-12.3+deb9u3]\nlibssl1.0.2/stable 1.0.2l-2+deb9u2 armhf [upgradable from: 1.0.2l-2+deb9u1]\nlibswresample-dev/stable 7:3.2.10-1~deb9u1+rpt1 armhf [upgradable from: 7:3.2.9-1~deb9u1]\nlibswresample2/stable 7:3.2.10-1~deb9u1+rpt1 armhf [upgradable from: 7:3.2.9-1~deb9u1]\nlibswscale-dev/stable 7:3.2.10-1~deb9u1+rpt1 armhf [upgradable from: 7:3.2.9-1~deb9u1]\nlibswscale4/stable 7:3.2.10-1~deb9u1+rpt1 armhf [upgradable from: 7:3.2.9-1~deb9u1]\nlibtiff5/stable 4.0.8-2+deb9u2 armhf [upgradable from: 4.0.8-2+deb9u1]\nlibtiff5-dev/stable 4.0.8-2+deb9u2 armhf [upgradable from: 4.0.8-2+deb9u1]\nlibtiffxx5/stable 4.0.8-2+deb9u2 armhf [upgradable from: 4.0.8-2+deb9u1]\nlibxml2/stable 2.9.4+dfsg1-2.2+deb9u2 armhf [upgradable from: 2.9.4+dfsg1-2.2+deb9u1]\nlibxml2-utils/stable 2.9.4+dfsg1-2.2+deb9u2 armhf [upgradable from: 2.9.4+dfsg1-2.2+deb9u1]\nraspberrypi-sys-mods/stable 20180103 armhf [upgradable from: 20171127]\nrobotd/unknown 0 armhf [upgradable from: 0]\nrsync/stable 3.1.2-1+deb9u1 armhf [upgradable from: 3.1.2-1]\nrunusb/unknown 0 all [upgradable from: 0]\nsb-vision/unknown 0 armhf [upgradable from: 0]\nsensible-utils/stable 0.0.9+deb9u1 all [upgradable from: 0.0.9]\nusbmount/unknown 0.0.24 all [upgradable from: 0.0.24]\n'
DEBUG:root:Performing update

EDIT 3: I feel dirty saying this, but I manually ran dpkg -i on all the debs in the update zip, ignored the scary errors, and now it's updated and directcommand() is working great. ¯\_(ツ)/¯ So I guess it's solved

[kierdavis]

Closing since it sounds like we reached a satisfactory (although perhaps not ideal) solution here. The longer-term issue of poor performance in the software stack is still open (#51).