Radio doesn't work with six robots

[justbuchanan]

After messing around with six robots several times, it seem like it's always the robot on the field with the highest shell number that loses communication.

Because robots only send packets back to the computer after they've received packets, it's possible that the issue is on the transmitting side. If one robot on the field is not sent a packet, it will not send packets back to the computer and show up in soccer with a 'No Rx' message. Maybe the packet sent from soccer to the bots is being truncated so that the last bot doesn't receive anything. If so, this may be an issue in the firmware for the transmitter.

[justbuchanan]

It may have to do with this line here: https://github.com/RoboJackets/robocup-software/blob/master/firmware/base_2011/device.c#L37

I'll try changing it to a 64 next time I'm in the shop to see if it fixes the issue.

[ehuang3]

@justbuchanan It's possible that there are multiple points of failure which cause the 6 comm bug.

I'll add to the series of observations:

• Last week, Justin and I toggled a robots shell number from 0-7 to 8-11 and watched a printf of the received Rx messages. The robot would send messages when the ID was in 0-7 and would barely send messages in 8-11. We could toggle between this behavior. Even stranger was that the robot's Tx light was on, confirming that it was receiving packets from the base station.
• From the same experiments last week, we watched robot comms fail when the shell number was above 8.
• Yesterday, we observed 5 robots running at once with some at shell numbers of 8 and above (at least 9 and 11 were present). Upon adding the 6th robot, the highest numbered shell dropped communications.

[justbuchanan]

Also, see the comment here: https://github.com/RoboJackets/robocup-software/blob/master/firmware/base_2011/device.c#L506-507

This seems to indicate that the 32 bit issue pointed out above may be even more relevant.

[ehuang3]

I spoke to Ben, to this effect:

There a communication protocol lower than the Rx/Tx command protocol.

The radio channel is half-duplex and the timing is strictly pre-arranged. The robots send in the order of their slots in the forward packet. This may explain why the failures start from the highest ID. Our soccer control loop run at 60 Hz; therefore, there may not be enough time between forward packets to send all 6 reverse packets.

I suggest running the control loop at a lower frequency first and seeing if all 6 packets start to show up. Ben suggested directly testing radio signals using a spectrum analyzer (an EE's job). Using the analyzer we would be able to see the transmission protocol.

Other notes:

• @justbuchanan That 32-bit is probably not it. That is the maximum size for the control endpoint. The radio packets go over other bulk endpoints. However, Ben confirmed that the base station code in firmware is what runs on the radio modules.
• The radio has it's own maximum packet size too. You can exceed it, but that takes much more complicated programming.
• Get a spectrum analyzer and put it in zero-span mode. Packets from each device will have different amplitudes so you can distinguish them (put them in a straight line).

[justbuchanan]

In the soccer program, the radio packets are sent once each time the Processor runloop iterates. I tried halving the rate (60Hz -> 30Hz) the Processor runs at by changing the number on this line, but the issue stayed the same.

This makes me think it's lower-level than soccer, probably something in the USB radio base station.

Also, I tried using lower shell numbers on the field and it does seem to always be the robot with the highest shell number that loses radio. In my case it was shell number 7, so I can confirm that it isn't just shell numbers above 8 that fail.

[ehuang3]

Just to anticipate future f's,

There is 3ms per slot for reverse traffic. With 6 slots, that's 18ms. A forward packet arrives every 16.7ms.

[justbuchanan]

Hmmm... does that mean that it shouldn't be working right now?

[ehuang3]

No, just putting that there for posterity.