Support Rover's Bluetooth Connection

[rirze]

See if existing SerialDevice can support such functionality. Original Issue now relocated to #14

Support sending commands over to Mirobot's rover bluetooth module. (Regular BT, not BT LE.)

[JorisvanDamTUD]

Not sure if this would already be capable of working, but I made some attempts anyways to connect with my rover/mirobot over (serial-over)bluetooth using this library. But perhaps the (odd) behaviour of the serial interface I describe below can be helpful in getting the BT functionality in mirobot-py working.

Short summary: bluetooth controlling the rover (and arm) over a standard TTY terminal works, with many quirks however, see below

control with mirobotpy library not yet, probably due to unexpected messages coming back from the grbl firmware on the rover shield and/or serial interface problems (e.g. exceptions due to 'expected letter command'). I suspect these throws the exceptions in the pyserial lib, which make py code stop.

Here are some findings: A) serial terminal

• I can connect to¬ the BT module (on the rovershield) with PuTTY, over a serial over Bluetooth com port (win10 laptop)
• rover commands via PuTTY to the Rover work (e.g. ‘W8’and ‘W0’ for forward stop)
• However, communication with the rover/arm/shield seems to be interfered by either buggy grbl firmware or serial communication errors: very often a command like $H reponds in ‘error: Expected command letter” or ‘error: Bad number format’
• Command ‘?’ returns the status as usual, but is always truncated‘ <Alarm,Angle(ABCDXYZ):0.000,0.000,0.000,0.000,0.000,0.000,0.000,Cartesian coordinate(XYZ RxRyRz):202.000,0.000,181.000,0.000,0.000,0.000,Pump v’
• Gcode command for arm movements do not work, all throw a ‘Expected command letter’ error
  B) Android serial terminal app
• same results n.b. after connection start, the rover always first send acknowledges ‘error: Expected command letter” followed directly by ‘error: Bad number format’

C) Test with mirobot-py lib:

• Attempt to connect results in: [COM13] [DEBUG] Attempting to open serial port COM13 [COM13] [DEBUG] Succeeded in opening serial port COM13 [COM13] [ERROR] 'utf-8' codec can't decode byte 0xfa in position 0: invalid start byte Traceback (most recent call last): File "c:\downloads\mirobot-py\mirobot\serial_device.py", line 99, in listen_to_device msg = str(msg.strip(), 'utf-8') UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfa in position 0: invalid start byte [COM13] [DEBUG] Attempting to close serial port COM13 [COM13] [DEBUG] Succeeded in closing serial port COM13

Process finished with exit code -1

Code used:

!/usr/bin/env python3

from mirobot import Mirobot import time with Mirobot(portname="COM13",debug=True,wait=True) as m:

time.sleep(10)

m.home_simultaneous()

[rirze]

Thank you for the detailed report!

I will definitely pour over this more in the next few days. While I originally opened this ticket for dealing with the arm over Bluetooth, including the rover functionality might be related as well.

Initial thoughts:

1. I don't have a rover, so much of the stuff I've implemented is theoretical on how this stuff stitches together.

2. I'm still unsure if the rover interface is a separate serial interface from the mirobot. If they are separate, then I need to rewrite the code (from your report, it seems like they are?).

3. You've mentioned command inputs are inconsistent-- could you clarify for me? Does it work sometimes or not at all?

4. If you get the same results from the Android app, then I'm not sure there's a way to get this behavior implemented presently. Generally, my guiding principle for this library has been to build on fundamental actions that can be performed with the Mirobot. If WKlata hasn't shown a way to perform these basic operations consistently, I'm not confident I'll be able to reverse-engineer a solution.

5. That status string is definitely concerning-- I'll have to consider that when dealing with rover operations.

6. The one thing I can do is look into the decoding error and see if there's some clever handling I can do to mitigate this.

Will reach back soon.

[JorisvanDamTUD]

Here are already some of my answers, see below:

1. don't have a rover: I understand, so I really appreciated your efforts. I can do testing offcourse, as Im really hoping to get the wireless operation and rover python coding working (because only cable tethered rover operation doesnt make much sense :) )

2. separate serial interface?: I think the system was designed to work as followed: bt on rover shield receives all commands for both rover and arm, rovershield operates the wheels if commands are for rover (eg. W8 or W0). For all other commands the firmware on rovershield forwards these commands to arm. over the 2-wire rxtx cable and extenderbox. I will ask Wlkata to publish rover firmware and details on interface. So, my conclusion now is there are not separate interfaces for rover and arm, but rover board is the master that forwards to arm.

3. Are command inputs are inconsistent?: for non rover commandsn i have had several occassions on which a $H command (send to the rover bt) does make the robot do homeing, but it only works after many repeated resends. So this would only work by resending $H command several times, the first couple of resends will result in errors ("exspected number" or the other error) and then suddenly the next resend will work, and the arm homes.

There seemed to be no structure in number of resends I had to do (sometime 3x, sometime 5x).

Possible explaination: It seemed to me that due my first sending $H, the rover grbl firmware gets into some kind of specific 'input state' in which it expects (and wait for) a specific input, but doesnt receive this as it gets the $H again. After several resends, the firmware then goes into the 'normal' state and can interpret $H correctly. But this is just a guess.
The reason that none of the 'M20 G21 .....' commands work could be that these commands have to be preceded with some unknown character or code. Or spaces should be replaced by underscores or whatever.

Getting the firmware from Wlkata hopefully will clear this all up. Ive looked at the arm firmware, and that based on standard CNC/GRBL code and much of this code is still in the Mirobot firmware and might lead to these unexapected behaviour.

[rirze]

Alright, so one step at a time:

I've uploaded a new commit that hopefully fixes the EncodingError-- let me know if you get past that error and see the same failures as you'd expect from the Android app.

[JorisvanDamTUD]

Thanks, the newest commit indeed seems to solve the problem with the encoding error, when using mirobot-py to serial interface. The python mirobot-py lib output now resembles the sequence on Android: [COM13] [DEBUG] Attempting to open serial port COM13 [COM13] [DEBUG] Succeeded in opening serial port COM13 [COM13] [DEBUG] [RECV] error: Expected command letter [COM13] [ERROR] Expected command letter [COM13] [DEBUG] Attempting to close serial port COM13 [COM13] [DEBUG] Succeeded in closing serial port COM13

So, one step done!

[rirze]

Ok, that's good to hear!

Now we have a baseline-- this library is producing the same results as WKlata's existing functionality.

Correct me if I'm wrong: The next step is wait for WKlata's solution to this problem or release of rover firmware. Is there anything else I can do to alleviate this issue?

[JorisvanDamTUD]

To investigate the sequence that seems to trigger the errors some more, I looked at the message stream with Realterm and captured the output. After making the connection (bt over serial to the rover) output is: "5-5-2020 16:55:55",error: Expected command letter "5-5-2020 16:55:55",ok "5-5-2020 16:55:55",error: Bad number format "5-5-2020 16:55:55",ok "5-5-2020 16:55:55",ok
(timestamps are added by Realterm)

I also did a hex output (text after the '>>' is my translation, to better understand sequence) "5-5-2020 17:01:02",6F6B0D0A >> ok "5-5-2020 17:01:02",6572726F723A20457870656374656420636F6D6D616E64206C65747465720D0A >> error: Expected command letter "5-5-2020 17:01:02",6F6B0D0A >> ok "5-5-2020 17:01:02",6572726F723A20426164206E756D62657220666F726D61740D0A >> error: Bad number format "5-5-2020 17:01:02",6F6B0D0A >> ok "5-5-2020 17:01:02",6F6B0D0A >> ok "5-5-2020 17:01:07",6572726F723A20457870656374656420636F6D6D616E64206C65747465720D0A >> error: Expected command letter "5-5-2020 17:01:07",6F6B0D0A >> ok "5-5-2020 17:01:07",6572726F723A20426164206E756D62657220666F726D61740D0A >> >> error: Bad number format "5-5-2020 17:01:07",6F6B0D0A >> ok "5-5-2020 17:01:07",6F6B0D0A >> ok

n.b. the hex-ouput seems to have some more output lines , and also an additional 'ok' at startup. Not sure if this difference is due to two separate capture sessions (can t do them simultaneously) or some decoding issue in Realterm

Will do some more testing later

[JorisvanDamTUD]

:)) LOL i think i figured it out.....

I decided to just do a bit of 'brute force' attempt....... I figured there probably had to be some kind of preceding character that the firmware expects. So i just opened another tty terminal and just wacked every key on my key board followed by Enter for a while.

The "~' plus enter is always followed by 'ok'. so, following up on this, I tried to input some Gcode command, but this time without space. so '~M20G91X10'

And that works!! Arm is moving

So I think the trick is to just have every command send to rover/arm begin with an '~' followed by the commands, but without the spaces. Offcourse the latter being quite a pain I suspects, as this is different from normal/direct arm command sequence.

[rirze]

I have so many questions.... but first, nice find!

To clarify, ~M20G91X10 works but ~M20 G91 X10 doesn't? I can change this, but .. I don't like the lack of readability.

If you can make sure this consistently works with all sorts of other commands, I can rig something together in the library to make this behavior possible.

[JorisvanDamTUD]

Sorry for the delay. And some bad news: i can not replicate my previous success in moving the arm with appending '~' and leaving out spaces :((( After a powercycle I just could not replicate it anymore. No matter what I tried. Next day I spend much more time on it again: no luck. There just is either some hidden state the rover board was in (perhaps emabled by some keystrokes?) or my rover board is just really crappy and only working properly now and than.

To make things (even) more confusion and frustrating: in the forum the people from Wlkata now say the app needs to be connected to the bt extension box! And not directly to rover bt (so then the bt extend box (or roverarm) probably forwards commands to rover.Instead of other way around).

But: the manual clearly shows that App BT connection should be to Rover BT module when using Rover.

So, this whole Wlkata Rover is quite a big failure for now.........

For now, I will just wait for the publishing of Rover grbl firmware (as you suggested ealier). So i can investigate how the rover firmware is programmed and should communicate. Hopefully that will make things more clear. That seems the best (and only) option for now.

In the mean time, I will be only using the arm, using python and your library, that works like a charm.

Thanks

[rirze]

Darn, I'm sorry to hear that-- if you ever find the patience to figure out how you did it the first time, I'd love to implement it in the library.

Until then, I think you're right; it's best to wait for WKlata to clarify.

[rirze]

Also, are you able to pair to the Arm's bluetooth box using a pin? I'm clueless what the default pin is or where to look for it...

[JorisvanDamTUD]

Ah, yes, the PIN, there's another challenge.....the bt extender box has no PIN! which works fine for an android phone, it just connects (and figures out there's no PIN needed apperently). Thats how I managed to wirelessly use a (android) serial terminal to control the arm (and the Wlkata app, but that is not really useable).

But: on Win10 a user is, since some updates ago, forced to enter a PIN when connecting; without a PIN you just simply can not connect to that BT device...... I already spend hours researching and trying to bypass this, but none of the solutions proposed work, including some CLI tools.

Not sure how no-PIN BT is on Linux.

For now i will only use the usb cabled connection to control arm from my win10 system, either Wlkata studio, serial terminal or python with your library.

Note: the Rover BT does have a PIN (1234), so had no problems there connecting from win10, but....sigh...... :(

[JorisvanDamTUD]

I discovered that the information I was given earlier by Wlkata forum support about the rover setup, serial connection and rover app has all been incorrect :( Originally, Wlkata stated on the forum that rover setup should be done with the BT expansion box and rover should work with the app. But, as you know, all this didn't work, how much I tried.

Much, much later it turned out that rover connection should be over the BT on the Rover but also that there is a special Rover version of the app (which they didnt publish yet). All this cost me a lot of time..... :( So, I’m quite frustrated by the bad support of Wlkata. But that’s another story……

The good news: in principal I can now control rover and arm over BT of the rover and also have a stable BT and serial connection, although still with some limitations regarding mirobot-py, see details below.

Lastly, one critical issue turned out to be the power supply for combination of rover and arm. In retrospect this has been factor in connection instability. In my earlier tests I used two power supplies for arm and rover, because the battery normally powering both was drained. With powersupplies, I had all kinds of strange serial problems. Switching back to the batterypack solved this, so I guess some grounding problem (for the serial connection from rover to arm) is introduced when using two separate power supplies.

[JorisvanDamTUD]

The results of a series of test in different configuration of Rover, Arm, extender box, connection box and python library.

1) Connect Mirobot over usb connection + nonBT extender box+ 2wire serial to Rover results using mirobot-py: a. working, can control arm (e.g. do some m.increment_axis(y=10, speed=3000) ) b. tried to control rover: not working (e.g. m._rover.move_backward)

2) Mirobot + bt extenderbox + 2wire serial to Rover results: a. can control arm, used the ‘connection_type=’bt’ parameter. takes a bit long to get a connection (up to ~ 6sec, sometimes no connection as described in the example comments) b. cannot control the Rover

3) connect via BT of Rover + connect rover to arm with BT extender box + 2wireserial + using the standard serial-python library: • s = serial.Serial('COM13',115200, parity='N', bytesize=8, stopbits=1, timeout=0, xonxoff=0, rtscts=0) • s.write(b"M20 G91 X1 Y1\r\n") • s.write(b"W8\r\n") results: a. I can control Rover wheels via standard serial lib b. but all arm commands are ignored c. rover commands only work if every command is ended with \r\n, otherwise rover command are ignored

4) connect via BT of Rover + connect rover to arm with BT extender box+ 2wireserial + mirobot-py lib (‘connection_type=’bt’ ) results: a. control of arm or rover does not work b. It raises the error “raise Exception('Could not find mirobot bt')”

5) connect via BT Rover + rover to arm connection over 'normal' (non-BT) extender box + using the standard serial-python library results: a. can control Rover wheels (via serial lib) b. AND can control the robot arm • all rover commands only work if every command is followed by \r\n (s.write(b"W8\r\n") • arm command also work, but only if ending with either \n or \r\n ( s.write(b"M20 G91 X1 Y1\r\n") or s.write(b"M20 G91 X1 Y1\n")

[rirze]

Thanks so much for the detailed testing! This really helps a lot!

So my thoughts:

• I have long suspected that \r\n is the preferred line break for these devices. On a fundamental level, this choice saddens me, but I understand why. I'm going to add a commit to just use \r\n for every message being sent.

• This new connection type (bt) only supports bluetooth low energy protocols at the moment. As such, the autodiscovery only covers the Mirobot's bluetooth extender box. Autodiscovery is nice for the first time someone connects, but I would personally recommend copying the MAC address that shows in debug logs (it's the first field in [..:..:..:..] from the left) and give that address to Mirobot: Mirobot(portname='34:1A:....'). That way, one doesn't have to wait 6 seconds every time.

• When connecting by Rover BT, do you have to set up the connection initially outside of this library? What is that process like? I would like to have the bluetooth on-boarding seamless for less savvy users.

I know I haven't documented much of the new functionality yet, but there are many convenient shortcuts for common Mirobot operations now. I would recommend using Mirobot.rover.wheel.[...].[...]() and Mirobot.rover.rotate.[...]() and Mirobot.rover.move.[...]() for rover operations. For Mirobot, one can use Mirobot.move.[...]() and Mirobot.increment.[...](). If you use autocomplete, I recommend exploring the different aliases available. Let me know what you think!

[JorisvanDamTUD]

Great that your going to add the \r\b line break that will support rover control, thanks!

on BT Rover connecting

• As it is a regular BT device, just BT connecting to it with (in my case) Windows Bluetooth device manager and provide the BT Rover PIN ('1234') is enough
• After this, the BT Rover can connect (e.g. from pyhton) to the serialport-over-Bluetooth COM-port (in my case: COM13, see picture) with help of the python serial library (e.g. s = serial.Serial('COM13',115200)

Connect BT Rover on serial-over-BT with Mirobot-py I tried use the COM13: with Mirobot(portname="COM13",debug=True,wait=True) as m:
m.home_simultaneous()

but that fails with debug log : [] [DEBUG] Attempting to open serial port COM13 [] [DEBUG] Succeeded in opening serial port COM13 [COM13] [DEBUG] [RECV] error: Expected command letter [COM13] [ERROR] Expected command letter [] [DEBUG] Attempting to close serial port COM13 [] [DEBUG] Succeeded in closing serial port COM13

on mirobot-py library rover commands/shortcuts
Odd. Somehow, autocomplete finding the examples you give doesnt work for me (im on mirobot-py lib version 2.0.0b0 on Pycharm) e.g. 'm.r' can autocomplete to m.rover, but after this (add a point at end > m.rover.) no more Mirobot items are listed for autocomplete. Nor wheel or rotate are shown in autocomplete (actually none for Mirobot at all)

Not sure im misinterpreting something here or Im on the wrong version?

.

[rirze]

No problem! The \r\n on rover commands is available, so let me know if that still doesn't work out.

Connection of BT Rover Yah, this sounds like the user should manage this. I don't think it's worth effort setting this up within the library.

Connect BT Rover over serial with mirobot-py Hmm, that doesn't seem too good. Oddly though, it seems like the connection is picking up errors before it even gets to the homing. Maybe there were some phantom messages from a previous execution?

Autocomplete for aliases Boy, this took me on a journey. While the aliases definitely work and are available, autocompletion is a tricky beast. As such, I've changed how the aliases are implemented, and tested it on a local PyCharm installation as well. It should show up now, with proper docstrings to boot.

Funny enough, my editor's autocomplete doesn't pick up the aliases even with the recent change. I've filed a bug report to see what they say, but PyCharm's autocomplete is bit more advanced in this regard.

Let me know if you still run into unexpected behavior?

[JorisvanDamTUD]

Some update after new tests:

Connect BT Rover on serial-over-BT with Mirobot-py > picking up errors before homing? When I connect to Rover BT with a serial terminal, than right after the connection is made, i can see following output: error: Expected command letter ok error: Bad number format ok ok

Apparently this is a bug in the Rover firmware code but causes the mirobot-py code to stop. When I temporarily comment out the SystemExit(-1) in exceptions.py, Im able to succesfully get past this and issue m.home_simultaneous() successfully, but after this code freezes (see below)

Output: [] [DEBUG] Attempting to open serial port COM13 [] [DEBUG] Succeeded in opening serial port COM13 system exit prevented [COM13] [DEBUG] [RECV] error: Expected command letter [COM13] [ERROR] Expected command letter [COM13] [DEBUG] [RECV] ok [COM13] [DEBUG] [SENT] $H [COM13] [DEBUG] [RECV] error: Bad number format [COM13] [ERROR] Bad number format [COM13] [DEBUG] [RECV] ok [COM13] [DEBUG] [RECV] ok system exit prevented

However, after this, the code does not run any further or terminate by itself, and I have to force quite the code execution. And by doing this close the still open COM port

BT Rover: single ok after each command After this, I looked also closer at command sequences with a serial terminal, and it seems that the Rover firmware only issues 1x ok after each command, except for Rover commands, there it zero (0) ok

so a sequence in&output would be: /////startup serial terminal error: Expected command letter ok error: Bad number format ok ok

//////input command from serial terminal>> $H ok //////input command from serial terminal >> M20 G90 X50 ok //////input command from serial terminal >> W8 //////input command from serial terminal >> W0 //////input command from serial terminal >> ? <Idle,Angle(ABCDXYZ):0.000,4.545,0.000,0.000,0.000,28.854,-33.406,Cartesian coordinate(XYZ RxRyRz):251.998,0.000,181.022,0.000,-0.007,0.000,Pu::E

Comments:

1. it is rather surprising (shocking?) to find out that in the Rover the command acknowledgments are Rover = 1x 'ok Mirobot arm = 2x 'ok' Rover = zero

But I think this is also the cause of the code freeze after the first command (home_simultaneous), as code keeps waiting for a second 'ok'

2. Observe the truncated string, as response to the '?' command, its always like this, same length. Not sure of the cause. But this could also cause errors in the _parse_status function?

The more I get to understand from the inner workings of the Rover, the more I wish Wlkata would just publish the rover firmware, even in its current state. That would make troubleshooting much easier and could give opportunity fix some things by changing the firmware.

[rirze]

Yah, this looks like a nightmare to workaround.

I'm gonna wait on firmware release to sort things out, for sure. This behavior does not seem sensible...