Characterization and Vision Questions

[icemannie]

Love the Romi features. Its a great training tool!

Couple of questions

Characterization requires Connect To Robot - won't connect to a Romi?

Vision from the raspberry pi is going to the Ethernet connector for the roborio - not to the Romi?

[zhiquanyeo]

For characterization: the frc-characterization tool doesn't currently have a template for the Romi. I have a rough Romi project that should work with the frc-characterization tool: https://github.com/zhiquanyeo/romi-characterization

In frc-characterization, make sure the team number is set to 0 before you connect to the robot. This sets up the tool in simulation mode, which will then attempt to connect to the NT server running on localhost.

You should be able to hook up USB or Raspberry Pi cameras to the Romi Pi and access it via the wireless connection (with the default vision application that runs on the Romi Pi)

[MrRSquared]

I just saw this. Thank you. You can see the camera stream in the web console (and use it as an IP camera in GRIP at http://wpilibpi.local:1181/), but for some reason, I can find neither the stream nor the camera data in the simulator or shuffleboard. This is true with the built-in simple example as well as more complex vision code.

[PeterJohnson]

That’s because the stream info is published via NetworkTables and the Pi vision program has no idea where the robot code is running. This could be fixed by configuring the Pi networktables to connect to your computer’s IP address, but unfortunately that doesn’t work right now because the configuration only accepts a team number, not an IP; we will look at fixing that in a future release.

[MrRSquared]

That makes sense. Would doing that interfere with the network tables instance that connectis the Romi to the simulator?

[PeterJohnson]

The Romi connects to the simulator with WebSockets, not NetworkTables, so no, it won't interfere.

[PeterJohnson]

I think I have a solution for this to make it automatic. The NT client has a feature (originally designed for use by dashboards) to discover the robot IP that connects to localhost port 1742. The DS provides a server at that port that sends a json string with the robot's IP address (and sends it again on change/disconnect). It's easy to add this to the vision programs (see wpilibsuite/wpilibpi#187), all the wpilib-ws-robot-romi server needs to do is run a server on this port that mocks the DS protocol and provides the robot IP (which it can discover from the remote address on the websocket connection).

[icemannie]

RomiCharaterization.zip

characterization-data20210110-1911.TXT

I made a characterization program also. I got it to run a couple of times but it failed the first time on a coding problem. The second time I ran it I held it in my hand until the Wheel width section. It ran straight then so there is at least one problem in there and probably more.

The JSON created is attached renamed to TXT. The Data Analyzer rejected in as a wrong format.

Connecting to the robot was the biggest problem. I was using Team number 0.

This came up in the Windows Powershell when trying to connect.

PS C:\Windows\system32> frc-characterization drive new 2021-01-10 19:40:21,802:802 INFO : nt : NetworkTables initialized in client mode invalid command name "2408772393800waitForConnection" while executing "2408772393800waitForConnection" ("after" script)

I'll keep testing.

[MrRSquared]

I used the robot characterization program @zhiquanyeo posted earlier. It worked for me after I changed the IP address. I forgot to set the units and the robot also moved from carpet to a yoga mat. The data was noisy, but understandably so due to the aforementioned issues. As such, I am relatively certain I would not trust the data. Yet as a proof of concept, it worked.

My guess regarding the going straight with the width calculation is that it is possible one of the motors is reversed that should not be.

[zhiquanyeo]

Romi Web Service 1.1.0 has the feature that Peter mentioned above. For existing vision programs, you can make the change that Peter did in his PR (https://github.com/wpilibsuite/WPILibPi/pull/187) to automatically get camera streaming in your dashboards.

To update your Romi software, download the .tgz file from the release link above, and then use the Web UI to update. Ensure that your Pi is in the writable state.

[zhiquanyeo]

@icemannie versions 1.0.3 and later of the web service (basically versions that came in the kickoff release of the image and later) have several changes to the motor directions to better match regular FRC setups. The characterization program I linked to above is basically a line-for-line copy of what gets generated via frc-characterization, with all the notable changes being in robotInit where the encoders, motors and gyros are setup. That should work out of the box

[MrRSquared]

Romi Web Service 1.1.0 has the feature that Peter mentioned above. For existing vision programs, you can make the change that Peter did in his PR (wpilibsuite/WPILibPi#187) to automatically get camera streaming in your dashboards.

Thank you. I updated and tried it, but still could not get it to work. I updated both the simple as well as the more complex vision sample like @PeterJohnson showed in his PR, and I updated to the new Romi software. I also rebooted the Romi and tried the Kill button on the web interface.

I get neither the video from the simple multicamer.py example nor the contours output from the vision code.

Here is the simple example.

multiCameraServer.py.txt

The complex can be found here.

[zhiquanyeo]

Does the camera information show up in the NT view in the Sim GUI? There should also be some trace in the Vision Status log that looks like "Overriding IP address to [some address]"

[MrRSquared]

I do not believe so. This is what I see in the Sim GUI. Sorry for all the extraneous info.

[zhiquanyeo]

Is your Pi set up as an AP or a bridge (connected to your own wifi network)?

[MrRSquared]

Just as you posted this, I was thinking about a few things that could have caused issues on my end, removed them, and now it is working.

I have the Pi set up as an access point. However, I am using two wifi cards on my computer (one connected to the Internet, and a USB dongle connected to the Romi). Also, I was using my school computer (which has a restrictive firewall that I frequently forget about) when I tried earlier.

So, I just tried on my home computer without the wifi dongle, and it worked. Both the video and the coordinates are coming through NT to both the simulator and Shuffleboard.

Then, I closed the program and tried adding the dongle. It seems to still work. So, I guess when I tried earlier, either the firewall was to blame, or, I needed to connect with the sole network interface first.

Either way, this works well. In addition, the network table entries for fetching the output stream are working in the robot program.

Sorry I did not realize the firewall issue this morning.

[zhiquanyeo]

Great! I am seeing an issue where the vision program isn't connecting to the NT server when my Pi is set up as a bridge, and I'll be taking a look at that in a bit

[icemannie]

It isn't connected to my own wifi. It is at the default 10.0.0.2. The SSIDs are set to WPILibn

I'm having the same problems with the Characterization not connecting to the Romis. It connected once this morning

I've tried 2 Romis.

Characterization connects immediately to my Roborio.

I plan on trying another laptop.

On Mon, Jan 11, 2021 at 11:03 AM Zhiquan Yeo notifications@github.com wrote:

Is your Pi set up as an AP or a bridge (connected to your own wifi network)?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wpilibsuite/wpilib-ws-robot-romi/issues/83#issuecomment-758088936, or unsubscribe https://github.com/notifications/unsubscribe-auth/AESZCOUBANQTSG42LSYGRMTSZMVMNANCNFSM4V4OHYKA .

[MrRSquared]

Are you using the code you posted before? I am not sure if it runs as I am still learning some of the ins and outs of java, but there are two Gyro classes each defining a public gyro called simGyroSim(). Only one is used, but if they are both created, I could see it being an issue. Could that be creating a runtime error? That kind of thing did create a similar issue we were having (IN our case, it was two sensors using the same pin for different things).

Did you try the code @zhiquanyeo posted? If so, is it exhibiting the same behavior?

You do need to change the build.prop to have this line.

sim {
  envVar "HALSIMWS_HOST", "10.0.0.2"  //His code had a different ip.
}

[icemannie]

Before I changed that line, I could start emulation but I could see something wasn't right because the gyro readings weren't there. I bring that up because I can get into that situation while switching this and other programs and running and starting emulations. recovery can be by power cycling the Romi or resaving the Romi Web Service file.

Other than that I have tried 2 Romis and 2 laptops with the same results. I have turned off the firewalls also.

I got one run this afternoon. It connected when I brought up the emulation when the Characterization was already trying to connect. Couldn't repeat it though.

Are you able to connect every time to Characterization? How long does it take to connect?

On Mon, Jan 11, 2021 at 1:24 PM Mr. Roth-Ritchie notifications@github.com wrote:

Are you using the code you posted before? I am not sure if it runs as I am still learning some of the ins and outs of java, but there are two Gyro classes each defining a public gyro called simGyroSim(). Only one is used, but if they are both created, I could see it being an issue. Could that be creating a runtime error? That kind of thing did create a similar issue we were having (IN our case, it was two sensors using the same pin for different things).

Did you try the code @zhiquanyeo https://github.com/zhiquanyeo posted? If so, is it exhibiting the same behavior?

You do need to change the build.prop to have this line.

sim { envVar "HALSIMWS_HOST", "10.0.0.2" //His code had a different ip. }

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wpilibsuite/wpilib-ws-robot-romi/issues/83#issuecomment-758169397, or unsubscribe https://github.com/notifications/unsubscribe-auth/AESZCOXGHCKEJGVZP6UK46LSZNF7HANCNFSM4V4OHYKA .

[MrRSquared]

I am relatively certain I can connect every time. The code runs fine. I think I have an issue somewhere with my input units, because I am getting rather large numbers for distance traveled (~445 inches). Yet, If you want to see what I arrived at, here is the output file.

characterization-data20210111-1615.json.txt test

There was one time where I had an issue where the characterization tool was not actually recording data, but that was because I had closed the lid of the laptop and reopened it without restarting the characterization tool. You should be able to see some good debug info in the command prompt you ran the characterization tool from too. That is what led me to realize I was not actually recording any data.

Other than that, I seem to get instantaneous connections to the sim every time. My procedure for running...

• Turn on the Romi
• Launch the robot program
• Launch the characterization tool from a normal Dos prompt (not the wonderful bash-enabled prompt included in VSCode)
• Load the config file
• Set units and units/rotation
• Launch Data Logger
• Enable Teleop and drive to a safe spot with the stick
• Ensure team # is set to 0
• Click Connect to robot
• Click the Quasistatic Forward
• Enable auto (or click the Ok button first).
• Stop before I run into things
• Repeat for each test

[icemannie]

You chose inches and how many units per rotation? If you divide your numbers by the circumference of the wheel, do they make more sense?

I don't think loading the config file can do anything. It is used to create the robot program only and is wrong because it has no choice for a Romi Gyro.

2021-01-11 16:16:45,232:232 INFO : nt : NetworkTables initialized in client mode

The above is what I get when I try to connect the robot. The status of the Characterization screen is "Robot Connecting"

On Mon, Jan 11, 2021 at 3:33 PM Mr. Roth-Ritchie notifications@github.com wrote:

I am relatively certain I can connect every time. The code runs fine. I think I have an issue somewhere with my input units, because I am getting rather large numbers for distance traveled (~445 inches). Yet, If you want to see what I arrived at, here is the output file.

characterization-data20210111-1615.json.txt https://github.com/wpilibsuite/wpilib-ws-robot-romi/files/5798434/characterization-data20210111-1615.json.txt test

There was one time where I had an issue where the characterization tool was not actually recording data, but that was because I had closed the lid of the laptop and reopened it without restarting the characterization tool. You should be able to see some good debug info in the command prompt you ran the characterization tool from too. That is what led me to realize I was not actually recording any data.

Other than that, I seem to get instantaneous connections to the sim every time. My procedure for running...

• Turn on the Romi
• Launch the robot program
• Launch the characterization tool from a normal Dos prompt (not the wonderful bash-enabled prompt included in VSCode)
• Load the config file
• Set units and units/rotation
• Launch Data Logger
• Enable Teleop and drive to a safe spot with the stick
• Ensure team # is set to 0
• Click Connect to robot
• Click the Quasistatic Forward
• Enable auto (or click the Ok button first).
• Stop before I run into things
• Repeat for each test

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wpilibsuite/wpilib-ws-robot-romi/issues/83#issuecomment-758238706, or unsubscribe https://github.com/notifications/unsubscribe-auth/AESZCOQJV3QJZLLDDH3BZPTSZNVBZANCNFSM4V4OHYKA .

[MrRSquared]

You chose inches and how many units per rotation? If you divide your numbers by the circumference of the wheel, do they make more sense?

Yep, I did so, and they seemed spot on. Thanks. I think it is because the encoder counts are adjusted in the code. But, I multiplied the wheel diameter by pi.

I don't think loading the config file can do anything. Probably, but I just do it in case it used any of the units.

I may have found something. What happens when you try to run the characterization tool with just the simulator and no Romi? Of course, you will get no data (unless your program has a simulated mechanism with feedback), but it will tell you if the tools are working.

Here is what I get... Command prompt...

2021-01-11 17:49:27,699:699 INFO    : nt                  : client: CONNECTED to server 127.0.0.1 port 1735
2021-01-11 17:49:27,700:700 INFO    : logger              : Data updated
2021-01-11 17:49:27,701:701 INFO    : logger              : Robot mode: disabled -> disabled
2021-01-11 17:52:10,129:129 INFO    : logger              : Robot mode: disabled -> disabled
2021-01-11 17:52:11,477:477 INFO    : nt                  : connect() to localhost port 1735 timed out

In the Vscode terminal, I get this...

********** Robot program starting **********
Default simulationInit() method... Override me!
Robot disabled
Collected: 0 in 0.419224 seconds
Default simulationPeriodic() method... Override me!
Warning at edu.wpi.first.wpilibj.IterativeRobotBase.printLoopOverrunMessage(IterativeRobotBase.java:301): Loop time of 0.005s overrun

Connection Attempt 2  //These are to be expected because there was no Romi on.
Connection Attempt 3
Connection Attempt 4
Connection Attempt 5
NT: server: client CONNECTED: 127.0.0.1 port 55456 //This was when I connected from the Characterization tool.

[PeterJohnson]

Version 1.1.1 now publishes the robot code IP. For a custom vision program you just need to add NetworkTableInstance.startDSClient() (see wpilibsuite/WPILibPi#187 for how to do this in each language).

[icemannie]

Tried your suggestion. I saw pretty much what you had with connection attempts but setting the Characterization connection did nothing.

On Mon, Jan 11, 2021 at 5:04 PM Mr. Roth-Ritchie notifications@github.com wrote:

You chose inches and how many units per rotation? If you divide your numbers by the circumference of the wheel, do they make more sense?

Yep, I did so, and they seemed spot on. I think it is because the encoder counts are adjusted in the code. But, I multiplied the wheel diameter by pi.

I don't think loading the config file can do anything. Probably not.

I may have found something. What happens when you try to run the characterization tool with just the simulator and no Romi? Of course, you will get no data (unless your program has a simulated mechanism with feedback), but it will tell you if the tools are working.

Here is what I get... Command prompt...

2021-01-11 17:49:27,699:699 INFO : nt : client: CONNECTED to server 127.0.0.1 port 1735 2021-01-11 17:49:27,700:700 INFO : logger : Data updated 2021-01-11 17:49:27,701:701 INFO : logger : Robot mode: disabled -> disabled 2021-01-11 17:52:10,129:129 INFO : logger : Robot mode: disabled -> disabled 2021-01-11 17:52:11,477:477 INFO : nt : connect() to localhost port 1735 timed out

In the Vscode terminal, I get this...

** Robot program starting ** Default simulationInit() method... Override me! Robot disabled Collected: 0 in 0.419224 seconds Default simulationPeriodic() method... Override me! Warning at edu.wpi.first.wpilibj.IterativeRobotBase.printLoopOverrunMessage(IterativeRobotBase.java:301): Loop time of 0.005s overrun

Connection Attempt 2 //These are to be expected because there was no Romi on. Connection Attempt 3 Connection Attempt 4 Connection Attempt 5 NT: server: client CONNECTED: 127.0.0.1 port 55456 //This was when I connected from the Characterization tool.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wpilibsuite/wpilib-ws-robot-romi/issues/83#issuecomment-758279834, or unsubscribe https://github.com/notifications/unsubscribe-auth/AESZCOTQVTS3CLNQAPDJABTSZN7YFANCNFSM4V4OHYKA .

[MrRSquared]

Sorry, I should have added that the characterization did show connected as well. Since you do not have a robot in doing that, none of the data will change, but for me it consistently showed connected. I am guessing that if this works without the robot, something is crashing the robot code which would cause the disconnect.

[icemannie]

The characterization shows it starting a network client. Wouldn’t it need a pi network server to connect?

On Mon, Jan 11, 2021 at 7:53 PM Mr. Roth-Ritchie notifications@github.com wrote:

Sorry, I should have added that the characterization did show connected as well. Since you do not have a robot in doing that, none of the data will change, but for me it consistently showed connected. I am guessing that if this works without the robot, something is crashing the robot code which would cause the disconnect.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wpilibsuite/wpilib-ws-robot-romi/issues/83#issuecomment-758341291, or unsubscribe https://github.com/notifications/unsubscribe-auth/AESZCOTIIUFG5EKVJJOZEHTSZOTRTANCNFSM4V4OHYKA .

[MrRSquared]

No. This is the beauty. The characterization tool does not connect to the Romi like it does to the Rio (and the VMX-Pi). It connects to the simulator through localhost. So, if that is not working for you, the setup may be wrong somehow. The data comes through the websocket from the Romi to the simulator.

Edit: I just noticed a difference between the project folder you uploaded and the one @zhiquanyeo uploaded (which is what I am using). His does not seem to have a SimEnabler() class whereas the uploaded version does. That could present a problem.

[icemannie]

Once I get Characterization to connect, the routine without the SimEnabler runs through the 5 sequences and produces a JSON file which the analyser accepts and produces the charts. They don’t look good but some of the problems are listed below.

Difficulty for me is getting Characterization to connect. It will only happen if Characterization is trying to connect when the simulation is started and then not often. The WebSocket is shown connected in the VSCode terminal.

Any troubleshooting I can do?

Autonomous Init. has what looks like valid code to reset the encoders but it doesn’t do that. They do not go to 0; The getBattery returns 10.2 volts with some jitter. Motor volts are estimated from commanded speed. The rightVolts needs to be negated.

[icemannie]

one JSON file as TXT and images

characterization-data20210112-1306.txt

[zhiquanyeo]

Hmm I've connected the characterization tool both ways (starting that first, or starting robot code first) and it's worked pretty much all the time. You're using the latest version of the characterization tool right?

The encoder resetting should work correctly, since it's handled in the HAL layer and never makes it over the wire. Were you seeing it continuing to count from where it was? or starting at some smaller numbers?

[icemannie]

I'm using 2021.1.3. What I'm finding now is that if I start one after the other as quickly as possible, then they connect.

The encoders are resetting. I was looking at the JSON file entries incorrectly.

[MrRSquared]

Hello, I wonder if something is killing a process somewhere on your computer then. Out of curiosity, if you open shuffleboard (or smart dashboard) while connected to the Romi, do you get consistent data there?

My graphs tend to resemble yours. There may be an element of noise inherent to unshielded hobby motor encoders on such a small base, but that is just a guess.

However, my numbers are a bit better (some of the constants in your file were so low, they invoked the all-powerful e). I had numbers that resembled yours when I did not give the robot enough space, or when it had surface changes during the test.

Here is the dataset that I feel comfortable with. Mind you, I have not begun working with PID yet or any feedforward control either (that is coming soon though). So, the constants could still be off a bit. characterization-data20210112-1447.json.txt

[icemannie]

Shuffleboard works correctly.

I’m going to take a break from Characterization and check out the vision updates. What should I be looking out for when I start?

On Wed, Jan 13, 2021 at 1:18 PM Mr. Roth-Ritchie notifications@github.com wrote:

Hello, I wonder if something is killing a process somewhere on your computer then. Out of curiosity, if you open shuffleboard (or smart dashboard) while connected to the Romi, do you get consistent data there?

My graphs tend to resemble yours. There may be an element of noise inherent to unshielded hobby motor encoders on such a small base, but that is just a guess.

However, my numbers are a bit better (some of the constants in your file were so low, they invoked the all-powerful e). I had numbers that resembled yours when I did not give the robot enough space, or when it had surface changes during the test.

Here is the dataset that I feel comfortable with. Mind you, I have not begun working with PID yet or any feedforward control either (that is coming soon though). So, the constants could still be off a bit. characterization-data20210112-1447.json.txt https://github.com/wpilibsuite/wpilib-ws-robot-romi/files/5810651/characterization-data20210112-1447.json.txt

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wpilibsuite/wpilib-ws-robot-romi/issues/83#issuecomment-759661879, or unsubscribe https://github.com/notifications/unsubscribe-auth/AESZCOW5YQH6QF5MS7HLDG3SZXWXZANCNFSM4V4OHYKA .

[MrRSquared]

To just get a camera running, if you are updated and make the change that @peterJohnson posted above, there is not much to do. If you want to use Grip, it took me awhile to figure out what to do with the output file. However, the WPILib Romi and Sim reveal video helped a lot. They not only show how to use Grip, the sample shows how to implement a pipeline on the pi. The examples I posted above work for me on the Romi that I have a camera (cheap so I camera) on.

Links are not working on my Phone, but here is the sample program.

https://wpilib.org/s/DemoFIRST-master.zip

[chauser]

A couple of observations about characterization and then a question that may be an issue report.

1. If you are running the characterization program on a Windows machine there is an issue with pynetwork tables that causes Connect'ing from the logger program to the robot code to hang. (Issue & pull request to fix have been submitted and accepted for pynetworktables.) As far as I can tell the problem is specific to Windows. Until there's a new version of frc-characterization that uses the new pynetworktables the only easy workaround I know is to disable IPv6 on the Windows machine (a google search will provide instructions). It's probably better to install virtualbox and a Linux VM and do everything there!
2. The track-width computation in the characterization analyzer program doesn't take into account the wheel diameter. I got track estimates that were 5X too big. Multiply the number the analyzer reports by the units per revolution number (0.22 for Romi) to get the correct estimate. An issue has been submitted and I understand a fix is in the works.
3. Finally, has anyone else noticed that on the Romi if you disable while it is moving, when you re-enable it begins moving again? For example, run the quasi-static forward test: enable auto and let it get moving at a good clip, disable, enable auto again -- the robot takes off. I produced the same behavior with teleop when I disabled with the robot moving. To do characterization I had to be careful to always choose the next test before re-enabling auto. Even then the robot would often make a sudden short move as if the previous settings were still in effect before beginning the behavior I expected of the next test. I did some breakpoint debugging and it appears in the Java code that 0 motor settings are being set as expected when disabled is entered, but somehow they don't get through to the Romi.

[zhiquanyeo]

@chauser for point 3, as you disable the robot, what does the /robot/autospeed NT entry read? The romi-characterization project re-reads the NT value during autonomousPeriodic so if that value is non-zero, it'll start moving

For the telop transition, that might be a bug. Could you put in an issue and I'll take a look? We do zero out the motors when disabled but there might be an edge case we're not catching somewhere.

[chauser]

autospeed is 0 at that point, and the java code is sending 0 to the drivetrain object for both motors. My guess is that it is the same issue as for teleop. I will put in an issue.

[icemannie]

Since I uninstalled the trial version of McAfeee that came with the laptop, connecting with Characterization hasn't been a problem. I had tried turning off their Firewall previously but it didn't help.

[jpokornyiii]

I think everything has been addressed here. If there are still problems, please open a new Issue.