error report: "internal issue with print and error tags." on the FRC Driver Station.

vitojan commented 8 months ago

Description of the Issue When the robot is enabled, it only moves for approximately 5 to 20 and then stops. Additionally, an error is reported: "internal issue with print and error tags." on the FRC Driver Station.

Steps to Reproduce After uploading the code, press [Enable].

Code Link: https://drive.google.com/file/d/1FnMK5D-NExMq3QSRd2-rVercRD_zdk9D/view?usp=sharing

Desktop Environment:

OS: Windows 10

Project Information:

plugins {
    id "java"
    id "edu.wpi.first.GradleRIO" version "2024.1.1"
}

java {
    sourceCompatibility = JavaVersion.VERSION_17
    targetCompatibility = JavaVersion.VERSION_17
}

{
    "fileName": "Phoenix6.json",
    "name": "CTRE-Phoenix (v6)",
    "version": "24.0.0-beta-7",
    "frcYear": 2024,
    "uuid": "e995de00-2c64-4df5-8831-c1441420ff19",
    "mavenUrls": [
        "https://maven.ctr-electronics.com/release/"
    ],
    "jsonUrl": "https://maven.ctr-electronics.com/release/com/ctre/phoenix6/latest/Phoenix6-frc2024-beta-latest.json"...
}

{
  "fileName": "WPILibNewCommands.json",
  "name": "WPILib-New-Commands",
  "version": "1.0.0",
  "uuid": "111e20f7-815e-48f8-9dd6-e675ce75b266",
  "frcYear": "2024",
  "mavenUrls": [],
  "jsonUrl": ""...
}

RCoder01 commented 8 months ago

We're having the same issue: after enabling, the robot immediately disables and the console shows "Internal issue with print and error tags".

Our code is here

Update: this issue occurred when deploying from a mac, but the code worked fine when deploying from windows 10. The code was deployed from both computers from the same project commit.

Update 2: the issue no longer occurs when deploying from either computer

AlexT2052 commented 8 months ago

Can you let us know what you did to solve it? I am experiencing this same issue for my Ri3D team and didn't find the solution when shuffling through your code.

RCoder01 commented 8 months ago

I don't know what we did to fix it; it just started working eventually.

Sometimes we still get the issue described in the original issue where after 5-20 seconds it stops, but it no longer instantly exits.

vitojan commented 8 months ago

When connecting via Type B, we found that the robot doesn't encounter this issue.

not-availiable commented 8 months ago

We have encountered this issue as well. We tried installing on different computers, connecting to the robot before opening driverstation, opening driverstation without a robot connected, making sure windows was up to date, removing any logs in our code, and reformatting the radio. Everything resulted in the robot returning to a disabled state anywhere between 2 and 30 seconds after enabling with the message "Internal issue with print and error tags." As a temporary fix, we found that the most recent 2023 version of driverstation is able to function as expected with code that is deployed using 2024 release software onto 2024 release firmware.

Gentry-H commented 8 months ago

My team is also having this issue, does seem to work fine when connecting through the USB port.

sciencewhiz commented 8 months ago

I think the driver station logs are going to be necessary. Particularly comparison from when it isn't working wireless to when it is working with USB.

NiranSchneller commented 8 months ago

We are also having this issue, Haven't found a fix.

ConnorJNorris commented 8 months ago

We have the same issue with our robot. I'm not exactly sure what it is. What I have found is that it gives me that error when I mess with the Ethernet cable on our robot. My driver station keeps disabling, and I noticed that when it disabled, it would sometimes say that there is no robot communication. Maybe something you all could test and see if it causes the same issue.

Debbie386 commented 8 months ago

Same problem here too. However we did notice that the problem has not appeared with our RoboRio 2.0. It has only happened with the original RoboRIO version. Using the USB A/B cable connection rather than WiFi to the robot does work around the problem.

RockyXRQ commented 8 months ago

Same problem here! We got "internal issue with print and error tags" issue report even we only deploy a single motor test code. We use IDEA to build and deploy the code, don't know if this matter

UriTabic commented 8 months ago

I also get "internal issue with print and error tags" when I try to enable, and the robot immediately goes to emergency stop. Haven't found a solution

sciencewhiz commented 8 months ago

and the robot immediately goes to emergency stop.

this is an old issue. Closing and restarting the Driver Station or rebooting the computer should resolve it.

yizhang24 commented 8 months ago

We are also experiencing this issue. Downgrading to 2023 Driver Station appears to have fixed it.

dawonn commented 8 months ago

We are also experiencing this issue. Downgrading to 2023 Driver Station appears to have fixed it.

Same.

NiranSchneller commented 8 months ago

Where can I find the 2023 Driverstation?

Silvxo commented 8 months ago

Where can I find the 2023 Driverstation?

You can find it at the FRC Game Tools Download page and select the desired version, which is indicated by the year. It doesn't have 2023 version, just 2022. But it is a workaround, probably WPI, CTRE or REV will be releasing a note or update fixing this issue soon.

maxlingenfelter commented 8 months ago

We are also experiencing this issue.

marquetterobotics7826 commented 8 months ago

We are experiencing this issue as well. We cannot determine whether or not it has an ill effect on the robot's performance, but we've been experiencing some weird suspicious behavior that could be related to this (command scheduler overrun).

tfield commented 8 months ago

We were seeing the same thing. We had deployed code from both a M1 Mac and the driver station. We rebooted the driver station computer and the problem went away.

dcowden commented 8 months ago

We are having this issue as well. Probably unrelated, but we have also noted that the roborio jvm crashes any time an exception occurs.

The work around to use 2024 code and firmware on the rio, but a 2022 driver station appears to have fixed xed it. MTBF with 2024 driver station was about 30 minutes, but we had no instances in 2 hours with the 2022 driver station

PeterJohnson commented 8 months ago

Thanks for all the reports! NI knows the cause of this and has a fix that will be in the next DS release (Game Tools update). Importantly, the message does not affect robot operation. The message is caused when the DS reports a message on its side intermixed with messages from the robot; it is not caused by and does not affect robot code.

For teams that are seeing the robot disable “at random”, the message is a symptom and not a cause. The disables are due to tighter timeouts on the DS added to improve control safety. The message is caused by the DS attempting to report on the timeout that caused the disable. To further investigate the disable, the most helpful thing is to look at the latency/timing plot (available through the gear icon); if there are large (eg 100s of ms) spikes there, that’s bad, and is what’s causing the disable. The fix is to isolate what other program (or background process) running on the machine is causing the spikes by systematically closing/stopping them until the spikes disappear. Common known culprits are Discord and the Autodesk updater. Sharing back here pictures of the plot and results of figuring out what program is causing it will help us better document this for other teams. Thanks!

ConnorJNorris commented 8 months ago

Thank you for the help, and for working with me.

dcowden commented 8 months ago

@PeterJohnson thank you for that summary! In reading your comment above, am I interpreting correctly that the DS fix will produce a more helpful message, but that the root cause ( the driver station taking too long to respond) will remain. Consequently, if we are having this issue, we have something running on our DS that's taking too long that we need to fix? We are not running discord, autodesk etc-- but we do have some older computers running.

Do you think that the new timing could mean that older computers are no longer fast enough to run the DS? (IE, maybe the minimum hardware requirements for the DS are now more important? )

PeterJohnson commented 8 months ago

Correct. New timeouts were added to the DS this year to address concerns raised last year regarding unsafe operating conditions (the DS remaining enabled even with unsafe latencies). The timeouts are set based on what should be safe operating limits, so there is currently no plan to change/remove them. It appears from team reports that more teams than we expected have likely been operating with unsafe latencies (at least occasionally) with no one realizing it.

dcowden commented 8 months ago

thanks @PeterJohnson

JonathanLindsey commented 8 months ago

It looks like opening File Explorer even while disabled causes trip times to increase for several seconds. Not that we have a need to be opening File Explorer while enabled, but maybe someone can use that info as a reproducible case.

I'll speculate there's some kind of IO blocking in the DS or underlying Windows stack.

sciencewhiz commented 8 months ago

It looks like opening File Explorer even while disabled causes trip times to increase for several seconds. Not that we have a need to be opening File Explorer while enabled, but maybe someone can use that info as a reproducible case.

What OS and computer specifications? What anti-virus? Wired or wireless? I don't see that on my computer, which admittedly is well specced.

JonathanLindsey commented 8 months ago

Tested on 2 different computers, one with Windows 10, the other with Windows 11.
- The Windows 10 computer was a fresh Windows 10 install a few years ago. I do some debloat of Windows. A few other things have been installed. 8 GB RAM and an i7 CPU from about 2014.
- The Windows 11 computer is known to have come with antivirus software but it has been uninstalled. This is a computer used for everyday tasks so has other software installed. Unknown specs, but from the last couple years.
Anti-virus: Both have Windows Defender and shouldn't have any other anti-virus. On the Windows 11 computer we tried disabling a bunch of things in Windows Defender.
Connection: Normal shop wireless setup with the OM5P-AC radio.
Robot: No co-processor or anything special. Just a RevRobotics MAXSwerve testbot.
Code: Reproduced this even with a blank java project.

rakar commented 8 months ago

We saw this check trip a couple of times. It appeared that the only high CPU app was Shuffleboard.

ajahan2008 commented 8 months ago

Same issue for us. Ends up removing both the connection and the robot code, then both come back after 30 or so seconds.

MadHatchetcoding commented 8 months ago

We are also experiencing this issue. Downgrading to 2023 Driver Station appears to have fixed it.

were you still able to use the 2024 Wpilib?

gcschmit commented 8 months ago

For teams that are seeing the robot disable “at random”, the message is a symptom and not a cause. The disables are due to tighter timeouts on the DS added to improve control safety. The message is caused by the DS attempting to report on the timeout that caused the disable. To further investigate the disable, the most helpful thing is to look at the latency/timing plot (available through the gear icon); if there are large (eg 100s of ms) spikes there, that’s bad, and is what’s causing the disable. The fix is to isolate what other program (or background process) running on the machine is causing the spikes by systematically closing/stopping them until the spikes disappear. Common known culprits are Discord and the Autodesk updater. Sharing back here pictures of the plot and results of figuring out what program is causing it will help us better document this for other teams. Thanks!

Here is an example from yesterday when we experienced the "random" robot disable. The spikes are not consistent. I thought we had the laptop pretty well locked down, but we will investigate what all is running.

Image_20240124_203926_505

spellingcat commented 7 months ago

opening the 24.0 driver station immediately gives me this error and also shows that i don't have comms even when connected to the robot's network, so i'm unable to deploy in the first place (even though vscode tells me it got deployed). did not have any issues on a computer w 2024 wpilib and 23.1 driver station (which I can't seem to find on NI anymore?)

dcowden commented 7 months ago

@PeterJohnson I'm sorry if I missed it, but what is the timeout value at which the robot will be disabled? That would really help

sciencewhiz commented 7 months ago

Game Tools 2024 Patch 1 adjusted to the safety mechanism slightly. It also fixed the bug that caused it to print internal issue with print and error tags instead of the message about why it was disabling. If you still have issues, please run through the troubleshooting steps at https://docs.wpilib.org/en/stable/docs/yearly-overview/known-issues.html#driver-station-randomly-disabled

spellingcat commented 7 months ago

installed ds 2024.0.1 and I'm still having the same issue on startup. (no other programs are running). the driver station is giving me this:

this is what the timing evaluation looks like:

i also received this error from NI when I tried to close the DS, not sure what it means:

sciencewhiz commented 7 months ago

@spellingcat Is the robot connected when you took that Timing window screenshot? Is this over Wi-Fi? Does it work when connected via USB? What of the troubleshooting steps in the known issues page have you done, and what were the results?

The message shown is not the message when the timing trips the threshold, so I'm suspecting you have something unique going on. You may get more help if you post on chiefdelphi.

spellingcat commented 7 months ago

The robot was connected over wifi when I took that particular screenshot, but the graph looks similar regardless of if I am connected or not (so I do suspect that maybe my computer is just bad). I haven't tried it over usb.

As for the troubleshooting steps, I've closed all other software and rebooted, wifi drivers are up to date, and no other robots are operating nearby. I'll check the bandwidth/wifi congestion later today.

I am curious-what does warning 44002 mean?

sciencewhiz commented 7 months ago

I am curious-what does warning 44002 mean?

https://docs.wpilib.org/en/stable/docs/software/driverstation/driver-station-errors-warnings.html#ping-status

downsviewSS commented 6 months ago

Today, Team 5031 developed this issue. I am still trying to figure out how to fix this issue. I formatted the roborio, and deleted NI twice. Please, someone help us. The team has two programs. Both programs can deploy, but one of the program driver station says that there are no codes and for other program codes are there, but as soon as we enable the codes to disappear. I guess I am going to download the 2023 driverstation. Team mentor : Sham.syal@TDSB.on.ca

spellingcat commented 6 months ago

im silly it was the firewall :skull:

rzblue commented 1 month ago

The "internal issue with print and error tags" issue was fixed in a game tools update.

wpilibsuite / allwpilib

error report: "internal issue with print and error tags." on the FRC Driver Station. #6174