search

PhotonVision / photonvision

PhotonVision is the free, fast, and easy-to-use computer vision solution for the FIRST Robotics Competition.
https://photonvision.org
GNU General Public License v3.0
262 stars 169 forks source link

Crashes under Simulation #823

Open randomstring opened 1 year ago

randomstring commented 1 year ago

I'm trying to test out the PhotonPoseEstimator with simulation, and am running into a series of crashes.

I tried this with two different PoseStrategy PNP and Lowest Ambiguity. The last thing in the stack trace from my code is calling setReferencePose() which should be somewhat harmless?

I am running two cameras, and I wonder if there's something not thread safe or the simcamera isn't expecting more than one instance.

with PNP

 Error at org.photonvision.PhotonPoseEstimator.reportFiducialPoseError(PhotonPoseEstimator.java:646): [PhotonPoseEstimator] Tried to get pose of unknown AprilTag: 0
Error at org.photonvision.common.dataflow.structures.Packet.decodeByte(Packet.java:135): Unhandled exception: java.lang.ArrayIndexOutOfBoundsException: Index 1066 out of bounds for length 1066
        at org.photonvision.common.dataflow.structures.Packet.decodeByte(Packet.java:135)
        at org.photonvision.targeting.PhotonTrackedTarget.decodeList(PhotonTrackedTarget.java:233)
        at org.photonvision.targeting.PhotonTrackedTarget.createFromPacket(PhotonTrackedTarget.java:268)
        at org.photonvision.targeting.PhotonPipelineResult.createFromPacket(PhotonPipelineResult.java:141)
        at org.photonvision.PhotonCamera.getLatestResult(PhotonCamera.java:186)
        at org.photonvision.PhotonPoseEstimator.update(PhotonPoseEstimator.java:252)
        at frc.lib.team3061.vision.Vision.updatePoseCombined(Vision.java:188)
        at frc.lib.team3061.vision.Vision.periodic(Vision.java:155)
        at edu.wpi.first.wpilibj2.command.CommandScheduler.run(CommandScheduler.java:280)
        at frc.robot.Robot.robotPeriodic(Robot.java:198)
        at edu.wpi.first.wpilibj.IterativeRobotBase.loopFunc(IterativeRobotBase.java:356)
        at org.littletonrobotics.junction.LoggedRobot.startCompetition(LoggedRobot.java:90)
        at edu.wpi.first.wpilibj.RobotBase.runRobot(RobotBase.java:349)
        at edu.wpi.first.wpilibj.RobotBase.lambda$startRobot$0(RobotBase.java:422)
        at java.base/java.lang.Thread.run(Thread.java:833)

with least ambiguity


Unhandled exception during listener callback: java.lang.ArrayIndexOutOfBoundsException: Index 714 out of bounds for length 714
java.lang.ArrayIndexOutOfBoundsException: Index 714 out of bounds for length 714
        at org.photonvision.common.dataflow.structures.Packet.decodeByte(Packet.java:135)
        at org.photonvision.targeting.PhotonTrackedTarget.decodeList(PhotonTrackedTarget.java:233)
        at org.photonvision.targeting.PhotonTrackedTarget.createFromPacket(PhotonTrackedTarget.java:268)
        at org.photonvision.targeting.PhotonPipelineResult.createFromPacket(PhotonPipelineResult.java:141)
        at org.photonvision.PhotonCamera.getLatestResult(PhotonCamera.java:186)
        at frc.lib.team3061.vision.VisionIOSim.lambda$0(VisionIOSim.java:70)
        at edu.wpi.first.networktables.NetworkTableInstance$ListenerStorage.lambda$startThread$0(NetworkTableInstance.java:776)
        at java.base/java.lang.Thread.run(Thread.java:833)
Error at org.photonvision.PhotonPoseEstimator.reportFiducialPoseError(PhotonPoseEstimator.java:646): [PhotonPoseEstimator] Tried to get pose of unknown AprilTag: 0
Error at org.photonvision.common.dataflow.structures.Packet.decodeByte(Packet.java:135): Unhandled exception: java.lang.ArrayIndexOutOfBoundsException: Index 1066 out of bounds for length 1066
        at org.photonvision.common.dataflow.structures.Packet.decodeByte(Packet.java:135)
        at org.photonvision.targeting.PhotonTrackedTarget.decodeList(PhotonTrackedTarget.java:233)
        at org.photonvision.targeting.PhotonTrackedTarget.createFromPacket(PhotonTrackedTarget.java:268)
        at org.photonvision.targeting.PhotonPipelineResult.createFromPacket(PhotonPipelineResult.java:141)
        at org.photonvision.PhotonCamera.getLatestResult(PhotonCamera.java:186)
        at org.photonvision.PhotonPoseEstimator.update(PhotonPoseEstimator.java:252)
        at frc.lib.team3061.vision.Vision.updatePoseCombined(Vision.java:188)
        at frc.lib.team3061.vision.Vision.periodic(Vision.java:155)
        at edu.wpi.first.wpilibj2.command.CommandScheduler.run(CommandScheduler.java:280)
        at frc.robot.Robot.robotPeriodic(Robot.java:198)
        at edu.wpi.first.wpilibj.IterativeRobotBase.loopFunc(IterativeRobotBase.java:356)
        at org.littletonrobotics.junction.LoggedRobot.startCompetition(LoggedRobot.java:90)
        at edu.wpi.first.wpilibj.RobotBase.runRobot(RobotBase.java:349)
        at edu.wpi.first.wpilibj.RobotBase.lambda$startRobot$0(RobotBase.java:422)
        at java.base/java.lang.Thread.run(Thread.java:833)
randomstring commented 1 year ago

OK, so I commented out the calls to setReferencePose() as they weren't getting used anyway. Still crashing.

"Index 362 out of bounds for length 362" would suggest improper bounds checking.

    Unhandled exception during listener callback: java.lang.ArrayIndexOutOfBoundsException: Index 362 out of bounds for length 362
java.lang.ArrayIndexOutOfBoundsException: Index 362 out of bounds for length 362
        at org.photonvision.common.dataflow.structures.Packet.decodeByte(Packet.java:135)
        at org.photonvision.targeting.PhotonTrackedTarget.decodeList(PhotonTrackedTarget.java:233)
        at org.photonvision.targeting.PhotonTrackedTarget.createFromPacket(PhotonTrackedTarget.java:268)
        at org.photonvision.targeting.PhotonPipelineResult.createFromPacket(PhotonPipelineResult.java:141)
        at org.photonvision.PhotonCamera.getLatestResult(PhotonCamera.java:186)
        at frc.lib.team3061.vision.VisionIOSim.lambda$0(VisionIOSim.java:70)
        at edu.wpi.first.networktables.NetworkTableInstance$ListenerStorage.lambda$startThread$0(NetworkTableInstance.java:776)
        at java.base/java.lang.Thread.run(Thread.java:833)
Error at org.photonvision.PhotonPoseEstimator.reportFiducialPoseError(PhotonPoseEstimator.java:646): [PhotonPoseEstimator] Tried to get pose of unknown AprilTag: 393900033
Error at org.photonvision.PhotonPoseEstimator.reportFiducialPoseError(PhotonPoseEstimator.java:646): [PhotonPoseEstimator] Tried to get pose of unknown AprilTag: 0
Unhandled exception during listener callback: java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException
Error at org.photonvision.PhotonPoseEstimator.reportFiducialPoseError(PhotonPoseEstimator.java:646): [PhotonPoseEstimator] Tried to get pose of unknown AprilTag: -2147483648
Unhandled exception: java.lang.ArrayIndexOutOfBoundsException
Warning at edu.wpi.first.wpilibj.RobotBase.runRobot(RobotBase.java:364): The robot program quit unexpectedly. This is usually due to a code error.
  The above stacktrace can help determine where the error occurred.
randomstring commented 1 year ago

I think the problem is here

https://github.com/PhotonVision/photonvision/blob/5b86360b9b911790bda9821da832a90a21e12323/photon-targeting/src/main/java/org/photonvision/common/dataflow/structures/Packet.java#L131-L136

I think line 132 needs to be:

if (packetData.length <= readPos) {
gerth2 commented 1 year ago

Concur the decodeByte implementation will read off the end of the array.

The questions in my head that I don't know offhand, but deserve investigation: 1) Does this happen on all decode operations? If not, why (what's the trigger here)? 2) Is it truly sim specific? or can the trigger be done on a real robot too (would be a major issue)

FWIW we have been running latest release for quite some time on the robot, with a variety of different NT connected/not-connected/reconnect and 0-4 targets visible. While this isn't proof positive that it always works, I would go as far to say that I don't expect all decode operations to be broken.

randomstring commented 1 year ago

Keep in mind my crashes are happening while simulating two cameras. Maybe there is a common buffer that's getting used by both cameras.

randomstring commented 1 year ago

So playing with simulation some more and I consistently get index out of bounds errors. Not sure how this happened, but PhotonPipelineResult reported 63 targets.

Unhandled exception during listener callback: java.lang.ArrayIndexOutOfBoundsException: Index 362 out of bounds for length 362 VISION left [2] right: [63] 19.797216, 19.77380182405305

The index off by one error is an issue, but I suspect the real issue is that somewhere it's getting random garbage in vision packets.

randomstring commented 1 year ago

I think the problem lies in the sim's network table listener synchronization. My code is based off of this code from team 3061. Which in turn is based off of code by 6328 Mechanical Advantage.

https://github.com/HuskieRobotics/3061-lib/blob/main/src/main/java/frc/lib/team3061/vision/VisionIOSim.java

Something about creating two listeners, one for each camera was causing the problem. As soon as I disabled one of the listeners all the bad buffers went away.