search

frc868 / 2017-robot

The code for FRC 868's 2017 robot, Ratchet
1 stars 1 forks source link

Power Management & Brownout Issues #104

Open karagenit opened 7 years ago

karagenit commented 7 years ago

This is where I'm going to be posting all of my problems/findings/solutions for various power management issues.

Reference Docs

RoboRIO Manual (See p.7) WPIlib Brownout Info CIM Datasheet CIM Specs miniCIM Specs 120A Breaker Datasheet 40A Breaker Datasheet PDP Datasheet

Key Info

RoboRIO Brownout Voltage is 6.8v 40A Breakers (at 80A overload or 200%) Trip Time is 1.5 to 3.9 seconds (I'm not sure if Trip Time is how long that current must be applied to trip the breaker or if it's the time it takes it to reset) 120A Breaker Time (at 300A or 250%) Time is 7.5 to 20 seconds (again, not sure what this "time" is) CIM stall current is 133 Amps miniCIM stall current is 89 Amps

Forum Posts

CD 6 CIM Brownouts CD Brownouts & Design CD 4 CIM Brownout CD 6 CIM Voltage Ramping CD Whitepaper CIM Current Draw

Experimental Data

Hammerhead

Driving Against Wall Test:

Ratchet (w/o miniCIMs)

Driving Against Wall Test:

Match Data:

Ratchet (w/ miniCIMs)

Match Data:

Solutions

Mechanical

Electrical

Programming

karagenit commented 7 years ago

Testing Results (20 March 2017)

Driving On Blocks Tests

Experienced some weird behavior while driving on blocks (low current draw) using code that is known to work.

First, the robot would occasionally automatically disable itself. At tippe this would happen after a match: on first boot, it would disable immediately after teleop was enabled. In our case today, it would happen after a minute or so of driving on blocks.

Second, we were experiencing jittering in driving, mostly with the loss-of-comm style issues documented below.

Both issues seemed to stop after testing for a couple minutes, so I just moved on to the next tests.

Driving Against Walls Tests

See PR #116

Power Management Thread successfully prevents current from exceeding 200 Amps continuously (total draw). This causes the 40A auto-reset breakers to pop after ~20 seconds.

While testing against the wall, the DS reported Brownout State on a couple of occasions and successfully disabled the motor outputs (causing a sort of jittering while pushing against the wall).

Rapid Driving Tests

We continued to have comm-loss type of issues, occurring especially after popping the 40A breakers (aka high current draw). To test, I drove the robot back & forth rapidly. The result was for anywhere from 1/4 to 2 seconds of loss of control where the Sparks would continue outputting whatever was last set to them (in other words, the RIO wasn't setting 0 to the motors, as it would/should if brownout state was reached). Current draw didn't exceed 200A, though spiked rapidly from 125-175 Amps. No breakers popped.

When this apparent loss of comm occurred, the status lights on the radio stayed the same. On most occurrences, the DS reported a Brownout State. Once, the DS reported Bad Robot Code (light next to "Robot Code" went red) - DS console didn't show a stack trace, didn't check the RIO log file and we've restarted since then so I think that's lost, and I couldn't replicate the issue.

Tried swapping the ethernet cable to the Radio -> RIO, didn't seem to help.

This is highly similar to what I experienced during a match at tippe. Interestingly, we couldn't replicate this after the matches (while testing) which may have been because we were tethered (indicating a radio issue, dropping wifi comm at a certain voltage).

Future Tests

It's possible this is an issue with the RIO and not the Radio - it could be that the RIO isn't setting 0 to all motors as it should when brownout is detected.

claytondetke commented 7 years ago

Testing Results (21 March 2017)

Today we tried swapping out the Radio with an older one from 2015.

We didn't have a chance to test the automatic brownout detection code, though this may help prevent voltages from dipping below ~7.5v (I think the RIO's cutoff of 6.8v is too low, and is resulting in the RIO dropping comm).

Driving Against Wall Tests

Before we started testing, I noted that we were using an uncharged battery (at ~12.4v before we started) so that likely affected the results of the following tests.

While driving against the wall, the current draw averaged ~150 Amps and the voltage often dipped below 6.8v, causing the RIO to disable motor outputs and causing a "jittering". Yesterday, we rarely saw this, likely because the battery was at a higher charge and thus was able to provide more current with less voltage drop.

I experienced what felt like a little bit of the loss-of-comm issues that we definitely had yesterday. Because of the significant voltage drop, the robot stuttered/jittered even when driving normally, so I can't be sure if the loss of control was because of the RIO disabling motor outputs or if it was the same issue as before.

This is very interesting, as it shows that the loss-of-comm issue is likely not caused by the voltage dropping so low that either the RIO or the Radio looses the ability to communicate with the DS. Instead, this seems to happen during times of excessive current draw.

Finally, we had two instances of the DS reporting No/Bad Robot Code while encountering high current draw/voltage drop (esp. the latter). The first time both the console and logfile showed nothing, leading me to believe that the excessive voltage drop may have forced a partial reboot of the RIO - it was displaying "No Robot Code" as it always does when it boots up.

The second time was different, as there was actually some info in the logfile:

Error at org.usfirst.frc.team868.robot.subsystems.PowerDrawSubsystem.getTotalCurrent(PowerDrawSubsystem.java:18): CTRE CAN Receive Timeout
edu.wpi.first.wpilibj.hal.PDPJNI.getPDPTotalCurrent(Native Method)
edu.wpi.first.wpilibj.PowerDistributionPanel.getTotalCurrent(PowerDistributionPanel.java:77)
org.usfirst.frc.team868.robot.subsystems.PowerDrawSubsystem.getTotalCurrent(PowerDrawSubsystem.java:18)
org.usfirst.frc.team868.robot.subsystems.DriveSubsystem$PowerThread.run(DriveSubsystem.java:232)

Error at org.usfirst.frc.team868.robot.subsystems.TurretRotationSubsystem.getPosition(TurretRotationSubsystem.java:200): CTRE CAN Receive Timeout
 at com.ctre.CanTalonJNI.GetSensorPosition(Native Method)
com.ctre.CANTalon.getPosition(CANTalon.java:884)
org.usfirst.frc.team868.robot.subsystems.TurretRotationSubsystem.getPosition(TurretRotationSubsystem.java:200)
org.usfirst.frc.team868.robot.subsystems.TurretRotationSubsystem$1.pidGet(TurretRotationSubsystem.java:54)
edu.wpi.first.wpilibj.PIDController.calculate(PIDController.java:265)
edu.wpi.first.wpilibj.PIDController$PIDTask.run(PIDController.java:129)
java.util.TimerThread.mainLoop(Timer.java:555)
java.util.TimerThread.run(Timer.java:505)

Java HotSpot(TM) Embedded Client VM warning: INFO: os::commit_memory(0xb4719000, 4096, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 4096 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /tmp/hs_err_pid2062.log
[thread -1399163808 also had an error]

The first two "errors" repeated themselves many times throughout the running of the robot, though they may have occurred only at times of high current draw or voltage drop - we have no way of knowing. The last error, however, only occurred once, and appears to be what caused the code reboot.

I haven't looked at the /tmp/hs_err_pid2062.log file yet, I'll do that tomorrow.

We had to issues with auto-disabling today. I wonder: if the RIO looses connection with the DS, does it automatically disable?

Future Tests

Additional Info

FMS Connectivity Troubleshooting Guide

That document notes that the most common communications issue is the power supply to the RIO.