search

frc868 / 2017-robot

The code for FRC 868's 2017 robot, Ratchet
1 stars 1 forks source link

Out of Memory Issues #119

Closed karagenit closed 7 years ago

karagenit commented 7 years ago

See Issue #104 

Java HotSpot(TM) Embedded Client VM warning: INFO: os::commit_memory(0xb4719000, 4096, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 4096 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /tmp/hs_err_pid2062.log
[thread -1399163808 also had an error]
karagenit commented 7 years ago

Testing Results (22 March 2017)

Started Robot, opened Camera stream on Smart Dashboard, did not enable teleop/auton. SSH'd to RIO and ran top command to monitor memory usage.

At 4:35: 75k RAM free At 4:43: 28k RAM free

Crashed shortly after hitting 28k free.

karagenit commented 7 years ago

Second test confirms this issue, 46k free after only 4 minutes. Crashed at ~29k free after 6 minutes.

karagenit commented 7 years ago

Java JRE - 174m NIwebserver - 61k MainAppThread - 47k SystemWebServer - 26k FRC Net Comm Daemon - 24k

After crash, JRE was at 177m, others seem unchanged.

karagenit commented 7 years ago

The CameraSubsystem patch that frees up used MATs didn't fix it. Interesting though, one time it actually appeared to correct itself: it jumped from ~40k free to ~55k free while I wasn't paying attention (without crashing the code).

Also, when the code crashes because of excessive memory usage it appears to not automatically restart itself like it usually does when an exception is thrown. Additionally, even after a re-deploy you must manually reboot the RIO to get it to work again. Maybe the JVM itself is dying?

karagenit commented 7 years ago

Testing by Disabling Threads

Free MATs: Did not fix

Power Manager Thread: Did not fix (though robot code did restart itself)

Camera Thread: Did not fix

Pixy Dump & SmartDashboard: Somewhat helped?

Just Pixy Dump: Seems to be declining

Other Tests

Simple Robot Project: Fine

Testing Ideas

Try teleop running Disable updatesmartdashboard empty project

karagenit commented 7 years ago

Based on the above, it seems like it may be the UpdateSmartDashboard command that is causing this issue (though we need further testing to confirm this). In the meantime, I'm going to try to write some fixes for the UpdateSmartDashboard command.

Testing Ideas