TheLogicMaster
commented
6 years ago So, would it be possible to just create a new thread and switch from a buffered writer to a manual one and just periodically flush it while making the writer synchronized? I would think that the overlap between when the main thread and the writing thread are both trying to access it would hold the main the thread less than just having it on one thread, especially if we substantially decreased the frequency it flushes at.
On Tue, Apr 3, 2018, 10:18 AM scarmain notifications@github.com wrote:
One potential issue I see. Currently, every robot loop, we log a bunch of variables to a CSV file. This is not inherently bad, and actually encouraged (our brake modules log 2200 channels at 5ms).
My concern is the file buffer writing/flushing. Currently, file writing occurs on the same thread as robot processing, and I wonder if some of the lag issues the drivers report while driving is related to this file writing and whatever the base Linux wants to do.
Is it possible to move at least the file writing to a new thread (so that Java hopefully would throw it to core 2 and our program runs smoother). I would only fix the update() function first (since it runs every loop, maybe a couple of the notify logs. I think the notification log might be fine as is because it isn't used much except for big transitions (like Auto to Teleop).
Multicore programming is tough, making sure you don't read/write variables that another core is using... The simplest way I could come up to solve this problem... Have a thread that sleeps for 20ms, then checks a List if it has any contents (put a lock around it). The logger code that is there is fine, all that needs to change is instead of writing directly to the file, fill the List buffer and let the thread empty it. (you might want to use ConcurrentList, since it is thread safe)
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/FRC2832/Robot_2018/issues/8, or mute the thread https://github.com/notifications/unsubscribe-auth/AMzf9DFRe8odkT6sfjQ38d7MRjIFiNAmks5tk4TBgaJpZM4TFKdU .
scarmain
One potential issue I see. Currently, every robot loop, we log a bunch of variables to a CSV file. This is not inherently bad, and actually encouraged (our brake modules log 2200 channels at 5ms).
My concern is the file buffer writing/flushing. Currently, file writing occurs on the same thread as robot processing, and I wonder if some of the lag issues the drivers report while driving is related to this file writing and whatever the base Linux wants to do.
Is it possible to move at least the file writing to a new thread (so that Java hopefully would throw it to core 2 and our program runs smoother). I would only fix the update() function first (since it runs every loop, maybe a couple of the notify logs. I think the notification log might be fine as is because it isn't used much except for big transitions (like Auto to Teleop).
Multicore programming is tough, making sure you don't read/write variables that another core is using... The simplest way I could come up to solve this problem... Have a thread that sleeps for 20ms, then checks a List if it has any contents (put a lock around it). The logger code that is there is fine, all that needs to change is instead of writing directly to the file, fill the List buffer and let the thread empty it. (you might want to use ConcurrentList, since it is thread safe)
Also, would it make sense to flush less? Do you really need it every 20ms? Could it be 1 sec? I guess if we browned out, the 20ms refresh rate would be nice, but the robot would be sitting at the end of the match for 30+ sec before shut off, so we don't really need that data. Only problem is longer flush = more data to process each flush = laggy robot unless it's on another core...