robotika / osgar

Open Source Garden (Autonomous) Robot
MIT License
13 stars 11 forks source link

osgar/logger.py - support gzipped logfiles #980

Closed m3d closed 3 months ago

m3d commented 4 months ago

Sometimes the content of OSGAR logs is wasteful, for example if you need to log raw sensor data and they are mostly zeros. One possibility is to gzip the stream, but an alternative (potentially faster for run-time) is to gzip OSGAR log file in some post-processing. I see now on robot Pat relatively large file and gzip helps a little bit there:

(osgar) robot@screwbot-ODYSSEY-X86J4125:~/git/osgar$ ls -lh *240302*
-rw-rw-r-- 1 robot robot 784M bře  2 15:38 fr07-followpath-240302_143722.log
-rw-rw-r-- 1 robot robot 677M bře  2 15:43 fr07-followpath-240302_144238.log
-rw-rw-r-- 1 robot robot 205M bře  2 15:46 fr07-followpath-240302_144603.log
-rw-rw-r-- 1 robot robot  52M bře  2 15:46 fr07-followpath-240302_144650.log
-rw-rw-r-- 1 robot robot 406M bře  2 15:49 fr07-followpath-240302_144850.log
-rw-rw-r-- 1 robot robot 373M bře  2 15:55 fr07-followpath-240302_145525.log
-rw-rw-r-- 1 robot robot 337M bře  2 15:58 fr07-followpath-240302_145817.log
-rw-rw-r-- 1 robot robot  63M bře  2 15:40 fr07-go-240302_144020.log
-rw-rw-r-- 1 robot robot 6,3M bře  2 15:41 fr07-go-240302_144126.log
-rw-rw-r-- 1 robot robot 107M bře  2 15:44 fr07-go-240302_144432.log

robot@screwbot-ODYSSEY-X86J4125:~/git/osgar$ ls -lh *240302*
-rw-rw-r-- 1 robot robot 540M bře  2 15:38 fr07-followpath-240302_143722.log.gz
-rw-rw-r-- 1 robot robot 469M bře  2 15:43 fr07-followpath-240302_144238.log.gz
-rw-rw-r-- 1 robot robot 136M bře  2 15:46 fr07-followpath-240302_144603.log.gz
-rw-rw-r-- 1 robot robot  31M bře  2 15:46 fr07-followpath-240302_144650.log.gz
-rw-rw-r-- 1 robot robot 281M bře  2 15:49 fr07-followpath-240302_144850.log.gz
-rw-rw-r-- 1 robot robot 255M bře  2 15:55 fr07-followpath-240302_145525.log.gz
-rw-rw-r-- 1 robot robot 227M bře  2 15:58 fr07-followpath-240302_145817.log.gz
-rw-rw-r-- 1 robot robot  42M bře  2 15:40 fr07-go-240302_144020.log.gz
-rw-rw-r-- 1 robot robot 3,9M bře  2 15:41 fr07-go-240302_144126.log.gz
-rw-rw-r-- 1 robot robot  71M bře  2 15:44 fr07-go-240302_144432.log.gz
m3d commented 4 months ago

p.s. well class LogIndexedReader is not supported and that's probably why I did not backported :( ... work in progress

tajgr commented 3 months ago

p.s. well class LogIndexedReader is not supported and that's probably why I did not backported :( ... work in progress

It is connected with lidarview. Add some support warning? What is current state?

m3d commented 3 months ago

well, I did not resolve this yet (other priorities, I will probably temporarily close it until I solve it). In the other project I use replay and draw option directly in modules so it does not matter that lidarview is not supported there ...

m3d commented 3 months ago

temporarily (?) paused, until lidarview is resolved