Requirements to run the scripts?

rwightman / udacity-driving-reader

Quick docker based reader for udacity rosbag self-driving dataset. Dumps to png/jpg + csv or Tensorflow examples.

Apache License 2.0

162 stars 64 forks source link

Requirements to run the scripts? #8

Closed ghost closed 6 years ago

ghost commented 6 years ago

Hello there, was wondering if there are many machine specs required to run the script? because I'm trying to run it on a large dataset (80 GBs) and it's stuck at processing dataset, never making any progress or writing any images and it causes my ubuntu to freeze. Thanks

rwightman commented 6 years ago

I have no idea what the specific reqs are, but the process of just opening the bag file can take a very long time with huge bag files that have lots of entries. It's not the most scalable file format and the whole index of all entries is scanned and parsed whenever you open it. I've seen this take many minutes on fast machines with SSD for very large files.