search

autorope / donkeycar

Open source hardware and software platform to build a small scale self driving car.
http://www.donkeycar.com
MIT License
3.16k stars 1.3k forks source link

Cannot train Donkeycar #815

Closed ender18g closed 3 years ago

ender18g commented 3 years ago

If I try the train command, I get the follow mmap error:

Any ideas what I could be doing wrong?

Total params: 817,028 Trainable params: 817,028 Non-trainable params: 0

None Using catalog /content/mycar/data/data/catalog_27.catalog Traceback (most recent call last): File "/content/mycar/train.py", line 28, in main() File "/content/mycar/train.py", line 24, in main train(cfg, tubs, model, model_type) File "/content/donkeycar/donkeycar/pipeline/training.py", line 101, in train dataset = TubDataset(cfg, all_tub_paths) File "/content/donkeycar/donkeycar/pipeline/types.py", line 78, in init for tub_path in self.tub_paths] File "/content/donkeycar/donkeycar/pipeline/types.py", line 78, in for tub_path in self.tub_paths] File "/content/donkeycar/donkeycar/parts/tub_v2.py", line 26, in init read_only=read_only) File "/content/donkeycar/donkeycar/parts/datastore_v2.py", line 254, in init self.current_catalog = Catalog(last_known_catalog, read_only=self.read_only, start_index=self.current_index) File "/content/donkeycar/donkeycar/parts/datastore_v2.py", line 144, in init self.seekable = Seekable(self.path.as_posix(), line_lengths=self.manifest.line_lengths(), read_only=read_only) File "/content/donkeycar/donkeycar/parts/datastore_v2.py", line 25, in init self.file = mmap.mmap(self.file.fileno(), length=0, access=mmap.ACCESS_READ) ValueError: cannot mmap an empty file

DocGarbanzo commented 3 years ago

@ender18g, can you add a comment how you solved it - maybe your tub was empty? Might be useful for other people who see the same error.

tikurahul commented 3 years ago

ValueError: cannot mmap an empty file is the error.

My guess is that the tub did not exist.

ender18g commented 3 years ago

The mmap error was a result of trying to train multiple tubs of data at once. Within my data directory, I had 4 tubs, which I wanted to all be included in my model training.

However, when I used the following command: donkey train --tub ./data/* --model ./models/mypilot.h5 I got the ValueError: cannot mmap an empty file

The only way I could avoid getting the error was to train each tub, one by one. donkey train --tub ./data/tub_1_2020_03_16 --model ./models/mypilot.h5

So, I got rid of the error, but according to the donkeycar docs, ./data/* should be a usable command to train using all tubs.

Also, none of the tubs were empty.

DocGarbanzo commented 3 years ago

It looks like you have a non-tub directory under your ./data/ directory, can you check? I think we can improve the error message in that case.

qasimalik134 commented 3 years ago

@ender18g Could you please share your dataset for training.

ender18g commented 3 years ago

Thanks for the responses. After some more investigation, it appears that all directories are indeed tub directories. However, the 1st tub (tub1.... ) was an empty directory. The other tubs had content.

The easy fix to is to simply find and delete the empty. I'm not exactly sure what causes an empty tub, but it could be visiting the donkey car web address and then refreshing the page without driving/producing any data.