search

dotchen / LearningByCheating

(CoRL 2019) Driving in CARLA using waypoint prediction and two-stage imitation learning
MIT License
304 stars 103 forks source link

Error in Training teacher model #24

Closed FitashUlHaq closed 4 years ago

FitashUlHaq commented 4 years ago

I have generated 100 episodes using the given script but when i try to train the birdview i am getting this error

MDB_INVALID: File is not an LMDB file

can you please guide me what can be the issue?

any help will be appreciated. Many thanks Fitash

dotchen commented 4 years ago

Make sure you pass the correct path to data when running the training script.

FitashUlHaq commented 4 years ago

the path is correct with two folders train and val both containing different episodes

dotchen commented 4 years ago

Can you print the fullpath here: https://github.com/dianchen96/LearningByCheating/blob/release-0.9.6/bird_view/utils/datasets/birdview_lmdb.py#L64 and see if it makes sense?

If yes then check the dataset size. If both make sense then the issue might be related to the lmdb installation.

dotchen commented 4 years ago

Closing it for now. If this is not solved feel free to reopen it and commen below.

FitashUlHaq commented 4 years ago

I tried:

only few episodes can be used, all others gave the same error

I am using ubunutu 18.04, python 3.6.9, lmdb 0.94

dotchen commented 4 years ago

Can you try running in the python environment we provided in the setup instructions? That uses python 3.5.

FitashUlHaq commented 4 years ago

I tried that and same error as

/media/fitash/Carladata/train/001: MDB_INVALID: File is not an LMDB file

i am using RTX 2080ti

FitashUlHaq commented 4 years ago

which ubuntu version are you using?

I am using 18.04

dotchen commented 4 years ago

Thanks for the details. I used 18.04 and 16.04 and both worked for me. Do the corrupt runs have the correct files in it (i.e lock.mdb etc.) & have the correct file size?

FitashUlHaq commented 4 years ago

the lock files are of the same size while the data file is sometime 2.0gbs sometime 1.8 and 2.2. Mostly the corrupt files are other than the last 6 files

dotchen commented 4 years ago

This is odd... the file sizes look correct to me. I will try your env and see if i can reproduce it. In the mean time can you try what if you just collect 1 episode and see if it is corrupted. Also maybe take a a look at https://stackoverflow.com/questions/44407659/how-to-force-64-bit-lmdb-to-generate-a-32-bit-database if your machine is 32-bit.