search

xinntao / EDVR

Winning Solution in NTIRE19 Challenges on Video Restoration and Enhancement (CVPR19 Workshops) - Video Restoration with Enhanced Deformable Convolutional Networks. EDVR has been merged into BasicSR and this repo is a mirror of BasicSR.
https://github.com/xinntao/BasicSR
1.49k stars 318 forks source link

About the create_lmdb_mp.py #39

Open zrongcheng opened 5 years ago

zrongcheng commented 5 years ago
  1. I created my dataset, my dataset had 200 folders(included val, total 120GB), but i got error, it's said that i didn't have enough memory; Thus, i want to know what the hardware configuration.
  2. Did you test your code with VMAF? The video train with rgb or yuv420 whether get the different result by VMAF?
zestloveheart commented 5 years ago

for 1........do you meet the error when you create lmdb?

xinntao commented 5 years ago

  1. The script reads all the images to the memory. If you want to read images one by one, you need to change some codes. Please refer to this.

  2. No, we have not. Did you mean this by VMAF?

zrongcheng commented 5 years ago
  1. Ok, thanks.
  2. Yes, VMAF is a good assessment norm.
xinntao commented 5 years ago

Thanks, I will have a look at VMAF.

crypticsymmetry commented 5 years ago

What changes to create_lmdb_mp.py did you make for it to use less memory? Thanks!

xinntao commented 5 years ago

@crypticsymmetry For fast IO speed during training.

crypticsymmetry commented 5 years ago

sorry, i meant what changes to Create lmdb did he make to not load all images at one time, as i have a max of 32gb of RAM. i checked out the basicSR version but didnt understand how to implement it to EDVR's create lmdb file. thanks for your time

xinntao commented 5 years ago

@crypticsymmetry I will update a version with limited memory consumption tmr.

xinntao commented 5 years ago

@crypticsymmetry I have update the create_lmdb_mp.py. You can use it to generate the lmdb files.