search

victorca25 / traiNNer

traiNNer: Deep learning framework for image and video super-resolution, restoration and image-to-image translation, for training and testing.
Apache License 2.0
293 stars 39 forks source link

lmdb has no valid image file #70

Closed MubarakHAlketbi closed 1 year ago

MubarakHAlketbi commented 2 years ago

i get this error

  File "C:\ManduScale\Train\codes\train.py", line 500, in <module>
    main()
  File "C:\ManduScale\Train\codes\train.py", line 487, in main
    dataloaders, data_params = get_dataloaders(opt)
  File "C:\ManduScale\Train\codes\train.py", line 134, in get_dataloaders
    dataset = create_dataset(dataset_opt)
  File "C:\ManduScale\Train\codes\data\__init__.py", line 79, in create_dataset
    dataset = D(dataset_opt)
  File "C:\ManduScale\Train\codes\data\aligned_dataset.py", line 41, in __init__
    self.A_paths, self.B_paths = get_dataroots_paths(self.opt, strict=False, keys_ds=self.keys_ds)
  File "C:\ManduScale\Train\codes\data\base_dataset.py", line 235, in get_dataroots_paths
    paths_A, paths_B = read_dataroots(opt, keys_ds=keys_ds)
  File "C:\ManduScale\Train\codes\data\base_dataset.py", line 168, in read_dataroots
    paths_A, paths_B = paired_dataset_validation(A_images_paths, B_images_paths,
  File "C:\ManduScale\Train\codes\data\base_dataset.py", line 99, in paired_dataset_validation
    A_paths = get_image_paths(data_type, paths[0], max_dataset_size)  # get image paths
  File "C:\ManduScale\Train\codes\dataops\common.py", line 82, in get_image_paths
    paths = sorted(_get_paths_from_images(dataroot, max_dataset_size=max_dataset_size))
  File "C:\ManduScale\Train\codes\dataops\common.py", line 43, in _get_paths_from_images
    assert images, '{:s} has no valid image file'.format(path)
AssertionError: C:\ManduScale\OPScale\DataSet\FourthSet\LR1.lmdb has no valid image file

my config is 

dataroot_HR: ['C:\ManduScale\OPScale\DataSet\FourthSet\HR', 
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR', 
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR', 
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR', 
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR', 
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR', 
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR', 
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR', 
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR', 
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR', 
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR', 
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR', 
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR', 
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR', 
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR', 
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR', 
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR', 
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR', 
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR',  
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR', 
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR', 
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR', 
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR', 
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR', 
    ]
    dataroot_LR: ['C:\ManduScale\OPScale\DataSet\FourthSet\LR1.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR2.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR3.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR4.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR5.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR6.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR7.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR8.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR9.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR10.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR11.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR12.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR13.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR14.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR15.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR16.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR17.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR18.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR19.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR20.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR22.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR23.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR24.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR25.lmdb',
    ]

anyone can help me fix this issue?

MubarakHAlketbi commented 2 years ago

i tried

    dataroot_B: ['C:\ManduScale\OPScale\DataSet\FourthSet\HR1\HR1.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\HR1\HR1.lmdb']
    dataroot_A: ['C:\ManduScale\OPScale\DataSet\FourthSet\LR1\LR1a.lmdb',
    'C:\ManduScale\OPScale\DataSet\FourthSet\LR1\LR1b.lmdb']

and i still game error AssertionError: C:\ManduScale\OPScale\DataSet\FourthSet\LR1\LR1a.lmdb has no valid image file

MubarakHAlketbi commented 2 years ago

the only thing that works is specifying 1 HR.lmdb and 1 LR.lmdb and i cant use it like the above example.

can this be fixed?

victorca25 commented 2 years ago

Hello! When using lmdb, only one lmdb for LR (input) images and one for HR (target) images is supported. Also, when using lmdb, it is only supported if both the LR and HR are a lmdb db.

Technically, it would be possible to have multiple DBs simultaneously, but it adds complexity and doesn't adds benefits during training. In this case I would recommend to create each of the lmdb with all the images that will be used during training, instead of multiple separate ones.