assert len(data_loader) > 0，AssertionError

[JoyeMing]

Thank you for your work. I encountered this kind of error when running the Imagenet dataset. Have you encountered any similar errors? How did you solve it? Traceback (most recent call last): File "train.py", line 207, in main(args) File "train.py", line 142, in main trainer = build_trainer(cfg) File "/home/dpsh/Dassl.pytorch/dassl/engine/build.py", line 11, in build_trainer return TRAINER_REGISTRY.get(cfg.TRAINER.NAME)(cfg) File "/home/dpsh/Dassl.pytorch/dassl/engine/trainer.py", line 319, in init self.build_data_loader() File "/home/dpsh/Dassl.pytorch/dassl/engine/trainer.py", line 342, in build_data_loader dm = DataManager(self.cfg) File "/home/dpsh/Dassl.pytorch/dassl/data/data_manager.py", line 128, in init test_loader = build_data_loader( File "/home/dpsh/Dassl.pytorch/dassl/data/data_manager.py", line 45, in build_data_loader assert len(data_loader) > 0 AssertionError

[KaiyangZhou]

more details on how you run the code?

[JoyeMing]

run the code with:bash scripts/coop/main.sh imagenet rn50_ep50 end 16 1 False and imagenet are configured according to datasets.md

[KaiyangZhou]

you can check the source code here https://github.com/KaiyangZhou/Dassl.pytorch/blob/master/dassl/data/data_manager.py#L38

the error you got, assert len(data_loader) > 0, means the data loader has zero length and hence no minibatches were loaded

it might happen when drop_last=True and batch_size is bigger than the dataset size

anyway, I'd suggest you take a look at your dataset and put some stop points like import pdb; pdb.set_trace() in the data_manager.py

what I'm sure about is you don't need to change any source code of dassl in order to get this CoOp code running

[JoyeMing]

Thank you for your reply. I rechecked the contents of the Imagenet dataset. There are 1000 files in the train folder that are consistent with the remarks of datasets.md. There is no error. What is the content of the Val folder? I use a 6.7g Val with 50000 pictures.

[KaiyangZhou]

A standard imagenet dataset has a train folder and a val folder. Isn't it?

[JoyeMing]

Good, thank you. It seems that my Val version is wrong. Thank you for your reply

[zhaoshitian]

Good, thank you. It seems that my Val version is wrong. Thank you for your reply

I meet the same problem. Did you fix it now?

[JoyeMing]

classifying pictures can solve the problem

[Pefect96]

classifying pictures can solve the problem

I meet the same problem, but after I classified the val fold, it still does not work.

[htzhang-code]

classifying pictures can solve the problem

I meet the same problem, but after I classified the val fold, it still does not work.

Did you fix it now?

[htzhang-code]

classifying pictures can solve the problem

I meet the same problem, but after I classified the val fold, it still does not work.

Did you fix it now?

I solve it. The reason is that the preprocessed.pkl was created, but no information was written. This causes datasets/imagenet.py to not execute correctly.

[lhrst]

classifying pictures can solve the problem

I meet the same problem, but after I classified the val fold, it still does not work.

Did you fix it now?

I solve it. The reason is that the preprocessed.pkl was created, but no information was written. This causes datasets/imagenet.py to not execute correctly.

whoa, thanks for solving the problem, this helps a lot!

[KevinLi-167]

Run './scripts/coop/main.sh imagenet rn50_ep50 end 16 1 False' faild.

Using https://kkgithub.com/soumith/imagenetloader.torch/blob/master/valprep.sh to divide the val.

After the val division, the good news is that the 4 PKLs have content. The bad news is that this error is still reported.

[ioslide]

imagenet.py changes line 78 to 91 to: `

    def read_data(self, classnames, split_dir):
            split_dir = os.path.join(self.image_dir, split_dir)
            folders = sorted(f.name for f in os.scandir(split_dir) if f.is_dir())
            items = []

    if len(folders) > 0:
        for label, folder in enumerate(folders):
            imnames = listdir_nohidden(os.path.join(split_dir, folder))
            classname = classnames[folder]
            for imname in imnames:
                impath = os.path.join(split_dir, folder, imname)
                item = Datum(impath=impath, label=label, classname=classname)
                items.append(item)
    else:
        imnames = listdir_nohidden(split_dir)
        for imname in imnames:
            impath = os.path.join(split_dir, imname)
            item = Datum(impath=impath, label=None, classname=None)
            items.append(item)

    return items

`