COCO checksum of downloaded file does not match the expected checksum

[sTranaeus]

❓ Questions and Help

When running !mmf_run config="experiment.yaml" model=mmbt dataset=coco I get a checksum error with the first download of COCO. Output below:

Overriding option config to experiment.yaml
Overriding option model to mmbt
Overriding option datasets to coco
Distributed Init (Rank 1): tcp://localhost:19541
Distributed Init (Rank 2): tcp://localhost:19541
Distributed Init (Rank 0): tcp://localhost:19541
Initialized Host nms-zeus.nms.kcl.ac.uk as Rank 1
Initialized Host nms-zeus.nms.kcl.ac.uk as Rank 2
Initialized Host nms-zeus.nms.kcl.ac.uk as Rank 0
Using seed 37206478
Logging to: ./save/logs/train_2020_07_16T11_57_37.log
Downloading annotations.tar.gz: 100%|██████| 62.3M/62.3M [00:07<00:00, 8.60MB/s]
Traceback (most recent call last):
  File "/home/k1762177/hateful_memes/bin/mmf_run", line 33, in <module>
    sys.exit(load_entry_point('mmf', 'console_scripts', 'mmf_run')())
  File "/home/k1762177/mmf/mmf_cli/run.py", line 108, in run
    nprocs=config.distributed.world_size,
  File "/home/k1762177/hateful_memes/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 200, in spawn
    return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
  File "/home/k1762177/hateful_memes/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 158, in start_processes
    while not context.join():
  File "/home/k1762177/hateful_memes/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 119, in join
    raise Exception(msg)
Exception: 

-- Process 0 terminated with the following error:
Traceback (most recent call last):
  File "/home/k1762177/hateful_memes/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 20, in _wrap
    fn(i, *args)
  File "/home/k1762177/mmf/mmf_cli/run.py", line 55, in distributed_main
    main(configuration, init_distributed=True, predict=predict)
  File "/home/k1762177/mmf/mmf_cli/run.py", line 41, in main
    trainer.load()
  File "/home/k1762177/mmf/mmf/trainers/mmf_trainer.py", line 38, in load
    super().load()
  File "/home/k1762177/mmf/mmf/trainers/base_trainer.py", line 38, in load
    self.load_datasets()
  File "/home/k1762177/mmf/mmf/trainers/mmf_trainer.py", line 62, in load_datasets
    self.dataset_loader.load_datasets()
  File "/home/k1762177/mmf/mmf/common/dataset_loader.py", line 17, in load_datasets
    self.train_dataset.load(self.config)
  File "/home/k1762177/mmf/mmf/datasets/multi_dataset_loader.py", line 113, in load
    self.build_datasets(config)
  File "/home/k1762177/mmf/mmf/datasets/multi_dataset_loader.py", line 130, in build_datasets
    dataset_instance = build_dataset(dataset, dataset_config, self.dataset_type)
  File "/home/k1762177/mmf/mmf/utils/build.py", line 106, in build_dataset
    builder_instance.build_dataset(config, dataset_type)
  File "/home/k1762177/mmf/mmf/datasets/base_dataset_builder.py", line 77, in build_dataset
    self.build(config, dataset_type, *args, **kwargs)
  File "/home/k1762177/mmf/mmf/datasets/mmf_dataset_builder.py", line 64, in build
    self._download_requirement(config, self.dataset_name, self.zoo_variation)
  File "/home/k1762177/mmf/mmf/datasets/mmf_dataset_builder.py", line 114, in _download_requirement
    resources, download_path, version, "annotations"
  File "/home/k1762177/mmf/mmf/datasets/mmf_dataset_builder.py", line 152, in _download_based_on_attribute
    self._download_resources(resources.get(attribute, []), path, version)
  File "/home/k1762177/mmf/mmf/datasets/mmf_dataset_builder.py", line 155, in _download_resources
    download.download_resources(resources, path, version)
  File "/home/k1762177/mmf/mmf/utils/download.py", line 399, in download_resources
    download_resource(resource, download_path)
  File "/home/k1762177/mmf/mmf/utils/download.py", line 408, in download_resource
    resource.download_file(download_path)
  File "/home/k1762177/mmf/mmf/utils/download.py", line 160, in download_file
    self.checksum(download_path)
  File "/home/k1762177/mmf/mmf/utils/download.py", line 126, in checksum
    f"[ Checksum for {self._file_name} from \n{self._url}\n"
AssertionError: [ Checksum for annotations.tar.gz from 
https://dl.fbaipublicfiles.com/mmf/data/datasets/coco/defaults/annotations/annotations.tar.gz
does not match the expected checksum. Please try again. ]

[apsdehal]

Hi,

Thanks for working with MMF. #416 should fix your issue.

On the other side, you are trying to run MMBT on COCO which won't work. What exactly you were trying to do?

[sTranaeus]

Thank you for the fix! I was trying a sanity check that MMF was working in the first place, but you're right - MMBT on COCO wouldn't work. Tried running with VISUALBert instead, and ran into this issue - please have a look if you can.