Can't download features.tar.gz

[nyj-ocean]

❓ Questions and Help

@apsdehal when I run this command mmf_run config=projects/m4c/configs/textvqa/defaults.yaml datasets=textvqa model=m4c run_type=train_val

it start to download the eatures.tar.gz but it will broke a few minutes later since eatures.tar.gz is too large (34.3G)

File "/home/gc/4-images/AB/mmf/mmf/utils/download.py", line 291, in download
    raise RuntimeWarning("Connection broken too many times. Stopped retrying.")
RuntimeWarning: Connection broken too many times. Stopped retrying.

How should I do？

[apsdehal]

Hi. This is a problem with your internet connection being flaky rather than mmf itself. We in general retry three times before we stop. Next time you run, it will again try for 3 more times using the existing .part file of partial download.

[nyj-ocean]

@apsdehal Thanks for your reply I want to download the features.tar.gz using a url in browser

I find a url https://docs.google.com/uc?export=download in download.py I try to open this url https://docs.google.com/uc?export=download in browser to download the features.tar.gz But it can't open

Can you provide a url to download the features.tar.gz ?

[apsdehal]

The url should be https://dl.fbaipublicfiles.com/mmf/data/datasets/textvqa/defaults/features/features.tar.gz.

Depending on if you are planning to use features with MMF, you will have replicate the proper directory structure for mmf and move the features.tar.gz there. MMF can then handle the extraction for you.

Feel free to close the issue if you query has been resolved.

[nyj-ocean]

@apsdehal Thanks for your help I am downloadingfeatures.tar.gz

I have anothor question when I finish download thefeatures.tar.gz , which directory should I put in on? Should I put in on the directory: /root/.cache/torch/mmf/data/dataset/hateful_memes/defaults/features/ ？

[apsdehal]

You are using textvqa dataset so you should put it in textvqa directory rather than hateful_memes. Rest of the path looks good.

[nyj-ocean]

@apsdehal Sorry to bother you again

When I run mmf_predict config=projects/m4c/configs/textvqa/defaults.yaml datasets=textvqa model=m4c run_type=test checkpoint.resume_zoo=m4c.textvqa.defaults

I meet an error

 File "/home/gc/4-images/AB/mmf/mmf/utils/phoc/build_phoc.py", line 3, in <module>
    from .cphoc import build_phoc as _build_phoc_raw
ModuleNotFoundError: No module named 'mmf.utils.phoc.cphoc'

I notice that in the directory /my-root/mmf/mmf/utils/phoc/, there is not a cphoc.py But I notice in the directory /my-root/mmf/mmf/utils/phoc/src/, there is a cphoc.c

So I try to change the build_phoc.py from from .cphoc import build_phoc as _build_phoc_raw #line 3 to from .src.cphoc import build_phoc as _build_phoc_raw #line 3

But it didn't work yet. How should I do?

[apsdehal]

Can you try cloning and installing again? We recently fixed this issue.

[nyj-ocean]

@apsdehal Sorry to bother you agian

When I run mmf_predict config=projects/m4c/configs/textvqa/defaults.yaml datasets=textvqa model=m4c run_type=test checkpoint.resume_zoo=m4c.textvqa.defaults

it can not download the detectron.lmdb,just like following

  File "/home/gc/anaconda3/lib/python3.7/site-packages/mmf/datasets/databases/readers/feature_readers.py", line 175, in __init__
    self.db_path
RuntimeError: /home/gc/.cache/torch/mmf/data/datasets/textvqa/defaults/features/open_images/detectron.lmdb path specified for LMDB features doesn't exists.

  0%|                                                                                           | 0/45 [00:00<?, ?it/s]

maybe it is the problem of my internet connection

• Can you provide me a url to download the detectron.lmdb

• I think I also need to download the ocr_en_frcn_features.lmdb ,according to the config.yaml. If so, can you also provide me the url to download the ocr_en_frcn_features.lmdb

[Ra0k]

A similar thing happens to me as well!

mmf_predict config=mmf/projects/lorra/configs/textvqa/defaults.yaml datasets=textvqa model=lorra run_type=test checkpoint.resume_file=model_data/lorra.pth

This is the full traceback: https://pastebin.com/bAQfcRbt

[nyj-ocean]

@Ra0k have you solved this problem? How do you solve it ?

[Ra0k]

@nyj-ocean

I couldn't solve it yet. :/ @apsdehal do you have any idea what causes the problem?

[apsdehal]

@Ra0k Your issue seems to be different than @nyj-ocean's. Let me debug it and get back to you.

@nyj-ocean The path to ocr_en_features: https://dl.fbaipublicfiles.com/mmf/data/datasets/textvqa/ocr_en/features/features.tar.gz. Extract them to /root/.cache/torch/mmf/data/datasets/textvqa/ocr_en/features/

Make sure your paths are properly setup for lmdb as you are getting lmdb file not found errors.

[apsdehal]

@Ra0k It seems like VQA Answer processor is broken for the case of LoRRA. Can you open up a separate issue as it is an unrelated to this one?