How to add my dataset to MMF for using existing multimodal models?

[KushRJ]

❓ Questions and Help

Forgive me but I am new to this package. I went through the official tutorial on how to add a dataset, but it is quite complex with multiple stages, as shown here: https://mmf.sh/docs/tutorials/dataset/ The webpage also says that '[Outdated] A new version of this will be uploaded soon', so is there any simple way to add my dataset comprising of text and images?

A fast response is highly appreciated. TIA!

[apsdehal]

Hi, More details on what the dataset is about would be helpful

For new tutorial have a look at this colab notebook where we add OKVQA dataset from scratch: https://colab.research.google.com/github/facebookresearch/mmf/blob/notebooks/notebooks/kdd_tutorial.ipynb

[KushRJ]

The dataset is Twitter data with tweets including image and text together.

[apsdehal]

Please have a look at the colab. You can follow the instructions in it to implement your own dataset.

[gireek]

Hi @apsdehal Is there a colab for loading a dataset on similar lines with the hateful memes dataset where there is an image and a text and you just have to classify it into multiple classes . Thanks

[apsdehal]

No, we don't have such a colab at the moment. But if dataset is similar to hateful memes, you can just use its code? Also, OKVQA example is already multiple classes and probably be adjusted for your use case?

[arjondas]

Try: !pip install --pre --upgrade mmf torch==1.5.0+cu101 torchvision==0.6.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html

Although I'm not sure if it's gonna break mmf

[sharduls007]

HI @apsdehal, I am trying to run colab notebook in this thread but getting below error while running run(opts)

Can you please help me out in resolving the error?

[Mayor2305]

Hi, More details on what the dataset is about would be helpful

For new tutorial have a look at this colab notebook where we add OKVQA dataset from scratch: https://colab.research.google.com/github/facebookresearch/mmf/blob/notebooks/notebooks/kdd_tutorial.ipynb

Hello,

I was trying this notebook but after running the last cell (code below) of building the dataset section:

from mmf.utils.build import build_dataset from mmf.utils.configuration import Configuration

Init configuration to register resolvers

Configuration() dataset = build_dataset("okvqa_colab", dataset_type="val") import matplotlib.pyplot as plt plt.rcParams["figure.figsize"] = (20, 20) dataset.visualize(num_samples=8, size=(512, 512), nrow=4)

I got an error:

FileNotFoundError: [Errno 2] No such file or directory: '/root/.cache/torch/mmf/data/datasets/okvqa_colab/defaults/annotations/annotations/imdb_val1.npy'

Please help me regarding the same...

ps: i am using google colab as the environment