[data request] Holistic Video Understanding Dataset (HVU)

[bhack]

• Name of dataset: Holistic Video Understanding Dataset
• URL of dataset: https://holistic-video-understanding.github.io/
• License of dataset:
• Short description of dataset and use case(s): HVU is organized hierarchically in a semantic taxonomy that focuses on multi-label and multi-task video understanding as a comprehensive problem that encompasses the recognition of multiple semantic aspects in the dynamic scene. HVU contains approx.~577k videos in total with ~13M annotations for training and validation set spanning over ~3k classes. HVU encompasses semantic aspects defined on categories of scenes, objects, actions, events, attributes and concepts, which naturally captures the real-world scenarios.

Folks who would also like to see this dataset in tensorflow/datasets, please thumbs-up so the developers can know which requests to prioritize.

And if you'd like to contribute the dataset (thank you!), see our guide to adding a dataset.

/cc @alidiba67 @MohsenFayyaz89

[bhack]

@MohsenFayyaz89 Do you plan to contribute your dateset?

[MohsenFayyaz89]

Would like to but unfortunately, I'm quite busy.

[bhack]

Too bad, it would have been useful for those who want to use your dataset but are too busy to create the adapter classes :stuck_out_tongue_winking_eye:

[bhack]

/cc @raviddoss do you know anyone internal that could be interested to take in charge this?

[Naman-Bhrgv]

@bhack I would like to contribute to this issue. Please assign this to me.Also, since I am new to this repo, please guide me through.

[bhack]

@Naman-Bhrgv Please start with standard guide https://www.github.com/tensorflow/datasets/tree/master/docs%2Fadd_dataset.md

[Naman-Bhrgv]

@bhack Thanks!

[NikhilBartwal]

@Naman-Bhrgv Are you currently working on this? @bhack I would like to take this up. Could you provide me with the current status?

[Naman-Bhrgv]

@NikhilBartwal I am currently not working on this.If you want you can take up this issue.

[bhack]

Ok. @NikhilBartwal do you want to prepare a PR?

[NikhilBartwal]

@bhack I would very much like to. As I'm new to this, I wanted to know if you could guide me through the process as the dataset rather contains YouTube video IDs, so I'm not sure about how to prepare the dataset after downloading the video IDs in TFDS. Could you help me through ? Thanks!

[bhack]

For the video set you can take an overview to: https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/video

[NikhilBartwal]

@bhack Thanks for the help! I will look into some examples and start working on it.

[NikhilBartwal]

Hey @bhack @ChanchalKumarMaji @Conchylicultor , I was trying out this script https://github.com/holistic-video-understanding/HVU-Downloader/blob/master/HVU_download.py and what i found was that no matter if I used the script directly from terminal for downloading or integrated the script code in my notebook, there were always sosme files which were not downloaded properly (around ~10% and were around 252 bytes). The script uses joblib.parallel for parallelisation, What do you think could be causing that?

[bhack]

@NikhilBartwal Have you tried to open a ticket at https://github.com/holistic-video-understanding/HVU-Downloader/?

[NikhilBartwal]

@bhack I have just opened one at https://github.com/holistic-video-understanding/HVU-Dataset/issues/3

[NikhilBartwal]

@bhack Hey, there was one doubt that I was having, what do you think would be the most efficient way of decoding the video to a numpy array inside a script?

[bhack]

numpy or Tensor? Have you seen https://www.tensorflow.org/datasets/api_docs/python/tfds/features/Video

https://www.tensorflow.org/io/api_docs/python/tfio/experimental/ffmpeg/decode_video

[NikhilBartwal]

@bhack Guess it was a typo :( I will have a look at it. Thanks !

[NikhilBartwal]

@bhack Well i tried it and looks like tensorflow-io only supports FFmpeg on Ubuntu 14.04, 16.04, and 18.04. as mentioned here Do you have any other idea?

[bhack]

I think that Tensorflow IO is what the other video datasets are using in this repo /cc @yongtang

[NikhilBartwal]

@bhack I checked the video datasets and unfortunately, I couldn't find them using tfio. Could you give a link to it?

[yongtang]

Well i tried it and looks like tensorflow-io only supports FFmpeg on Ubuntu 14.04, 16.04, and 18.04. as mentioned here Do you have any other idea?

@NikhilBartwal Do you have a specific platform (such as Ubuntu 20.04?) you want to use?

[bhack]

@Naman-Bhrgv I supposed that they was reusing TF-IO API but it seems that they are controlling ffmpeg subprocess: https://github.com/tensorflow/datasets/blob/master/tensorflow_datasets/core/features/video_feature.py#L116

[NikhilBartwal]

@yongtang I was working on a video dataset for TF-DS and so i was hoping for a platform independent way of decoding a video file.

[NikhilBartwal]

@bhack I will have a look at it. Thanks!

[NikhilBartwal]

@bhack @Conchylicultor Could you review the PR?

[bhack]

@NikhilBartwal thank you for the PR. I am on vacation. It could be nice if a TFDS maintainer could do a first pass in the meantime.

[NikhilBartwal]

@bhack I didn't know that. Sorry for the disturbance :(