The files in this repository contain verb and sense annotations for images taken from MSCOCO and TUHOI datasets.

Creation of the Verse dataset and the unsupervised model proposed to use the Multimodal features are described more detail in the paper:

Spandana Gella, Mirella Lapata, and Frank Keller. 2016. Unsupervised Visual Sense Disambiguation for Verbs using Multimodal Embeddings. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2016). San Diego, CA.

Note that this repository just includes the Verse dataset annotations and not the images.

• gold_all_final_lemma_image_sense_annotations.csv: This file has verb and verse sense annotated for every image in the verse dataset.

• verse_visualness_labels.csv: This file has visualness label annotated for OntoNotes senses of 150 visual verbs (commonly observed in image descriptions and image action datasets)

• sense_specific_search_engine_queries.csv: Human annotated sense specific search engine queries to retrieve images related to visual senses.

• motion_verbs.csv: Set of motion verbs mentioned in the paper

• non_motion_verbs.csv: Set of motion verbs mentioned in the paper License

  This work is licensed by the University of Edinburgh under a Creative Commons Attribution 4.0 International License.