Create dataset vicon_visim400

[albertvillanova]

• uid: vicon_visim400
• type: processed
• description:
  • name: Vietnamese Datasets for Evaluating Semantic Models of (Dis-)Similarity and Relatedness (ViCon and ViSim-400)
  • description: This dataset consists of two kinds of datasets: The first dataset, namely ViCon, comprises pairs of synonyms and antonymys across noun, verb, and adjective classes, offerring data to distinguish between similarity and dissimilarity. The second dataset ViSim-400 is a dataset of semantic relation pairs which contains degrees of similarity across five semantic relations, as rated by human judges.
  • homepage: https://www.ims.uni-stuttgart.de/forschung/ressourcen/experiment-daten/vnese-sem-datasets/
  • validated: True
• languages:
  • language_names:
  • Vietnamese
  • language_comments:
  • language_locations:
  • Western Europe
  • Germany
  • validated: False
• custodian:
  • name: Thang Vu
  • in_catalogue:
  • type: A university or research institution
  • location: Germany
  • contact_name: Thang Vu
  • contact_email: ngoc-thang.vu@ims.uni-stuttgart.de
  • contact_submitter: False
  • additional:
  • validated: False
• availability:
  • procurement:
  • for_download: Yes - it has a direct download link or links
  • download_url: https://www.ims.uni-stuttgart.de/documents/ressourcen/experiment-daten/ViData.zip
  • download_email:
  • licensing:
  • has_licenses: Unclear
  • license_text:
  • license_properties:
  • license_list:
  • pii:
  • has_pii: No
  • generic_pii_likely:
  • generic_pii_list:
  • numeric_pii_likely:
  • numeric_pii_list:
  • sensitive_pii_likely:
  • sensitive_pii_list:
  • no_pii_justification_class: general knowledge not written by or referring to private persons
  • no_pii_justification_text:
  • validated: False
• processed_from_primary:
  • from_primary: Original data
  • primary_availability:
  • primary_license:
  • primary_types:
  • validated: False
• media:
  • category:
  • text
  • text_format:
  • audiovisual_format:
  • image_format:
  • database_format:
  • .ZIP
  • text_is_transcribed: No
  • instance_type: article
  • instance_count: 100<n<1K
  • instance_size: 100<n<10,000
  • validated: False
• fname: vicon_visim400.json

[albertvillanova]

DONE: https://huggingface.co/datasets/bigscience-catalogue-data/vicon_visim400

[albertvillanova]

It needs support for ZIP:

• huggingface/datasets#3375

[albertvillanova]

ERROR:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x87 in position 23: invalid start byte

[albertvillanova]

These datasets just contain pairs of words:

• ViCon: synonyms and antonymys
• ViSim-400: degrees of similarity

  Word1   Word2   POS Sim1    Sim2    STD
  biến    ngập    V   3.13    5.22    0.72
  nhà_thi_đấu nhà N   3.07    5.12    1.18
  động    tĩnh    V   0.6 1.0 0.95

I don't think these are appropriate to train a Language Model. CC: @yjernite