EricGuo5513 / text-to-motion

Official implementation for "Generating Diverse and Natural 3D Human Motions from Texts (CVPR2022)."
MIT License
484 stars 40 forks source link

Which model does HumanML3D use for POS-tagging? #34

Closed miibotree closed 1 year ago

miibotree commented 1 year ago

Refer to the HumanML3D dataset, I want to label my own dataset, and fine-tune the text/motion encoder for my own task. Here I noticed the dataset use POS-tagging and extra-dictionary:

  POS_enumerator = {
      'VERB': 0,        
      'NOUN': 1,        
      'DET': 2,     
      'ADP': 3,     
      'NUM': 4,     
      'AUX': 5,     
      'PRON': 6,        
      'ADJ': 7,     
      'ADV': 8,     
      'Loc_VIP': 9,
      'Body_VIP': 10,
      'Obj_VIP': 11,
      'Act_VIP': 12,
      'Desc_VIP': 13,
      'OTHER': 14,
  }

  Loc_list = ('left', 'right', 'clockwise', 'counterclockwise', 'anticlockwise', 'forward', 'back', 'backward',
              'up', 'down', 'straight', 'curve')

  Body_list = ('arm', 'chin', 'foot', 'feet', 'face', 'hand', 'mouth', 'leg', 'waist', 'eye', 'knee', 'shoulder', 'thigh')

  Obj_List = ('stair', 'dumbbell', 'chair', 'window', 'floor', 'car', 'ball', 'handrail', 'baseball', 'basketball')

  Act_list = ('walk', 'run', 'swing', 'pick', 'bring', 'kick', 'put', 'squat', 'throw', 'hop', 'dance', 'jump', 'turn',
              'stumble', 'dance', 'stop', 'sit', 'lift', 'lower', 'raise', 'wash', 'stand', 'kneel', 'stroll',
              'rub', 'bend', 'balance', 'flap', 'jog', 'shuffle', 'lean', 'rotate', 'spin', 'spread', 'climb')

  Desc_list = ('slowly', 'carefully', 'fast', 'careful', 'slow', 'quickly', 'happy', 'angry', 'sad', 'happily',
               'angrily', 'sadly')

Because I can't find relevant information in the paper and supplementary materials, I want to know:

  1. which model is used for POS-tagging?
  2. how does the extra-dictionary constructed?
miibotree commented 1 year ago

According to author's reply: Refer to this line for POS tagging. Using spacy for text processing. The extra dictionary is constructed manually.