Interacting with 90M Blenderbot gives "RuntimeError: error in LoadLibraryA"

[Weilin37]

I'm trying to load Blenderbot 90M like so:

python parlai/scripts/safe_interactive.py -t blended_skill_talk -mf zoo:blender/blender_90M/model
11:34:54 WARN | overriding opt['task'] to blended_skill_talk (previously: internal:blended_skill_talk,wizard_of_wikipedia,convai2,empathetic_dialogues)
11:34:54 WARN | overriding opt['model_file'] to C:\Users\WW\Documents\GitHub\ParlAI\data\models\blender/blender_90M/model (previously: /checkpoint/edinan/20200210/baseline_BST_retnref/lr=7.5e-06_attention-dropout=0.0_relu-dropout=0.0/model)
11:34:54 INFO | Using CUDA
11:34:54 ERRO | You set --fp16 true with --fp16-impl apex, but fp16 with apex is unavailable. To use apex fp16, please install APEX from https://github.com/NVIDIA/apex.
11:34:54 INFO | loading dictionary from C:\Users\WW\Documents\GitHub\ParlAI\data\models\blender/blender_90M/model.dict
11:34:54 INFO | num words = 54944
11:34:54 INFO | TransformerGenerator: full interactive mode on.
11:34:54 WARN | DEPRECATED: XLM should only be used for backwards compatibility, as it involves a less-stable layernorm operation.
11:34:56 INFO | Total parameters: 87,508,992 (87,508,992 trainable)
11:34:56 INFO | Loading existing model params from C:\Users\WW\Documents\GitHub\ParlAI\data\models\blender/blender_90M/model
[ optional arguments: ]
[  display_examples: False ]
[  display_ignore_fields: label_candidates,text_candidates ]
[  display_prettify: False ]
[  interactive_task: True ]
[ Main ParlAI Arguments: ]
[  batchsize: 16 ]
[  datapath: C:\Users\WW\Documents\GitHub\ParlAI\data ]
[  datatype: train ]
[  download_path: C:\Users\WW\Documents\GitHub\ParlAI\downloads ]
[  dynamic_batching: None ]
[  hide_labels: False ]
[  image_mode: raw ]
[  init_opt: None ]
[  loglevel: info ]
[  multitask_weights: [1.0, 3.0, 3.0, 3.0] ]
[  numthreads: 1 ]
[  show_advanced_args: False ]
[  task: blended_skill_talk ]
[ ParlAI Model Arguments: ]
[  dict_class: parlai.core.dict:DictionaryAgent ]
[  init_model: /checkpoint/parlai/zoo/new_reddit/newreddit_trained20190909_usedfordodeca/model ]
[  model: transformer/generator ]
[  model_file: C:\Users\WW\Documents\GitHub\ParlAI\data\models\blender/blender_90M/model ]
[ Safe Local Human Arguments: ]
[  safety: all ]
[ Local Human Arguments: ]
[  local_human_candidates_file: None ]
[  single_turn: False ]
[ ParlAI Image Preprocessing Arguments: ]
[  image_cropsize: 224 ]
[  image_size: 256 ]
[ Transformer Arguments: ]
[  activation: gelu ]
[  attention_dropout: 0.0 ]
[  dropout: 0.1 ]
[  embedding_size: 512 ]
[  embeddings_scale: True ]
[  ffn_size: 2048 ]
[  learn_positional_embeddings: True ]
[  model_parallel: False ]
[  n_decoder_layers: -1 ]
[  n_encoder_layers: -1 ]
[  n_heads: 16 ]
[  n_layers: 8 ]
[  n_positions: 512 ]
[  n_segments: 0 ]
[  output_scaling: 1.0 ]
[  relu_dropout: 0.0 ]
[  share_word_embeddings: True ]
[  variant: xlm ]
[ Torch Generator Agent: ]
[  beam_block_list_filename: None ]
[  beam_block_ngram: 3 ]
[  beam_context_block_ngram: 3 ]
[  beam_delay: 30 ]
[  beam_length_penalty: 0.65 ]
[  beam_min_length: 20 ]
[  beam_size: 10 ]
[  compute_tokenized_bleu: False ]
[  inference: beam ]
[  skip_generation: False ]
[  temperature: 1.0 ]
[  topk: 10 ]
[  topp: 0.9 ]
[ TorchAgent Arguments: ]
[  add_p1_after_newln: False ]
[  delimiter:
 ]
[  embedding_projection: random ]
[  embedding_type: random ]
[  force_fp16_tokens: True ]
[  fp16: True ]
[  fp16_impl: apex ]
[  gpu: -1 ]
[  history_add_global_end_token: None ]
[  history_size: -1 ]
[  interactive_mode: True ]
[  label_truncate: 128 ]
[  no_cuda: False ]
[  person_tokens: False ]
[  rank_candidates: False ]
[  split_lines: False ]
[  text_truncate: 512 ]
[  truncate: -1 ]
[  use_reply: label ]
[ Optimizer Arguments: ]
[  adafactor_eps: [1e-30, 0.001] ]
[  adam_eps: 1e-08 ]
[  betas: [0.9, 0.999] ]
[  gradient_clip: 0.1 ]
[  learningrate: 7.5e-06 ]
[  momentum: 0 ]
[  nesterov: True ]
[  nus: [0.7] ]
[  optimizer: adamax ]
[  weight_decay: None ]
[ Dictionary Arguments: ]
[  bpe_debug: False ]
[  dict_endtoken: __end__ ]
[  dict_file: C:\Users\WW\Documents\GitHub\ParlAI\data\models\blender/blender_90M/model.dict ]
[  dict_initpath: None ]
[  dict_language: english ]
[  dict_lower: True ]
[  dict_max_ngram_size: -1 ]
[  dict_maxtokens: -1 ]
[  dict_minfreq: 0 ]
[  dict_nulltoken: __null__ ]
[  dict_starttoken: __start__ ]
[  dict_textfields: text,labels ]
[  dict_tokenizer: bpe ]
[  dict_unktoken: __unk__ ]
[ BPEHelper Arguments: ]
[  bpe_add_prefix_space: None ]
[  bpe_merge: None ]
[  bpe_vocab: None ]
[ Learning Rate Scheduler: ]
[  invsqrt_lr_decay_gamma: -1 ]
[  lr_scheduler: reduceonplateau ]
[  lr_scheduler_decay: 0.5 ]
[  lr_scheduler_patience: 3 ]
[  max_lr_steps: -1 ]
[  update_freq: 1 ]
[  warmup_rate: 0.0001 ]
[  warmup_updates: -1 ]
[ BST Interactive World: ]
[  display_partner_persona: True ]
[  include_initial_utterances: False ]
[  include_personas: True ]
[  safe_personas_only: True ]
Enter [DONE] if you want to end the episode, [EXIT] to quit.
11:34:56 WARN | overriding opt['model'] to transformer/classifier (previously: transformer_classifier)
11:34:56 WARN | overriding opt['model_file'] to C:\Users\WW\Documents\GitHub\ParlAI\data\models\dialogue_safety/single_turn/model (previously: /checkpoint/edinan/20190828/safety_reddit/contiguous-dropout=0_multitask-weights=0.5,0.1,0.1,0.4,0.2_lr=5e-05_lr-scheduler-patience=3_lr-scheduler-decay=0.9_warmupupdates=1000/model)
11:34:56 WARN | overriding opt['print_scores'] to True (previously: False)
11:34:56 INFO | Using CUDA
11:34:56 INFO | loading dictionary from C:\Users\WW\Documents\GitHub\ParlAI\data\models\dialogue_safety/single_turn/model.dict
11:34:57 INFO | num words = 54944
11:34:59 INFO | Loading existing model parameters from C:\Users\WW\Documents\GitHub\ParlAI\data\models\dialogue_safety/single_turn/model
11:35:00 INFO | creating task(s): blended_skill_talk
[ loading personas.. ]

  [NOTE: In the BST paper both partners have a persona.
         You can choose to ignore yours, the model never sees it.
         In the Blender paper, this was not used for humans.
         You can also turn personas off with --include-personas False]

[context]: your persona: my brother in law runs a major hollywood studio.
your persona: i build aquariums for a living.
Hollywood
Enter Your Message: Hi!

and I get this error message:

Traceback (most recent call last):
  File "parlai/scripts/safe_interactive.py", line 94, in <module>
    SafeInteractive.main()
  File "c:\users\ww\documents\github\parlai\parlai\scripts\script.py", line 79, in main
    return cls._run_args(None)
  File "c:\users\ww\documents\github\parlai\parlai\scripts\script.py", line 66, in _run_args
    return script.run()
  File "parlai/scripts/safe_interactive.py", line 89, in run
    return safe_interactive(self.opt, print_parser=self.parser)
  File "parlai/scripts/safe_interactive.py", line 70, in safe_interactive
    world.parley()
  File "c:\users\ww\documents\github\parlai\parlai\tasks\interactive\worlds.py", line 78, in parley
    acts[1] = agents[1].act()
  File "c:\users\ww\documents\github\parlai\parlai\core\torch_agent.py", line 1880, in act
    response = self.batch_act([self.observation])[0]
  File "c:\users\ww\documents\github\parlai\parlai\core\torch_agent.py", line 1930, in batch_act
    output = self.eval_step(batch)
  File "c:\users\ww\documents\github\parlai\parlai\core\torch_generator_agent.py", line 853, in eval_step
    beam_preds_scores, _ = self._generate(batch, self.beam_size, maxlen)
  File "c:\users\ww\documents\github\parlai\parlai\core\torch_generator_agent.py", line 1091, in _generate
    decoder_input = torch.cat([decoder_input, selection], dim=-1)
RuntimeError: error in LoadLibraryA

[stephenroller]

This looks like a problem with your pytorch and/or cuda installation.

[Weilin37]

I am using pytorch 1.4, so I guess it could be my cuda installation? Which cuda version should I be using then? Sorry, it's not obvious how it is related exactly to the installation.

[stephenroller]

https://stackoverflow.com/questions/60478862/how-to-avoid-runtimeerror-error-in-loadlibrarya-for-torch-cat

Try 1.5.1

[github-actions[bot]]

This issue has not had activity in 30 days. Please feel free to reopen if you have more issues. You may apply the "never-stale" tag to prevent this from happening.