search

facebookresearch / ParlAI

A framework for training and evaluating AI models on a variety of openly available dialogue datasets.
https://parl.ai
MIT License
10.49k stars 2.1k forks source link

ModuleNotFoundError: No module named 'parlai_internal' #4157

Closed nastya236 closed 2 years ago

nastya236 commented 2 years ago

I have downloaded the model (blender fine-tuned on ed) and have tried to interact with pretained model, however faced a problem.

The command is:

python3 parlai/scripts/interactive.py -mf '../Downloads/v1.0/model'

13:53:53 | loading dictionary from /Users/anastasiafilippova/Downloads/v1.0/model.dict
13:53:53 | num words = 54944
13:53:53 | Polyencoder: full interactive mode on.
13:53:53 | Building candidates file as they do not exist: /Users/anastasiafilippova/Downloads/v1.0/model.cands-parlai_internal.tasks.empathetic_dialogues.agents:EmpatheticDialoguesTeacher.cands
13:53:53 | Opt:
13:53:53 |     activation: gelu
13:53:53 |     adafactor_eps: '(1e-30, 0.001)'
13:53:53 |     adam_eps: 1e-08
13:53:53 |     add_p1_after_newln: False
13:53:53 |     aggregate_micro: False
13:53:53 |     allow_missing_init_opts: False
13:53:53 |     attention_dropout: 0.1
13:53:53 |     batch_length_range: 5
13:53:53 |     batch_sort_cache_type: pop
13:53:53 |     batch_sort_field: text
13:53:53 |     batchsize: 1
13:53:53 |     betas: '[0.9, 0.999]'
13:53:53 |     bpe_add_prefix_space: None
13:53:53 |     bpe_debug: False
13:53:53 |     bpe_dropout: None
13:53:53 |     bpe_merge: None
13:53:53 |     bpe_vocab: None
13:53:53 |     candidates: batch
13:53:53 |     cap_num_predictions: 100
13:53:53 |     checkpoint_activations: False
13:53:53 |     codes_attention_num_heads: 4
13:53:53 |     codes_attention_type: basic
13:53:53 |     data_parallel: True
13:53:53 |     datapath: /Users/anastasiafilippova/HCI/lib/python3.9/site-packages/data
13:53:53 |     datatype: train:evalmode
13:53:53 |     delimiter: '\n'
13:53:53 |     dict_build_first: True
13:53:53 |     dict_class: parlai.core.dict:DictionaryAgent
13:53:53 |     dict_endtoken: __start__
13:53:53 |     dict_file: /Users/anastasiafilippova/Downloads/v1.0/model.dict
13:53:53 |     dict_include_test: False
13:53:53 |     dict_include_valid: False
13:53:53 |     dict_initpath: None
13:53:53 |     dict_language: english
13:53:53 |     dict_loaded: True
13:53:53 |     dict_lower: True
13:53:53 |     dict_max_ngram_size: -1
13:53:53 |     dict_maxexs: -1
13:53:53 |     dict_maxtokens: -1
13:53:53 |     dict_minfreq: 0
13:53:53 |     dict_nulltoken: __null__
13:53:53 |     dict_starttoken: __start__
13:53:53 |     dict_textfields: text,labels
13:53:53 |     dict_tokenizer: bpe
13:53:53 |     dict_unktoken: __unk__
13:53:53 |     display_add_fields: 
13:53:53 |     display_examples: False
13:53:53 |     display_prettify: False
13:53:53 |     download_path: None
13:53:53 |     dropout: 0.1
13:53:53 |     dynamic_batching: None
13:53:53 |     embedding_projection: random
13:53:53 |     embedding_size: 768
13:53:53 |     embedding_type: random
13:53:53 |     embeddings_scale: False
13:53:53 |     encode_candidate_vecs: False
13:53:53 |     encode_candidate_vecs_batchsize: 256
13:53:53 |     eval_batchsize: 16
13:53:53 |     eval_candidates: inline
13:53:53 |     evaltask: None
13:53:53 |     ffn_size: 3072
13:53:53 |     fixed_candidate_vecs: reuse
13:53:53 |     fixed_candidates_path: None
13:53:53 |     force_fp16_tokens: True
13:53:53 |     fp16: True
13:53:53 |     fp16_impl: safe
13:53:53 |     gpu: -1
13:53:53 |     gradient_clip: 0.1
13:53:53 |     hide_labels: False
13:53:53 |     history_add_global_end_token: None
13:53:53 |     history_reversed: False
13:53:53 |     history_size: 20
13:53:53 |     ignore_bad_candidates: False
13:53:53 |     image_cropsize: 224
13:53:53 |     image_mode: raw
13:53:53 |     image_size: 256
13:53:53 |     inference: max
13:53:53 |     init_model: /private/home/ems/GitHub/facebookresearch/ParlAI/data/models/pretrained_transformers/poly_model_huge_reddit/model
13:53:53 |     init_opt: None
13:53:53 |     interactive_candidates: fixed
13:53:53 |     interactive_mode: True
13:53:53 |     interactive_task: False
13:53:53 |     invsqrt_lr_decay_gamma: -1
13:53:53 |     is_debug: False
13:53:53 |     label_truncate: 72
13:53:53 |     learn_embeddings: True
13:53:53 |     learn_positional_embeddings: True
13:53:53 |     learningrate: 2e-06
13:53:53 |     local_human_candidates_file: None
13:53:53 |     log_every_n_secs: 20.0
13:53:53 |     log_keep_fields: all
13:53:53 |     loglevel: info
13:53:53 |     lr_scheduler: reduceonplateau
13:53:53 |     lr_scheduler_decay: 0.7
13:53:53 |     lr_scheduler_patience: 1
13:53:53 |     max_train_time: 200000.0
13:53:53 |     memory_attention: sqrt
13:53:53 |     metrics: default
13:53:53 |     model: transformer/polyencoder
13:53:53 |     model_file: /Users/anastasiafilippova/Downloads/v1.0/model
13:53:53 |     model_parallel: False
13:53:53 |     momentum: 0
13:53:53 |     multitask_weights: [1]
13:53:53 |     mutators: None
13:53:53 |     n_decoder_layers: -1
13:53:53 |     n_encoder_layers: -1
13:53:53 |     n_heads: 12
13:53:53 |     n_layers: 12
13:53:53 |     n_positions: 1024
13:53:53 |     n_segments: 2
13:53:53 |     nesterov: True
13:53:53 |     no_cuda: False
13:53:53 |     normalize_sent_emb: False
13:53:53 |     num_epochs: 50.0
13:53:53 |     numthreads: 1
13:53:53 |     numworkers: 4
13:53:53 |     nus: [0.7]
13:53:53 |     optimizer: adamax
13:53:53 |     outfile: /Users/anastasiafilippova/Downloads/v1.0/model.cands-parlai_internal.tasks.empathetic_dialogues.agents:EmpatheticDialoguesTeacher.cands
13:53:53 |     output_scaling: 0.04
13:53:53 |     override: {}
13:53:53 |     parlai_home: /checkpoint/ems/all_in_one_dialogue/scripts/s2019_08_08__bert_polyencoder_training/05_wider_interacting_sweep/006/ParlAI
13:53:53 |     person_tokens: False
13:53:53 |     poly_attention_num_heads: 4
13:53:53 |     poly_attention_type: basic
13:53:53 |     poly_n_codes: 300
13:53:53 |     polyencoder_type: n_first
13:53:53 |     pytorch_context_length: -1
13:53:53 |     pytorch_datapath: None
13:53:53 |     pytorch_include_labels: True
13:53:53 |     pytorch_preprocess: False
13:53:53 |     pytorch_teacher_batch_sort: False
13:53:53 |     pytorch_teacher_dataset: None
13:53:53 |     pytorch_teacher_task: None
13:53:53 |     rank_candidates: True
13:53:53 |     rank_top_k: -1
13:53:53 |     reactions_only: True
13:53:53 |     reduction_type: mean
13:53:53 |     relu_dropout: 0.15
13:53:53 |     remove_political_convos: False
13:53:53 |     repeat_blocking_heuristic: True
13:53:53 |     return_cand_scores: False
13:53:53 |     save_after_valid: True
13:53:53 |     save_every_n_secs: -1
13:53:54 |     save_format: conversations
13:53:54 |     share_encoders: False
13:53:54 |     share_word_embeddings: True
13:53:54 |     short_final_eval: False
13:53:54 |     show_advanced_args: False
13:53:54 |     shuffle: True
13:53:54 |     single_turn: False
13:53:54 |     special_tok_lst: None
13:53:54 |     split_lines: False
13:53:54 |     starttime: Aug25_23-31
13:53:54 |     task: parlai_internal.tasks.empathetic_dialogues.agents:EmpatheticDialoguesTeacher
13:53:54 |     tensorboard_comment: 
13:53:54 |     tensorboard_log: False
13:53:54 |     tensorboard_metrics: None
13:53:54 |     tensorboard_tag: None
13:53:54 |     text_truncate: 360
13:53:54 |     topk: 5
13:53:54 |     train_experiencer_only: False
13:53:54 |     train_predict: False
13:53:54 |     truncate: 1024
13:53:54 |     update_freq: 1
13:53:54 |     use_memories: False
13:53:54 |     use_reply: label
13:53:54 |     validation_cutoff: 1.0
13:53:54 |     validation_every_n_epochs: 0.5
13:53:54 |     validation_every_n_secs: -1
13:53:54 |     validation_max_exs: 8000
13:53:54 |     validation_metric: accuracy
13:53:54 |     validation_metric_mode: max
13:53:54 |     validation_patience: 10
13:53:54 |     validation_share_agent: False
13:53:54 |     variant: xlm
13:53:54 |     verbose: False
13:53:54 |     warmup_rate: 0.0001
13:53:54 |     warmup_updates: 100
13:53:54 |     weight_decay: None
13:53:54 |     wrap_memory_encoder: False
13:53:54 | Current internal commit: 74dfeebcef90068223880d938a9e26a7a732eb7d
13:53:54 | Current fb commit: 74dfeebcef90068223880d938a9e26a7a732eb7d
13:53:54 | creating task(s): parlai_internal.tasks.empathetic_dialogues.agents:EmpatheticDialoguesTeacher
Traceback (most recent call last):
  File "/Users/anastasiafilippova/HCI/lib/python3.9/site-packages/parlai/core/worlds.py", line 1389, in _create_task_agents
    my_module = load_task_module(opt['task'])
  File "/Users/anastasiafilippova/HCI/lib/python3.9/site-packages/parlai/core/loader.py", line 247, in load_task_module
    task_module = importlib.import_module(module_name)
  File "/usr/local/Cellar/python@3.9/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'parlai_internal'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/anastasiafilippova/HCI/ParlAI/parlai/scripts/interactive.py", line 123, in <module>
    Interactive.main()
  File "/Users/anastasiafilippova/HCI/lib/python3.9/site-packages/parlai/core/script.py", line 129, in main
    return cls._run_args(None)
  File "/Users/anastasiafilippova/HCI/lib/python3.9/site-packages/parlai/core/script.py", line 101, in _run_args
    return cls._run_from_parser_and_opt(opt, parser)
  File "/Users/anastasiafilippova/HCI/lib/python3.9/site-packages/parlai/core/script.py", line 108, in _run_from_parser_and_opt
    return script.run()
  File "/Users/anastasiafilippova/HCI/ParlAI/parlai/scripts/interactive.py", line 118, in run
    return interactive(self.opt)
  File "/Users/anastasiafilippova/HCI/ParlAI/parlai/scripts/interactive.py", line 84, in interactive
    agent = create_agent(opt, requireModelExists=True)
  File "/Users/anastasiafilippova/HCI/lib/python3.9/site-packages/parlai/core/agents.py", line 468, in create_agent
    model = create_agent_from_opt_file(opt)
  File "/Users/anastasiafilippova/HCI/lib/python3.9/site-packages/parlai/core/agents.py", line 421, in create_agent_from_opt_file
    return model_class(opt_from_file)
  File "/Users/anastasiafilippova/HCI/lib/python3.9/site-packages/parlai/agents/transformer/polyencoder.py", line 121, in __init__
    super().__init__(opt, shared)
  File "/Users/anastasiafilippova/HCI/lib/python3.9/site-packages/parlai/core/torch_ranker_agent.py", line 190, in __init__
    super().__init__(opt, shared)
  File "/Users/anastasiafilippova/HCI/lib/python3.9/site-packages/parlai/core/torch_agent.py", line 840, in __init__
    self.set_interactive_mode(opt.get('interactive_mode', False), shared)
  File "/Users/anastasiafilippova/HCI/lib/python3.9/site-packages/parlai/core/torch_ranker_agent.py", line 310, in set_interactive_mode
    path = self.get_task_candidates_path()
  File "/Users/anastasiafilippova/HCI/lib/python3.9/site-packages/parlai/core/torch_ranker_agent.py", line 334, in get_task_candidates_path
    build_cands(opt)
  File "/Users/anastasiafilippova/HCI/lib/python3.9/site-packages/parlai/scripts/build_candidates.py", line 55, in build_cands
    world = create_task(opt, agent)
  File "/Users/anastasiafilippova/HCI/lib/python3.9/site-packages/parlai/core/worlds.py", line 1441, in create_task
    world = create_task_world(opt, user_agents, default_world=default_world)
  File "/Users/anastasiafilippova/HCI/lib/python3.9/site-packages/parlai/core/worlds.py", line 1405, in create_task_world
    task_agents = _create_task_agents(opt)
  File "/Users/anastasiafilippova/HCI/lib/python3.9/site-packages/parlai/core/worlds.py", line 1393, in _create_task_agents
    return create_task_agent_from_taskname(opt)
  File "/Users/anastasiafilippova/HCI/lib/python3.9/site-packages/parlai/core/teachers.py", line 2666, in create_task_agent_from_taskname
    teacher_class = load_teacher_module(opt['task'])
  File "/Users/anastasiafilippova/HCI/lib/python3.9/site-packages/parlai/core/loader.py", line 281, in load_teacher_module
    task_module = load_task_module(taskname)
  File "/Users/anastasiafilippova/HCI/lib/python3.9/site-packages/parlai/core/loader.py", line 247, in load_task_module
    task_module = importlib.import_module(module_name)
  File "/usr/local/Cellar/python@3.9/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'parlai_internal'
klshuster commented 2 years ago

For now, specify -t empathetic_dialogues, as it looks as though the opt file for the model is still pointing to an internal teacher.

Which model from the zoo is this?

github-actions[bot] commented 2 years ago

This issue has not had activity in 30 days. Please feel free to reopen if you have more issues. You may apply the "never-stale" tag to prevent this from happening.