search

facebookresearch / diplomacy_cicero

Code for Cicero, an AI agent that plays the game of Diplomacy with open-domain natural language negotiation.
Other
1.28k stars 157 forks source link

Model parameter size mismatch #30

Open yingtaoluo opened 1 year ago

yingtaoluo commented 1 year ago

(diplomacy_cicero) luoyingtao@CRIPAC-iota:~/diplomacy_cicero$ python run.py --adhoc --cfg conf/c01_ag_cmp/cmp.prototxt Iagent_one=agents/cicero.prototxt use_shared_agent=1 power_one=TURKEY I0713 01:47:05 [run:154] Config: conf/c01_ag_cmp/cmp.prototxt I0713 01:47:05 [run:155] Overrides: ['Iagent_one=agents/cicero.prototxt', 'use_shared_agent=1', 'power_one=TURKEY'] I0713 01:47:05 [run:163] Exp dir: /home/luoyingtao/diplomacy_experiments/results/diplomacy/adhoc/2023-07-13T014705.372767/c01_ag_cmp/cmp/Iag@agents_cicero.prototxt@pow@TURKEY@use@1_a400902b I0713 01:47:05 [run:164] Job status [before run]: Status.NOT_STARTED I0713 01:47:05 [util:575] Changing cwd to /home/luoyingtao/diplomacy_experiments/results/diplomacy/adhoc/2023-07-13T014705.372767/c01_ag_cmp/cmp/Iag@agents_cicero.prototxt@pow@TURKEY@use@1_a400902b I0713 01:47:05 [run:139] Machine IP Address: 169.254.3.1 I0713 01:47:05 [run:141] Using GPU: 4,5 I0713 01:47:05 [run:147] Cwd: /home/luoyingtao/diplomacy_experiments/results/diplomacy/adhoc/2023-07-13T014705.372767/c01_ag_cmp/cmp/Iag@agents_cicero.prototxt@pow@TURKEY@use@1_a400902b I0713 01:47:05 [run:148] Task: compare_agents

      overrides {
        inference: "nucleus"
        beam_size: 1
        topp: 0.8999999761581421
        set_player_rating: 5
      }
    }
    model_pseudo_orders {
      model_path: "models/imitation_intent"
      overrides {
        inference: "topk"
        set_player_rating: 5
      }
    }
    model_sleep_classifier {
      model_path: "models/sleep_classifier"
      overrides {
        set_player_rating: 5
      }
    }
    ensemble_nonsense_classifier {
      nonsense_classifiers {
        name: "location"
        nonsense_classifier {
          model_path: "models/nonsense_ensemble/location"
          overrides {
            set_player_rating: 5
            threshold: 0.05000000074505806
          }
        }
      }
      nonsense_classifiers {

      nonsense_classifiers {
        name: "hvwm"
        nonsense_classifier {
          model_path: "models/nonsense_ensemble/hvwm"
          overrides {
            set_player_rating: 5
            threshold: 0.5
          }
        }
      }
      nonsense_classifiers {
        name: "hvm_NEW_justifications_2seeds"
        nonsense_classifier {
          model_path: "models/nonsense_ensemble/humanvsmodel_nonsense_classifier_denoising_justifications"
          overrides {
            set_player_rating: 5
            threshold: 0.699999988079071
          }
        }
      }
      nonsense_classifiers {
        name: "non_sequitur_detector"
        nonsense_classifier {
          model_path: "models/nonsense_ensemble/nonsequitur_detector"
          overrides {
            set_player_rating: 5
            threshold: 0.8999999761581421
          }
        }
      }
      nonsense_classifiers {
        n
  use_truthful_pseudoorders: true
  skip_policy_evaluation_for_truthful_pseudoorders: true
  use_greedy_po_for_rollout: true
  br_corr_bilateral_search {
    enable_for_pseudo_order: true
    enable_for_final_order: true
    bilateral_search_num_cond_sample: 20
    use_all_power_for_p_joint: true
    br_regularize_lambda: 0.003000000026077032
    min_unnormalized_weight: 0.019999999552965164
    max_unnormalized_weight: 10.0
  }
  message_search {
    n_messages: 8
    strategy: FILTER
    filter_top_k: 5
  }
}
num_player_types: 6
agent_type: 2
agent_type_is_public: false
player_types {
  log_uniform {
    min_lambda: 0.001
    max_lambda: 0.3
  }
}
scale_lambdas_1901_spring: 10.0
dynamic_lambda_stdev_espilon: 0.009999999776482582
dynamic_lambda_stdev_baseline: 0.05000000074505806
dynamic_lambda_stdev_num_samples: 100
do_bayesian_updates: false

} } agent_six { base_strategy_model { model_path: "models/blueprint.pt" temperature: 0.10000000149011612 } } power_one: TURKEY out: "output.json" seed: 0 use_shared_agent: true draw_on_stalemate_years: 5

I0713 01:47:05 [util:234] Git revision: f9ccaf6718bf132ca9bbcff8fee4b21020242668 I0713 01:47:06 [util:250] Found unsubmitted diff. Saving to /home/luoyingtao/diplomacy_experiments/results/diplomacy/adhoc/2023-07-13T014705.372767/c01_ag_cmp/cmp/Iag@agents_cicero.prototxt@pow@TURKEY@use@1_a400902b/workdir.diff I0713 01:47:06 [run:152] Is on slurm: False I0713 01:47:06 [run:153] Job env: JobEnvironment(job_id=1, hostname=CRIPAC-iota, local_rank=0(1), node=0(1), global_rank=0(1)) I0713 01:47:06 [run:156] Is master: True I0713 01:47:06 [run:40] Set seed to 0 I0713 01:47:06 [load_model:155] Loading base_strategy_model from models/rl_search_orders.ckpt I0713 01:47:11 [load_model:155] Loading base_strategy_model from models/rl_value_function.ckpt I0713 01:47:12 [searchbot_agent:441] Performing qre regret minimization with eta=10.0 and lambda=0.0 with target pi=BLUEPRINT I0713 01:47:12 [searchbot_agent:445] Using lambda 0.0 simplifies qre to regular hedge I0713 01:47:12 [searchbot_agent:492] Setting up parlai orders model... I0713 01:47:12 [factory:93] Loading allorderindependentrollout wrapper for model trained on task: message_history_orderhistorysincelastmovementphase_shortstate_allorderindependentrollout_chunk I0713 01:47:12 [base_wrapper:138] [ParlAIAllOrderIndependentRolloutWrapper] Using overrides: {'interactive_mode': True, 'skip_generation': False, 'beam_length_penalty': 0, 'inference': 'topk', 'topk': 10, 'temperature': 0.75, 'set_player_rating': 5, 'model': 'bart_custom_inference'} 01:47:12 | Overriding opt["inference"] to topk (previously: greedy) 01:47:12 | Overriding opt["skip_generation"] to False (previously: True) 01:47:12 | Overriding opt["temperature"] to 0.75 (previously: 1.0) 01:47:12 | Overriding opt["model"] to bart_custom_inference (previously: bart) 01:47:12 | Overriding opt["beam_length_penalty"] to 0 (previously: 0.65) 01:47:12 | Overriding opt["set_player_rating"] to 5 (previously: -1) [downloading BART models: /home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages/data/models/bart] 01:47:12 | Downloading http://dl.fbaipublicfiles.com/fairseq/models/bart.large.tar.gz to /home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages/data/models/bart/bart.large.tar.gz Downloading bart.large.tar.gz: 100%|████████████████████| 3.70G/3.70G [53:14<00:00, 1.16MB/s] 02:41:15 | Using CUDA 02:41:15 | Downloading https://dl.fbaipublicfiles.com/fairseq/gpt2_bpe/vocab.bpe to /home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages/data/gpt2/vocab.bpe Downloading vocab.bpe: 0.00B [00:01, ?B/s] 02:41:16 | Downloading https://dl.fbaipublicfiles.com/fairseq/gpt2_bpe/encoder.json to /home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages/data/gpt2/encoder.json Downloading encoder.json: 0.00B [00:01, ?B/s] 02:41:23 | Total parameters: 406,286,336 (406,286,336 trainable) 02:41:24 | Saving dictionary to /home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages/data/models/bart/bart_large/model.dict 02:41:25 | Opt: 02:41:25 | datapath: /home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages/data 02:41:25 | no_backup: True 02:41:25 | override: "{'path': '/home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages/data/models/bart/bart_large/model', 'no_backup': True}" 02:41:25 | parlai_home: /home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages 02:41:25 | path: /home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages/data/models/bart/bart_large/model 02:41:25 | starttime: Jul13_02-41 02:41:25 | Loading /home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages/data/models/bart/bart_large/model 02:41:26 | Deleting key optimizer 02:41:26 | Deleting key optimizer_type 02:41:26 | Deleting key lr_scheduler 02:41:26 | Deleting key lr_scheduler_type 02:41:26 | Deleting key warmup_scheduler 02:41:26 | Deleting key number_training_updates 02:41:26 | Remaining keys: model 02:41:26 | Saving to /home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages/data/models/bart/bart_large/model {'id': 'Bart', 'episode_done': False, 'text': "What's your favorite kind of ramen?", 'beam_texts': [("What's your favorite kind of ramen?", -0.00019120505021419376)], 'metrics': {'clen': AverageMetric(9), 'ctrunc': AverageMetric(0), 'ctrunclen': AverageMetric(0), 'gen_n_toks': AverageMetric(11)}} 02:41:32 | your model is being loaded with opts that do not exist in the model you are initializing the weights with: evaltask: None,final_extra_opt: ,eval_batchsize: None,eval_dynamic_batching: None,num_workers: 8,display_examples: False,num_epochs: -1,max_train_time: -1,max_train_steps: 250000,log_every_n_steps: 100,validation_every_n_secs: -1,validation_every_n_steps: 1000,save_every_n_secs: 3600.0,save_after_valid: True,validation_every_n_epochs: -1,validation_max_exs: -1,short_final_eval: False,validation_patience: 30,validation_metric: ppl,validation_metric_mode: min,validation_cutoff: 1.0,validation_share_agent: False,metrics: default,aggregate_micro: False,world_logs: ,save_format: conversations,log_keep_fields: all,tensorboard_log: False,tensorboard_logdir: None,wandb_log: True,wandb_name: 2fb,wandb_project: 20220513_allorderindependentrollout_prefix_bilateral,wandb_entity: fairdiplomacy,dict_maxexs: -1,dict_include_valid: False,dict_include_test: False,log_every_n_secs: 120.0,distributed_world_size: 128,ddp_backend: ddp,port: 38316,allorders_mark_all_holds: True,filter_all_holds: True,train_on_message_prefixes: True,train_two_powers_view_orders: True,n_chunks: -1,counting_examples: False,include_task_token: False,message_history_truncation: 2048,task_version: 3,include_game_info: True,include_player_ratings: True,include_draw_info: False,include_draw_state: True,hide_empty_draw_state: True,include_centers_state: True,include_builds_state: False,player_rating_max: 5,player_rating_percentiles: games_played,set_player_rating: 5,include_player_chattiness: False,set_player_chattiness: -1,only_phase: None,only_game_id: None,only_chunk: -1,skip_input_validation: False,input_validation_check_pct: 0.1,lie_detector_annotations_dir: None,lie_detector_filter_above_stdev: None,chunk_size: 80,rank: 0,interactive_mode: True,datapath: /home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages/data 02:41:32 | your model is being loaded with opts that differ from the model you are initializing the weights with. Add the following args to your run command to change this: --task None --batchsize 1 --attention-dropout 0.1 --n-positions 1024 --beam-length-penalty 0.65 --inference greedy --temperature 1.0 --fp16-impl safe --optimizer sgd --learningrate 1 --text-truncate None --label-truncate None --special-tok-lst None --gpu -1 --lr-scheduler reduceonplateau --warmup-updates -1 --model bart --parlai-home /home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages --dict-loaded False 02:41:32 | Using CUDA 02:41:32 | loading dictionary from /home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages/data/models/bart/bart_large/model.dict 02:41:32 | num words = 50264 02:41:32 | BartCustomInference: full interactive mode on. 02:41:37 | Total parameters: 408,470,528 (408,470,528 trainable) 02:41:37 | Loading existing model params from /home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages/data/models/bart/bart_large/model I0713 02:41:38 [transformer:380] Resizing token embeddings from 50264 to 50349 Traceback (most recent call last): File "/home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages/parlai/core/torch_agent.py", line 2045, in load_state_dict self.model.load_state_dict(state_dict) File "/home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1052, in load_state_dict self.class.name, "\n\t".join(error_msgs))) RuntimeError: Error(s) in loading state_dict for BartModel: size mismatch for embeddings.weight: copying a param with shape torch.Size([50264, 1024]) from checkpoint, the shape in current model is torch.Size([50349, 1024]). size mismatch for encoder.embeddings.weight: copying a param with shape torch.Size([50264, 1024]) from checkpoint, the shape in current model is torch.Size([50349, 1024]). size mismatch for encoder.position_embeddings.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([2048, 1024]). size mismatch for decoder.embeddings.weight: copying a param with shape torch.Size([50264, 1024]) from checkpoint, the shape in current model is torch.Size([50349, 1024]). size mismatch for decoder.position_embeddings.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([2048, 1024]).

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "run.py", line 166, in heyhi.parse_args_and_maybe_launch(main) File "/home/luoyingtao/diplomacy_cicero/heyhi/run.py", line 122, in parse_args_and_maybe_launch maybe_launch(main, exp_root=get_exp_dir(PROJECT_NAME), overrides=overrides, *kwargs) File "/home/luoyingtao/diplomacy_cicero/heyhi/run.py", line 173, in maybe_launch util.run_with_config(main, exp_handle, cfg, overrides, ckpt_dir, log_level) File "/home/luoyingtao/diplomacy_cicero/heyhi/util.py", line 608, in run_with_config callable() File "/home/luoyingtao/diplomacy_cicero/heyhi/util.py", line 383, in wrapped result = f(args, *kwargs) File "run.py", line 162, in main return TASKStask File "run.py", line 44, in compare_agents agent_one = build_agent_from_cfg(cfg.agent_one) File "/home/luoyingtao/diplomacy_cicero/fairdiplomacy/agents/init.py", line 47, in build_agent_from_cfg return AGENT_CLASSESwhich_agent File "/home/luoyingtao/diplomacy_cicero/fairdiplomacy/agents/bqre1p_agent.py", line 367, in init cfg.base_searchbot_cfg, skip_base_strategy_model_cache=skip_base_strategy_model_cache File "/home/luoyingtao/diplomacy_cicero/fairdiplomacy/agents/searchbot_agent.py", line 493, in init self.parlai_model_orders = load_order_wrapper(cfg.parlai_model_orders) File "/home/luoyingtao/diplomacy_cicero/parlai_diplomacy/wrappers/factory.py", line 137, in load_order_wrapper model = parlai_wrapper_factory(cfg) File "/home/luoyingtao/diplomacy_cicero/parlai_diplomacy/wrappers/factory.py", line 103, in parlai_wrapper_factory ret = ParlAIAllOrderIndependentRolloutWrapper(wrapper_args) File "/home/luoyingtao/diplomacy_cicero/parlai_diplomacy/wrappers/base_wrapper.py", line 91, in init model_path=model_path, frozen_overrides=frozen_args File "/home/luoyingtao/diplomacy_cicero/parlai_diplomacy/wrappers/base_wrapper.py", line 56, in load_parlai_agent_cached return create_agent_from_model_file(model_path, override_opts) File "/home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages/parlai/core/agents.py", line 347, in create_agent_from_model_file return create_agent_from_opt_file(opt) File "/home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages/parlai/core/agents.py", line 421, in create_agent_from_opt_file return model_class(opt_from_file) File "/home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages/parlai/agents/bart/bart.py", line 72, in init super().init(opt, shared) File "/home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages/parlai/core/torch_generator_agent.py", line 517, in init states = self.load(init_model) File "/home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages/parlai/core/torch_agent.py", line 2078, in load self.load_state_dict(states['model']) File "/home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages/parlai/core/torch_agent.py", line 2051, in load_state_dict self.model.load_state_dict(state_dict) File "/home/luoyingtao/miniconda3/envs/diplomacy_cicero/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1052, in load_state_dict self.class.name, "\n\t".join(error_msgs))) RuntimeError: Error(s) in loading state_dict for BartModel: size mismatch for encoder.position_embeddings.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([2048, 1024]). size mismatch for decoder.position_embeddings.weight: copying a param with shape torch.Size([1024, 1024]) from checkpoint, the shape in current model is torch.Size([2048, 1024])

yingtaoluo commented 1 year ago

The Cicero paper mentioned that 1024 has been extended to 2048, but the Bart Model is still 1024. Not sure why we need to download Bart Model because I have downloaded all the models via download instructions already.

yingtaoluo commented 1 year ago

The error occurred when "python run.py --adhoc --cfg conf/c01_ag_cmp/cmp.prototxt Iagent_one=agents/cicero.prototxt use_shared_agent=1 power_one=TURKEY".

yingtaoluo commented 1 year ago

@jonathathan @alexholdenmiller Could I ask for help from you? I am really interested in Cicero and want to learn and simulate the game.