search

stanfordnlp / wge

Workflow-Guided Exploration: sample-efficient RL agent for web tasks
https://stanfordnlp.github.io/wge/
Other
108 stars 33 forks source link

error when running python main.py #4

Closed khliu1238 closed 6 years ago

khliu1238 commented 6 years ago

hi:

when running python main.py configs/default-base.txt --task click-tab-2 the following error occurred, without changing any single line of code / data:

TrainingRun configuration:
gamma = 1.0
discount_negative_reward = false
train {
  max_control_steps = 50000
  learning_rate = 0.001
  behavioral_cloning = false
  replay = 5
  replay_steps = 30
  reinforce_program = true
  reinforce_neural = false
}
explore {
  program = 1
  neural = 10
  best_first_search = 10
  max_steps_per_episode = 10
  best_first_beam_size = 10
}
log {
  episodes_to_evaluate_small = 20
  episodes_to_evaluate_big = 500
  evaluate = 100
  evaluate_big = 500
  explore = 10
  replay = 10
  trace_evaluate = 1000
  trace_explore = 1000
  trace_replay = 1000
  save = 1000000000
  visualize_attention = 100000
  record_screenshots = false
}
replay_buffer {
  size = 1000
  min_size = 100
}
policy {
  query_type = "structured"
  episodes_to_replay = 5
  update_rule = "use-whole-episode"
  baseline = 0.1
  use_critic = true
  attn_dim = 64
  context_embed_dim = 96
  dom_attention_for_state = true
  scoring_batch_size = 64
  utterance_embedder {
    type = "glove"
    vocab_size = 100000
    lstm_dim = 64
  }
  dom_embedder {
    tag_embed_dim = 16
    value_embed_dim = 16
    tampered_embed_dim = 8
    classes_embed_dim = 16
  }
}
program_policy {
  type = "program"
  parameterization = "softmax"
  weight_init = 1.0
  learning_rate = 0.8
  alpha = 0.1
  init_v = 0.0
  max_programs = 50
}
env {
  domain = "miniwob"
  num_instances = 6
  subdomain = "click-tab-2"
  return_image = false
  headless = false
  wait_ms = 0
  block_on_reset = true
  refresh_freq = 50
  reward_processor {
    type = "time_independent"
  }
}
demonstrations {
  min_raw_reward = 1.0
  base_dir = "2017-10-16_second-turk"
  parser = "chunk-shortcut"
  max_to_use = 1000000000000
}
LOADING DEMOS FROM: /Users/wge/third-party/miniwob-demos/2017-10-16_second-turk/click-tab-2
Loading demonstrations: 100%|█████████████| 32/32 [00:00<00:00, 74.19it/s]
Loading GloVe embeddings: 100%|█▉| 99608/100000 [00:10<00:00, 9878.71it/s]No GPUs detected. Sticking with CPUs.
No checkpoint to reload. Initializing fresh.
No uncommitted changes.
New TrainingRun created at: /Users/data/experiments/2_unnamed
                                                              ERROR:root:############################################################/s]
ERROR:root:=== SOMETHING WRONG HAPPENED !!! === (Retry attempt 1 / 5)
Traceback (most recent call last):
  File "/Users/wge/wge/episode_generator.py", line 62, in __call__
    seeds, record_screenshots)
  File "/Users/wge/wge/episode_generator.py", line 93, in _get_episodes
    actions = policy.act(states, test_policy)
  File "/Users/wge/wge/wob_policy.py", line 425, in act
    action_scores_batch = self.score_actions(states)
  File "/Users/wge/wge/wob_policy.py", line 192, in score_actions
    states, force_dom_attn, force_type_values)
  File "/Users/wge/wge/wob_policy.py", line 221, in _score_actions
    dom_embeds, dom_elems = self._dom_embeds(states)
  File "/Users/wge/wge/wob_policy.py", line 502, in _dom_embeds
    dom_elems, aligned_dom_embeds)
  File "/Users/env/lib/python2.7/site-packages/torch/nn/modules/module.py", line 325, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/wge/wge/miniwob/embeddings.py", line 406, in forward
    pixel_neighbor_embeds, allow_empty=True)
  File "/Users/wge/gtd/ml/torch/seq_batch.py", line 174, in reduce_mean
    weights = mask / sums.expand(*mask.size())
RuntimeError: The expanded size of the tensor (9) must match the existing size (96) at non-singleton dimension 1. at /Users/soumith/code/builder/wheel/pytorch-src/torch/lib/TH/generic/THTensor.c:309
ERROR:root:Will restart the environment and try again ...
ERROR:root:############################################################
ERROR:root:############################################################
ERROR:root:=== SOMETHING WRONG HAPPENED !!! === (Retry attempt 2 / 5)
Traceback (most recent call last):
  File "/Users/wge/wge/episode_generator.py", line 62, in __call__
    seeds, record_screenshots)
  File "/Users/wge/wge/episode_generator.py", line 93, in _get_episodes
    actions = policy.act(states, test_policy)
  File "/Users/wge/wge/wob_policy.py", line 425, in act
    action_scores_batch = self.score_actions(states)
  File "/Users/wge/wge/wob_policy.py", line 192, in score_actions
    states, force_dom_attn, force_type_values)
  File "/Users/wge/wge/wob_policy.py", line 221, in _score_actions
    dom_embeds, dom_elems = self._dom_embeds(states)
  File "/Users/wge/wge/wob_policy.py", line 502, in _dom_embeds
    dom_elems, aligned_dom_embeds)
  File "/Users/env/lib/python2.7/site-packages/torch/nn/modules/module.py", line 325, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/wge/wge/miniwob/embeddings.py", line 406, in forward
    pixel_neighbor_embeds, allow_empty=True)
  File "/Users/wge/gtd/ml/torch/seq_batch.py", line 174, in reduce_mean
    weights = mask / sums.expand(*mask.size())
RuntimeError: The expanded size of the tensor (7) must match the existing size (90) at non-singleton dimension 1. at /Users/soumith/code/builder/wheel/pytorch-src/torch/lib/TH/generic/THTensor.c:309
ERROR:root:Will restart the environment and try again ...
ERROR:root:############################################################
ERROR:root:############################################################
ERROR:root:=== SOMETHING WRONG HAPPENED !!! === (Retry attempt 3 / 5)
Traceback (most recent call last):
  File "/Users/wge/wge/episode_generator.py", line 62, in __call__
    seeds, record_screenshots)
  File "/Users/wge/wge/episode_generator.py", line 93, in _get_episodes
    actions = policy.act(states, test_policy)
  File "/Users/wge/wge/wob_policy.py", line 425, in act
    action_scores_batch = self.score_actions(states)
  File "/Users/wge/wge/wob_policy.py", line 192, in score_actions
    states, force_dom_attn, force_type_values)
  File "/Users/wge/wge/wob_policy.py", line 221, in _score_actions
    dom_embeds, dom_elems = self._dom_embeds(states)
  File "/Users/wge/wge/wob_policy.py", line 502, in _dom_embeds
    dom_elems, aligned_dom_embeds)
  File "/Users/env/lib/python2.7/site-packages/torch/nn/modules/module.py", line 325, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/wge/wge/miniwob/embeddings.py", line 406, in forward
    pixel_neighbor_embeds, allow_empty=True)
  File "/Users/wge/gtd/ml/torch/seq_batch.py", line 174, in reduce_mean
    weights = mask / sums.expand(*mask.size())
RuntimeError: The expanded size of the tensor (6) must match the existing size (90) at non-singleton dimension 1. at /Users/soumith/code/builder/wheel/pytorch-src/torch/lib/TH/generic/THTensor.c:309
ERROR:root:Will restart the environment and try again ...
ERROR:root:############################################################
ERROR:root:############################################################
ERROR:root:=== SOMETHING WRONG HAPPENED !!! === (Retry attempt 4 / 5)
Traceback (most recent call last):
  File "/Users/wge/wge/episode_generator.py", line 62, in __call__
    seeds, record_screenshots)
  File "/Users/wge/wge/episode_generator.py", line 93, in _get_episodes
    actions = policy.act(states, test_policy)
  File "/Users/wge/wge/wob_policy.py", line 425, in act
    action_scores_batch = self.score_actions(states)
  File "/Users/wge/wge/wob_policy.py", line 192, in score_actions
    states, force_dom_attn, force_type_values)
  File "/Users/wge/wge/wob_policy.py", line 221, in _score_actions
    dom_embeds, dom_elems = self._dom_embeds(states)
  File "/Users/wge/wge/wob_policy.py", line 502, in _dom_embeds
    dom_elems, aligned_dom_embeds)
  File "/Users/env/lib/python2.7/site-packages/torch/nn/modules/module.py", line 325, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/wge/wge/miniwob/embeddings.py", line 406, in forward
    pixel_neighbor_embeds, allow_empty=True)
  File "/Users/wge/gtd/ml/torch/seq_batch.py", line 174, in reduce_mean
    weights = mask / sums.expand(*mask.size())
RuntimeError: The expanded size of the tensor (7) must match the existing size (96) at non-singleton dimension 1. at /Users/soumith/code/builder/wheel/pytorch-src/torch/lib/TH/generic/THTensor.c:309
ERROR:root:Will restart the environment and try again ...
ERROR:root:############################################################
ERROR:root:############################################################
ERROR:root:=== SOMETHING WRONG HAPPENED !!! === (Retry attempt 5 / 5)
Traceback (most recent call last):
  File "/Users/wge/wge/episode_generator.py", line 62, in __call__
    seeds, record_screenshots)
  File "/Users/wge/wge/episode_generator.py", line 93, in _get_episodes
    actions = policy.act(states, test_policy)
  File "/Users/wge/wge/wob_policy.py", line 425, in act
    action_scores_batch = self.score_actions(states)
  File "/Users/wge/wge/wob_policy.py", line 192, in score_actions
    states, force_dom_attn, force_type_values)
  File "/Users/wge/wge/wob_policy.py", line 221, in _score_actions
    dom_embeds, dom_elems = self._dom_embeds(states)
  File "/Users/wge/wge/wob_policy.py", line 502, in _dom_embeds
    dom_elems, aligned_dom_embeds)
  File "/Users/env/lib/python2.7/site-packages/torch/nn/modules/module.py", line 325, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/wge/wge/miniwob/embeddings.py", line 406, in forward
    pixel_neighbor_embeds, allow_empty=True)
  File "/Users/wge/gtd/ml/torch/seq_batch.py", line 174, in reduce_mean
    weights = mask / sums.expand(*mask.size())
RuntimeError: The expanded size of the tensor (7) must match the existing size (108) at non-singleton dimension 1. at /Users/soumith/code/builder/wheel/pytorch-src/torch/lib/TH/generic/THTensor.c:309
ERROR:root:Will restart the environment and try again ...
ERROR:root:############################################################
ERROR:root:############################################################
ERROR:root:=== SOMETHING WRONG HAPPENED !!! === (Retry attempt 6 / 5)
Traceback (most recent call last):
  File "/Users/wge/wge/episode_generator.py", line 62, in __call__
    seeds, record_screenshots)
  File "/Users/wge/wge/episode_generator.py", line 93, in _get_episodes
    actions = policy.act(states, test_policy)
  File "/Users/wge/wge/wob_policy.py", line 425, in act
    action_scores_batch = self.score_actions(states)
  File "/Users/wge/wge/wob_policy.py", line 192, in score_actions
    states, force_dom_attn, force_type_values)
  File "/Users/wge/wge/wob_policy.py", line 221, in _score_actions
    dom_embeds, dom_elems = self._dom_embeds(states)
  File "/Users/wge/wge/wob_policy.py", line 502, in _dom_embeds
    dom_elems, aligned_dom_embeds)
  File "/Users/env/lib/python2.7/site-packages/torch/nn/modules/module.py", line 325, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/wge/wge/miniwob/embeddings.py", line 406, in forward
    pixel_neighbor_embeds, allow_empty=True)
  File "/Users/wge/gtd/ml/torch/seq_batch.py", line 174, in reduce_mean
    weights = mask / sums.expand(*mask.size())
RuntimeError: The expanded size of the tensor (7) must match the existing size (102) at non-singleton dimension 1. at /Users/soumith/code/builder/wheel/pytorch-src/torch/lib/TH/generic/THTensor.c:309
ERROR:root:Will restart the environment and try again ...
ERROR:root:############################################################
Traceback (most recent call last):
  File "main.py", line 103, in <module>
    run.train()
  File "/Users/wge/wge/training_run.py", line 209, in train
    control_step)
  File "/Users/wge/wge/training_run.py", line 306, in _explore
    test_policy=False)
  File "/Users/wge/wge/episode_generator.py", line 73, in __call__
    raise RuntimeError('Envionment died too many times!')
RuntimeError: Envionment died too many times!

the root cause seems to be File "/Users/wge/gtd/ml/torch/seq_batch.py", line 174, in reduce_mean weights = mask / sums.expand(*mask.size()) RuntimeError: The expanded size of the tensor (7) must match the existing size (108) at non-singleton dimension 1. at /Users/soumith/code/builder/wheel/pytorch-src/torch/lib/TH/generic/THTensor.c:309 but I'm not sure what wrong from here.

any idea what's wrong?

ezliu commented 6 years ago

I think the likely cause is that you're running with a different version of PyTorch. Can you confirm that you're using Python2.7 and that your version of PyTorch is 0.1.12_2?

Run: 

python -c "import torch; print torch.__version__"

This should output: 0.1.12_2.

If you have a different version of PyTorch, the README has instructions for installing this version.

khliu1238 commented 6 years ago

that's right, I just discovered conda doesn't work with virtualenv, have to use conda virtual environment instead to install pytorch 0.1.12 in a virtual environment. It's working now. Thanks.