Closed Mithrandir2k18 closed 3 years ago
lanctot
commented
3 years ago I believe it's because back then we had dm-haiku listed as a git link since there was no pip package.
I think it should be as easy as replacing the last line in requirements.txt from:
git+git://github.com/deepmind/dm-haiku@master
to
dm-haiku == 0.0.1
lanctot
commented
3 years ago Any luck @Mithrandir2k18 ?
Mithrandir2k18
commented
3 years ago 2 of the tests fail, but if I skip the tests I can build the container now and run the example of the concepts.md and print the terminal state at the end without any problems.
Find below the working Dockerfile and the output of the tests.
Dockerfile:
FROM pytorch/pytorch:1.6.0-cuda10.1-cudnn7-devel
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y libx11-dev curl gnupg2 git python3-setuptools clang-10 python3-dev sudo python3-wheel make cmake gcc g++
RUN DEBIAN_FRONTEND="noninteractive" apt-get -y install tzdata
# Install pip
RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && python get-pip.py; rm get-pip.py
# Install Python dependencies
RUN pip install --upgrade pynput PyYAML numpy matplotlib nox cmake
## Install Open_Spiel
RUN git clone https://github.com/deepmind/open_spiel.git /openspiel
WORKDIR /openspiel
RUN git checkout tags/v0.1.0 -b build
RUN sed -i '$ d' requirements.txt && cat requirements.txt && echo "dm-haiku == 0.0.1" >> requirements.txt && cat requirements.txt
RUN pip install --upgrade setuptools testresources
RUN yes | ./install.sh
RUN python3 -m pip install .
# RUN nox -s testsTests Output. Uncomment the last line of dockerfile to run tests during build.
Using /openspiel/.nox/tests/lib/python3.7/site-packages
Finished processing dependencies for pyspiel==0.0.1rc2
nox > cd build/temp.linux-x86_64-3.7
nox > ctest -j96 --output-on-failure
Test project /openspiel/build/temp.linux-x86_64-3.7
Start 1: best_response_test
Start 2: cfr_test
Start 3: cfr_br_test
Start 4: corr_dist_test
Start 5: deterministic_policy_test
Start 6: evaluate_bots_test
Start 7: external_sampling_mccfr_test
Start 8: get_all_states_test
Start 9: get_legal_actions_map_test
Start 10: history_tree_test
Start 11: is_mcts_test
Start 12: matrix_game_utils_test
Start 13: minimax_test
Start 14: outcome_sampling_mccfr_test
Start 15: state_distribution_test
Start 16: tabular_exploitability_test
Start 17: tensor_game_utils_test
Start 18: trajectories_test
Start 19: benchmark_game_test
Start 20: cfr_example_test
Start 21: example_test
Start 22: matrix_example_test
Start 23: mcts_example_test
Start 24: value_iteration_example_test
Start 25: backgammon_test
Start 26: blotto_test
Start 27: breakthrough_test
Start 28: bridge_test
Start 29: catch_test
Start 30: chess_test
Start 31: cliff_walking_test
Start 32: coin_game_test
Start 33: connect_four_test
Start 34: coop_box_pushing_test
Start 35: coordinated_mp_test
Start 36: cursor_go_test
Start 37: deep_sea_test
Start 38: efg_game_test
Start 39: first_sealed_auction_test
Start 40: gin_rummy_test
Start 41: go_test
Start 42: goofspiel_test
Start 43: havannah_test
Start 44: hex_test
Start 45: kuhn_poker_test
Start 46: leduc_poker_test
Start 47: liars_dice_test
Start 48: markov_soccer_test
Start 49: matching_pennies_3p_test
Start 50: matrix_games_test
Start 51: negotiation_test
Start 52: oshi_zumo_test
Start 53: othello_test
Start 54: oware_test
Start 55: pentago_test
Start 56: phantom_ttt_test
Start 57: pig_test
Start 58: quoridor_test
Start 59: skat_test
Start 60: tic_tac_toe_test
Start 61: laser_tag_test
Start 62: tiny_bridge_test
Start 63: tiny_hanabi_test
Start 64: trade_comm_test
Start 65: y_test
Start 66: turn_based_simultaneous_game_test
Start 67: misere_test
Start 68: coop_to_1p_test
Start 69: efg_writer_test
Start 70: normal_form_extensive_game_test
Start 71: spiel_test
Start 72: python_api_test
Start 73: python_playthrough_test
Start 74: python_action_value_test
Start 75: python_action_value_vs_best_response_test
Start 76: python_evaluator_test
Start 77: python_model_test
Start 78: python_best_response_test
Start 79: python_cfr_br_test
Start 80: python_cfr_test
Start 81: python_deep_cfr_test
Start 82: python_discounted_cfr_test
Start 83: python_double_oracle_test
Start 84: python_dqn_test
Start 85: python_eva_test
Start 86: python_evaluate_bots_test
Start 87: python_expected_game_score_test
Start 88: python_exploitability_descent_test
Start 89: python_exploitability_test
Start 90: python_fictitious_play_test
Start 91: python_gambit_test
Start 92: python_generate_playthrough_test
Start 93: python_get_all_states_test
Start 94: python_rl_losses_test
Start 95: python_lp_solver_test
Start 96: python_masked_softmax_test
1/151 Test #1: best_response_test ................................ Passed 1.49 sec
Start 97: python_mcts_test
2/151 Test #4: corr_dist_test .................................... Passed 1.50 sec
Start 98: python_minimax_test
3/151 Test #5: deterministic_policy_test ......................... Passed 1.51 sec
Start 99: python_neurd_test
4/151 Test #9: get_legal_actions_map_test ........................ Passed 1.51 sec
Start 100: python_nfsp_test
5/151 Test #13: minimax_test ...................................... Passed 1.53 sec
Start 101: python_noisy_policy_test
6/151 Test #15: state_distribution_test ........................... Passed 1.53 sec
Start 102: python_outcome_sampling_mccfr_test
7/151 Test #19: benchmark_game_test ............................... Passed 1.53 sec
Start 103: python_policy_aggregator_joint_test
8/151 Test #21: example_test ...................................... Passed 1.54 sec
Start 104: python_policy_aggregator_test
9/151 Test #22: matrix_example_test ............................... Passed 1.54 sec
Start 105: python_policy_gradient_test
10/151 Test #35: coordinated_mp_test ............................... Passed 1.44 sec
Start 106: python_projected_replicator_dynamics_test
11/151 Test #49: matching_pennies_3p_test .......................... Passed 1.30 sec
Start 107: python_best_response_oracle_test
12/151 Test #63: tiny_hanabi_test .................................. Passed 1.16 sec
Start 108: python_strategy_selectors_test
13/151 Test #12: matrix_game_utils_test ............................ Passed 1.59 sec
Start 109: python_generalized_psro_test
14/151 Test #20: cfr_example_test .................................. Passed 1.60 sec
Start 110: python_rectified_nash_response_test
15/151 Test #31: cliff_walking_test ................................ Passed 1.64 sec
Start 111: python_random_agent_test
16/151 Test #37: deep_sea_test ..................................... Passed 1.67 sec
Start 112: python_rcfr_test
17/151 Test #47: liars_dice_test ................................... Passed 1.74 sec
Start 113: python_sequence_form_lp_test
18/151 Test #56: phantom_ttt_test .................................. Passed 1.76 sec
Start 114: python_value_iteration_test
19/151 Test #60: tic_tac_toe_test .................................. Passed 1.81 sec
Start 115: python_bluechip_bridge_test
20/151 Test #64: trade_comm_test ................................... Passed 1.82 sec
Start 116: python_bluechip_bridge_uncontested_bidding_test
21/151 Test #87: python_expected_game_score_test ................... Passed 1.65 sec
Start 117: python_is_mcts_test
22/151 Test #75: python_action_value_vs_best_response_test ......... Passed 1.84 sec
Start 118: python_uniform_random_test
23/151 Test #23: mcts_example_test ................................. Passed 2.68 sec
Start 119: python_alpharank_test
24/151 Test #38: efg_game_test ..................................... Passed 2.58 sec
Start 120: python_alpharank_visualizer_test
25/151 Test #67: misere_test ....................................... Passed 2.28 sec
Start 121: python_dynamics_test
26/151 Test #68: coop_to_1p_test ................................... Passed 2.27 sec
Start 122: python_heuristic_payoff_table_test
27/151 Test #91: python_gambit_test ................................ Passed 2.02 sec
Start 123: python_utils_test
28/151 Test #92: python_generate_playthrough_test .................. Passed 2.02 sec
Start 124: python_visualization_test
29/151 Test #26: blotto_test ....................................... Passed 2.77 sec
Start 125: python_catch_test
30/151 Test #33: connect_four_test ................................. Passed 2.73 sec
Start 126: python_cliff_walking_test
31/151 Test #36: cursor_go_test .................................... Passed 2.72 sec
Start 127: python_data_test
32/151 Test #50: matrix_games_test ................................. Passed 2.70 sec
Start 128: python_tic_tac_toe_test
33/151 Test #54: oware_test ........................................ Passed 2.70 sec
Start 129: python_bot_test
34/151 Test #59: skat_test ......................................... Passed 2.71 sec
Start 130: python_games_bridge_test
35/151 Test #45: kuhn_poker_test ................................... Passed 3.05 sec
Start 131: python_games_sim_test
36/151 Test #101: python_noisy_policy_test .......................... Passed 1.87 sec
Start 132: python_matrix_game_utils_test
37/151 Test #111: python_random_agent_test .......................... Passed 1.73 sec
Start 133: python_policy_test
38/151 Test #115: python_bluechip_bridge_test ....................... Passed 1.51 sec
Start 134: python_pyspiel_test
39/151 Test #122: python_heuristic_payoff_table_test ................ Passed 0.98 sec
Start 135: python_rl_environment_test
40/151 Test #123: python_utils_test ................................. Passed 1.15 sec
Start 136: python_tensor_game_utils_test
41/151 Test #116: python_bluechip_bridge_uncontested_bidding_test ... Passed 1.66 sec
Start 137: python_file_logger_test
42/151 Test #118: python_uniform_random_test ........................ Passed 1.53 sec
Start 138: python_lru_cache_test
43/151 Test #52: oshi_zumo_test .................................... Passed 3.67 sec
Start 139: python_spawn_test
44/151 Test #44: hex_test .......................................... Passed 4.19 sec
Start 140: python_examples_bridge_supervised_learning
45/151 Test #83: python_double_oracle_test ......................... Passed 3.84 sec
Start 141: circular_buffer_test
46/151 Test #121: python_dynamics_test .............................. Passed 1.85 sec
Start 142: data_logger_test
47/151 Test #138: python_lru_cache_test ............................. Passed 0.68 sec
Start 143: file_test
48/151 Test #137: python_file_logger_test ........................... Passed 0.77 sec
Start 144: json_test
49/151 Test #41: go_test ........................................... Passed 4.57 sec
Start 145: logger_test
50/151 Test #125: python_catch_test ................................. Passed 1.97 sec
Start 146: lru_cache_test
51/151 Test #127: python_data_test .................................. Passed 1.90 sec
Start 147: run_python_test
52/151 Test #55: pentago_test ...................................... Passed 4.52 sec
Start 148: stats_test
53/151 Test #128: python_tic_tac_toe_test ........................... Passed 1.91 sec
Start 149: tensor_view_test
54/151 Test #98: python_minimax_test ............................... Passed 3.46 sec
Start 150: thread_test
55/151 Test #126: python_cliff_walking_test ......................... Passed 2.15 sec
Start 151: threaded_queue_test
56/151 Test #141: circular_buffer_test .............................. Passed 0.53 sec
57/151 Test #32: coin_game_test .................................... Passed 5.00 sec
58/151 Test #139: python_spawn_test ................................. Passed 1.13 sec
59/151 Test #142: data_logger_test ..................................***Failed 0.50 sec
Spiel Fatal Error: /openspiel/open_spiel/utils/data_logger_test.cc:35 CHECK_FALSE(file::Exists(dir))
60/151 Test #143: file_test ......................................... Passed 0.48 sec
61/151 Test #144: json_test ......................................... Passed 0.41 sec
62/151 Test #145: logger_test ....................................... Passed 0.43 sec
63/151 Test #146: lru_cache_test .................................... Passed 0.43 sec
64/151 Test #148: stats_test ........................................ Passed 0.38 sec
65/151 Test #147: run_python_test ...................................***Failed 0.56 sec
/openspiel/.nox/tests/bin/python3: Error while finding module specification for 'open_spiel.utils.run_python_test_file' (ModuleNotFoundError: No module named 'open_spiel')
Spiel Fatal Error: /openspiel/open_spiel/utils/run_python_test.cc:25 CHECK_TRUE(RunPython(test_module, {"--return_value", "0"}))
66/151 Test #150: thread_test ....................................... Passed 0.35 sec
67/151 Test #149: tensor_view_test .................................. Passed 0.52 sec
68/151 Test #151: threaded_queue_test ............................... Passed 0.45 sec
69/151 Test #135: python_rl_environment_test ........................ Passed 1.85 sec
70/151 Test #17: tensor_game_utils_test ............................ Passed 6.13 sec
71/151 Test #74: python_action_value_test .......................... Passed 5.62 sec
72/151 Test #40: gin_rummy_test .................................... Passed 6.12 sec
73/151 Test #46: leduc_poker_test .................................. Passed 6.67 sec
74/151 Test #96: python_masked_softmax_test ........................ Passed 6.26 sec
75/151 Test #120: python_alpharank_visualizer_test .................. Passed 4.36 sec
76/151 Test #94: python_rl_losses_test ............................. Passed 6.45 sec
77/151 Test #119: python_alpharank_test ............................. Passed 4.89 sec
78/151 Test #51: negotiation_test .................................. Passed 7.35 sec
79/151 Test #103: python_policy_aggregator_joint_test ............... Passed 6.65 sec
80/151 Test #130: python_games_bridge_test .......................... Passed 5.27 sec
81/151 Test #8: get_all_states_test ............................... Passed 8.99 sec
82/151 Test #27: breakthrough_test ................................. Passed 9.42 sec
83/151 Test #129: python_bot_test ................................... Passed 6.56 sec
84/151 Test #62: tiny_bridge_test .................................. Passed 9.96 sec
85/151 Test #42: goofspiel_test .................................... Passed 10.23 sec
86/151 Test #29: catch_test ........................................ Passed 10.93 sec
87/151 Test #104: python_policy_aggregator_test ..................... Passed 9.54 sec
88/151 Test #76: python_evaluator_test ............................. Passed 10.75 sec
89/151 Test #53: othello_test ...................................... Passed 11.42 sec
90/151 Test #124: python_visualization_test ......................... Passed 9.07 sec
91/151 Test #132: python_matrix_game_utils_test ..................... Passed 8.54 sec
92/151 Test #84: python_dqn_test ................................... Passed 11.29 sec
93/151 Test #108: python_strategy_selectors_test .................... Passed 11.68 sec
94/151 Test #78: python_best_response_test ......................... Passed 12.71 sec
95/151 Test #24: value_iteration_example_test ...................... Passed 13.78 sec
96/151 Test #86: python_evaluate_bots_test ......................... Passed 13.41 sec
97/151 Test #34: coop_box_pushing_test ............................. Passed 14.36 sec
98/151 Test #11: is_mcts_test ...................................... Passed 14.59 sec
99/151 Test #16: tabular_exploitability_test ....................... Passed 15.17 sec
100/151 Test #100: python_nfsp_test .................................. Passed 14.17 sec
101/151 Test #99: python_neurd_test ................................. Passed 14.40 sec
102/151 Test #85: python_eva_test ................................... Passed 15.68 sec
103/151 Test #48: markov_soccer_test ................................ Passed 17.16 sec
104/151 Test #69: efg_writer_test ................................... Passed 17.38 sec
105/151 Test #97: python_mcts_test .................................. Passed 20.42 sec
106/151 Test #57: pig_test .......................................... Passed 22.93 sec
107/151 Test #102: python_outcome_sampling_mccfr_test ................ Passed 21.90 sec
108/151 Test #6: evaluate_bots_test ................................ Passed 23.44 sec
109/151 Test #39: first_sealed_auction_test ......................... Passed 23.49 sec
110/151 Test #25: backgammon_test ................................... Passed 24.05 sec
111/151 Test #81: python_deep_cfr_test .............................. Passed 23.82 sec
112/151 Test #113: python_sequence_form_lp_test ...................... Passed 22.93 sec
113/151 Test #106: python_projected_replicator_dynamics_test ......... Passed 23.53 sec
114/151 Test #66: turn_based_simultaneous_game_test ................. Passed 25.85 sec
115/151 Test #131: python_games_sim_test ............................. Passed 24.16 sec
116/151 Test #43: havannah_test ..................................... Passed 27.29 sec
117/151 Test #58: quoridor_test ..................................... Passed 28.84 sec
118/151 Test #105: python_policy_gradient_test ....................... Passed 29.37 sec
119/151 Test #136: python_tensor_game_utils_test ..................... Passed 27.10 sec
120/151 Test #95: python_lp_solver_test ............................. Passed 33.44 sec
121/151 Test #117: python_is_mcts_test ............................... Passed 32.07 sec
122/151 Test #140: python_examples_bridge_supervised_learning ........ Passed 30.95 sec
123/151 Test #82: python_discounted_cfr_test ........................ Passed 34.97 sec
124/151 Test #65: y_test ............................................ Passed 35.94 sec
125/151 Test #10: history_tree_test ................................. Passed 36.44 sec
126/151 Test #107: python_best_response_oracle_test .................. Passed 35.24 sec
127/151 Test #30: chess_test ........................................ Passed 38.90 sec
128/151 Test #73: python_playthrough_test ........................... Passed 38.57 sec
129/151 Test #112: python_rcfr_test .................................. Passed 37.64 sec
130/151 Test #110: python_rectified_nash_response_test ............... Passed 38.63 sec
131/151 Test #71: spiel_test ........................................ Passed 41.52 sec
132/151 Test #93: python_get_all_states_test ........................ Passed 43.43 sec
133/151 Test #14: outcome_sampling_mccfr_test ....................... Passed 47.92 sec
134/151 Test #70: normal_form_extensive_game_test ................... Passed 48.41 sec
135/151 Test #7: external_sampling_mccfr_test ...................... Passed 48.88 sec
136/151 Test #88: python_exploitability_descent_test ................ Passed 55.65 sec
137/151 Test #134: python_pyspiel_test ............................... Passed 52.81 sec
138/151 Test #3: cfr_br_test ....................................... Passed 63.98 sec
139/151 Test #77: python_model_test ................................. Passed 63.95 sec
140/151 Test #114: python_value_iteration_test ....................... Passed 65.74 sec
141/151 Test #28: bridge_test ....................................... Passed 68.98 sec
142/151 Test #18: trajectories_test ................................. Passed 69.26 sec
143/151 Test #90: python_fictitious_play_test ....................... Passed 69.30 sec
144/151 Test #79: python_cfr_br_test ................................ Passed 70.81 sec
145/151 Test #109: python_generalized_psro_test ...................... Passed 75.31 sec
146/151 Test #2: cfr_test .......................................... Passed 78.94 sec
147/151 Test #133: python_policy_test ................................ Passed 77.93 sec
148/151 Test #61: laser_tag_test .................................... Passed 83.02 sec
149/151 Test #80: python_cfr_test ................................... Passed 84.53 sec
150/151 Test #89: python_exploitability_test ........................ Passed 91.16 sec
151/151 Test #72: python_api_test ................................... Passed 244.57 sec
99% tests passed, 2 tests failed out of 151
Total Test time (real) = 245.07 sec
The following tests FAILED:
142 - data_logger_test (Failed)
147 - run_python_test (Failed)
Errors while running CTest
nox > Command ctest -j96 --output-on-failure failed with exit code 8
nox > Session tests failed.
The command '/bin/sh -c nox -s tests' returned a non-zero code: 1Ok, cool. Both of these tests interact with the filesystem in non-trivial ways, so my guess is the container setup is preventing such access (possibly that access needs to be enabled somehow, not an expert here).
If all the others are passing, you should be ok now. So, I will close this as fixed but please re-open if you have other problems.
I am trying to install basing from other docker containers that provide PyTorch/Tensorflow-GPU respectively. However during build I get this error:
Dockerfile: