search

uber-research / deep-neuroevolution

Deep Neuroevolution
Other
1.63k stars 298 forks source link

Worker receives old task_id from pipe #14

Closed njustesen closed 6 years ago

njustesen commented 6 years ago

WorkerClient.get_current_task() doesn't return the new task_id when the master's work_id changes, but instead keeps returning the old task_id. The workers cached_task_id is thus never updated. What can be the reason for this?

njustesen commented 6 years ago

I run the following in four different windows:

redis-server redis_config/redis_master.conf
redis-server redis_config/redis_local_mirror.conf
python -m es_distributed.main master --master_socket_path /tmp/es_redis_master.sock --algo es --exp_file 'configurations/frostbite_es.json'
python -m es_distributed.main workers --master_host localhost --relay_socket_path /tmp/es_redis_relay.sock --algo es --num_workers 2

Sometimes it runs a few iterations but then the workers stop getting the newest task_id.

VashishtMadhavan commented 6 years ago

This issue has been resolved in the latest commit: fe24443927fd23d644a847c6d68733bf27f4877e

njustesen commented 6 years ago

Thank you for looking into this. However, I think you may have forgotten to push dist.py as I now get the following error:

File "/Users/noju/git/deep-neuroevolution/es_distributed/es.py", line 210, in run_master temp_taskid, = master.pop_result(flushing=True) TypeError: pop_result() got an unexpected keyword argument 'flushing'