Debug

[umair-nasir14]

Good day @ruiwang2uber

Hope you are doing well. I am using poet for my master's thesis and found two errors. The errors and solutions are following:

1)

Error:

Missing lower and upper bounds for cap_score() (line 597 and 600).

Solution:

Added lower and upper bound to cap_score().

2)

Error:

niches[optim_id].delete_env(env_name)

Solution:

niches[self.optim_id].delete_env(env_name)

Hope this helps, thank you!

With best regards, Muhammad

[CLAassistant]

All committers have signed the CLA.

[umair-nasir14]

The second Update es.py commit should be viewed as the previous one Debug had a mistake

[erdisayar]

After fixing this error, I am getting the following one. Any idea how to fix this ?

INFO:poet_distributed.poet_algo:Iter=449 Optimizer flat theta_mean 318.9991343427439 best po 320.3475929971013 iteration spent 449
INFO:poet_distributed.poet_algo:health_check
INFO:poet_distributed.poet_algo:niche flat created at 0 start_score -93.02073542368042 current_self_evals 318.9991343427439
INFO:poet_distributed.poet_algo:list of niches to reproduce
INFO:poet_distributed.poet_algo:['flat']
INFO:poet_distributed.poet_algo:list of niches to delete
INFO:poet_distributed.poet_algo:[]
/home/nrp/Documents/publications/poet/poet_distributed/stats.py:22: RuntimeWarning: invalid value encountered in true_divide
  y /= (x.size - 1)
INFO:poet_distributed.poet_algo:we pick to mutate: flat and we got 882c9023-a365-4d4e-b959-5c805205c92d back
INFO:poet_distributed.poet_algo:we pick to mutate: flat and we got 3b17566b-260a-447d-9a17-9980af148df3 back
INFO:poet_distributed.poet_algo:we pick to mutate: flat and we got 48e0aeea-243f-44c9-bea0-4bf294fb3d0b back
INFO:poet_distributed.poet_algo:we pick to mutate: flat and we got 0bece951-cc6f-4f8d-8884-66a1d28fea5f back
INFO:poet_distributed.poet_algo:we pick to mutate: flat and we got e3a48cdc-6833-4ea4-849d-12de167c95f4 back
INFO:poet_distributed.poet_algo:we pick to mutate: flat and we got cd4f9f5d-1ba2-4bd0-a515-d56398fa0516 back
INFO:poet_distributed.poet_algo:we pick to mutate: flat and we got 84110d94-c66c-4b9e-88c1-4781cdb8bca7 back
INFO:poet_distributed.poet_algo:we pick to mutate: flat and we got 3d08cef7-4e6d-4f27-9e42-89713adac983 back
INFO:poet_distributed.logger:Creating data logger at /home/nrp/logs/poet_final_test/poet_final_test.882c9023-a365-4d4e-b959-5c805205c92d.log
INFO:poet_distributed.es:Optimizer 882c9023-a365-4d4e-b959-5c805205c92d created!
Traceback (most recent call last):
  File "master.py", line 78, in <module>
    main()
  File "master.py", line 75, in main
    run_main(args)
  File "master.py", line 38, in run_main
    steps_before_transfer=args.steps_before_transfer)
  File "/home/nrp/Documents/publications/poet/poet_distributed/poet_algo.py", line 347, in optimize
    max_num_envs=self.args.max_num_envs)
  File "/home/nrp/Documents/publications/poet/poet_distributed/poet_algo.py", line 318, in adjust_envs_niches
    if self.pass_mc(score_archive):
  File "/home/nrp/Documents/publications/poet/poet_distributed/poet_algo.py", line 238, in pass_mc
    if score < self.args.mc_lower or score > self.args.mc_upper:
TypeError: '<' not supported between instances of 'NoneType' and 'int'

[umair-nasir14]

in pass_mc() function just write a check first that:

if score == None:
    return False

This should do it.

[erdisayar]

It works thanks, Approximately after how many iterations , the envrionment is changing from flat to the something else ?

[umair-nasir14]

It works thanks, Approximately after how many iterations, the environment is changing from flat to the something else ?

You are welcome. So it checks every 150 iterations if there is a possibility to create a new env. I think for author's standard ES it creates in 300-450 iterations.

[erdisayar]

I am in 804 th iteration and used this command to train the model ./run_poet_local.sh final_test . Logs are still showing flat output.

It is because something we change and it affected the algorithm's learning ?

INFO:poet_distributed.poet_algo:Iter=804 Optimizer flat theta_mean 331.87367868082106 best po 333.19986814360766 iteration spent 804
INFO:poet_distributed.poet_algo:Iter=804 Optimizer 8b4f6442-9ece-4353-b2f3-7b2b48009119 theta_mean 307.36649731125027 best po 307.8356657658848 iteration spent 504
INFO:poet_distributed.poet_algo:Iter=804 Optimizer 7307230c-ee49-4248-be24-40949b06ac36 theta_mean 317.1629876322855 best po 313.02190838114836 iteration spent 354
INFO:poet_distributed.poet_algo:Iter=804 Optimizer 3c0fd31a-885a-4d79-a8bd-48bba715ece3 theta_mean 311.2954170824174 best po 309.33581614951606 iteration spent 204
INFO:poet_distributed.poet_algo:Iter=804 Optimizer 781c723a-198c-46e6-93d5-b38093a4e3e5 theta_mean 316.5226709072094 best po 309.8612311332749 iteration spent 54

[umair-nasir14]

So it has created 4 more optimisers. There should be a logs file which will have five files, each for an optimiser. Optimiser here means environment as well as agent.

On Fri, 29 Apr 2022 at 21:33, Erdi @.***> wrote:

I am in 804 th iteration and used this command to train the model ./run_poet_local.sh final_test . Logs are still showing flat output.

INFO:poet_distributed.poet_algo:Iter=804 Optimizer flat theta_mean 331.87367868082106 best po 333.19986814360766 iteration spent 804 INFO:poet_distributed.poet_algo:Iter=804 Optimizer 8b4f6442-9ece-4353-b2f3-7b2b48009119 theta_mean 307.36649731125027 best po 307.8356657658848 iteration spent 504 INFO:poet_distributed.poet_algo:Iter=804 Optimizer 7307230c-ee49-4248-be24-40949b06ac36 theta_mean 317.1629876322855 best po 313.02190838114836 iteration spent 354 INFO:poet_distributed.poet_algo:Iter=804 Optimizer 3c0fd31a-885a-4d79-a8bd-48bba715ece3 theta_mean 311.2954170824174 best po 309.33581614951606 iteration spent 204 INFO:poet_distributed.poet_algo:Iter=804 Optimizer 781c723a-198c-46e6-93d5-b38093a4e3e5 theta_mean 316.5226709072094 best po 309.8612311332749 iteration spent 54

— Reply to this email directly, view it on GitHub https://github.com/uber-research/poet/pull/8#issuecomment-1113655418, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQHQ6LT4VGIHGNWMNG4LMIDVHQ2PJANCNFSM5RM26JQA . You are receiving this because you authored the thread.Message ID: @.***>

[erdisayar]

Yes I have 5 files. I am in 1041 th iteration and output in the terminal still shows the flat environment. I uploaded the log folder here poet_final_test.zip

[umair-nasir14]

Yes these are the logs per optimizers and best.json are the checkpoints I guess. I did briefly try to use them once as a checkpoint and wasn't successful. Let me know if you find something about it.

[erdisayar]

I would like to understand If the environment is changeing or not ( because log shows always flat) I will dive into the codes and try to understand them.

I will let you know once I am successfull about the checkpoints

If you want , we can keep communication about the poet. Here is my email: erdisayar@hotmail.com

[umair-nasir14]

So there are other logs in your zipped file as well. all these 16-digits.logs are each for one environment.

I believe the code is a bit over-complicated yet understandable.

I believe there is a line logger.debug('{} passed mc, novelty score {}'.format(...)) in the function 'get_child_list()' in 'poet_algo.py', make it logger.info and now when your new environment is created (ignore the first two times) it should have some novelty score which will indicate that your new environment is actually not just the same one.

Sure I would love to, I'll email you and we can keep in communication.