perciplex / raas

Reality as a Service: OpenAI Gym environments in real life
https://raas.perciplex.com
7 stars 1 forks source link

Stepper encoder randomly resets #37

Closed declanoller closed 4 years ago

declanoller commented 4 years ago

The stepper angle is determined by its self.step variable. It sets this to 0 in its __init__(). However, it appears to reset during running programs, so then it sets its zero point to a different place than when it started. This will make learning impossible.

We probably have to either make it more robust (not restart randomly), or at least have a reset command in motor_controller that sets self.step = 0 again, after you pause and let it settle.

declanoller commented 4 years ago

Killing motor_controller.py while running reproduced this, so that's a good guess. If you do ps aux | grep motor_con, it has different entries for whether it's being run by the system process or manually by the user. Maybe it cuts in and out if it's not a priority or something.

declanoller commented 4 years ago

Okay, I reproduced it by running motor_controller.py manually, so it's not that it's run through the service.

I also made it print the state at the beginning of an ep, after waiting 15 seconds to settle down, so you can see if it has reset.

It could be coincidence, but it was doing fine until it moved fast and went over the top, and then it was messed up:

Episode 4   Average score: -1266.58 Average Q: 0.00
Sleeping for 15s to let it reset...
state: [-1.0000000e+00 -1.2246468e-16  0.0000000e+00]
Go!
Step 0
Step 20
Step 40
Step 60
Step 80
Step 100
Step 120
Step 140
Step 160
Step 180

Episode 5   Average score: -1303.41 Average Q: 0.00
Sleeping for 15s to let it reset...
state: [-1.0000000e+00 -1.2246468e-16  0.0000000e+00]
Go!
Step 0
Step 20
Step 40
Step 60
Step 80
Step 100
Step 120
Step 140
Step 160
Step 180

Episode 6   Average score: -1327.33 Average Q: 0.00
Sleeping for 15s to let it reset...
state: [-0.93358043 -0.35836795  0.        ]
Go!
Step 0
Step 20
Step 40
Step 60
Step 80
Step 100
^CTraceback (most recent call last):
  File "ddpg.py", line 216, in <module>
    main()

We think it might be pigpiod's polling frequency.

declanoller commented 4 years ago

So we changed the pigpiod parameter to -s 1, and it still messed up after a few episodes. AGAIN, it was after an episode where it swung very fast and went over the top.

MaximilianKing commented 4 years ago

We were on the wrong service Type. As a result, the service was restarting every 90 seconds (the default wait time for a waiting service). Changing Type=notify to Type=simple resolved the issue.