Things to Do

[joon0503]

We are simplifying the problem right now. Followings are a list of improvements we must do.

• [x] Randomizing goal points
• [x] Find the bottleneck

Algorithms

• [x] Add heading to estimation & control
• [x] Add sensors to interactive plotting
• [x] Add estimated vehicle position and sensor measurements to plot
• [x] Use optimal input for the estimation
• [ ] Add more structure to NN that estimates the position. (e.g., use veh x,y position instead of goal points, only use veh pos and action to estimate next vehicle position, use scaled action instead of one-hot-encoding, always make prev position as origin)
• [ ] Implement the curiosity-based algorithm
• [ ] Distributed optimization + parallel processing?
• [ ] Policy gradients
• [ ] Train another neural net to estimate the success chance

Performance

• [x] Implementation using the PyBullet
• [x] Improve performance of applying steering action by using custom scripts
• [x] Parallelize applySteeringAction function
• [x] Increase the number of parallel vehicles and check the performance -> performance gain is minimal
• [ ] Implement parallel processing

Training

• [x] Scale up LIDAR count (to 19?)
• [x] Add penalty for extreme actions
• [x] T-intersection & Random lane width
• [ ] Correct scaling for PyBullet

• [ ] Incorporate actuator dynamics

• [x] Fix position of obstacle for half of track? Perhaps the training data is skewed to ones that passes automatically without much effort
• [x] Adding cost to change of input to reduce oscillatory behavior
• [ ] Adding control for speed/acceleration
• [ ] Fix estimation memorization issue
• [ ] Moving obstacles?
• [ ] Change scenario to T intersection & randomize goal point
• [ ] Add noise the LIDAR data for robustness

Getting Results

• [ ] Use multiple frames

Coding

• [ ] Modularize code
• [ ] Use Keras

[joon0503]

Adding more issues.

[joon0503]

Adding more issues related to curiosity-driven learning

[joon0503]

Added sensors. See joon0503@e3cd43f9794fe4ec188800637bc6048cf2cfe4cd and joon0503@cc4d3a09e83802bc9aee155bb8eb3274408840a9

[joon0503]

• Estimation & Control may not need heading
• Added performance issue

[joon0503]

Improved performance of applySteeringAction doing heavy work in vrep.

[joon0503]

Now estimates uses the optimal action. See joon0503@f1a4fb6e81b97a0a4873a93548a72ba43f82cbd3

[joon0503]

Implemented PyBullet for better performance. Also includes random lane width.

See 372b881e6a2ab975ceb74898110ab355e9e30152