Implement PSO

[kymckay]

• [x] Particle class
  • [x] Has position
  • [x] Has velocity
  • [x] Remembers current best
  • [x] Has informants
• [x] Swarm class
  • [x] Initialises particles uniformly in the search space
  • [x] Performs search steps
  • [x] Update information (knowledge of best positions)
  • [x] Update position with velocity and step size
    • [x] Handles search space bounds
  • [x] Update velocity (important: stochastic)
  • [x] Run to completiton

[linarietuma]

After re-watching the lecture on PSOs and Part 2 of CW, here's a summary of the suggested options for an improved mark:

1. Investigate different variants of PSOs:
   • Adaptive swarm sizes (unfit particles die off/ fit ones are replicated)
   • Adaptive weights
   • Multi swarm (useful for avoiding premature convergence)
   • Bare-bones PSO (a simplified version of PSO, uses Gaussian distribution instead of velocities)
2. Optimise additional ANN params in addition to weights (e.g. activation functions)
3. Investigate different methods for boundary handling:
   • Truncate values to fit within bounds
   • Reject dimension updates that do not fit within the bounds
   • Bounce particles off the boundaries
   • Randomly re-initialise particles that go out of bounds

[linarietuma]

We're doing point 2. and at least some version of point 3., so I think we're good.

[kymckay]

Note from slides:

Typically, PSO terminates after a fixed number of iterations