Test stability of assignment algorithm

[BZ-BowenZhang]

According to the discussion on 7th AUG between @BZ-BowenZhang @sgreenbury and @Hussein-Mahfouz, the difference between the assigning results across each run is needed to test and prove this algorithm's stability. This may involve at least two parts:

• [ ] Observe the difference between each run with the same input configuration to prove the stability.
• [ ] Observe the difference in travel flow distributions with different input configurations to test the sensibility for specific configurations.

[sgreenbury]

I have started looking at this to see if the same RNG seed generates outputs deterministically.

It looks like the scripts up to 3.2.3 are currently deterministic but not for secondary locations with PAM.

[sgreenbury]

Exploring this would also be simplified through implementing #53 first

[sgreenbury]

From discussion with @BZ-BowenZhang and @Hussein-Mahfouz, we'd like to measure three things:

1. Running with same seed, leads to identical outputs
2. Running with different seeds, generates different stochastic samples from the pipeline's distribution. Invariance of properties will measure the stability.
3. Running with different config and different seed generates outputs with different assumptions allowing a measure of the sensitivity.

[Hussein-Mahfouz]

@sgreenbury non determinism in 3.2.3_assign_secondary_zone.py may be due to the following:

• we are using our internal update_population_plans here, which is a wrapper around pam DiscretionaryTrips class. We use the function update_plan from the class, which itselef uses DiscretionaryTripsODand DiscretionaryTripsRound.
• Both of these classes use the sample_weighted function to select a zone. That function uses random (see here)

[sgreenbury]

Thanks @Hussein-Mahfouz for identifying these calling random.choice(), I think that's it. I've updated the init_rng() method to include setting the seed for random too and this now produces deterministic outputs for a given seed. I'll open a PR!