ActivitySim / activitysim

An Open Platform for Activity-Based Travel Modeling
https://activitysim.github.io
BSD 3-Clause "New" or "Revised" License
189 stars 96 forks source link

School zone alternatives are far away from home for high school student #815

Open i-am-sijia opened 4 months ago

i-am-sijia commented 4 months ago

Describe the bug When doing benchmarking runs with full size prototype mtc extended, the model crashed in the trip mode choice for some school-related trips. When looking at the trip mode choice trace files, I noticed that the child is going to their high school (taz 1402) that is 146 miles away from home (taz 690). This is not directly causing the mode choice to fail, but it is odd to go to school 146 miles away from home for a high school student.

Traced the school location choice model. Attaching the trace files.

choosers-5c2274.csv alternatives-5c2274.csv 2156922-5c2274.csv interaction_utilities-5c2274.csv probs-5c2274.csv rands-5c2274.csv utilities-5c2274.csv

This high school child only has 8 school zone alternatives, and all 8 of them are far far away from home.

Hypothesis:

  1. The new Shadow Pricing mechanism could be causing this. It could be that all the schools close to home are full and they are turned off in the next shadow pricing simulation.
  2. Others?

To Reproduce Steps to reproduce the behavior:

  1. Use main branch @ff3e96b.
  2. Run full size prototype mtc extended model with 100% household.
  3. Trace hh id 1098914

Additional context

If it is the new Shadow Pricing mechanism that's causing this, should we not run Shadow Pricing for high school students? For work, it might be ok to commute 146 miles, but for school, one probably prefer going over the size term than sending students far away from home...

It would be good to check in the model result how many students are attending schools far away from home.

jpn-- commented 4 months ago

I ran this and confirmed it is the new shadow pricing algorithm that is sending this high school kids musical-chairs style to a school 143 miles away from home, after being booted out from 7 other closer schools due to overcrowding.

I don't think this is a "bug" per se, I think it is an expected outcome of this algorithm, and I recall we did talk about this potentially happening back when we were discussing the implementation. We agreed back then that it was not a problem to have a smattering of people end up sent pell mell across the region, as it's expected to be only a small handful. @i-am-sijia has a good point on the school location though... I believe when RSG initially looked at these effects they looked at work locations, which are a bit more homogeneously distributed through the region than schools, esp high schools. As an example in this model with 1454 zones, nearly every zone has grade school and employment attractions, but there are only 218 zones with high school enrollment.

dhensle commented 4 months ago

The 6 Oct 22 Presentation that RSG gave goes into good detail about how the simulation-based mandatory location choice constraint algorithm works and some plots on convergence.

Pasted below is an excerpt from that presentation. It shows that the school tour lengths are a little bit longer than the old "ctramp" shadow pricing method showed. If you dig more into the presentation you will see that the simulation-based constraint mechanism converges much better than the shadow pricing mechanism. The thought at the time was that the shadow pricing would eventually converge to a value close to the simulation-based mechanism. We did not see any red flags when looking in aggregate at school level trips and the trend for longer tour lengths was present for workplace location as well.

image

There is a setting for percent tolerance which gets applied here as

# shadow prices are set to -999 if overassigned or 0 if the zone still has room for this segment self.shadow_prices[segment] = np.where( (sprice <= 1 + percent_tolerance / 100), -999, 0 )

We could adjust the percent tolerance setting and see if this makes a difference? (The default value is 5% which I think is actually relatively large for a one zone system running a 100% sample.)

cc: @aletzdy

joecastiglione commented 4 months ago

David,

Thanks so much for pointing to this presentation - it was helpful to get a refresh on the technical details

It's not within anyone's current scope to deal with this, but in looking back at the convergence chart, I think that there is clearly a bigger issue with school location choice than work location choice.

For work location, the simulation method produces location average distances about 4% longer than using CT-RAMP shadow pricing method.

But for school location, the simulation method produces location average distances about more than 30% longer than using CT-RAMP shadow pricing method. What's not clear to me is whether this is just a calibration issue, or whether there's something else going on.

I'm interested in exploring the sensitivity to the percent tolerance (is the current MTC extended example using the 5% setting?), but after we reach the end of the current phase of optimization work.

Thanks,

Joe [image: Screenshot 2024-02-29 at 7.11.50 AM.png]

Joe Castiglione Deputy Director for Technology, Data, and Analysis

sfcta.org https://www.sfcta.org/ | sign up for our newsletter https://www.sfcta.org/stay-connected

On Wed, Feb 28, 2024 at 8:23 PM David Hensle @.***> wrote:

The 6 Oct 22 Presentation https://github.com/ActivitySim/activitysim/wiki/Project-Meeting-2022.10.06#presentation that RSG gave goes into good detail about how the simulation-based mandatory location choice constraint algorithm works and some plots on convergence.

Pasted below is an excerpt from that presentation. It shows that the school tour lengths are a little bit longer than the old "ctramp" shadow pricing method showed. If you dig more into the presentation you will see that the simulation-based constraint mechanism converges much better than the shadow pricing mechanism. The thought at the time was that the shadow pricing would eventually converge to a value close to the simulation-based mechanism. We did not see any red flags when looking in aggregate at school level trips and the trend for longer tour lengths was present for workplace location as well.

image.png (view on web) https://github.com/ActivitySim/activitysim/assets/51132108/7158df3e-bb6d-42ca-af3f-0297010b3712

There is a setting for percent tolerance which gets applied here https://github.com/ActivitySim/activitysim/blob/main/activitysim/abm/tables/shadow_pricing.py#L868 as

shadow prices are set to -999 if overassigned or 0 if the zone still has

room for this segment self.shadow_prices[segment] = np.where( (sprice <= 1

  • percent_tolerance / 100), -999, 0 )

We could adjust the percent tolerance setting https://github.com/ActivitySim/activitysim/blob/main/activitysim/abm/tables/shadow_pricing.py#L118 and see if this makes a difference? (The default value is 5% which I think is actually relatively large for a one zone system running a 100% sample.)

cc: @aletzdy https://github.com/aletzdy

— Reply to this email directly, view it on GitHub https://github.com/ActivitySim/activitysim/issues/815#issuecomment-1970370695, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDVIHZATNWU37FJWSOZEKDYV77C3AVCNFSM6AAAAABDVYT2QCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZQGM3TANRZGU . You are receiving this because you are subscribed to this thread.Message ID: @.***>