search

ActivitySim / activitysim

An Open Platform for Activity-Based Travel Modeling
https://activitysim.github.io
BSD 3-Clause "New" or "Revised" License
189 stars 96 forks source link

fix bug in interaction_simulate #839

Closed jpn-- closed 3 months ago

jpn-- commented 3 months ago

We recently fixed a problem where sharrow was holding on to data references, sometimes preventing memory from being freed.

This introduced a bug in the sharrow-test error handler of the interaction simulate code, where sharrow re-runs data if it does not match utility values with the original non-sharrow code. This PR fixes the bug.

jpn-- commented 3 months ago

@i-am-sijia this is the fix for the intermediate error we discussed yesterday.

i-am-sijia commented 3 months ago

Thank you. This fixes the NoneType error, but Sharrow is still crashing at the non_mandatory_tour_scheduling.vectorize_tour_scheduling.tour_6.interaction_sample_simulate.eval_interaction_utils, due to the difference in utilities. See attached logs.

activitysim_nonetype_error.log activitysim.log

You mentioned Sharrow can find and print out the utility expression(s) that have the biggest value difference. It will be helpful in this case. What setting should I use?

I did not get this error when running the full-scale benchmarking run with Sharrow. So this might just be happening with the small test example. Should I open a separate issue?

i-am-sijia commented 3 months ago

Would this PR have any memory implication?

jpn-- commented 3 months ago

You mentioned Sharrow can find and print out the utility expression(s) that have the biggest value difference. It will be helpful in this case. What setting should I use?

You just run in debug mode setting a breakpoint here: https://github.com/ActivitySim/activitysim/blob/2540ede5d6718c20dbd65143205e81c9aabaaf1d/activitysim/core/interaction_simulate.py#L582 and the variable look_for_problems_here should point the way.

jpn-- commented 3 months ago

Would this PR have any memory implication?

There are no implications for memory usage unless sharrow and non-sharrow disagree on what the utility values are. In that case, sharrow does gobble up a bunch of memory to write out all the expression values into memory so you can peruse them in the debugger, but this is obviously desirable in this situation.

i-am-sijia commented 3 months ago

Would this PR have any memory implication?

There are no implications for memory usage unless sharrow and non-sharrow disagree on what the utility values are. In that case, sharrow does gobble up a bunch of memory to write out all the expression values into memory so you can peruse them in the debugger, but this is obviously desirable in this situation.

That makes sense. I believe the utility comparison only happens in the sharrow compile mode. This change should not impact if the user is running sharrow production mode, which is when we are usually concerned about memory.