Processing of frequency aggregation is slow, taking 10 seconds for sample data containing 1580 trips.
This is about 20 times longer than the time to read stops and routes (0.5s).
Cause
In frequency aggregation, the processing time of the stop aggregation accounts for 94% of the total.
Most of the time is spent on __get_similar_stop_tuple().
The cause is that __get_similar_stop_tuple() is called for the number of stops by map() for the stops data frame.
__get_similar_stop_tuple() is slow because it searches and sorts all of the stops on each call.
Problem
Processing of frequency aggregation is slow, taking 10 seconds for sample data containing 1580 trips. This is about 20 times longer than the time to read stops and routes (0.5s).
Cause
In frequency aggregation, the processing time of the stop aggregation accounts for 94% of the total. Most of the time is spent on
__get_similar_stop_tuple()
. The cause is that__get_similar_stop_tuple()
is called for the number of stops bymap()
for the stops data frame.__get_similar_stop_tuple()
is slow because it searches and sorts all of the stops on each call.Profiling results
Solution
I consider that the process will be much faster if the process is done for the entire stops data frame at once.
Sample data
GTFS: feed_chitetsu_chitetsubus_20240326_191913.zip Results of cProfile: chitetsu_prof.txt