ITSLeeds / UK2GTFS

Convert UK transport data (TransXchange / ATOC CIF) to GTFS format in R
https://itsleeds.github.io/UK2GTFS/
GNU General Public License v3.0
37 stars 13 forks source link

performance of bind #51

Closed oweno-tfwm closed 1 year ago

oweno-tfwm commented 1 year ago

on a full timetable file the call to bind takes slightly more than 10 minutes, which is longer than the multi threaded section before it.

changing to call data.table::rbindlist reduces this to 20 seconds or so. In order for this to work the columns need to stay in the same order - which we achieve by flipping the join round so that the target is on the left hand side of the output instead of the right hand side - which means that the columns stay in the same order, so we don't need to match on names either.

0001-performance(first).patch