search

ITSLeeds / UK2GTFS

Convert UK transport data (TransXchange / ATOC CIF) to GTFS format in R
https://itsleeds.github.io/UK2GTFS/
GNU General Public License v3.0
37 stars 13 forks source link

atoc2gtfs fails because makeCalendar downloads all data in each thread #61

Open mpadge opened 10 months ago

mpadge commented 10 months ago

This package is cleverly set up to download all of the release data each time it is loaded. But that means these lines can fail: https://github.com/ITSLeeds/UK2GTFS/blob/5094f1f4a447d65f2e14cea0b494c9c602192a24/R/atoc_export.R#L386-L388

In my case, I've got 36 threads, and those lines cause each thread to download the full dataset, which then maxes out my available bandwidth, causing failure of threads and therefore the whole routine. (I can get around this by running fewer threads, but would prefer a proper solution to this.)

Edit: Actually, that whole call to checkrows(res.calendar.days) takes hardly any time even in single threaded, so maybe the easiest solution would be just removing the multi-core version there? Everything else then works fine.

mem48 commented 4 months ago

This is lazy coding on my part, it is not necessary to load the whole package so this can be fixed by specifying the relevant functions