"JavaScript heap out of memory"

[k-yle]

I tried your tool on the Auckland Transport GTFS data (for Auckland, New Zealand), but I got this failure:

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

This is what the task manager shows:

Complete logs:

Generating HTML schedules for AT
Starting GTFS import for 1 file
AT: Importing GTFS from ./gtfs/
AT: Importing - agency.txt - 17 lines imported
AT: Importing - calendar_dates.txt - 149624 lines imported
AT: Importing - calendar.txt - 42664 lines imported
AT: Importing - fare_attributes.txt - No file found
AT: Importing - fare_rules.txt - No file found
AT: Importing - feed_info.txt
AT: Importing - frequencies.txt - No file found
AT: Importing - routes.txt - 2524 lines imported
AT: Importing - shapes.txt - 2148670 lines imported
AT: Importing - stop_times.txt - 1488810 lines imported
AT: Importing - stops.txt - 9896 lines imported
AT: Importing - transfers.txt - No file found
AT: Importing - trips.txt - 42664 lines imported
AT: Post Processing data
AT: Completed
Completed GTFS import for 1 file

<--- Last few GCs --->

[11640:000002382E0AFBE0]   741617 ms: Mark-sweep 1412.6 (1479.7) -> 1412.6 (1448.7) MB, 2345.1 / 0.0 ms  (+ 2.0 ms in 1 steps since start of marking, biggest step 2.0 ms) last resort gc
[11640:000002382E0AFBE0]   742567 ms: Mark-sweep 1412.6 (1448.7) -> 1412.6 (1448.7) MB, 949.7 / 0.0 ms  last resort gc

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 000003F9A92C0D51 <JS Object>
    1: deserializeObject(aka deserializeObject) [C:\Users\Kyle\AppData\Roaming\npm\node_modules\mongoose\node_modules\mongodb-core\node_modules\bson\lib\bson\parser\deserializer.js:~39] [pc=000001190B8DEDB5](this=000003F9A9204311 <undefined>,buffer=000003C89C087439 <an Uint8Array with map 0000017C2101AA99>,index=12,options=000003C89C087409 <an Object with map 0000001786A30939>,isArray=000003F...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

config.json file:

{
    "mongoUrl": "mongodb://127.0.0.1:27017/gtfs",
    "agencies": [{
        "agency_key": "AT",
        "path": "./gtfs/"
    }],
    "effectiveDate": "December 19, 2016",
    "noServiceSymbol": "—",
    "requestStopSymbol": "***",
    "showMap": true,
    "showOnlyTimepoint": false,
    "showStopCity": false,
    "zipOutput": false,
    "verbose": true
}

you can download the GTFS data from here.

[3vivekb]

There is a workaround here: https://github.com/brendannee/gtfs-to-html/issues/27

[brendannee]

I just pushed an update that makes showOnlyTimepoint default to true. This should speed up timetable generation (as it will only consider stops that are timepoints) and I think is what most people would want as default.

Try setting "showOnlyTimepoint" to true in your config.json and see if this works as expected.

[k-yle]

I tried both methods, however I still get the "JavaScript heap out of memory" error. The task manager shows that node.js uses around 3200 MB before it stops (with a limit of --max_old_space_size=3000, the highest I can allocate).

[brendannee]

I pushed an update which should solve this memory error - now each timetable is built in series and so it gets to a point where it is unable to find trips for a specific route.

This GTFS file is interesting in that is has 10,000s of service_ids for 1000s of routes - which is pretty intense. It looks like each route has its own service id - which makes the lookup take a while.

Give it a try and let me know where you get.

Note that you can now create a timetables.txt and timetable_pages.txt file as detailed in the readme to specify which routes and directions get created - so if you don't need all the thousands of routes in this file then you can build just the ones you need by including these files.

[k-yle]

This happens now - see gist (too long to copy-paste in a comment)

[brendannee]

I got the same message when running. So the memory error is gone, its now complaining that for a specific route, direction and set of service_ids, it can't find any trips.

You can try creating a timetables.txt file and adding it to the folder with your GTFS - this allows you to specify a route_id, direction_id and specific days of the week for each timetable you want to generate. See this file for an example: https://github.com/BlinkTagInc/gtfs-to-html/blob/master/examples/timetables.txt

[k-yle]

Yes, Auckland Transport tends to remove trips but keep all the other info - e.g. in the school holidays, the school bus services are removed from the trips.txt file but kept everywhere else.

Is there an easier way to generate timetables based on the route_short_name property? For example, if I wanted a timetable for the 881 bus service (881 is the route_short_name)?

[brendannee]

Currently, if you just wanted a timetable for one route, you could create a timetables.txt file with just one line and it would only generate one timetable. You have to use route_id instead of route_short_name - so you'll have to check in routes.txt first to figure that out.

Give it a try and let me know if you have questions.