TypeError: Cannot read property 'end' of undefined

[missinglink]

reported by @heffergm May 20th

/mnt/pelias/pelias-osm/releases/20150520094242/node_modules/pelias-admin-lookup/lib/master.js:298
    lookup.end();
          ^
TypeError: Cannot read property 'end' of undefined
    at DestroyableTransform.end [as _flush] (/mnt/pelias/pelias-osm/releases/20150520094242/node_modules/pelias-admin-lookup/lib/master.js:298:11)
    at DestroyableTransform.<anonymous> (/mnt/pelias/pelias-osm/releases/20150520094242/node_modules/pelias-admin-lookup/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:135:12)
    at DestroyableTransform.g (events.js:199:16)
    at DestroyableTransform.emit (events.js:129:20)
    at finishMaybe (/mnt/pelias/pelias-osm/releases/20150520094242/node_modules/pelias-admin-lookup/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:371:12)
    at endWritable (/mnt/pelias/pelias-osm/releases/20150520094242/node_modules/pelias-admin-lookup/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:378:3)
    at DestroyableTransform.Writable.end (/mnt/pelias/pelias-osm/releases/20150520094242/node_modules/pelias-admin-lookup/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:356:5)
    at DestroyableTransform.onend (/mnt/pelias/pelias-osm/releases/20150520094242/node_modules/through2/node_modules/readable-stream/lib/_stream_readable.js:523:10)
    at DestroyableTransform.g (events.js:199:16)
    at DestroyableTransform.emit (events.js:129:20)

[missinglink]

I think this is fixed, leaving open to track as grant reported it again, I was not able to reproduce this bug in dev

[missinglink]

I saw this error again, in geonames, it may be due to the problematic streaming zip decompression module

[thebabelfish]

I just pulled everything today from the https://github.com/pelias/openstreetmap and ran it on Ubuntu 14.04 using nodejs-legacy from apt-get repositories and get this error.

  /home/vagrant/openstreetmap/node_modules/pelias-admin-lookup/lib/master.js:298
      lookup.end();
             ^
  TypeError: Cannot call method 'end' of undefined
      at DestroyableTransform.end [as _flush] (/home/vagrant/openstreetmap/node_modules/pelias-admin-lookup/lib/master.js:298:12)
      at DestroyableTransform.<anonymous> (/home/vagrant/openstreetmap/node_modules/pelias-admin-lookup/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:135:12)
      at DestroyableTransform.g (events.js:180:16)
      at DestroyableTransform.EventEmitter.emit (events.js:117:20)
      at finishMaybe (/home/vagrant/openstreetmap/node_modules/pelias-admin-lookup/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:371:12)
      at endWritable (/home/vagrant/openstreetmap/node_modules/pelias-admin-lookup/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:378:3)
      at DestroyableTransform.Writable.end (/home/vagrant/openstreetmap/node_modules/pelias-admin-lookup/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:356:5)
      at DestroyableTransform.onend (/home/vagrant/openstreetmap/node_modules/through2/node_modules/readable-stream/lib/_stream_readable.js:539:10)
      at DestroyableTransform.g (events.js:180:16)
      at DestroyableTransform.EventEmitter.emit (events.js:117:20)

Any ways around this?

[missinglink]

I've had a look at this a few times now (it happens infrequently) and what I can tell you is:

a cosmetic fix is to change https://github.com/pelias/admin-lookup/blob/master/lib/master.js#L298 to:

if( lookup && 'function' === typeof lookup.end ) lookup.end();

although this doesn't really fix the underlying issue, which is that either:

• your data source contains 0 valid records and so end() is called prematurely
• the async logic for creating the workers in sub processes is not correctly blocking the stream flow

I'm going to bump up the priority on this as it's been reported several times although I likely won't get to it until after our current sprint.

everyone please continue to report this issue when it occurs so I can get a better scope of the problem, also please let me know the pbf file (or whatever data source) you are using, I suspect this only occurs for smaller extracts.

the cosmetic fix above will mask the issue until it's fixed properly.

[thebabelfish]

@missinglink thanks for the cosmetic fix. Much appreciated. I was running an Arizona pbf file downloaded from here:

http://download.geofabrik.de/north-america/us/arizona-latest.osm.pbf

And an important note, I ran the same install steps and data on my OSX Yosemite and it worked great. The problem occurred only on 14.04 Ubuntu.

Thanks again.

[missinglink]

thanks for the info @thebabelfish, I'm pretty convinced it's not Ubuntu as I run that OS and only see the error very very occasionally.

it's one of those nasty heisenbugs that only happens intermittently, I think the real cause is a mix of race conditions and complex asynchronous code.

[riordan]

This code will be deprecated in favor of wof-admin-lookup, so hopefully we won't have these issues any longer.