jacktuck / unfurl

Metadata scraper with support for oEmbed, Twitter Cards and Open Graph Protocol for Node.js :zap:
MIT License
481 stars 52 forks source link

Error: write after end #35

Closed shaunc869 closed 6 years ago

shaunc869 commented 6 years ago

I am getting a sporadic error:

Error: write after end at writeAfterEnd (_stream_writable.js:236:12) at PassThrough.Writable.write (_stream_writable.js:287:5) at IncomingMessage.ondata (_stream_readable.js:639:20) at emitOne (events.js:116:13) at IncomingMessage.emit (events.js:211:7) at addChunk (_stream_readable.js:263:12) at readableAddChunk (_stream_readable.js:250:11) at IncomingMessage.Readable.push (_stream_readable.js:208:10) at HTTPParser.parserOnBody (_http_common.js:130:22) at TLSSocket.socketOnData (_http_client.js:440:20) at emitOne (events.js:116:13) at TLSSocket.emit (events.js:211:7) at addChunk (_stream_readable.js:263:12) at readableAddChunk (_stream_readable.js:250:11) at TLSSocket.Readable.push (_stream_readable.js:208:10) at TLSWrap.onread (net.js:597:20)

It seems like if I introduce a wait of some kind of the code by putting a break point it works, but if I don't have the break point the error will occur.

The website I am getting this on is:

https://www.lexhelper.com

And the method in typescript is:

async function get_site_data(scanReport: ScanReport, url: string) { let result = await unfurl(url); log.info("Got unfurl data " + result); scanReport.unfurlData = JSON.stringify(result); await scanReport.save() return scanReport; };

It dies on the very first line and I was using 1.18.beta2 but it still happens with beta3 as well.

Thanks!

jacktuck commented 6 years ago

Good catch!

I think i worked it out. I've gone a head and published 1.1.8-beta.4. Also I'm working on 2.0 at the moment which will be much better in terms of schema.

Let me know if this works for you - i'll leave this open for now.

shaunc869 commented 6 years ago

Looks like it worked, awesome job man!

jacktuck commented 6 years ago

Good to hear! :)

github-actions[bot] commented 3 years ago

:tada: This issue has been resolved in version 1.0.0 :tada:

The release is available on:

Your semantic-release bot :package::rocket: