search

freaktechnik / twitch-chatlog

Fetch the chatlog to a twitch VOD from your command line.
https://www.npmjs.com/package/twitch-chatlog
MIT License
83 stars 15 forks source link

JSON.stringify() range error - invalid string length #157

Closed josh-channin closed 3 years ago

josh-channin commented 3 years ago

When attempting to fetch a chatlog for a VOD with over ~600k-700k+ chats, e.g. a 24 hour XQC VOD (888381698), the following error is thrown. 

RangeError: Invalid string length
    at JSON.stringify (<anonymous>)
    at C:\....\node_modules\twitch-chatlog\bin\twitch-chatlog:90:21

This appears to be the max string size or max possible string size that can be "stringified"

freaktechnik commented 3 years ago

Thank you for the report. A workaround for now would be to call the CLI with smaller time ranges and then assemble in your consumer.

freaktechnik commented 3 years ago

https://www.npmjs.com/package/bfj seems like a package that would fit the needs quite well. Will have to experiment with its performance and if I can tweak it, since blocking the event loop isn't a concern.

josh-channin commented 3 years ago

Awesome news, thanks!

freaktechnik commented 3 years ago

This is now implemented in the main branch. However, there's no release yet, since I also need to update the tests. I'll publish a release once that's done. If you feel adventurous (it should be fine, but I haven't checked if I can load the entire VOD for the example you posted, then I wouldn't even have an implementation yet ;) ) you can of course use the state in the main branch for now.

freaktechnik commented 3 years ago

And there you are, now available from NPM.