Parsing / Indexing Fix for WARC record boundaries

webrecorder / warcio.js

JS Streaming WARC IO optimized for Browser and Node

MIT License

30 stars 6 forks source link

Closed ikreymer closed 1 year ago

ikreymer commented 1 year ago

Simplify readToNextRecord()

Fix indexing issues where incorrect offset is stored if new gzip member is started
Fix indexing of WARCs that don't have spaces after record result in skipped first line
Remove Buffer and just use btoa() to avoid extra dependency in browsers.