ikreymer
commented
1 year ago Yeah, one of the main reasons for this it to have deterministic output when downloading the WACZ, so that the same file is produced on subsequent downloads. Currently, ArchiveWeb.page should produce the same exact byte-for-byte WACZ, if nothing has changed (of course, if version is updated, it will change). This also becomes important when generating WACZ files to put on IPFS. I suppose the name could be set to the initial creation time, which is also available. data-<collection-creation-date-timestamp>.warc as a possible option.. will check
I was looking at WACZ files generated from ReplayWeb.page and noticed that the WARC file under the
archive/folder is always nameddata.warc.gz(ordata.warc). It looks like the file name is hard-coded.https://github.com/webrecorder/awp-sw/blob/3e5bf49fe5d6e4028fb5e34483208ef7300a25f6/src/downloader.js#L78
Would it be practical to add datetime or some other pseudo-unique or unique strings (like a hash or GUID) to the name (e.g.,
data-YYYYMMDDhhmmss.warc.gz)? One issue I can see is that if the same data is downloaded multiple times, a new timestamp might be added, making the WACZ file bytes non-reproducible, unless we use some deterministic identifier (like the datetime of the first capture or the hash of the WARC file itself, which might require renaming the file after writing data).If people were to extract WARC files from more than one WACZ files and place them in a single folder, there is a chance of overwriting due to name collisions.