The current archiving script crawls through all version of the duotang.html file, including the ones that are dev previews. This is causing the archive to explode in size causing deployment issues with github pages.
Short term mediation: I manually removed dev versions from the archive.
Long term mediation: Need to scan through all the PRs to identify dates in which dev > main merge happened and scan for only those htmls to archive.
The current archiving script crawls through all version of the duotang.html file, including the ones that are dev previews. This is causing the archive to explode in size causing deployment issues with github pages.
Short term mediation: I manually removed dev versions from the archive.
Long term mediation: Need to scan through all the PRs to identify dates in which dev > main merge happened and scan for only those htmls to archive.