derekantrican
commented
4 years ago Ideas:
- Hashing the HTML and if it has not changed, skipping all of the
IHtmlDocument/parsing/etc lines (note that we will probably still need to get theIHtmlDocumentfor subareas/routes so that we can check those as well)- Need to check speed of hashing algorithms and see if there would actually be any time savings here to "hash HTML and check hash against previous" or "just load the
IHtmlDocumentand do all the parsing anyway"
- Need to check speed of hashing algorithms and see if there would actually be any time savings here to "hash HTML and check hash against previous" or "just load the
- Maybe there's a faster way to download HTML from a URL? Possibly: https://stackoverflow.com/a/54502464/2246411 (may need to investigate a couple different options)
- See what happens if I take off the
awaits beforeParseRouteAsyncandParseAreaAsync(since I don't actually need to wait before those things complete. Last time I tried this I think this meant that theTask.WaitAllactually finished too quickly because all the DestAreas finished before any of the children
See if I can take some hints from buildxl to only rebuild things that have actually changed.
For instance: if a route has not changed, we may be able to avoid all the HTML parsing and instead skip it