without this changes, grance once is not available, especially for errors. example:
Website A has a link to B. The Website crashes and the SpoiledReferenceStrategy is GRACE_ONCE.
200 ok crawl: A and B are committed
First error crawl, the cache is filled with A and B: A has an error and B has an error.
Second error crawl: without the change, the cache is empty, but:
A is a Start-URL, so it is crawled and deleted, because of some bug-fix on AbstractCrawler line 694
B is not in the cache, but was committed, is never deleted and is unknown forever
With this fix, the AbstractCrawler line 694 is not nessessary anymore, but its also not wrong.
without this changes, grance once is not available, especially for errors. example: Website A has a link to B. The Website crashes and the SpoiledReferenceStrategy is GRACE_ONCE. 200 ok crawl: A and B are committed First error crawl, the cache is filled with A and B: A has an error and B has an error. Second error crawl: without the change, the cache is empty, but:
With this fix, the AbstractCrawler line 694 is not nessessary anymore, but its also not wrong.