When scanning a large site:
If the same url is on multiple pages:
The same url will be scanned for each page.
If there is a way to remember that the url has already been scanned so it can skip the url when it is found an 1+n time, this would save a lot of time.
When scanning a large site: If the same url is on multiple pages: The same url will be scanned for each page.
If there is a way to remember that the url has already been scanned so it can skip the url when it is found an 1+n time, this would save a lot of time.