It is unclear whether *Extra pages* are recursively scanned for more pages

[hroncok]

Reading the README, it is not clear to me whether additional pages only reachable from Extra pages will be recursively scanned for more pages. My guts tell me they should be, but the wording does not make that super obvious.

[encukou]

They will. If you don't want that:

• for files that aren't part of the app (like deployment config), you want extra_files.
• for pages where links should be ignored (like possibly the pyladies.cz legacy area), add the Freezeyt-URL-Finder: None HTTP header (main branch only, let us know if you want a PyPI release).

The README should be updated.