Iframes are sometimes used to have parts of sites controlled by a CMS.
Here we add the option of inspecting the iframe's content and for any links that are to the site being indexed, having those included. It takes into account the tag, since if the base tag matches the site being indexed, then all relative URLs should be crawled.
If you want to enable this option, you can just add the flag:
Iframes are sometimes used to have parts of sites controlled by a CMS.
Here we add the option of inspecting the iframe's content and for any links that are to the site being indexed, having those included. It takes into account the tag, since if the base tag matches the site being indexed, then all relative URLs should be crawled.
If you want to enable this option, you can just add the flag:
Resolves #90