Assumption:
Most interesting information exists at the nearest depth to the base url
Reasoning:
As you go deeper into the website, the information becomes more specialized to specific function such as the integration service #2 such that the information no longer applies to the description of the institution
set the level of how deep a crawler goes
Assumption: Most interesting information exists at the nearest depth to the base url
Reasoning: As you go deeper into the website, the information becomes more specialized to specific function such as the integration service #2 such that the information no longer applies to the description of the institution