Open Veilkrand opened 3 years ago
I have a website with millions of categorized records, it will be useful if I could limit the number of urls to parse per section. E.g. the 900,000 first urls under /products/toys/ section but not from a higher category.
/products/toys/
I have a website with millions of categorized records, it will be useful if I could limit the number of urls to parse per section. E.g. the 900,000 first urls under
/products/toys/section but not from a higher category.