Open BurnzZ opened 2 years ago
I think that's a good idea, but probably it would make sense to wait a bit, when a real-world use case would pop up. Then we can think about how to help solving it.
For the stated issue, I wonder if an opt-in setting in scrapy-poet that enables logging a debug message indicating which page object is used for any given URL and requested output, and why, could do the trick.
Background
Following the acceptance of https://github.com/scrapinghub/web-poet/pull/27, developers could now use URL patterns to declare which Page Objects would work on specific URL patterns (reference code).
Problem
For large code bases, there might be hundreds of Page Objects which in turn could also result in hundreds of
OverrideRule
created using the@handle_urls
annotation.This could be unwieldy especially when they're spread out across multiple different subpackages and submodules within a Page Object Project. A project could utilize other Page Objects from other external packages, leading to a deeper roots.
Moreover, overlapping rules (e.g. POs improving on older POs) could add another layer of complexity. It should be immediately clear which PO would be executed according to URL pattern and priority.
Idea
There should be some sort of collection of utility functions that could interact with the
List[OverrideRule]
from the registry. Suppose that we have:We could then have something like:
This could help lead in creating test suites in projects that utilize other Page Object projects:
Other Notes:
rule_match.find()
is quite similar to how thePageObjectRegistry.search_override()
method behaves (_reference_).List[OverrideRule]
is not created by thedefault_registry
(or some custom registry). For example, it could merely be a simple configuration file containing all of theList[OverrideRule]
that is manually maintained.rule_match.find()
that is explored above aims to have an actual URL instead of a Pattern (whichPageObjectRegistry.search_overrides()
has)