handrews
commented
1 year ago Looking more closely at rfc3987, the function I tried defaults to non-validating parsing, but it's trivial to enable validation, which seems to be appropriately strict. It seems like the best candidate for validation, but it does not offer any sort of convenience class packaging up relevant functionality.
URI and IRI support in python is a dumpster fire of sadness. It is "good enough" for most day-to-day purposes, but for a project specifically focusing on correctness and compliance, that's not really good enough. Considering some of the more commonly used libraries:
urllibdoesn't even pretend to be compliant, and automatically encodes things that should be errors.urllib3also auto-encodes instead of detecting errorsrfc3986comes a lot closer to compliance, but still does undesirable auto-encoding, although in some circumstance you can work around that with more elaborate use of their validation classrfc39862.0.0 also emits deprecation warnings when you use certain non-deprecated functions. I have contributed a fix for this, but it's unclear when they will get around to publishing a new release.For IRIs, all of the above means that non-URI unicode characters are encoded, rendering the IRI unreadable as an IRI. It's not entirely clear if there's any need for true IRI support for OAS 3.x compliance, but presumably it will be a concern for 4.0. The options for supporting unencoded IRIs is very sketchy:
rfc3986.IRIReferenceis experimental, and incomprehensibly does the same encoding asrfc3986.URIReferencerfc3987is a completely separate package fromrfc3986; it handles unicode correctly but is otherwise very minimal and poorly documented, and is still too permissive about some thingsNone of this even gets into scheme-specific parsing, or media-type-specific fragment parsing.
To the best of my knowledge, the only code that handles everything correctly is the
abnfpackage, which includes ABNF support for RFC 3986 and 3987, but produces a huge parse tree as a result and is probably significantly slower (although I haven't measured it). Its error reporting is also incomprehensible.To add further confusion:
rdflilbwantsrdflib.URIRefinstances (which are actually IRIs and not URI-references, for but calledURIReffor historical reasons), which subclass fromstr.jschonhas its ownjschon.URIclass which is a wrapper aroundrfc3986.URIReferenceSo at minimum there are two URI-ish classes floating around the code, one of which wraps a third, none of which are the standard library, and all of which have different interfaces.
┻━┻︵ \(`Д´)/ ︵ ┻━┻
Clearly the only answer is to write another library. More seriously, wrap this dumpster fire in a facade so that we can be less fiddly but still change implementations until finding the right correctness/convenience/performance balance. We are unlikely to know what constitutes "reasonable" performance until the system is near-complete and can process enormous specs like the GitHub API. A facade will make it feasible to quickly measure different alternative implementations.
Requirements:
application/x-www-form-urlencoded-style stringsjschon.JSONPointerclass, although there are at least three other JSON Pointer packages as alternatives; however we also need Relative JSON Pointer support which is more rare, but well-supported byjschon.RelativeJSONPointer, to which I contributed fixes injschon0.10.0)http(s),file,urnespeciallyurn:uuid, and possiblytagandmailto- this can be deferred and added as the use cases arise, but it needs to be planned)application/schema+json, andapplication/yaml; JSON Schema plain name fragments, possibly other things to support External Documentation links?)rdflib.URIRefandjschon.URI, as well as plain strings