DECODE-URL and url! syntax don't obey the url encoding rules

[rebolbot]

Submitted by: BrianH

Hex-encoded characters in URLs are supposed to stay encoded until the URL is broken into its component parts. This does not happen correctly with REBOL URLs: They are decoded too early. This causes a problem when one or more of the component parts have characters in them that would be mistaken for structural characters. This happens pretty often nowadays with many sites that use a full email address as a user name, or that allow passwords with less restricted character sets.

This can probably be solved by fixing DECODE-URL. Note: This problem also exists in R2, and could use a similar solution.

>> http://user%40rebol.com:blah@www.rebol.com/
== http://user@rebol.com:blah@www.rebol.com/
; should be http://user%40rebol.com:blah@www.rebol.com/
>> decode-url http://user%40rebol.com:blah@www.rebol.com/
== [scheme: 'http user: "user" host: "rebol.com" path: ":blah@www.rebol.com/"]
; should be [scheme: 'http pass: "blah" user: "user@rebol.com" host: "www.rebol.com" path: "/"]

^{CC - Data [ Version: alpha 97 Type: Bug Platform: All Category: Mezzanine Reproduce: Always Fixed-in:none ]}

[rebolbot]

Submitted by: BrianH

Gabriele wrote a correct parser here: http://www.rebol.it/power-mezz/parsers/uri-parser.html

Perhaps we can adapt that to R3 style, or at least analyze the code to get the correct rules to follow. We can't use it directly because it decodes the URLs too far, decoding the path and fragment identifier when we don't want that in this case. But it shouldn't be too hard to adapt, as long as it doesn't use too much of the rest of the Power Mezz stuff.

[rebolbot]

Submitted by: meijeru

The following tickets also apply: #482, #1327, #1333.

[rebolbot]

Submitted by: Carl

This does not seem like hex decoding problem, but more like a field decoding problem in DECODE-URL. For example, in the example, I could have used the @ directly, without the %40.

[rebolbot]

Submitted by: BrianH

But what if the hex was %2F or some other character? All hex characters need to be allowed when specified in hex form.

[rebolbot]

Submitted by: Ladislav

"This can probably be solved by fixing DECODE-URL." - this provably can't be solved by "fixing" DECODE-URL in any way.

[rebolbot]

Submitted by: BrianH

Agreed, the internal treatment of url! values by LOAD and the other url! manipulation functions would need to be changed first, because url! values are getting corrupted long before DECODE-URL ever sees them. Once those other functions are fixed, and possibly a new internal model for the url! type is chosen (see #2014), then DECODE-URL can be fixed to work on the new url! data model.