AddQuerystringParam with cyrillic value characters

[delros]

Hi everyone,

Obviously, the root cause most likely is not the VBA-tools side, however, I would like to double-check with the community if anyone experienced the same case and already has a solution.

The issue occurred when the value from the cell was read to a variable which was then passed to AddQuerystringParam. As a result, in debug in always shown as "find?q=%3F%3F%3F%3F%3F%3F%3F%3F". Alternatively, I tried to directly add this param to the URL string to avoid "AddQuerystringParam", and in such case, the method "GetFullUrl" returned a correctly looking value, but API still received the request like "/find?q=????????"

Could anyone suggest a possible approach to resolve this problem or advise the direction for future investigation?

Thanks for the help, Ernesto

[RichardWein]

H Ernesto.

I've just taken a look at the VBA-Web code, and it seems it doesn't handle non-ASCII characters in URLs. I'm a little surprised that I've never noticed this before.

If you look at the function UrlEncode in module WebHelpers, you can see this comment:

            ' TODO For non-ASCII characters,
            '
            ' FormUrlEncoded:
            '
            ' Replace the character by a string consisting of a U+0026 AMPERSAND character (&), a "#" (U+0023) character,
            ' one or more ASCII digits representing the Unicode code point of the character in base ten, and finally a ";" (U+003B) character.
            '
            ' Else:
            '
            ' Encode to sequence of 2 or 3 bytes in UTF-8, then percent encode
            ' Reference Implementation: https://www.w3.org/International/URLUTF8Encoder.java

I'm now thinking of adding this code myself!

[delros]

Oh wow... thanks for the reply. Looks like I would need to update from your side if that might be possible since I'm not an expert in vbs at all.

[RichardWein]

Hi. I'm working on it. It turned out to be more complicated than I first thought (as usual!). I've got it working for encoding more common character sets, like Cyrillic. But I'd like to make it handle all Unicode characters. And I'd like to make it decode URLs as well as encoding them.

[delros]

@RichardWein, if, or once, you have some draft or "work in progress" branch - I can try to help with validation or sort of testing, at least with Cyrillic characters. And yes, thanks for help

[RichardWein]

@delros Hi Ernesto. I'm attaching a file with my code so far. It contains a replacement for the UrlEncode function in WebHelpers, and 2 new functions, called DecodeUTF8 and EncodeUTF8, which you can put in the same module or another one. DecodeUTF8 is not used at present, but it may be in the future, if I upgrade UrlDecode as well.

This version should handle all Unicode characters. It doesn't handle "FormUrlEncoding". I don't even know what that would be used for! But I will look into it when I have time. I also don't know when you would want to use URLDecode, but I'll probably look into that too.

New UrlEncode (v1).txt

[delros]

It works like a magic, thanks a lot @RichardWein

[mwoollen]

There's an interesting discussion about UNICODE characters in URL/URIs... https://stackoverflow.com/questions/2742852/unicode-characters-in-urls.