search

VBA-tools / VBA-Web

VBA-Web: Connect VBA, Excel, Access, and Office for Windows and Mac to web services and the web
http://vba-tools.github.io/VBA-Web/
MIT License
2.01k stars 494 forks source link

Needs conversion to Unicode before sending to server #157

Open mauriciojxs opened 9 years ago

mauriciojxs commented 9 years ago

Hi Tim,

I'm having trouble to send chars with accentuation to the TODOist API. This chars are part of the "task description", but the server doesn't recognize the command when I do so. Here is my logging:

--> Request - 15:35:57
GET https://todoist.com/API/v6/sync?token=MY_TOKEN_HERE&seq_no=0&seq_no_global=0&commands=[{"type": "item_add", "temp_id": "C8B1DA6F-7D54-41B4-8C69-686A9BF71FB6", "uuid": "C8B1DA6F-7D54-41B4-8C69-686A9BF71FB6", "args": {"content": "HO%20enviará%20reporte%20de%20rendimientos%20comparando%20primeros%20lotes%20(2012)%20con%20los%20últimos%20como%20PIR.", "project_id": 148429714, "date_string": "2015-09-20", "labels": [145450]}}]
User-Agent: VBA-Web v4.0.15 (https://github.com/VBA-tools/VBA-Web)
Content-Type: application/json
Accept: application/json
Content-Length: 0

ERROR - WebHelpers.ParseByFormat: 11000, An error occurred during parsing
10001: Error parsing JSON:
<html><body
^
Expecting '{' or '['
ERROR - WebHelpers.ParseByFormat: 11000, An error occurred during parsing
10001: Error parsing JSON:
<html><body
^
Expecting '{' or '['
<-- Response - 15:35:57
400 Bad request
Cache-Control: no-cache
Connection: keep-alive
Transfer-Encoding: chunked
Content-Type: text/html

<html><body><h1>400 Bad request</h1>
Your browser sent an invalid request.
</body></html>

If I create the task in TODOist website, and get it from VBA-WEB, I could tell they use Unicode (á becomes \u00E1). I tried encoding my string to unicode before adding to the url, with StrConv, but no sucess.

Is there any feature on VBA-WEB that would help me with that, or any suggestion from your side?

timhall commented 9 years ago

Thanks for raising this issue. I think json_Encode in WebHelpers could help with this encoding. It is currently a private method, but it's come up before so I may make it public in the future to help with cases like this. In the meantime, I would recommend making json_Encode public in your Workbook and then use that on your parameters before adding them to the url.

mauriciojxs commented 9 years ago

Great, that option worked!

Thanks.

timhall commented 9 years ago

I've been reading more about this and it looks something like json_Encode needs to be the default, I'll fix this soon. http://stackoverflow.com/questions/912811/what-is-the-proper-way-to-url-encode-unicode-characters

mauriciojxs commented 9 years ago

Tim, take a look at the reply from Todoist support staff. Apparently, they would accept chars with accents, don't know what could have gone wrong the first time (without json_encode) https://todoist.com/Support/show/98060/ Also, I copied the Resource string and pasted in Chrome, and it work as expected.

mohawk2 commented 4 years ago

Just FYI and/or for the record, to put bytes in a URL needs them to be URL-safe, achieved with URL-encoding. If the data is Unicode characters (which are different from bytes), it will first need converting into bytes. Probably the most widely-used way to do that is with UTF-8 encoding, though Todoist would need to confirm that's what they expect. Note Windows libraries tend to use UTF-16.

I assume that years later this problem is solved. If so, I suggest closing this issue. If not, say so and I will be pleased to make a PR to fix! (presumably on VBA-JSON)