Import error

[lucas-gaitzsch]

After Wakatime import I get following logs:

[ERROR] failed to fetch heartbeats for day '2020-11-20 00:00:00 +0000 UTC' and user 'xxxxx' – &v%!(EXTRA *json.SyntaxError=invalid character '<' looking for beginning of value)

I use wakapi in a docker container with MySQL (Maria DB) and behind an nginx reverse proxy.

[muety]

Thanks for bringing this up! I'll investigate...

Did the import work eventually or did it fail entirely?

[lucas-gaitzsch]

Yes, some days has data but the days from the log errors has no data.

[muety]

I'm trying to debug this right now. Can you please do me a favor? Please perform the above request manually and post the response here. Use Postman, cURL, etc. to make the following request:

GET https://wakatime.com/api/v1/users/current/heartbeats?date=2020-11-20

with header:

Authorization: Basic your-base64-encoded-api-token-here

[lucas-gaitzsch]

I will send you the response via mail

[muety]

Thanks! The data you sent me looks fine, especially it's valid JSON. The above error message suggests that HTML might have been returned instead (because of the <). At first sight, three possible causes for the issue come to my mind:

1. A temporary WakaTime outage -> WakaTime might have returned a 500 HTML error page
2. You ran into a WakaTime request limit -> WakaTime might have returned a 429 HTML error page

Initially I thought that the request might also have timed out (> 10 seconds), but that's not an option as otherwise the error would be a different one.

Could you please try the following?

1. Manually connect to your database and bring up the key_string_value table
2. Find the entry with key = last_import_<your-username>
3. Set its value to 19 Nov 20 00:00 CET (one day before the day that failed above)
4. Try running the import again and see if the error still occurs for that certain day

I know it's quite some effort, so please only do it if you like to!

Frankly, I'm quite sure that the problem is on WakaTime's side, so we might not actually be able to do a lot here.

[lucas-gaitzsch]

I get this output:

[WARN ] did not insert key 'last_import_lucas', maybe just updated?
[INFO ] downloaded 342 heartbeats for user 'lucas' (0 actually imported)
[INFO ] clearing summaries for user 'lucas'
[INFO ] generating summaries

And following was logged for every day:

...
[INFO ] successfully generated summary (2021-03-20 00:00:00 +0000 UTC, 2021-03-21 00:00:00 +0000 UTC, lucas)
...

But the data dont appear in the database and the dashboard.

[muety]

Alright, thanks! Maybe it helps to entirely revert the import and then run it once again?

[lucas-gaitzsch]

I get the same errors with different dates. Your guess seems plausible.

• A temporary WakaTime outage -> WakaTime might have returned a 500 HTML error page
• You ran into a WakaTime request limit -> WakaTime might have returned a 429 HTML error page

[muety]

Turns out Wakatime has a rate limit of 10 req / second on average over 5 minutes. I'll throttle the import. Stay tuned for the next release.

[lucas-gaitzsch]

thank you!