Error: Failed to delete old mastodon favs

[bajtos]

I have finally went ahead and configured a cron job to regularly sync my twitter and mastodon posts. I have also enabled removal of old post via delete_older_statuses and delete_older_favs.

It looks like re-posting works well both ways.

However, after the initial pruning of all my Twitter and Mastodon history, subsequent runs of the delete algorithm are failing with the following error:

Error: Failed to delete old mastodon favs
Because: Serde(Error("data did not match any variant of untagged enum EmptyOrMeta", line: 1, 
column: 32780))

What's the recommended way for debugging this issue to find the root cause?

[klausi]

Maybe you ran into Mastodon API rate limiting by deleting too many favs?

Debugging is a bit tricky in this case because you need go into the Mammut library to look at the actual Mastodon API response.

1. Use Mammut git checkout in Cargo.toml https://doc.rust-lang.org/cargo/reference/overriding-dependencies.html#testing-a-bugfix
2. Add a dbg!() statement in Mammut right before Serde deserialization runs
3. check debug output what the Mastodon API returned.

[bajtos]

Thank you @klausi for the pointers, I'll take a look. Hopefully later this week.

[klausi]

in https://github.com/XAMPPRocky/Mammut/issues/47 XAMPPRocky said that he added request logging to Mammut, so you also might be able to get more output by enabling some log environment variable or similar.

[bajtos]

Thank you for the pointer!

Here is the relevant commit adding debug logs: https://github.com/XAMPPRocky/Mammut/commit/f8dfca50432a4cfe498e52a182a97690f1af29c0

IIUC, Mammut is using the crate log, which does not print any logs by default. To enable debug logging in mastodon-twitter-sync, we need to initialize a logger - see In executables. The crate env_logger seems to work nicely, I'll open a PR to set it up.

Here is the output:

$ RUST_LOG=mammut=debug cargo run -- --skip-existing-posts`:
[2020-05-06T16:03:29Z DEBUG mammut] REQUEST: RequestBuilder { client: Client, request: Ok(Request { method: GET, url: "https://mastodon.social/api/v1/accounts/verify_credentials", headers: {"authorization": "Bearer <redacted>"} }) }
[2020-05-06T16:03:29Z DEBUG mammut] RESPONSE: Response { url: "https://mastodon.social/api/v1/accounts/verify_credentials", status: 200, headers: {"date": "Wed, 06 May 2020 16:03:27 GMT", "content-type": "application/json; charset=utf-8", "transfer-encoding": "chunked", "connection": "keep-alive", "server": "Mastodon", "x-frame-options": "DENY", "x-content-type-options": "nosniff", "x-xss-protection": "1; mode=block", "x-ratelimit-limit": "300", "x-ratelimit-remaining": "286", "x-ratelimit-reset": "2020-05-06T16:05:00.057517Z", "cache-control": "no-cache, no-store", "set-cookie": "_mkra_stck=<redacted>; path=/; max-age=10; expires=Wed, 06 May 2020 16:03:37 GMT; HttpOnly", "vary": "Accept-Encoding, Origin", "strict-transport-security": "max-age=31536000", "etag": "W/\"d914a9dcdd9f62aca242c89119d27b48\"", "x-request-id": "<redacted>", "x-runtime": "0.051615", "x-cached": "MISS"} }
Error: Failed to delete old mastodon favs
Because: Serde(Error("data did not match any variant of untagged enum EmptyOrMeta", line: 1, column: 38450))
error: process didn't exit successfully: `target\debug\mastodon-twitter-sync.exe --skip-existing-posts` (exit code: 1)

Not very useful, perhaps because the response uses Transfer-Encoding: chunked. I'll have to dig deeper.

[bajtos]

I'll open a PR to set it up.

Done, see https://github.com/klausi/mastodon-twitter-sync/pull/22