gopro sync fails

[antonsn]

command gopro sync fails with Error below . At the same time gopro upload works fine.

I: 6028 new items
I: Storing batch of 100
gopro: HttpExceptionRequest Request {
  host                 = "images-02.gopro.com"
  port                 = 443
  secure               = True
  requestHeaders       = [("Content-Type","application/json"),("Accept","application/vnd.gopro.jk.media+json; version=2.0.0"),("Authorization","<REDACTED>"),("User-Agent","github.com/dustin/gopro 0.1")]
  path                 = "/resize/450wwp/qweyJhbGciOiJIUzI1NiJ9.ehgggyJtZWRpdW1fasssWQiOiIxOTU1Mzc1MTc2ODIwNDU4NTM2Iiwib3duZXIiOiI1NDVlYzZiYS03YWJlLTQ0YTctOTZkNy1iMTFlYTQxN2Q0NTUiLCJpc19wdWJsaWMiOmZhbHNlLCJvIjoxLCJ0cmFucyI6bnVsbCwicmVnaW9uIjoidXMtd2VzdC0yIiwidGh1bWJuYWlsX3VwZGF0ZWRfZGF0ZSI6bnVsbH0.3k-ro9BJl5OK-Z1duI4xLT6qCMd6KbUWuwuBAMEDRm4"
  queryString          = ""
  method               = "GET"
  proxy                = Nothing
  rawBody              = False
  redirectCount        = 10
  responseTimeout      = ResponseTimeoutDefault
  requestVersion       = HTTP/1.1
}
 (StatusCodeException (Response {responseStatus = Status {statusCode = 404, statusMessage = "Not Found"}, responseVersion = HTTP/1.1, responseHeaders = [("Content-Type","image/jpeg"),("Content-Length","272"),("Connection","keep-alive"),("Date","Wed, 22 Sep 2021 08:02:21 GMT"),("Server","nginx"),("Vary","Accept-Encoding, Origin"),("Access-Control-Allow-Credentials","true"),("Strict-Transport-Security","max-age=31536000; includeSubDomains"),("X-Cache","Error from cloudfront"),("Via","1.1 2f194b62c8c43859cbf5af8e53a8d2a7.cloudfront.net (CloudFront)"),("X-Amz-Cf-Pop","FRA2-C2"),("X-Amz-Cf-Id","Um2L9ajXeqJdT88GlVNQnGzyQQ7TfOdSpwqJT61ZHQKX3uKB-xuO0g==")], responseBody = (), responseCookieJar = CJ {expose = []}, responseClose' = ResponseClose}) "\255\216\255\224\NUL\DLEJFIF\NUL\SOH\SOH\NUL\NULH\NULH\NUL\NUL\255\219\NUL\132\NUL\SOH\SOH\SOH\SOH\SOH\SOH\STX\SOH\SOH\STX\ETX\STX\STX\STX\ETX\EOT\ETX\ETX\ETX\ETX\EOT\ACK\EOT\EOT\EOT\EOT\EOT\ACK\a\ACK\ACK\ACK\ACK\ACK\ACK\a\a\a\a\a\a\a\a\b\b\b\b\b\b\t\t\t\t\t\v\v\v\v\v\v\v\v\v\v\SOH\STX\STX\STX\ETX\ETX\ETX\ENQ\ETX\ETX\ENQ\v\b\ACK\b\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\v\255\192\NUL\DC1\b\NUL\n\NUL\n\ETX\SOH\"\NUL\STX\DC1\SOH\ETX\DC1\SOH\255\196\NULL\NUL\SOH\SOH\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\a\DLE\SOH\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\SOH\SOH\SOH\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\b\t\DC1\SOH\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\255\218\NUL\f\ETX\SOH\NUL\STX\DC1\ETX\DC1\NUL?\NUL\155\128v%[\255\217")
root@005ceca2fddf:/mnt/Scans# gopro sync
I: 6028 new items
I: Storing batch of 100

[dustin]

Huh. That's interesting. We have a thumbnail reference, but they're not serving a thumbnail. I assume you have recent auth?

I can try all thumbnail servers and then fail with a default, but I don't have a solid plan for what to do if they give us media with no thumbnail. I'm curious if the other artifacts are available.

The actionable things I can do here are:

• [ ] Try all the image servers.
• [x] Mark broken items
• [x] Have a "default" thumbnail (or just use the junk they send in their 404)
• [ ] Provide a bit more

Skipping suggests just ignoring them and not marking them in the DB at all so they'd show up on the next full sync. However, in practice it'd mean there'd be media that exists on their end that you may not be getting.

By marking them, I can have a means of doing extra checks when something doesn't seem right. e.g., has this medium been deleted or something? What kind of broken state are we in?

Thanks for the report. It's not something I've seen, but it's apparently something that can happen that seems inconsistent on the cloud end. I'll hopefully have some time later today to handle this case so we can get the rest of your media to sync and get a useful report to GoPro to fix this service issue.

[dustin]

Apparently I got distracted writing that list this morning. heh.

I think the change I just made should improve things, but I don't have any such broken things.

I didn't bother trying all the image servers, but if we can't get an image, we just store null along with the rest of the metadata and serve that on error.

I did test this by modifying fetchThumbail to produce an invalid URL one some of my media and got a sync to complete and the "missing thumbnail" thumbnail to show up. You can find the broken media in the DB doing something like:

sqlite3 gopro.db 'select media_id from media where thumbnail is null'