TumblThreeApp / TumblThree

A Tumblr and Twitter Blog Backup Application
https://TumblThreeApp.github.io
MIT License
612 stars 75 forks source link

tumblr sometimes returns an empty json array for Attribution rather than an object #579

Open mjharwood opened 4 days ago

mjharwood commented 4 days ago

Describe the bug

Unhandled exception parsing the json object returned from tumblr.

instead of:

 "content": [
    {
       "attribution": {
       "type": "link",
       "url": "https://64.media.tumblr.com/....."
        },
   },

it returns:

content": [
  {
     "attribution": [
      ],

To Reproduce seems to be on occasional posts, no particular pattern to trigger it

Expected behavior post is successfully downloaded

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

Additional context

This, and many other fields, are not used by the application so could just be removed / disabled in TumblrTaggedSearchJson : Content to prevent this issue. The json parser just skips fields that aren't mapped.

I can produce a PR if that would be helpful.

thomas694 commented 4 days ago

Hello, I would not remove the fields so that the complete data object can still be saved, but change it so that either an object or an array can be processed in the respective field. Yes, you can produce a PR.

mjharwood commented 4 days ago

Is there some debugging that saves the complete data object? Otherwise it's less overhead to just drop the unwanted fields.

Unless there's some json.net magic I'm not aware of, mapping an array and a typed object to the same field in C# is going to be very difficult and not worth it for data the application doesn't need.

I might be missing something obvious, it's a while since I used json.net for parsing.

thomas694 commented 3 days ago

There's an option "dump crawler data" to store the complete JSON data.

I rather think that there has already been done something similar for a property of one of the crawlers before, with a custom JSON converter I guess.