Handling of failed downloads (posters, banners, (maybe) fanart...)

GoogleCodeExporter commented 9 years ago

What steps will reproduce the problem?
1. Start import
2. Some banners or posters are found online
3. Download of poster or banner fails or something else (i can't really
say, happened to me on Family Guy banners)
4. You still get DB entry for that image
5. In plug-in configuration you can select that image in the combo box for
the series but there is an error loading image because it doesn't exist,
download failed..

What is the expected output? What do you see instead?
If download fails i expect not to add it in the DB.

What version of the product are you using? On what operating system?
Using the latest SVN of MP with 2.2.3361 of MP-TVSeries.dll

Original issue reported on code.google.com by fred.doo...@gmail.com on 18 Mar 2009 at 8:44

Merged into: #263

GoogleCodeExporter commented 9 years ago

Yeah, good idea. I noticed the same issue.

Its add it to the DB so it can download next time if its not found locally. I 
think a
check should be in place though which removes it from the list of viewable 
banners.

Original comment by damien.haynes@gmail.com on 18 Mar 2009 at 9:34

Changed state: Accepted

GoogleCodeExporter commented 9 years ago

i had it few banners like this in db with no local file for quite a few but it 
never
showed any logs of trying to redownload it.. or this check is yet to be added?

Original comment by fred.doo...@gmail.com on 18 Mar 2009 at 9:51

GoogleCodeExporter commented 9 years ago

It attempts to download every scan when you see Refresh\Download Artwork...I 
guess
the question is are they still missing from your computer?

Original comment by damien.haynes@gmail.com on 18 Mar 2009 at 10:20

GoogleCodeExporter commented 9 years ago

files aren't there.. i double, tripple checked..

snippet from the log on mediaportal start..

00000019 - 18.03.2009 11:25:16 - ***************     Checking for missing 
artwork   
 ***************
00000019 - 18.03.2009 11:25:16 - Looking for artwork on 4 Series
00000019 - 18.03.2009 11:25:16 - Downloading artwork for "Everybody Hates Chris"
00000019 - 18.03.2009 11:25:18 - Downloading artwork for "Fifth Gear"
00000019 - 18.03.2009 11:25:19 - Downloading artwork for "The Real Hustle"
00000019 - 18.03.2009 11:25:19 - Downloading artwork for "Top Gear Australia"

family guy isn't even mentioned

00000001 - 18.03.2009 11:28:32 - ImageLoadFast threw an error
00000001 - 18.03.2009 11:28:32 - ImageLoading threw an error: C:\Documents and
Settings\All Users\Application Data\Team
MediaPortal\MediaPortal\Thumbs\MPTVSeriesBanners\Family
Guy\-langen-graphical/348-g2.jpg - C:\Documents and Settings\All 
Users\Application
Data\Team MediaPortal\MediaPortal\Thumbs\MPTVSeriesBanners\Family
Guy\-langen-graphical/348-g2.jpg
00000001 - 18.03.2009 11:28:33 - ImageLoadFast threw an error
00000001 - 18.03.2009 11:28:33 - ImageLoading threw an error: C:\Documents and
Settings\All Users\Application Data\Team
MediaPortal\MediaPortal\Thumbs\MPTVSeriesBanners\Family
Guy\-langen-graphical/75978-g.jpg - C:\Documents and Settings\All 
Users\Application
Data\Team MediaPortal\MediaPortal\Thumbs\MPTVSeriesBanners\Family
Guy\-langen-graphical/75978-g.jpg

Original comment by fred.doo...@gmail.com on 18 Mar 2009 at 10:29

GoogleCodeExporter commented 9 years ago

All series should be mentioned there so that's strange! Do you have 5 series or 
more,
can you should be a full debug log after a scan?

Original comment by damien.haynes@gmail.com on 18 Mar 2009 at 10:33

GoogleCodeExporter commented 9 years ago

All series should be mentioned there so that's strange! Do you have 5 series or 
more,
can you send me a full debug log after a scan?

You may need to delete Family Guy from configuration and try re-import

Original comment by damien.haynes@gmail.com on 18 Mar 2009 at 10:36

GoogleCodeExporter commented 9 years ago

i have 18 series right now.. here is debug+sql log:

http://pastebin.com/m2fa7dd71

Original comment by fred.doo...@gmail.com on 18 Mar 2009 at 10:39

GoogleCodeExporter commented 9 years ago

Thanks, but the SQL logs are really hard to go through unless you need it. Can 
you
just do 'Debug'

Original comment by damien.haynes@gmail.com on 18 Mar 2009 at 10:44

GoogleCodeExporter commented 9 years ago

sure thing.. here is just Debug one

http://pastebin.com/m3264596a

Original comment by fred.doo...@gmail.com on 18 Mar 2009 at 10:52

GoogleCodeExporter commented 9 years ago

this issue i'm having is also related to issue 151
http://code.google.com/p/mptvseries/issues/detail?id=151

for series fifth gear and the real hustle no posters were downloaded before.. 
now i
added a poster for both on thetvdb but no mather what i do (force or manual 
online
scan) the posters don't get downloaded.. 

something is fishy in local vs online artwork check.. 
1. not all series is checked on "Checking for new artwork" or "Checking for 
missing
artwork"
2. even if the series is checked no new stuff is downloaded

Original comment by fred.doo...@gmail.com on 18 Mar 2009 at 12:26

GoogleCodeExporter commented 9 years ago

Ok, there is some problems online again that's why the banners are not 
downloading.
Although I need to put some extra checking in so it continue to download 
banners for
other series.

Original comment by damien.haynes@gmail.com on 19 Mar 2009 at 3:41

GoogleCodeExporter commented 9 years ago

Ignore what I said!

'Checking for new artwork' should only occur when doing a update scan...it 
should do
this for all your series (I think). Were you running a import from 
configuration,
because that will do an automatic update scan as well.

'Checking for missing artwork' should only download artwork for series that 
contain
no artwork for either Posters and Banners. So this should be very rare

Now I have discovered that the logic that handles both these cases is a bit 
off, its
not returning the correct number of series in both cases....

I will get back to you soon.

Original comment by damien.haynes@gmail.com on 19 Mar 2009 at 4:08

GoogleCodeExporter commented 9 years ago

Small correction on 'Checking for new artwork'. This does not do it on all 
series, it
use to work that way but now it only does it on what the online database tell 
us has
changed by requesting a updates zip.

Original comment by damien.haynes@gmail.com on 19 Mar 2009 at 4:25

GoogleCodeExporter commented 9 years ago

the logs were created by starting the mediaportal and collecting the logs 
afterwards..

later on i cleared the logs, then started the update from the plugin itself
(menu->force local scan).. but i havent sent you that one, just commented on the
results in my last reply (10)..

i'm waiting for the fixes.. will not touch my db before we can test it..

Original comment by fred.doo...@gmail.com on 19 Mar 2009 at 7:39

GoogleCodeExporter commented 9 years ago

As far as I can tell 'Checking for new artwork' is doing exactly as it should. 
It
gets a list of series that have updated since your last update and attempt to
download banners for them.

If for some reason there was a problem online and you did a update, its 
possible you
will never get any new banners for a particular series (at least until that 
series
online gets updated again)...the only option would be to reset timestamps in the
configuration. There was a bug with the manual reset of timestamps which I 
fixed today.

When you reset update timestamps, the next update will retrieve updates for 
series
done in the last month as opposed to Day or Week depending on last time update 
was run.

Currently the only outstanding bug with banner downloads is in 'Checking for 
missing
artwork'. This is only suppose to update series that contain no banners, 
instead it
tries to do it on some series that already have them! This is not a serious 
bug, but
is a waste of time. Its a bug in the SQL logic but I could not figure out what.

Now, none of this has anything to do with your original fault. I guess what I 
will do
is not write the entry to the DB when download fails.

You should also be aware, there are lots of problems with the online server at 
the
moment..so anything you update online might not make it through to the API. 
Last I
heard they know of the issue and trying to resolve it.

Original comment by damien.haynes@gmail.com on 19 Mar 2009 at 10:43

GoogleCodeExporter commented 9 years ago

ok so how can i check why the new posters i uploaded yesterday for the real 
hustle
and fifth gear don't get downloaded in "checking for new artwork"? i want to 
see if
the error is online or in the plugin...

Original comment by fred.doo...@gmail.com on 19 Mar 2009 at 10:49

GoogleCodeExporter commented 9 years ago

You need an apikey, which you can request from theTVDB.com.

If you didnt reset timestamps then most likely the plugin will request the daily
update file:

http://thetvdb.com/api/<apikey>/updates/updates_day.xml

where <apikey> should be replaced with you apikey.

If the banners are not in that file, then there is an issue online. If you reset
timestamps then updates_all.xml is used instead....but as I said in comment 15, 
this
was not working.

Original comment by damien.haynes@gmail.com on 19 Mar 2009 at 11:00

GoogleCodeExporter commented 9 years ago

i just did what you said + reset timestamps.. but as you said (reset timestamps 
isnt
working) i got updates_day instead updates_all.. 

nevertheless the file contained series 78998 and my uploaded poster.. 

then i got http://thetvdb.com/api/<apikey>/series/78998 /all/en.zip but there 
is no
poster in there.. maybe that's the problem..

Original comment by fred.doo...@gmail.com on 19 Mar 2009 at 11:07

GoogleCodeExporter commented 9 years ago

Here is the zip file for updates_day.xml

Im having a look at the code that processes this file and see if its missing
anything. From what I have seen it can be improved.

Original comment by damien.haynes@gmail.com on 19 Mar 2009 at 11:08

Attachments:

updates_day.zip

GoogleCodeExporter commented 9 years ago

If the poster is not in there, then the plugin will not know about it.

Original comment by damien.haynes@gmail.com on 19 Mar 2009 at 11:10

GoogleCodeExporter commented 9 years ago

thought so but still..it's listed in updates_day.xml

bt at least we isolated the problem to thetvdb.. updated poster is listed in 
updates_
and not in series xml..

Original comment by fred.doo...@gmail.com on 19 Mar 2009 at 11:13

GoogleCodeExporter commented 9 years ago

just a snippet from updates_day

...
<Series><id>78998</id><time>1237376970</time></Series>
...
<Banner>
  <Series>78998</Series>
  <format>680x1000</format>
  <path>posters/78998-1.jpg</path>
  <time>1237376970</time>
  <type>poster</type>
</Banner>
...

why don't you use this file to parse updates instead of main series one? all 
updates
are listed here anyhow..

Original comment by fred.doo...@gmail.com on 19 Mar 2009 at 11:15

GoogleCodeExporter commented 9 years ago

The data that the plugin uses is from the series en.zip file as that is suppose 
to
contain everything. the updates zip is used to just get what it needs.

So I imagine that your log would be trying to get Fifth Gear banners\posters 
but wont
find any new ones because it doesnt have the path from en.zip

I know the onlinepath to the banner is located in the updates file, but we 
shouldnt
need to use that.

Original comment by damien.haynes@gmail.com on 19 Mar 2009 at 11:20

GoogleCodeExporter commented 9 years ago

fyi: http://forums.thetvdb.com/viewtopic.php?f=4&t=762

Original comment by damien.haynes@gmail.com on 19 Mar 2009 at 11:24

GoogleCodeExporter commented 9 years ago

indeed, updates_day contains info about new poster but fifth gear xml does not.

ideally your approach should work nicely.. so it doesn't need to be changed.

curious about why the series xml doesn't get updated with new stuff thought

oh well at lteast its problem solved.. 
original fault still pending :)

Original comment by fred.doo...@gmail.com on 19 Mar 2009 at 11:26

GoogleCodeExporter commented 9 years ago

oh thats great! thank you!

Original comment by fred.doo...@gmail.com on 19 Mar 2009 at 11:28

GoogleCodeExporter commented 9 years ago

Regarding comment 15 when doing: 'Checking for missing artwork'

After further analysis this is doing the correct thing, I never realized some 
of my
series are also missing season banners as well :)

Original comment by damien.haynes@gmail.com on 19 Mar 2009 at 11:52

GoogleCodeExporter commented 9 years ago

Easier to track from other issue

Original comment by damien.haynes@gmail.com on 30 Oct 2009 at 5:57

Changed state: Duplicate

mwheelerjr / mptvseries

Handling of failed downloads (posters, banners, (maybe) fanart...) #152