incorrect number of notification in GUI + Notification like Twitter / Mail

GoogleCodeExporter commented 9 years ago

I took this out of a previous issue as it might not be related or might be.

How to reproduce
Make sure you have a TV serie where multiple subs are not yet available
than 1 sub must be made available, than the issue can be seen.

For me:
TV Series: Artic Air, until today there were 7 subs not available yet

Today, 1 of the subs (EP04) arrived and Autosub noticed it.
But.. this episode 4 was notified (and/or downloaded) 8 times !
7x at 15.41 and 1 time 1 hour later

logfile it starts here:
2012-08-05 15:39:01,420 DEBUG  scanDir: File Arctic Air - S01E04 - 720p HDTV 
x264-2HD.mkv is missing a subtitle
2012-08-05 15:39:01,421 DEBUG  ProcessFileName: Dumping show_info dict for 
debug {'season': u'01', 'title': u'Arctic Air', 'episode': u'04', 'extra_info': 
u'720p HDTV x264-2HD'}
2012-08-05 15:39:01,421 DEBUG  ProcessFileName: Dumping dict for debug 
{'episode': u'04', 'title': u'Arctic Air', 'season': u'01', 'releasegrp': 
u'2hd', 'source': u'hdtv', 'codec': u'h264', 'quali$
2012-08-05 15:39:01,421 INFO  scanDir: Dutch subtitle wanted for Arctic Air - 
S01E04 - 720p HDTV x264-2HD.mkv and added to wantedQueue
2012-08-05 15:39:01,422 DEBUG  scanDir: File Arctic Air - S01E05 - 720p PROPER 
HDTV x264-2HD.mkv is missing a subtitle
2012-08-05 15:39:01,422 DEBUG  ProcessFileName: Dumping show_info dict for 
debug {'season': u'01', 'title': u'Arctic Air', 'episode': u'05', 'extra_info': 
u'720p PROPER HDTV x264-2HD'}
2012-08-05 15:39:01,422 DEBUG  ProcessFileName: Dumping dict for debug 
{'episode': u'05', 'title': u'Arctic Air', 'season': u'01', 'releasegrp': 
u'2hd', 'source': u'hdtv', 'codec': u'h264', 'quali$
2012-08-05 15:39:01,422 INFO  scanDir: Dutch subtitle wanted for Arctic Air - 
S01E05 - 720p PROPER HDTV x264-2HD.mkv and added to wantedQueue
2012-08-05 15:39:01,423 DEBUG  scanDir: File Arctic Air - S01E06 - 720p HDTV 
x264-2HD.mkv is missing a subtitle
2012-08-05 15:39:01,423 DEBUG  ProcessFileName: Dumping show_info dict for 
debug {'season': u'01', 'title': u'Arctic Air', 'episode': u'06', 'extra_info': 
u'720p HDTV x264-2HD'}
2012-08-05 15:39:01,423 DEBUG  ProcessFileName: Dumping dict for debug 
{'episode': u'06', 'title': u'Arctic Air', 'season': u'01', 'releasegrp': 
u'2hd', 'source': u'hdtv', 'codec': u'h264', 'quali$
2012-08-05 15:39:01,424 INFO  scanDir: Dutch subtitle wanted for Arctic Air - 
S01E06 - 720p HDTV x264-2HD.mkv and added to wantedQueue

I only can relate the multiple notifications to the availability of the number 
of open srt's needed.

otherwise it is not to understand this: I think there is somewhere an incorrect 
checking at the database for the downloadable file. 

2012-08-05 15:39:11,808 DEBUG  scanDir: Finished round of local disk checking
2012-08-05 15:41:00,815 DEBUG  downloadSubs: Something in downloadqueue, 
Starting downloadSubs
2012-08-05 15:41:00,987 DEBUG  downloadSubs: Trying to download the following 
subtitle 
http://www.bierdopje.com/downloads/sub/100091/arctic.air.s01e04.720p.hdtv.x264-2
hd.srt/apikey/BB442E7744E9B54$
2012-08-05 15:41:01,347 INFO  downloadSubs: DOWNLOADED: /media/usenet/TV/Arctic 
Air/Season 1/Arctic Air - S01E04 - 720p HDTV x264-2HD.srt
2012-08-05 15:41:01,541 DEBUG  Notify: Trying to send notifications. Language: 
nl Srt: /media/usenet/TV/Arctic Air/Season 1/Arctic Air - S01E04 - 720p HDTV 
x264-2HD.srt Video: /media/usenet/TV/Arc$
2012-08-05 15:41:01,541 DEBUG  Notify: Twitter is enabled
2012-08-05 15:41:01,541 DEBUG  Twitter: Trying to send a tweet
2012-08-05 15:41:02,517 INFO  Twitter: Tweet sent
2012-08-05 15:41:02,517 DEBUG  Notify: Mail is enabled
2012-08-05 15:41:02,517 DEBUG  Mail: Trying to send a mail
2012-08-05 15:41:03,163 INFO  Mail: Mail sent
2012-08-05 15:41:03,164 DEBUG  downloadSubs: Postprocess: running python 
PostProcess.py "/media/usenet/TV/Arctic Air/Season 1/Arctic Air - S01E04 - 720p 
HDTV x264-2HD.srt" "/media/usenet/TV/Arctic$
2012-08-05 15:41:03,164 INFO  downloadSubs: Running PostProcess
2012-08-05 15:41:04,445 DEBUG  downloadSubs: PostProcess Output:Sending the 
following tweet AutoSub Downloaded: Arctic Air - S01E04 - 720p HDTV x264-2HD.srt
ERROR: Could not send tweet, make sure token_key & token_secret are correct.

2012-08-05 15:41:04,582 DEBUG  downloadSubs: Trying to download the following 
subtitle 
http://www.bierdopje.com/downloads/sub/100091/arctic.air.s01e04.720p.hdtv.x264-2
hd.srt/apikey/BB442E7744E9B54$
2012-08-05 15:41:04,719 INFO  downloadSubs: DOWNLOADED: /media/usenet/TV/Arctic 
Air/Season 1/Arctic Air - S01E04 - 720p HDTV x264-2HD.srt
2012-08-05 15:41:04,866 DEBUG  Notify: Trying to send notifications. Language: 
nl Srt: /media/usenet/TV/Arctic Air/Season 1/Arctic Air - S01E04 - 720p HDTV 
x264-2HD.srt Video: /media/usenet/TV/Arc$
2012-08-05 15:41:04,866 DEBUG  Notify: Twitter is enabled
2012-08-05 15:41:04,866 DEBUG  Twitter: Trying to send a tweet
2012-08-05 15:41:05,733 ERROR  Twitter: Failed to send a tweet
2012-08-05 15:41:05,734 DEBUG  Notify: Mail is enabled
2012-08-05 15:41:05,734 DEBUG  Mail: Trying to send a mail

Note: there is nothing in the log which states that it still needs EP04 .. but 
it is not removed from downloadqueue .. 

So here lays somewhere an issue

Original issue reported on code.google.com by aroundmyroom on 5 Aug 2012 at 5:16

Merged into: #131

GoogleCodeExporter commented 9 years ago

nou .. what I thought is not valid for all
today: perception, 7 times announced / notified (mail send so I can see how 
much time it is processed. )
where swubs of covert affairs (s03e04) and Necessary Roughness (s02E03) are 
announced just once.

is there a way to see the downloadqueue? and wantedqueue

I see that 1 hour before the sub was found but not yet downloaded. Could it be 
that every check before the actual download is added to the wantedqueue?
and autosub does not see it is already in the wantedqueue?

Original comment by aroundmyroom on 6 Aug 2012 at 1:52

GoogleCodeExporter commented 9 years ago

at least somewhere found a bug

if there is a sub found through the RSS but or the api calls are empty or maybe 
by default the sub is added to the wantedqueue, so far so good, but from this 
wanted queue there is no DownloadSub action 

If than there is a normal subcheck, than the same sub can and will be found 
again. the existing sub in the wantedqueue is ignored and a new sub will be 
added

in log form;

2012-08-10 18:16:33,444 INFO  checkRss: Got a match, matching file is: 
Anger.Management.S01E08.720p.HDTV.x264-IMMERSE
2012-08-10 18:16:33,444 DEBUG  checkRss: Dumping downloadlink for debug 
purpose: 
http://www.bierdopje.com/downloads/sub/100594/Anger.Management.S01E08.720p.HDTV.
x264-IMMERSE.srt/apikey/BB442Exxx
2012-08-10 18:16:33,444 INFO  checkRSS: The episode Anger Management - Season 
01 Episode 08 has a matching subtitle on the RSSFeed, adding to toDownloadQueue
2012-08-10 18:16:33,733 DEBUG  checkRSS: Removed item from the wantedQueue at 
index 79
2012-08-10 18:16:33,733 DEBUG  checkRSS: Finished round of RSS checking
2012-08-10 18:23:04,374 ERROR  downloadSubs: out of api calls

Above you see the sub is found and added to the wantedqueue

But than .. 
012-08-10 18:28:48,310 DEBUG  scanDir: File Anger Management - S01E08 - 720p 
HDTV x264-IMMERSE.mkv is missing a subtitle
2012-08-10 18:28:48,310 DEBUG  ProcessFileName: Dumping show_info dict for 
debug {'season': u'01', 'title': u'Anger Management', 'episode': u'08', 
'extra_info': u'720p HDTV x264-IMMERSE'}
2012-08-10 18:28:48,310 DEBUG  ProcessFileName: Dumping dict for debug 
{'episode': u'08', 'title': u'Anger Management', 'season': u'01', 'releasegrp': 
u'immerse', 'source': u'hdtv', 'codec': u'h26$
2012-08-10 18:28:48,311 INFO  scanDir: Dutch subtitle wanted for Anger 
Management - S01E08 - 720p HDTV x264-IMMERSE.mkv and added to wantedQueue

And here number 2 is added

As I am debugging the log of the DownloadQueue I see an index 0 and 1

and yes.. 
the log is showing it
012-08-10 18:33:09,826 DEBUG  checking what WAS in THE downloadqueue 
downloadsubs.py file
2012-08-10 18:29:52,931 DEBUG
[{'lang': [], 'episode': u'08', 'title': u'Anger Management',
'season': u'01', 'downloadLink': 
u'http://www.bierdopje.com/downloads/sub/100594/Anger.Management.S01E08.720p.HDT
V.x264-IMMERSE.srt/apikey/BB442E7744E9B541/',
'releasegrp': u'immerse', 'source': u'hdtv', 'codec': u'h264',
'originalFileLocationOnDisk': u'/media/usenet/TV/Anger Management/Season 
1/Anger Management - S01E08 - 720p HDTV x264-IMMERSE.mkv', 'downlang': 'nl',
'destinationFileLocationOnDisk': u'/media/usenet/TV/Anger Management/Season 
1/Anger Management - S01E08 - 720p HDTV x264-IMMERSE.srt',
'quality': u'720p'},

012-08-10 18:33:09,825 DEBUG  checking what WAS in THE downloadqueue 
downloadsubs.py file
{'lang': [], 'episode': u'08', 'title': u'Anger Management',
'season': u'01', 'downloadLink': 
u'http://www.bierdopje.com/downloads/sub/100594/Anger.Management.S01E08.720p.HDT
V.x264-IMMERSE/apikey/BB442E7744E9B541',
'releasegrp': u'immerse', 'source': u'hdtv', 'codec': u'h264',
'originalFileLocationOnDisk': u'/media/usenet/TV/Anger Management/Season 
1/Anger Management - S01E08 - 720p HDTV x264-IMMERSE.mkv', 'downlang': 'nl',
'destinationFileLocationOnDisk': u'/media/usenet/TV/Anger Management/Season 
1/Anger Management - S01E08 - 720p HDTV x264-IMMERSE.srt',
'quality': u'720p'}]

So, one of the issues is that after the RSS check the download is not directly 
started causing a duplicate sub to be added to the downloadqueue.

Original comment by aroundmyroom on 10 Aug 2012 at 5:25

GoogleCodeExporter commented 9 years ago

Denk dat ik er wel uit ben .. 
Ik heb voor het debuggen de API even erg opgehoogd om zo even geen problemen te 
hebben.
En wat zie ik: gewoon na de RSS check komt de downloadsubs script in actie, dus 
geen issue met geen beschikbare API, dus dat betekent dat de bug de volgende is:

wanneer er geen API calls meer zijn en er zijn tijdens de RSS check (of bijv. 
de normale scan) wel nieuwe subs gevonden, dan zullen deze gevraagde subs in de 
downloadqueue blijven zitten. Echter wanneer dan de API weer gereset wordt en 
er een nieuwe scan volgt (RSS of download) dan worden de gevraagde subs gewoon 
extra in de downloadqueue gezet. 

Wat er zou moeten gebeuren is dat wanneer de API calls op zijn, de 
downloadqueue leeggemaakt moet worden en wanneer er weer API calls beschikbaar 
zijn dan kunnen ze gewoon weer gedownload worden alvorens verwijderd.

Of je gaat checken wat er al in de dowloadqueue staat en als er iets nieuws bij 
moet komen dan zorg je ervoor dat er geen duplicaat mag zijn.

Ik hou eea nog in de gaten dit weekend om te zien of dit juist is. (ophogen API 
calls kan ook maar dat wil je denk ik niet).

Original comment by aroundmyroom on 11 Aug 2012 at 7:23

GoogleCodeExporter commented 9 years ago

Mee eens, voor me zelf:
Leeg download queue wanneer er geen API calls zijn.

Original comment by romke.va...@gmail.com on 30 Aug 2012 at 7:48

Changed state: Accepted
Added labels: Milestone-Release0.5.6

GoogleCodeExporter commented 9 years ago

Original comment by romke.va...@gmail.com on 30 Aug 2012 at 7:48

GoogleCodeExporter commented 9 years ago

Original comment by romke.va...@gmail.com on 4 Oct 2012 at 1:43

Changed state: Duplicate

chenqingyu / auto-sub

incorrect number of notification in GUI + Notification like Twitter / Mail #121