search

XrXr / YoutubeSubscriptionChecker

A Firefox add-on that checks for new uploads from Youtube channels
4 stars 0 forks source link

Subscription checker seems to check the first 10 items only. #16

Closed HAL42b closed 8 years ago

HAL42b commented 8 years ago

I have 100+ subscriptions. I seem to get updates for the first 10 items in the list only. Never any updates for items lower in the list.

Not sure but this might have happened after one subscriber (CCCen, number 33 in my list) dumped 100+ videos in a single day. After that there were no updates from any other users until I deleted those 100 videos from the list.

After that I seem to be getting updates from the first 10 items only.

So I suppose the question is "Under what conditions does the code stop checking for other updates?"

Many thanks for your work btw, if it wasn't for the subscription checker YT would have been a pain in the nether regions. Good Job!

XrXr commented 8 years ago

hmm.. The only thing I can think of off the top of my head is the storage limit on simple-storage, which is about 5mb. If you have a lot of unwatched video, that might be a cause. How many unwatched videos do you usually have?

I actually can't think of any plausible reason... Even if you are over the storage limit, you should still get the notifications, supposedly. I've never tested pushing it that hard though so maybe that's it...

If you are okay with it, could you post an export of your config? You can export through the settings menu.

If not, could you try starting a new Firefox profile, adding a few channels that are below the top 10 in your list and see if you get updates for them? (You will probably have to start multiple FF instances for this.) This will at least tell me whether the issue is specific to your main profile or happen in general.

HAL42b commented 8 years ago

Profile attached.

I will also start a new Firefox profile and see if I get updates that way.

I usually have max 5 unwatched videos. The exception was CCCen dumping more than 100 in a single day. They are kinda important so I didn't delete them straight away with intention of watching one or two every day. So they ended up staying in the list for more than a couple of weeks.

Another quirk I have is that I'm running my Firefox profile in tmpfs in Linux (basically the profile stays in the RAM and is not written to disk). I really do not expect this to be an issue but I will check to make sure.

I will also try to add a few channels on a fresh profile (not in tmpfs) and see if I get updates.

YTSubCheckerConfig.txt

XrXr commented 8 years ago

Thank you! I will look into this deeper this weekend.

HAL42b commented 8 years ago

Resetting the Firefox profile seems to have solved the problem. Currently using the above profile and receiving videos from many channels. (Had to wait a few hours for people to post videos)

Given that, it would be nice if Subscription Checker gave some sort of a warning that not all is right, or maybe not depend on the FF profile so much (is this even possible?)

Maybe storing the config in a human readable text file separate from the FF profile?

XrXr commented 8 years ago

Unfortunately if I switch away from using the sdk's storage api for storage, and start rolling my own, things get nasty really quickly. I will run into cross platform issues and if anything it's probably even less robust than what it is right now.

I do have the option to use indexed-db for storage though, which should be more robust than what I use right now, which basically stores json text in a text file in the profile..

XrXr commented 8 years ago

I hope you still have the profile that originally were having issues, at this point I'm still not sure if it's a problem with storage. Nothing in the doc says trying to storage extra things when at quota will cause an exception, so you should at the very least get the desktop notification for videos..

One thing to maybe look out for is whether channels below the first 10 still show in the desktop notification. If it only shows up in there but not in the hub then it's probably a storage issue

If you still have that profile, could you check the size of {your profile path}/jetpack/youtube-subscription-checker@xrxr/simple-storage/store.json?

HAL42b commented 8 years ago

I still have the old profile. Tell me if you need me to do something with it.

I was not getting notification btw.

Indexed Json might be a good idea. This way I can at least export it to a human readable file, then add channel icons, contact info of the owner and so on. (All this because I don't want a YT account but I still like to stay in contact with some of those people) Many of them have blogs or their own sites, RSS or at least mail.

Also most of them are starting to post to alternative sites as YT is getting more and more draconian. So if a video is removed from YT it would still be available at another site.

XrXr commented 8 years ago

I already store channel info in the JSON file ({your profile path}/jetpack/youtube-subscription-checker@xrxr/simple-storage/store.json) But the information you get there for channels is really minimal. It's just the channel name and the channel ID. But hey that might be enough for you.

Can you tell me the size of that JSON file?

HAL42b commented 8 years ago

Oh! That seems very useful. I can do things with that.

I could parse it into some sort of table and add the fields I need. It would only benefit myself though, which is a bummer.

Need to learn Java and Json first. Programming is not my area of expertise.

XrXr commented 8 years ago

You might have missed my edits, could you tell me the size of that JSON file?

HAL42b commented 8 years ago

/jetpack had two folders in it, one was the subscription checker the other seems to be Reddit-RES (which is notably buggy but not used for more than a year)

There is no way for me to sanitize the RES file so I can't post it but it's size is 124Kb

youtube-subscription-checker@xrxr is 64Kb [attached]

youtube-subscription-checker.zip

edit: Channel notifications do show up in the new fresh profile. They definitely did not show up for the items 11 and greater in the old broken profile.

XrXr commented 8 years ago

Thank you for the profile! I was able to find the issue and fix it. Here a special build which contains the fix. Could you try it on the broken profile and see if the issue is resolved? special-build.zip

HAL42b commented 8 years ago

Wow, great news. I need to load the old profile and wait for new videos to be posted though, don't I? It might take some time.

HAL42b commented 8 years ago

Installed old profile.

2 new videos show up immediately but they belong to the first 10 in the list. There were unwatched videos further down on the list but those do not show up. Do I need to wait or something?

Need to install the patch first! DOH.

HAL42b commented 8 years ago

Success!!!!

All the unwatched videos show up now! All 387 of them!

Thank You very Much!

XrXr commented 8 years ago

Awesome! I will send the update to AMO in a few days.

In the meantime you probably want to reinstall the addon from AMO. I know it's a bummer, but if you don't, Firefox won't automatically update the add-on when new versions get approve on AMO. Feel free to keep using it if you don't mind though.

HAL42b commented 8 years ago

No problem. Freshening up the FF profile was kinda neat anyway.

This is Great! Sooo many new videos!

Man, I owe you big time. One of these days I'll learn this Java thing and submit patches if I can.

XrXr commented 8 years ago

You are welcome. I don't want to miss videos either. Also the language the addon uses is called JavaScript, which is very different from Java, despite what the names imply.

HAL42b commented 8 years ago

Hence the need to learn.

The only interpreted language I was exposed to was BASIC in my childhood and that scarred me for life. Since then I believe in programming with my soldering iron :)