FMaz008 / VineHelper

Browser extension to improve Vine Experience
https://www.vinehelper.ovh
MIT License
32 stars 23 forks source link

New Error Pop Up #40

Closed s2daxlgn closed 5 months ago

s2daxlgn commented 6 months ago

Hi! Thank you for building a great extension. I have a physical disability and it has made my life so much easier. I search for things I feel like many others don't so I have a lot of pages to wade through. The ability to hide items I am not interested in has made my struggles much easier. Thank you

That being said, I have been worried about Amazon not being "okay" with the extension/script and I don't want to lose my Vine ability. The hide feature is a feature I actually asked Amazon for "way back" when I started with Vine. It is something they should have on their pages themselves so they shouldn't be upset that someone created an extension that does this.

Today I started getting this error often when trying to hide items. (see photo attachment below) Does this mean that Amazon can see that we are using the extension and it is reporting back to Amazon?

I truly hope Amazon doesn't have an issue. For people like me who have a struggle and only use it to hide items - we need something like this to wade through the 25,000+items that aren't segregated very well. For that, I thank you.

Have you see this error?

Not sure if this helps: I am using the latest version of Google Chrome. I am in the US. I am using a Windows Computer (not mobile or mac)

s2daxlgn commented 6 months ago

Photo attachment of error: vinehelpererrorreport

s2daxlgn commented 6 months ago

Oh and the the error came up when I was trying to click the "Hide All" link sorry I failed to mention that. It happened with the link at the top of the page and the link on the bottom of the page.

FMaz008 commented 6 months ago

So this error is from the extension, not amazon per say. It just says amazon because that's the website it's happening on, but definitely just a problem with the extension.

It's from a last resort error catch clause when trying (but failing) to saving to local storage.

Is it happening every time you try to hide an item?

FMaz008 commented 6 months ago

Code origin: https://github.com/FMaz008/VineHelper/blob/2478ddbf160d4c942351eec99aa4ee719e145cad/scripts/HiddenListMgr.js#L66

FMaz008 commented 6 months ago

Is it possible that you have more than 100,000 items hidden? That's roughly how many hidden items it would take to fill the alloted 10mb of local storage, which could be causing this error.

skotl commented 6 months ago

Just browsing, but that's an interesting potential problem.

Do you have a timestamp on hidden items? If so it might be useful to have a scheduled task (which could be no more complex than "my page is being refreshed; oh - it-s been 30 days since I had a clean up") to remove items more than 30ish days old.

Thanks, by the way, for an awesome helper.

FMaz008 commented 6 months ago

Just browsing, but that's an interesting potential problem.

Do you have a timestamp on hidden items? If so it might be useful to have a scheduled task (which could be no more complex than "my page is being refreshed; oh - it-s been 30 days since I had a clean up") to remove items more than 30ish days old.

Thanks, by the way, for an awesome helper.

Yes for the dates. There's a garbage collection in place set for 90 days, which is the known maximum lifespan of a vine enrollment.

But the extension has not existed for 90 days yet and he seemed to have capped the local storage. If OP hides everything, he may have exceeded the storage before the garbage collection did anything.

Looking at implementing a rolling delete system as well, so old items are removed from the local hidden list when the storage approach its limit.

s2daxlgn commented 6 months ago

Hi! I have only been using the extension maybe 2 weeks or so? It could be possible that I have more than 100,000 items but I really doubt it. Maybe? it depends on how many new items Amazon has dropped. I don’t think I’ve seen anything older than 30 days so the difference between 30 days and 90 days would be a moot point for the garbage drop point because nothing over 30 days would be hidden so nothing to dump and to clear up space. I don’t know if I said that right? Is it possible to hide more than 100,000 items by clearing up space elsewhere that I have? If I’m breached the max 100,000 items in just two weeks then I would definitely hit it by the 90 day mark. Hiding is mainly what I use the extension for. As I said earlier, I’m very grateful for it. I don’t think it’s made me have any advantage with Vine since I’m mainly getting things to make my life easier with my disability and hiding things that would never help me. Storage containers, shower, benches, dispensers, automatic trash cans.. Any thoughts? Thank you! Sincerely.

FMaz008 commented 6 months ago

The extension itself has not existed for 90 days yet, so very mood point indeed. Amazon US has around 43k product live per day. Unclear how big the daily drops are, but since jan 17th they had over 340k products IIRC.

If you are hidding everything, it is very possible you hid 100k items in 30 days and capped the local storage.

Short term fix: V 2.2 added a setting reset button at the bottom of the "?" Tab. This will wipe all the local storage, including hidden items. It won't impact remote stored items. This would solve your problem in the short term, but you would have to reconfigure your extension and re hide everything if you were not using remote storage.

Long term fix (work in progress): We are currently working on 2 ideas: one is to clear the hidden items when the list is full. The other, better solution, is to implement a rolling delete: delete the oldest items to always keep the list under 9mb. Most older items will be delisted, some rare items may show back up if they are to stay for the full 90 days. A minor inconvennience that can be solved by enabling remote storage, or just re-hidding them.

Again I want to reassure you that this empty error is NOT from amazon and has zero impact on amazon. They don't even know this error happened.

I'm sorry this confusing error happened, a different error (and solution) was supposed to be displayed, and it was supposed to show an error message.

Thank you again for your bug report, as I believe for every person who take the time to fill one, 10 people live the same issue but don't say anything. we are currently working on it and will let you know when a fix has been implemented

FMaz008 commented 6 months ago

Another possibility, which would make sense given I had code to try to catch storage limit being exceeded, is that you triggered one of those errors:

Any change your usage could have been fast paced enough to have triggered one of those errors ?

FMaz008 commented 6 months ago

Version 2.2.1 published, I implemented a new garbage collector which should help clearing out old items if you are using too much local storage, as well as new entries in the debug windows (press d to display it), at the very bottom is will show the size usage for various internal variables. If you could provide me the numbers it would be helpful.

skotl commented 5 months ago

Thanks for the continued improvements here. Just some stats for you to help understand the garbage collection: I pretty much "Hide all" on the first 5-8 pages on the UK site and debug states "Storage used by hiddenItems: 332.38 KB representing 3912 items" or 84 bytes per item.

So I find it hard to comprehend that anyone could use up 10MB of local storage.

FMaz008 commented 5 months ago

@skotl the US have a lot more items. Around 190k per month if I'm not mistaken. Some people hide everything, and so the count can add up faster than my own experience could have anticipated.

FMaz008 commented 5 months ago

V2.2.2 fails to fix the issue. Reworked the code, hopeful about v2.2.3.

Sorry for the delay in fixing this, we're trying..

FMaz008 commented 5 months ago

V2.2.5 published and should fix the problem. Please advise if it did the trick for you :)

s2daxlgn commented 5 months ago

Now I am getting this error LOL @FMaz008 Can't hide at all. vinhelper031224

s2daxlgn commented 5 months ago

I honestly don't think I hide that much. Nowhere near 100K - if that helps troubleshoot. I put in keywords and hide. I don't go page by page by page. Only 36 on a page Thanks for your help :)

Update: Okay I just realized. Of the 10 things I search they only have like 3 or 4 pages each. But let's say they have 10 pages each. 10 x 10 x 36 = 3600 items. There is nooooooo way I am anywhere near 100,000 LOL Even if it were 20 items x 10 x 36 that would only be 7200 and over 2 weeks. I don't think that would be 100,000 do you? If so, that's really not a lot of items being hidden

FMaz008 commented 5 months ago

What version are you using ? Can you hit "d", it should show a debug popup. Scroll all the way down to the very last page and send me the hidden items info, as to how many item you have and how much data they are using ?

Also if you use Chrome v113 or earlier you only have 5mb available instead of 10. Found that out today. The new version allow you to set your cache limit to stay under what ever maximum you reached.

s2daxlgn commented 5 months ago

Hi! I will check now. I did check Chrome earlier - this is what I have but it says up to date? Is there a later Version?

Chrome is up to date Version 122.0.6261.112 (Official Build) (64-bit)

FMaz008 commented 5 months ago

You need to have the keyboard shortcuts enabled in the options. What version of VineHelper are you using?

jenifav commented 5 months ago

Where is the local storage located? Is it possible to get in there and manually delete items? I am definitely over the limit, lol.

Thanks for a great extension! Going to donate!

FMaz008 commented 5 months ago

Where is the local storage located? Is it possible to get in there and manually delete items? I am definitely over the limit, lol.

Thanks for a great extension! Going to donate!

That is a good question, and I don't have the answer. The browser handle that... somewhere.

Modern browsers have a 10MB limit.

But with v2.2.6 and up, if you are getting near 9Mb of space used, the extension will automagically delete 1MB of the oldest hidden items.

Some new features you might not know about:

If somehow you were to exceed the storage space, you would get a nasty alert message and you could no longer save anything: hidden items, settings, etc ... essentially makes Vine Helper useless. It's a critical problem.

So that's why I went to great length to implement a solution ASAP, as some people I like to call "Mega Hiders" were stuck with an unusable extension after hidding North of 130,000 items. Going a little too fast on publishing a fix, it lead to 3 problematic releases in the same day.

Hope this answer your questions.

Ps: thank you for your consideration :)

jenifav commented 5 months ago

Yes, I discovered a bunch of these new features. I am getting the popup that my hidden items are being trimmed and I don't mind the oldest 1MB being sacrificed! I was being a Mega Hider mostly to stress test it and see what would happen. I appreciate all your hard work on getting these releases out and being responsive. I went to donate via PayPal and it says "This organization's page is broken."

s2daxlgn commented 5 months ago

V2.2.2 fails to fix the issue. Reworked the code, hopeful about v2.2.3.

Sorry for the delay in fixing this, we're trying..

No apologies needed :) I am very grateful for your app and help. Oh to confirm I did see you say it is just a vinehelper error and not amazon - thank you for the reassurance. I am no longer getting the error. I am using VineHelper 2.3.0 now. There is one odd thing happening - I hide items but sometimes the items I hid come back up again (randomly) when I search that keyword again later. Doesn't seem to be duplicates of the same items. It is intermittent. I will comment here if it happens again today with the steps I took. THANK YOU for your help with the other error notification I was getting. It is gone now. (I am sure it was gone when you fixed it but I have been offline)

Thanks! Dax (female)

FMaz008 commented 5 months ago

Hi Dax, I'm very sorry if I miss-used your gender (french here)

I will keep my eyes for that problem with the hidden items, but for the sake of keeping the bug tracker tidy, I will mark this issue as resolved.

If you happen to figure a clear steps to reproduce the problem, absolutely do not hesitate to reopen a ticket.

s2daxlgn commented 5 months ago

No worries on the gender. I thought it was funny :) No way you would have known either way. You could refer to me as a lizard or elephant and I wouldn't care as long as I could keep using VineHelper LOL IF the hiding keeps happening I will open a new ticket. Yes, this one can be closed now :)

Maybe this is right? Merci d'avoir créé votre extension pour que je puisse l'utiliser. Je suis très reconnaissant. (Thank you for creating your extension for me to use. I am very grateful.)

s2daxlgn commented 5 months ago

Oh! I failed to mention. I looked at items hidden and they were no where near 100K and it was more than 2 weeks. Here is what it says now. I have hidden a LOT more since - now that I know I am only at 1/2:

65455ms: Storage used by arrTemplate: 38.7 KB representing 70 items 66118ms: Storage used by hiddenItems: 4.65 MB representing 54802 items 66136ms: Storage used by latestProduct: 21 Bytes representing 19 items 66139ms: Storage used by settings: 1.21 KB

FMaz008 commented 5 months ago

Quick note just in case, but we had a user who was using an older version of Chrome. Before Chrome v113, the memory limit was 5MB, not 10MB, so in his case, he needs to set the memory limit to 4MB. Most people should be fine with 9MB though...

s2daxlgn commented 5 months ago

I pay google for extra space - I will see if that includes extensions. Stupid question. Is there anything in VineHelper to unhide everything and start over? It isn't hiding the items I have hidden today. Just an example - I have hidden these same things multiple times over the last 7 hours they keep popping up - it is the same ones not new ones.

FMaz008 commented 5 months ago

Yes there is an option to clear locally stored hidden items in the System tab of the option window :)

I fixed a complex bug today where people using multiple tabs at once would get items hidden instead of shown and vice versa. If you are using multiple tabs that might be why things were showing back up.

s2daxlgn commented 5 months ago

Hi! Sorry to bother you - I know you are super busy. If I delete VineHelper - will it delete all the hidden items and then if I reinstall VineHelper I can start all over hiding from fresh?

(My items are still not staying hidden. Want to see if this would work?)

s2daxlgn commented 5 months ago

Not sure if this helps anyone else? If I have more storage room in Google than I am using - is it possible to start using some of that storage for VineHelper extension in addition to the 9MB? image

FMaz008 commented 5 months ago

Reinstalling won't change anything. And yes it will wipe everything AND give you a new ASIN, something you can do with the factory reset.

This means you will no longer be linked to your remotely stored hidden items if you were using that feature.

And no, the google storage has nothing to do with VH and VH has no way of utilizing it. But even if you could, locally storing 10mb of items represented approximately 130k items. Going much above that would present performance problem as you need to load all that data, and search it 36 times per page load.

That's why the remote storage is better suited for the task because it is handled by a proper database.

Hope this help :)

s2daxlgn commented 5 months ago

Update: Doing the "factory reset" worked. Everything is staying hidden now. Thank you for your help!!!