sonigy / YCS

Chrome Extension: YCS. Search for comments and authors in YouTube videos
https://chrome.google.com/webstore/detail/pmfhcilikeembgbiadjiojgfgcfbcoaa
MIT License
112 stars 5 forks source link

Loading Comments stopped working. #56

Closed programmer1o1 closed 3 months ago

programmer1o1 commented 7 months ago

Clicking "reload" on the comments doesn't load the comments anymore or just loading doesn't work anymore. Please fix it. Thanks you so much.

NicPalacio commented 7 months ago

Exactly it happen for me recently too , it does not work in any browser

@sonigy , @knoajp, @Letssharearow any idea why ? If you need help I will be happy to contribute but I think that it has to do with something more technical.

chuimoi commented 7 months ago

For me, it doesn't work with firefox

JackOfSpade commented 7 months ago

Guys, the author has probably died by now. Just move on to another chrome extension. There are plenty.

programmer1o1 commented 7 months ago

Guys, the author has probably died by now. Just move on to another chrome extension. There are plenty.

To be fair, it's the best one right now. There aren't many good extensions like these in Firefox, and maybe Chrome too.

NicPalacio commented 7 months ago

@JackOfSpade mm not sure why you say that xD, but there us not really an alternative to this extension, only this one does the work

RenegadeScooter commented 7 months ago

I use Brave (version 1.64.113, Chromium 123.0.6312.86, 64-bit). The extension loads comments for normal windows, but not private windows.

knoajp commented 7 months ago

@NicPalacio Thank you for mentioning me😆, by quick investigation, it does load comments but only fails to handle/show. Although I haven't checked the extension code yet, it could be caused by just a JSON structural modification by YouTube.

As well as making a quick patch on the YCS, I expect some nice alternatives to appear, like the IYCS project. 😉 https://github.com/StellarSt0rm/IYCS @StellarSt0rm

knoajp commented 7 months ago

🤔 It seems to take much effort to solve it...

YCS source code on Mac Chrome (minified): Library/Application Support/Google/Chrome/Default/Extensions/pmfhcilikeembgbiadjiojgfgcfbcoaa/1.1.12_0/web-resources/wresources.js

JSON API: https://www.youtube.com/youtubei/v1/next?key=xxxx

Currently, comments are in: JSON.frameworkUpdates.entityBatchUpdate.mutations

But there are no such property names in wresources.js. Still, I can find JSON.onResponseReceivedEndpoints in wresources.js, which might haven't changed.

knoajp commented 7 months ago

My biggest reason for using YCS is to browse popular comments by Likes order. So my temporal workaround is using the CSS below to highlight popular comments.

#vote-count-middle{
  color: black !important;
  border-radius: .5em;
  padding: .25em .5em;
}
/*1000-*/#like-button:has(:is(button[aria-label*="K "], button[aria-label*="M "])) + #vote-count-middle{
  font-weight: bold;
  background: #f008;
  border: 1px solid #f00f;
}
/*100-999*/#vote-count-middle{
  background: #f004;
  border: 1px solid #f008;
}
/*10-99*/#like-button:has(:is(
  button[aria-label*=" 10 "], button[aria-label*=" 11 "], button[aria-label*=" 12 "], button[aria-label*=" 13 "], button[aria-label*=" 14 "], button[aria-label*=" 15 "], button[aria-label*=" 16 "], button[aria-label*=" 17 "], button[aria-label*=" 18 "], button[aria-label*=" 19 "],
  button[aria-label*=" 20 "], button[aria-label*=" 21 "], button[aria-label*=" 22 "], button[aria-label*=" 23 "], button[aria-label*=" 24 "], button[aria-label*=" 25 "], button[aria-label*=" 26 "], button[aria-label*=" 27 "], button[aria-label*=" 28 "], button[aria-label*=" 29 "],
  button[aria-label*=" 30 "], button[aria-label*=" 31 "], button[aria-label*=" 32 "], button[aria-label*=" 33 "], button[aria-label*=" 34 "], button[aria-label*=" 35 "], button[aria-label*=" 36 "], button[aria-label*=" 37 "], button[aria-label*=" 38 "], button[aria-label*=" 39 "],
  button[aria-label*=" 40 "], button[aria-label*=" 41 "], button[aria-label*=" 42 "], button[aria-label*=" 43 "], button[aria-label*=" 44 "], button[aria-label*=" 45 "], button[aria-label*=" 46 "], button[aria-label*=" 47 "], button[aria-label*=" 48 "], button[aria-label*=" 49 "],
  button[aria-label*=" 50 "], button[aria-label*=" 51 "], button[aria-label*=" 52 "], button[aria-label*=" 53 "], button[aria-label*=" 54 "], button[aria-label*=" 55 "], button[aria-label*=" 56 "], button[aria-label*=" 57 "], button[aria-label*=" 58 "], button[aria-label*=" 59 "],
  button[aria-label*=" 60 "], button[aria-label*=" 61 "], button[aria-label*=" 62 "], button[aria-label*=" 63 "], button[aria-label*=" 64 "], button[aria-label*=" 65 "], button[aria-label*=" 66 "], button[aria-label*=" 67 "], button[aria-label*=" 68 "], button[aria-label*=" 69 "],
  button[aria-label*=" 70 "], button[aria-label*=" 71 "], button[aria-label*=" 72 "], button[aria-label*=" 73 "], button[aria-label*=" 74 "], button[aria-label*=" 75 "], button[aria-label*=" 76 "], button[aria-label*=" 77 "], button[aria-label*=" 78 "], button[aria-label*=" 79 "],
  button[aria-label*=" 80 "], button[aria-label*=" 81 "], button[aria-label*=" 82 "], button[aria-label*=" 83 "], button[aria-label*=" 84 "], button[aria-label*=" 85 "], button[aria-label*=" 86 "], button[aria-label*=" 87 "], button[aria-label*=" 88 "], button[aria-label*=" 89 "],
  button[aria-label*=" 90 "], button[aria-label*=" 91 "], button[aria-label*=" 92 "], button[aria-label*=" 93 "], button[aria-label*=" 94 "], button[aria-label*=" 95 "], button[aria-label*=" 96 "], button[aria-label*=" 97 "], button[aria-label*=" 98 "], button[aria-label*=" 99 "],
dummy)) + #vote-count-middle{
  background: #f002;
  border: 1px solid #f004;
}
/*1-9*/#like-button:has(:is(
  button[aria-label*=" 1 "], button[aria-label*=" 2 "], button[aria-label*=" 3 "], button[aria-label*=" 4 "], button[aria-label*=" 5 "], button[aria-label*=" 6 "], button[aria-label*=" 7 "], button[aria-label*=" 8 "], button[aria-label*=" 9 "],
dummy)) + #vote-count-middle{
  background: #f001;
  border: 1px solid #f002;
}
xNasuni commented 7 months ago

this is happening to me aswell. from my investigation the API is returning the data, and the extension does get all the comments, but it never actually puts them in the UI. this might be due to a change in the structure of how they return stuff that could've changed recently which made this issue arise but until I or someone else does more research this will stay broken. image

for some reason this repository does not contain the source code of the YCS extension which is a little sad, but I will make my own version of this extension on github if I do manage to fix it, with credits to the original creator ofcourse. (the extension that you can download from webstore is minified)

suprisingly still works on my laptop, which has had the extension for a long time, weird...

xNasuni commented 7 months ago

i've figured out atleast 1 issue.

this extension stops working some versions after chromium 122, i've tested this extension and it's worked on 121.0.6156.0 (chromium build), and 122.0.6261.137 (chrome build) but stopped working around 123.0.6312.106 (chrome build) which released to the stable channel around the time this issue was posted.

when i mention chromium i also mean chrome as chrome is built on top of chromium, so they are practically interchangeable.

the statement above and list below may not be correct as the extension has become unresponsive after a day on the same instance. see my latest issue on 04/07/2024 to read more

Version Working
≤122.0.6261.137 (less than or equal to)
any version inbetween may or may not work
≥123 (greater than or equal to)

YCS on chrome 121, looked the same on chromium 122, worked like it used to image YCS on chrome 123, basically just wouldn't load at all image

this means that the extension Youtube Comment Search is not broken, but the versions/releases of chrome after or on 123 are breaking it.

i will try fixing the extension to work in these new versions, but for now the only solution if you want to use YCS, is to go back/downgrade to 121 or 122.

mauigirl commented 7 months ago

I'm running Chrome 116 on three older laptops and it stopped working on all those installations too, so it's not a v123 issue for me. Same in Firefox. I've turned off other extensions, cleared history, sometimes that will get a few comments or chat to load briefly but never the transcript anymore and even the partial comments don't last long (minutes).

Slowlyness commented 7 months ago

I have the same issue with Firefox 124.0.2 (64-bit). Reload-Button of "comments" is grayed out. Search in comments is not working. I thought it might be an issue with uMatrix etc. which could block a script from a change on the Youtube.com website. But it was not the case. Even when all extensions in FF are disabled, the issue with YCS persists. Unfortunately, I'm not a web developer who could parse the issue deeper. Hope a solution will be found soon. I like the YCS extension => Thanks to the maintainer(-s)!!! 👍

NicPalacio commented 7 months ago

It is very true @xNasuni two weeks ago stopped working for me in Edge and continued to work for me in Chrome but with the update this morning it stopped working in Chrome too which shows that it is a problem with the new chronium.

NicPalacio commented 7 months ago

Hi @xNasuni , how are you, any updates?

I've been reviewing it, but I can't seem to understand the error in the code. The problem is that all browsers have already updated to version 123 of Chromium, so as of now, it's not working on any browser. I also looked into your advice on downgrading Chrome, but I'm not sure how I could do that. I found tools like FileHippo, but the latest version available for download is 104, and YCS doesn't work on that version either. Do you know of any way to specifically use Chromium 122?

RenegadeScooter commented 7 months ago

It's not working for me at all anymore. Brave version 1.64.116, Chromium 123.0.6312.105, 64-bit

ahmed2m commented 7 months ago

Did youtube change the API interface? What's happening, anyone knows how to access the extension's devtools?

xNasuni commented 7 months ago

Hi @xNasuni , how are you, any updates?

I've been reviewing it, but I can't seem to understand the error in the code. The problem is that all browsers have already updated to version 123 of Chromium, so as of now, it's not working on any browser. I also looked into your advice on downgrading Chrome, but I'm not sure how I could do that. I found tools like FileHippo, but the latest version available for download is 104, and YCS doesn't work on that version either. Do you know of any way to specifically use Chromium 122?

I usually use npx to install chromium builds for testing by using the command npx @puppeteer/browsers install chrome@version and in this case I would run npx @puppeteer/browsers install chrome@122.0.6261.137 but I think my original hypothesis was incorrect.

Both versions that I tested it on have stopped working as of currently and nothing I can do gets it to work unless I completely reset the browser which goes inline with one of the reviews that said some of these fixes only work for about a day or so then it stops working.

I guess that's what I am facing with now, using a brand new browser with no previous data or similar fixes it for a day or so and then it breaks again. I will continue trying to find a fix for this, and in the mean time I have sent an inquiry to the developer's personal email and also sent the same email to the contact email for this project ycs.ext@gmail.com which I doubt I'll get a reply from.

ahmed2m commented 7 months ago

I usually use npx to install chromium builds for testing by using the command npx @puppeteer/browsers install chrome@version and in this case I would run npx @puppeteer/browsers install chrome@122.0.6261.137 but I think my original hypothesis was incorrect.

Both versions that I tested it on have stopped working as of currently and nothing I can do gets it to work unless I completely reset the browser which goes inline with one of the reviews that said some of these fixes only work for about a day or so then it stops working.

I guess that's what I am facing with now, using a brand new browser with no previous data or similar fixes it for a day or so and then it breaks again. I will continue trying to find a fix for this, and in the mean time I have sent an inquiry to the developer's personal email and also sent the same email to the contact email for this project ycs.ext@gmail.com which I doubt I'll get a reply from.

Yea, it's not a browser thing it seems, maybe switching versions somehow serves you a different version of youtube web api temporarily because they're using a/b testing for different versions/changes.

A classic thing for such extensions is that YT has done an API change or patched something that the extension was using and likely it can't be solved with either a fix from the author or a fork of the extension with a fix.

NicPalacio commented 7 months ago

Thank you @xnasuni for your response and your work. I will try to find a fix as well, but to be honest, I don't have too much technical knowledge. If anyone discover some alternative or workaround to fetch all comments of a Youtube video with the recently changes, please let us know. I use the extension a lot to search for timestamped comments, most liked comments, and the YouTube experience decreases a lot without it :(

Slowlyness commented 7 months ago

Hi there, For guys who want to test with other browser emulations or older versions of Edge: I know, that in Edge it is possible to emulate other browser/versions etc.

Here's the how to (extract from copilot - I checked it for another purpose last week):

Open Microsoft Edge1. Press Ctrl+Shift+P (Windows, Linux) or Command+Shift+P (macOS) to open the Command Menu1. Type network conditions, select Show Network conditions, and then press Enter to open the Network conditions tool1. In the User agent section, clear the Use browser default checkbox1. Select a user agent from the dropdown list, or enter a custom user agent1

If someone is searching for older firefox versions: click

Source with more information for firefox downgrading: click

Hope it helps someone. 🙂

NicPalacio commented 7 months ago

@Slowlyness Thank you for the info. I tried changing the network conditions, but it only shows the latest version of Edge. Do you know how I can specify a previous version, like 122 or something?"

image

I attempted to install an older version of Firefox, but none of them worked. It's probably because the issue stems from the Chromium version in which the extension runs, rather than the specific version of Firefox itself.

NicPalacio commented 7 months ago

@xNasuni I'm not sure if you've noticed, but the extension loads Chat replay correctly. Perhaps this gives an idea of why it's not able to do the same with comments.

Additionally, I found this other 2 extensions that do a similar job :

Interestingly, they fetch the comments properly and the first one even orders them by likes. The only issue with the first one is that it's a pop-up and doesn't stay on top always, plus the UI isn't that good and additionally I notice that it does not always fetch ALL the comments (sometimes only 70% of the total), while the second one fetches all comments. However, I think reviewing the code of those extensions could provide some valuable clues to help resolve the problem of YCS not fetching the comments.

Slowlyness commented 7 months ago

Do you know how I can specify a previous version, like 122 or something?"

Unfortunately no. Mhhh.

Slowlyness commented 7 months ago

I checked YCS on my FireFox portable version 122.1.1 64-bit on my mediacenter pc. It's still working there. I also check to click the "reload" button for comments. On my mediacenter pc the buttons is labelled "load". After this, the YCS is searching only the transscript of the video. Not sure how it behaved before.

Aldo113A commented 7 months ago

Eugene Gubar

Has anyone yet? This is too valuable an extension to lose

Miwer commented 7 months ago

I saw another post somewhere, that the distributed .xpi files of a firefox add-on, is basically just a zip file. So I got inspired to try to extract it, and I have posted the embedded source code here: https://github.com/Miwer/YCS-firefox-source

The code is compacted into very unreadable format, but I managed to run it through some beautifiers to make it more readable.

Maybe somebody can use it to make a new fork or something like that. wink wink

Enjoy ;)

Aldo113A commented 7 months ago

I saw another post somewhere, that the distributed .xpi files of a firefox add-on, is basically just a zip file. So I got inspired to try to extract it, and I have posted the embedded source code here: https://github.com/Miwer/YCS-firefox-source

The code is compacted into very unreadable format, but I managed to run it through some beautifiers to make it more readable.

Maybe somebody can use it to make a new fork or something like that. wink wink

Enjoy ;)

When's this become a useful replacementz is it when it's Chrome-adapted

drhouse commented 7 months ago

When's this become a useful replacementz is it when it's Chrome-adapted

why not just unzip the YCS Chrome .crx file? here

I'm confused, are recent comments coming from well-intentioned people who don't know how to code and are trying to help? The problem has never been getting the code... it's not behaving the way it's expected, an abnormal problem, it's supposed to work, it's getting the data, but the output isn't consistently reaching the page and no one can figure out why, except that it might have to do with updated Chrome version changes

ImprovedTube commented 7 months ago

hi guys! always planned to integrate this: ImprovedTube#Roadmap:Continuously integrate awesome code &hoping @songiy would join that, since the repo is stating an MIT license)

Lets brainstorm and list/rank more/all alternatives! - In the meantime i'm sending this conversation to the four email addresses belonging to the projects mentioned above.


YouTube Comment Search Extensions

Interestingly, they fetch the comments properly and the first one even orders them by likes. The only issue with the first one is that it's a pop-up and doesn't stay on top always, plus the UI isn't that good and additionally I notice that it does not always fetch ALL the comments (sometimes only 70% of the total), while the second one fetches all comments. However, I think reviewing the code of those extensions could provide some valuable clues to help resolve the problem of YCS not fetching the comments.


Editing www.youtube.com to undo change

Generally, while we often have to bother with the DOM, you might be able to undo some changes to comments with a uBlock rule, such as www.youtube.com##+js(set,yt.config_.EXPERIMENT_FLAGS.kevlar_watch_flexy_comments_manager, false) (no sure which one, if it works with any)

please check what other yt.config_.EXPERIMENT_FLAGS you might get and add them here https://docs.google.com/spreadsheets/d/1GidvMduxTl6jXpDCKj-sOPg8KSqDfCYO2OlCdBADaSI/edit#gid=0 ( @code-charity/contributors )

NewsGuyTor commented 6 months ago

WTF, why/how did this start working again? I see it hasn't been updated, glad to see that it works again though.

tnmdynamiq commented 6 months ago

WTF, why/how did this start working again? I see it hasn't been updated, glad to see that it works again though.

Elaborate on this please. I am still unable to use it. Which version of Chrome & what's what machine?

NewsGuyTor commented 6 months ago

WTF, why/how did this start working again? I see it hasn't been updated, glad to see that it works again though.

Elaborate on this please. I am still unable to use it. Which version of Chrome & what's what machine?

Chrome?! 🤮

I'm using Firefox ofc, 126.0b4.

tnmdynamiq commented 6 months ago

WTF, why/how did this start working again? I see it hasn't been updated, glad to see that it works again though.

Elaborate on this please. I am still unable to use it. Which version of Chrome & what's what machine?

Chrome?! 🤮

I'm using Firefox ofc, 126.0b4.

Shut up! 🤣 Also thank you

tnmdynamiq commented 6 months ago

WTF, why/how did this start working again? I see it hasn't been updated, glad to see that it works again though.

Elaborate on this please. I am still unable to use it. Which version of Chrome & what's what machine?

Chrome?! 🤮

I'm using Firefox ofc, 126.0b4.

My machine (Apple Silicon) will only let me go up to version 125.0.2 & it's still not working so I would imagine that is the reason. Are you running on AMD or Intel? Or are you using Firefox Beta?

EDIT: I just realized the b4 probably means beta my bad

NewsGuyTor commented 6 months ago

WTF, why/how did this start working again? I see it hasn't been updated, glad to see that it works again though.

Elaborate on this please. I am still unable to use it. Which version of Chrome & what's what machine?

Chrome?! 🤮 I'm using Firefox ofc, 126.0b4.

My machine (Apple Silicon) will only let me go up to version 125.0.2 & it's still not working so I would imagine that is the reason. Are you running on AMD or Intel? Or are you using Firefox Beta?

Developer Edition, on Macbook Air M1.

tnmdynamiq commented 6 months ago

126.0b4

Tried on my machine which is essentially the same as yours... No luck.

NewsGuyTor commented 6 months ago

Screenshot 2024-04-26 at 18 19 33

These are my PREF cookie values, perhaps they do the trick.

Miwer commented 6 months ago

Screenshot 2024-04-26 at 18 19 33

These are my PREF cookie values, perhaps they do the trick.

Tried your PREF settings - did not make YCS work for me. Only F5 and F7 are retained when refreshing. F4 and F6 are removed.

Firefox 125.0.1 on Linux

Slowlyness commented 6 months ago

Where to set those values? Can anybody provide a short explanation about the F6 f5 etc.? I've never seen such cookie denotation before and have no clue... 🤪 Thx

Miwer commented 6 months ago

Where to set those values? Can anybody provide a short explanation about the F6 f5 etc.? I've never seen such cookie denotation before and have no clue... 🤪 Thx

It's just a standard cookie for the youtube.com domain called PREF - how to edit it will depend on which browser you use. I use an extension called Cookie Quick Manager.

Screenshot_20240427_100936

Slowlyness commented 6 months ago

OK. Thanks. I understood, that PREF would be the PREFERRED settings... Honestly: I do not believe that a value in a cookie from youtube will have any impact of YCS.

Regarding PREF:

YouTube uses the ‘PREF’ cookie to store information such as a user’s preferred page configuration and playback preferences like autoplay, shuffle content, and player size. For YouTube Music, these preferences include volume, repeat mode, and autoplay. This cookie expires 8 months from a user’s last use.

Source: https://cookiedatabase.org/cookie/youtube/pref/

Don't I remember right that there is a website programmer's security rule, to avoid that one integrated script can access cookies from another script/domain??? I think there was a Java-Flag for this (I'm not a web developer).

NewsGuyTor commented 6 months ago

Well, it works for me, so there's gotta be something about my setup that makes it work. Other than the browser version talk above, that cookie seemed the most likely to impact things. I doubt they do it by IP location.

Do you guys have any idea about where I should look?

It

Miwer commented 6 months ago

... Honestly: I do not believe that a value in a cookie from youtube will have any impact of YCS.

Well, technically, it is indeed possible for youtube to change the layout of the web page, or the workings of the information presented, based on a cookie setting. There has been reporting about the F6=8 setting to get "Youtube classic" back no longer working, which seems to indicate that youtube does in fact have (or at least had) different layouts based on a cookie setting. Now since I do not know the inner workings of YCS, I'm not saying that this is the cause of YCS not working for us, but it very much COULD have the potential to break an extension like YCS. So it's fair to try different settings to see if it makes a difference. It could also just as well be a red herring, and then you would be absolutely correct in your belief. :)

programmer1o1 commented 6 months ago

Hey there, guys. So, this extension is not going to be fixed. So, I figure I will just make another one similar to it. (Do note that you need your own youtube api key for this but I will find if you actually don't need one) Here is a preview of this: image

tnmdynamiq commented 6 months ago

Well, technically, it is indeed possible for youtube to change the layout of the web page, or the workings of the information presented, based on a cookie setting. There has been reporting about the F6=8 setting to get "Youtube classic" back no longer working, which seems to indicate that youtube does in fact have (or at least had) different layouts based on a cookie setting.

Best response I have seen yet!

drhouse commented 6 months ago

Hey there, guys. So, this extension is not going to be fixed. So, I figure I will just make another one similar to it. (Do note that you need your own youtube api key for this but I will find if you actually don't need one)

excellent news. Let us know if there's any estimate on when we might be able to try it or any way we can help.

ahmed2m commented 6 months ago

Hey there, guys. So, this extension is not going to be fixed. So, I figure I will just make another one similar to it. (Do note that you need your own youtube api key for this but I will find if you actually don't need one)

@programmer1o1 Any news? Are you developing it in public (preferred, so that community can contribute especially if you're going to make the user input their token then it should be public code for sure) - check out this project https://github.com/raj-khare/yt-migrate it uses the whole youtube API and has documentation/guide on how to do so for the end users (GCP app etc..) but it's not an extension.

On a side note, I think this YCS extension creator just reverse-engineered the youtube components requests in the same tab and injected elements that utilize the same personal tokens for the same endpoints to scroll through the comments' pages and harvest for search. But the author is no longer active for some reason and it broke due to an API change so the served data structure was changes (for the comments calls) and no proper error handling was put in place hence when one failed all the other functionalities broke with it.

programmer1o1 commented 6 months ago

Hey there, guys. So, this extension is not going to be fixed. So, I figure I will just make another one similar to it. (Do note that you need your own youtube api key for this but I will find if you actually don't need one)

@programmer1o1 Any news? Are you developing it in public (preferred, so that community can contribute especially if you're going to make the user input their token then it should be public code for sure) - check out this project https://github.com/raj-khare/yt-migrate it uses the whole youtube API and has documentation/guide on how to do so for the end users (GCP app etc..) but it's not an extension.

On a side note, I think this YCS extension creator just reverse-engineered the youtube components requests in the same tab and injected elements that utilize the same personal tokens for the same endpoints to scroll through the comments' pages and harvest for search. But the author is no longer active for some reason and it broke due to an API change so the served data structure was changes (for the comments calls) and no proper error handling was put in place hence when one failed all the other functionalities broke with it.

Hi, I am still developing it but I am busy as I have something to do, but I will eventually provide the source code if I have enough time to do so. Thanks for providing the project link for guide.

5koder commented 5 months ago

The extension works for me now. I had it disabled for a couple of months. I removed it today and then installed it again. I went to test if it was working and it to my surprise, it was. Only thing that doesn't seem to work is the "Transcript video" but I never needed that anyway.

I'm on Firefox Developer Edition v127.0b6.