Open D3SOX opened 2 years ago
according to the discord the backend code will be uploaded once api keys are cleaned out of it
Three days later, the backend's source is still closed :/
according to the discord the backend code will be uploaded once api keys are cleaned out of it
That's strange. Normally you don't store api keys in the source code but rather in .env
files. Seems a bit suspicious to me.
Since the maintainers are not even giving an explanation everyone can see and are closing issues directly, I am not anymore going to support this project and rather help the archiveteam archiving dislikes. As already said here, this extension willl probably only have a few thousand videos archived, once Youtube shuts down the api, while the archiveteam already has stored around 60 million videos so far (see http://tracker.archiveteam.org/youtube-dislikes/)
@Myzel394 I kinda feel the same and also understand the opinion Brodie Robertson shared in his video This with the fact that they always close such issues with no comment made me uninstall the user script a while ago.
I just hope that the extension will never do anything malicious, and the server doesn't log IPs with videos watched or something like that.
@Anarios why did you close this issue without answering? What makes you want to hide the backend source code?
Update from the dev: https://addons.mozilla.org/fr/firefox/addon/return-youtube-dislikes/reviews/1789930/
The issue was closed as a duplicate - check the existing issues before opening a new one. Sorry if I failed to link the duplicate.
The backend code is not yet ready to be made publicly available.
We don't know what informations are kept.
Everything that is being sent by the extension (you can see it in network tab). Unique (but random and non-identifiable) userId, videoId that you voted for and your vote.
Privacy policy is coming. It's just that I'm doing this mostly alone, and writing a huge legally-binding text in English takes some time.
Sorry, there were at least 3 clones of this, so this one got closed.
Since it has the longest discussion thread, I'll reopen it
The main issue right now is that despite using micro-tasks as proof of work and other measures - API is quite vulnerable to botting. There is no good way around it (there is a huge related discussion, that concludes with nothing)
So, despite security through obscurity being the worst imaginable approach - I'm keeping backend closed for at least few more weeks.
If your main concern about disclosing backend code is your own privacy - I have to assure you that disclosed sources do nothing for your privacy, except a false sense of security - you could not know which code actually runs on the backend.
I start to think that in the end - we'll have to end up with authorization through google oAuth to verify votes. Though I don't want to do it - it would discourage too many users.
If your main concern about disclosing backend code is your own privacy - I have to assure you that disclosed sources do nothing for your privacy, except a false sense of security - you could not know which code actually runs on the backend.
This applies to all open-source servers. We don't know if Mozilla ships Firefox with the given code, we don't know whether Signal is built with the given code, we don't know if Android is compiled with the given code, etc.. However, at least we have the possibility to build everything for ourselves. And if we're suspicious that you are doing nasty things with our data, we can simply host our own server.
API is quite vulnerable to botting
While I understand this concern, it is also a statement that can be applied to basically every open-source project (see Invidious, Nitter, Signal server code, Whoogle, etc.).
If you could at least share the database with the dislikes, that would make you look a little bit better. And if you can't handle the server code on your own, you should seek out to trustworthy people that can help you.
we can simply host our own server.
You wouldn't have the data, though. And even if took a published db dump - it would get outdated.
And if you can't handle the server code on your own
I didn't say I couldn't handle it.
that would make you look a little bit better
and what makes me look bad? Providing a free service with open source client code and a pledge to disclose backend code when it's ready?
By the way - a bunch of copycat-extensons died today once I enabled IP rate limiting.
They were just calling my api in their backend - no own DB, no caching - nothing. Just pretending to provide a service while in reality they didn't. Now imagine they had a DB dump and server code - what good would it make - more userbase fragmentation, less reliable votes? And all while using my work for free.
By the way - a bunch of copycat-extensons died today once I enabled IP rate limiting.
That's a decision I support! Because unfortunately, many copycat-extensions include malware or tracking. However, your extension is on place 1 in both the Chrome & Firefox AddOn store.
By the way - a bunch of copycat-extensons died today once I enabled IP rate limiting.
While there will always be bad guys everywhere, it would do much more good than it would do bad. The archiveteam already helped archiving dislikes and for archiving purposes, it would be much better if you share your archived dislikes. Sure, there will be a slight chance that fake extensions will use this database to scam people, but if you're really worried about people getting scammed, we'd have to shut down the internet.
oh and btw:
They were just calling my api in their backend - no own DB, no caching - nothing. Just pretending to provide a service while in reality they didn't
The archiveteam already provides a db dump (with probably contains much more dislikes than you have archived) and these extensions didn't bother loading them, so I strongly doubt they'd do it if you share your dump.
The archiveteam already provides a db dump (with probably contains much more dislikes than you have archived) and these extensions didn't bother loading them, so I strongly doubt they'd do it if you share your dump.
And it's already a few weeks olds, and users complain when dislikes are delayed by even an hour.
By the way - a bunch of copycat-extensons died today once I enabled IP rate limiting.
Fun thing - when I search for it, a dysfunctional copycat is above mine, despite having a worse score and download numbers.
it would be much better if you share your archived dislikes.
I shared every ID with archive team - so all my videos are archived by them as well.
Now imagine worst case scenario - say, I was as evil as it gets - what could I track? A random ID that you can regenerate at any time and an IP that can be dynamic\behind NAT\behind VPN.
So if you use a dynamic IP or a VPN - there is nothing I can track. This would be a real solution to privacy concern. Unlike a non-solution of posting server sources.
There is a similar discussion #344 and since it's very related - I cross-post some messages.
Fun thing - when I search for it, a dysfunctional copycat is above mine, despite having a worse score and download numbers.
When I searched for it in a new incognito tab (no extensions loaded), it was all on place 1. I guess in your case it's based on your search and view history probably.
I shared every ID with archive team - so all my videos are archived by them as well.
Sorry, but there is not proof that you did this. I know this sounds harsh, but since this is a sensitive topic, we need to make sure that the given information is correct.
So if you use a dynamic IP or a VPN - there is nothing I can track. This would be a real solution to privacy concern. Unlike a non-solution of posting server sources.
A VPN doesn't matter in this case since we all have an unique id generated.
Now imagine worst case scenario - say, I was as evil as it gets - what could I track? A random ID that you can regenerate at any time and an IP that can be dynamic\behind NAT\behind VPN.
Please don't get me wrong. I really like the idea of the extension and would love to support it. However, based on your recent activities this extension really seems a bit suspicious to me (closed issues without any explanation, no source code but advertised, generic arguments; also, your question "what could I track" - while it could really be that you don't know what you could track, such questions are often used to make the other person silent, since most people don't know either what you could track, since they don't know that much about tracking. You could track what each unique id watches. You could know exactly who watched what video and probably if they watched it completely or are currently jumping from video to video)
Sorry, but there is not proof that you did this.
I don't see a reason why I would even need to provide proof, but okay, lets play this game - you could ask a member of archive team Jopik, he took dumps from me.
A VPN doesn't matter in this case since we all have an unique id generated.
And how would it get tied to you by someone who wants to use it, for example, to show you ads?
while it could really be that you don't know what you could track, such questions are often used to make the other person silent, since most people don't know either what you could track, since they don't know that much about tracking. You could track what each unique id watches. You could know exactly who watched what video and probably if they watched it completely or are currently jumping from video to video)
Actually the words that you disliked so much are words of Ajay (SponsorBlock author) used regarding this extension.
You could know exactly who watched what video
The point is that I don't. I know a random id associated with it, and, in worst case scenario - an IP - but how do I tie it to a user?
Again - if you're paranoid - use VPN and reset extension storage from time to time.
closed issues without any explanation
Because there were 3 clones.
generic arguments
Which specifics do you need?
no source code
Do I owe it to you or to anyone? Are you in a position to demand me to share my work just because a stranger on internet asked me to? Sharing source code would change nothing for the privacy of the users. So stop using this argument as if it means something.
The only ones for whom it would change something are the copycats and the ones who want to affect votes. Are you, by any chance one of them?
such questions are often used to make the other person silent,
No, the question was asked for you to think what value I could actually get.
Because there were 3 clones.
"Duplicate of #issue" and then you close
Solved :)
Because there were 3 clones.
"Duplicate of #issue" and then you close
Solved :)
yep, I apologized for failing to do so. And reopened this one since it had longest discussion
I will quote Ajay here.
from what I understand from offers I have received while running sponsorblock, there are only two things you can sell
- personal watch history associated with some id that can be related to a user account on another service (which won't happen here) OR
- anonymous view paths. IE. after watching this video, what do you watch next
for 2, they really need proof that there actually is a referral. so, I believe the data collected wouldn't be too much use to sell
so, he shares my opinion that there is nothing to sell from what I have here. If you have a different opinion - I'm happy to discuss. WIth actual arguments, please. And not baseless accusations.
So if you use a dynamic IP or a VPN - there is nothing I can track. This would be a real solution to privacy concern. Unlike a non-solution of posting server sources.
So you are telling people to get a VPN or change ISPs?
an IP - but how do I tie it to a user?
By comparing the watch history to other data, like last.fm or comments under the video.
So you are telling people to get a VPN or change ISPs?
I didn't suggest changing ISPs, but if you're really concerned about exposing your IP - yes, VPN is the only real solution.
By comparing the watch history to other data, like last.fm or comments under the video.
Sounds like bs to me. Explain in detail - 1. where would I get last.fm data or data on comments. 2. How would I tie it to watch history that I have.
Sounds like bs to me. Explain in detail - 1. where would I get last.fm data or data on comments. 2. How would I tie it to watch history that I have.
A new video appears. Few views, 1/10 as many comments. Comments at the artist's last.fm page from a similar username, the artist is in the last.fm user's top artists (list for a random user who left a comment at a popular artist's page). Or a user watches artists strongly intersecting with a user at sites like last.fm, libre.fm or Bandcamp. Sure, you could say it was stupid of them to use those sites in the first place and not change their tastes over time…
Sounds like bs to me. Explain in detail - 1. where would I get last.fm data or data on comments. 2. How would I tie it to watch history that I have.
A new video appears. Few views, 1/10 as many comments. Comments at the artist's last.fm page from a similar username, the artist is in the last.fm user's top artists (list for a random user who left a comment at a popular artist's page). Or a user watches artists strongly intersecting with a user at sites like last.fm, libre.fm or Bandcamp. Sure, you could say it was stupid of them to use those sites in the first place and not change their tastes over time…
Extension doesn't see your comments on youtube. To tie a random like on youtube to a random account on last fm by his preferences in music would be a miracle, to say the least. And all it would give me - just a last fm username, which is useless. Man, you're imagining something extremely weird. I could give you all my likes on youtube and give you a thousand dollars if you find my account on last fm from that data.
but okay, lets play this game
This is starting to get emotional. I'm sorry if I offended you in any way, but it feels like that you are very angry about this discussion (especially https://github.com/Anarios/return-youtube-dislike/issues/45#issuecomment-997272796) and I think such important discussions should be discussed objectively.
You said you don't want "baseless accusations":
And not baseless accusations
but did exactly that:
Again - if you're paranoid
and
The only ones for whom it would change something are the copycats and the ones who want to affect votes. Are you, by any chance one of them?
So, you can decide to respond or not, as I'm not gonna continue this discussion anymore (for some time). However, here's my point summarized:
I would like you to make the source code for the backend open source. While the data (that could be collected) may not be much, it is still valuable data. You can of course always say: "Use a VPN if you don't want to get tracked" - but that's often not feasible for normal people (and normal people often don't want to pay money for anything) - so a developer should take the responsibility to use the least amount of tracking as possible.
You're completely right that we still wouldn't know if your server is really running the code you are hosting on GitHub. However, we could host our own servers and then we would know that we were safe.
I understand that keeping some code private to avoid spambots is not a bad idea, but the rest of the code should be open for everyone.
This all also seems a bit suspicious since you closed issues asking about the backend code directly without giving any reasons why (yes you did give an explanation a few hours ago: https://github.com/Anarios/return-youtube-dislike/issues/45#issuecomment-997273868, but why so late?). Also, you're giving reasons why not to make the source code open source that are typical arguments and have been debilitated often:
what could I track? A random ID that you can regenerate at any time and an IP that can be dynamic\behind NAT\behind VPN.
Asking "What could I track" and then saying what you have is like Facebook (Whatsapp) saying: "We don't know your chat content, so what can we track?" - The normal guy would say that Facebook isn't tracking anything. However, only later did we find out that it's so much more.
Providing a free service with open source client code and a pledge to disclose backend code when it's ready?
Here you are trying to make you look innocent by making me look like the bad guy that's trying to destroy the little guy who just wants to be altruistic.
Do I owe it to you or to anyone? Are you in a position to demand me to share my work just because a stranger on internet asked me to? Sharing source code would change nothing for the privacy of the users. So stop using this argument as if it means something.
First part is emotional; Second part would indeed change something (see my text above)
This is starting to get emotional. I'm sorry if I offended you in any way, but it feels like that you are very angry about this discussion (especially #45 (comment)) and I think such important discussions should be discussed objectively.
Yes, I got annoyed by the " that would make you look a little bit better." part, that is plain rude. To make me look "a little bit better" - you'd have to make me look bad in some way first.
I would like you to make the source code for the backend open source.
Please explain how it would make user privacy better (while using this extension).
However, we could host our own servers and then we would know that we were safe.
Yes, and I don't want it to happen, thanks. It's my work, I'm not obliged to give it out for free. It would be giving out my work, to make my own project work worse - kinda counter-productive, don't you think?
Asking "What could I track" and then saying what you have is like Facebook (Whatsapp) saying
That was an honest question. To which you still didn't reply constructively.
You said that you don't want to get emotional, and your last comment brought only emotions, and zero discussion of technical aspects.
but did exactly that: Again - if you're paranoid use VPN
How is that an accusation? It's an advise in case you worry about exposing your public IP. Actually - the only advise that actually works.
as I'm not gonna continue this discussion anymore?
Because you have zero constructive arguments. You didn't even come up with any kind of data that could be sold.
so a developer should take the responsibility to use the least amount of tracking as possible.
And I did. Which can be clearly seen in frontend code.
but why so late?
Because I don't actively track every issue? Because you can still reopen the issue if it bothers you so much and I would look at it again? Because my time is limited, and today, instead of new in-app reporting system for bugs in extension - all users get is this and couple other conversations that I spen most of my day on?
Extension doesn't see your comments on youtube.
Somebody else might, and the database might leak (especially if there are logs).
To tie a random like on youtube to a random account on last fm by his preferences in music would be a miracle, to say the least.
It certainly is not easy, and not possible for a non-programmer like me, but I'm pretty sure it doesn't take a Google to do it if you know what to look for and the user is playing most videos on YouTube itself. If you make a list of a videos a specific person is likely to like, you can compare that list to entries in the database. #72 and a quick on/off switch would make it harder.
And all it would give me - just a last fm username, which is useless.
That's like the government saying some data is not personally identifying information because it is not stored with the person's full name in a particular database.
Extension doesn't see your comments on youtube.
Somebody else might, and the database might leak.
To tie a random like on youtube to a random account on last fm by his preferences in music would be a miracle, to say the least.
It certainly is not easy, and not possible for a non-programmer like me, but I'm pretty sure it doesn't take a Google to do it if you know what to look for and the user is playing most videos on YouTube itself. If you make a list of a videos and music a specific person is likely to like, you can compare that list to entries in the database. #72 and a quick on/off switch would make it harder.
And all it would give me - just a last fm username, which is useless.
That's like the government saying some data is not personally identifiable because it is not stored with the person's full name in a particular database.
okay, lets imagine what you said is true. Your suggestions?
okay, lets imagine what you said is true. Your suggestions?
72 and a quick on/off switch would make it harder.
on/of switch of entire extension is in extensions tab.
If you don't wan't to submit your votes to extension - a PR for this is already open. There will be a toggle. Which would make your votes not counted, though.
If you make a list of a videos a specific person is likely to like, you can compare that list to entries in the database
you would find 3 users out of a million. And ONLY their last fm name. I wonder - how much could I sell dislikes stats for 3 users, tied only to their lastFM username.
72 only works when you requests dislikes for a video (i.e. watch it)
That's significant. One could find that somebody watches something "bad" (without rating it) and get him cancelled, fired or something.
on/of switch of entire extension is in extensions tab.
The Firefox UI scrolls imperfectly, kinda annoying. But I guess it's good enough if you move the toolbar button to a prominent location.
If you don't wan't to submit your votes to extension - a PR for this is already open. There will be a toggle. Which would make your votes not counted, though.
OK.
One could find that somebody watches something "bad" (without rating it) and get him cancelled, fired or something.
One can't find out who this user is, though. It's a random id, and all you know about this ID - is it's watch history, and it's likes. You can't identify a person from it. I could give you data for my ID and would love to see you do anything with it at all.
Actually, I can post all data that's stored about me - you might see how useless it is. Give me a sec.
VideoId UserId Value Challenge IsConfirmed
Zd3T4yAuYoE yGYMR9EvtInAj7OQK6jbzJFhZSZpBC9Wv2TN 0 >ÙiV$ªæz «emáÜC true aA0KgaW513g gVLAvO1g4jW1orh6x4g2HZWsPdG2oT6XqOPz 1 y¶Â0rGuo°J C true 7zyswFi777E SoDIcqahMOSrujEtnQV6npZItspmPYt9RaaJ 1 S÷¥ þ ) Ñ÷ ® ñ true rojZqvG7ElE q6jQRI5whvtn1a13y2bAwhUkbkSIOahTgT4R 1 b æµ Y\ Ǹñ| true D_Q_6V10mTU xFyhcktJtgepNqSqRXCOZsXUtS3NgUTkvy6K 1 tNT`ô+?®kÞ[$1× true Qr_eFaYMoXw xFyhcktJtgepNqSqRXCOZsXUtS3NgUTkvy6K 1 $òYhM ~ÖÕ ð¦ true IUqQ_rVnSXg xFyhcktJtgepNqSqRXCOZsXUtS3NgUTkvy6K 1 Þðé §§ Ï MÊÐ ¨ true 9EgAc8fXU2o xFyhcktJtgepNqSqRXCOZsXUtS3NgUTkvy6K 1 ª½Ñ( °» äÃUT» true e6IhsHnVUGA xFyhcktJtgepNqSqRXCOZsXUtS3NgUTkvy6K 1 Õ ¨vu&BwB å ± true ReDXHIf6cuc xFyhcktJtgepNqSqRXCOZsXUtS3NgUTkvy6K 1 <+ ô©yåÆ=ô¤ W true 4HhK0DJmbP8 xFyhcktJtgepNqSqRXCOZsXUtS3NgUTkvy6K 1 T : í ë/âÚ®^Óo false
Multiple UserIds because I was testing some things.
Do you think this is, by any means, sellable? If you want to get somebodies interests - you can just parse their subscriptions. That would give you way more info
Actually, I can post all data that's stored about me - you might see how useless it is. Give me a sec.
Do you have no access logs?
When you retrieve a video stats by id? No. Only in-memory storage for rate limits
I think I just came up with a solution. I will confirm with everyone in #72, but it should work even better.
Lets make extension never request a single video id. It is planned to display a ratio bar near every thumbnail. So why not request all videos at once, therefore making the watch history unavailable for server (because server will never know which single video out of 50 requested you actually watch).
Randomize id order before request, and voila
One can't find out who this user is, though. It's a random id, and all you know about this ID - is it's watch history, and it's likes. You can't identify a person from it. I could give you data for my ID and would love to see you do anything with it at all.
You can get somebody to watch an unpopular video. If acting purposefully, you can get them to watch a video they will dislike immediately.
When you retrieve a video stats by id? No. Only in-memory storage for rate limits
Oh, that's good. If it's encrypted, it might be safe for a time (with exceptions you can guess).
Does it not save the likes made without the extension on the videos you watch with it?
You can get somebody to watch an unpopular video. If acting purposefully, you can get them to watch a video they will dislike immediately.
this would give you one user. And you need millions to sell something of value :) Think scale, not a single user.
If it's encrypted, it might be safe for a time (with exceptions you can guess).
Encrypting a public IP that's stored only in memory seems excessive. I honestly doubt anyone at all does that.
this would give you one user. And you need millions to sell something of value :) Think scale, not a single user.
Not to sell as big data, but to
get him cancelled, fired or something.
Remember what SJWs do to cancel people. In case of Richard Stallman (who is, of course, unlikely to use your extension), they used minor things from decades ago and turned them into slander they sent to mass media in order to sway opinions. It seems to have taken them over a year to have a significant effect in his case despite a concerted attack, but small people are not as immune to much smaller things.
Encrypting a public IP that's stored only in memory seems excessive. I honestly doubt anyone at all does that.
Encrypting the video id sent over the network (that is, using something like HTTPS).
Encrypting the video id sent over the network (that is, using something like HTTPS).
yep, https is used by default.
(who is, of course, unlikely to use your extension)
Who knows :D
This all also seems a bit suspicious since you closed issues asking about the backend code
Wouldn't I delete the issue and block all participants, if I had a malicious intent? Instead I reopened it as soon as I saw the discussion
Totally agree Anarios on the privacy matter. Only the client-side (extension) source code needs to be inspectable to address any privacy concerns. You should always assume whatever data the frontend is sending is logged by the server. If any service sends sensitive data and then promises you privacy on the server side, don't trust them - even if the server source is open
But even so, I would like to see the server code open-sourced (eventually - once the hype dies down), if only for academic reasons. If botting remains a big concern, maybe don't share the anti-bot code. The other parts of the code are bound to be interesting nonetheless
So why not request all videos at once, therefore making the watch history unavailable for server (because server will never know which single video out of 50 requested you actually watch).
Perhaps a hash can be send similar to the SponsorBlock API https://wiki.sponsor.ajay.app/w/API_Docs#GET_.2Fapi.2FskipSegments.2F:sha256HashPrefix
As for when voting, some sort of unique identifier has to be recorded to prevent botting. I don't believe there is any way out of this (and no, sharing server source code still doesn't help).
As for the IP, whenever you are making any network request to any website/service, you are always sharing your IP with them. There is nothing to be done about it other than blindly trust that they are not logging it. If you are using a VPN, you will not be giving your IP to the websites, but will still give it to the VPN provider. If you really care about not leaking your IP in a zero-trust environment, using a Tor network is the only thing you can do
Because there were 3 clones.
"Duplicate of #issue" and then you close
You are being entitled here. It is the user's responsibility to check for duplicate issues before opening a new one. How many times do you expect the maintainers to find and link duplicate issues for you, just because you are too lazy to search for them yourself? If any maintainer does so, it is a courtesy, not your right to demand it!
You are being entitled here. It is the user's responsibility to check for duplicate issues before opening a new one. How many times do you expect the maintainers to find and link duplicate issues for you, just because you are too lazy to search for them yourself? If any maintainer does so, it is a courtesy, not your right to demand it!
The thing is I actually did search, and the original one had a bad title, so I was unable to find it
It is the user's responsibility to check for duplicate issues before opening a new one.
To be fair - I messed up here, and I think this is the oldest issue for disclosing backend code (or at least the oldest I could find with a quick search right now), so I shouldn't have closed it.
But I didn't think it would cause any fuss - because it can always be reopened, by me or by the user who created it.
Also it was a very intense period where I needed to do a lot of things to keep the extension alive under exponentially growing load. Solutions which were perfectly fine for 10k users didn't work so well on 100k, and then again when we started closing to a million. It was always a race to retrieve enough reliable data for all the users, and to keep servers alive under the growing load.
So I might have acted without thinking it through.
"ah, another "disclose sources" issue, I think I've seen 5 of them, i will just close this one and finally go to bed after 40 hours without sleep" was my line of thought at the moment when I was closing it (I actually remember it quite well)).
So yeah, as I said multiple times - no foul play was involved here. And there are much better ways to actually hide somebody's opinion if I didn't want it to be seen.
@Anarios I just wanted to chime in on your concerns with giving other people your code for free. I really do understand that you want to keep this as your thing and prevent copycats that possibly scam people, but the benefits of open-sourcing far outweigh the drawbacks. You create trust among users and creators and allow others to contribute (and with a project as popular as this one, no doubt there would be contributors).
If it were the case where the integrity of the project is harmed by open sourcing, then why is it that so many successful products have open-source backends like Canvas, for instance?
thanks for responding to my LTT youtube comment btw :)
If it were the case where the integrity of the project is harmed by open sourcing
It really depends on the state and stage of the project. If I know that all it takes to tamper with video scores is just reading the code of backend - can I disclose it right now?
As I previously mentioned - as soon as we switch to oAuth and votes are somewhat safer from tampering - disclosing backend is a must.
Extension or Userscript?
Both
Request or suggest a new feature!
I want to suggest open-sourcing the backend so that people can self-host it and also maybe contribute to it.
Ways to implement this!
Release the source code on GitHub
Can you work on this?