Possible Issue With Data Request Throttling

playlaughlovelearn commented 9 years ago

Possible Issue/Bug

Data Gathering for Thumbnail Data Overlay Is Not Throttled to a level consistent with Google TOU #4 Section H

Notes

Recently my channel PlayLaughLoveLearn was terminated by YouTube for a violation of their Terms of Use; that of causing requests to be sent to their servers more quickly than a human could do in normal usage using a web browser.

Unfortunately I have not been able to (nor do I expect to be able to) get an answer as to whether it was one of two things I suspect [YTC or a large Twitch.TV export] or some other thing that I am not aware of.

However, I did have at least a few times when I got "retreiving data too fast" (or something like that) errors on the thumbnail overlays. and thought little of it. Unfortunately, I do not remember the exact wording, and I do not care to reproduce the errors as I do not wish to risk having another account terminated, nor having my entire account terminated rather than a single YouTube account on my Google account.

I did notify Twitch, and I am also letting you know, just so you can take appropriate steps, if necessary, to throttle all requests to meet the "human while using a web browser" requirements. I doubt there is anything that can ever be done for my account at this point; they have denied my request several times, so at this point my main concern is that it not happen to anyone else.

Again I am not even sure that it was due to YTC, I actually suspect it could have been a combination of the Twitch.TV export plus the YTC usage, as searches do not produce any hits for either of them alone regarding account terminations. However it could have been that both together went over some set amount that ended up triggering something; either way, I didn't figure it would hurt to report it here.

Final note

This is also the section of the TOU that is used to remove videos and terminate accounts for purchased views; however, I checked my final socialblade stats and I am not sure whether the view counts on those days are enough that it would cause a problem or not... (somewhere around 50 per day). This does seem inconsistent with how many I would have expected, but, of course, I have no way of tracking where that traffic came from, so I have no way to examine it.

Regardless, the messages I had received saying I was getting data "too fast" seem to indicate a problem one way or other.

(To clarify: I did not purchase views, but a quick Google search on TOU #4 Section H let me to lots of hits on the problems that have been had with this policy: many accounts at this point have indeed had their videos removed/accounts terminated due to this policy that likely have never purchased views.)

there are many people Thank you.

Yonezpt commented 9 years ago

I really doubt YTC is responsible for your account termination for one simple reason: Even if you are not using any account you can still use YTC with the same features that you suspect of being responsible for your account termination, which completely defeats the purpose of account termination as a means to stop this.

It would be best if you posted your issue in their forums instead: https://productforums.google.com/forum/#!forum/youtube

playlaughlovelearn commented 9 years ago

Well, agreed that it is unlikely; however, I did several times get the message saying "too fast" so I still think this should be an open issue. Shall I cull the termination-related stuff from the report?

Yonezpt commented 9 years ago

The message you received is the default Youtube data api mechanism that prevents data request overloads, it is what is used to avoid users from requesting too much data at once or during a short period of time. Since this exists then there is absolutely no way that an account could be terminated due to the user doing too many requests. If that protection did not exist then maybe it would be possible for them to terminate accounts in order to prevent service abuse, but that would still be useless unless they also banned the IP.

The reason for your account termination is related to something else, in which case you will have more luck of finding what was it if you post in the youtube forums and with some luck you will be able to recover your account.

If I am not mistaken, unless in very extreme cases, first-time account terminations are temporary, so if you still don't manage to recover your account it is possible that in a few months (can't tell you how many months exactly) your account will be restored, so try to use it every month or so to check wether it is working again or not.

playlaughlovelearn commented 9 years ago

Okay, thank you for your response; I will try those things. It's nice to know that there's any chance as it was very disheartening since I had never knowingly done anything that would trigger a violation; so it was just like a mystery thing that happened to me and not even a way to avoid it. I wouldn't have even posted an issue if I hadn't seen those messages and had so few ideas as to what had happened, but as you clearly have a lot more knowledge of policy, etc. I will defer to your judgement on the matter.

However, even assuming that everything is kosher with YouTube once those messages are appearing, I don't want to ever be in a situation where they were reviewing somet activity on my account and saw then for them to see this slew of notes in their logs saying (obviously, paraphrased) "User ABC with IP W.X.Y.Z: Request Rejected Due to Requests Being Sent Faster Than Allowed" on top of whatever else was being reviewed...

At best it couldn't help. So, as it stands, I am very uncomfortable about it. I don't intend to continue to argue the point, but please let me make one final attempt at giving the best possible statement of, and reasoning behind, my opinion that this is indeed a bug.

In #4, Section H, of The YouTube Terms of Service, a.k.a., the Community Guidelines, a.k.a the TOU, you agree

not to use or launch any automated system [...] that sends more [...] request messages to the YouTube servers in a given period of time than a human can reasonably produce in the same period by using a conventional on-line web browser.

As it's not well defined, it seems that YouTube Center would meet the definition of an "automated system", and the way YTC was structuring its requests to YouTube in at least some cases (e.g. requesting data for thumbnails of videos) caused me on multiple occasions to get some sort of "data requested too fast" rejection messages from YouTube instead of the data that was requested.

Though I cannot be sure of the reason, as none was displayed (given?), YouTube did reject the requests, and it seems reasonable to believe [given that the content of the rejection stated that requests were coming too quickly] that it's automated rejection came due to there being more requests generated that this is because there were more requests sent to YouTube for data in a given amount of time than it allows...

Regardless of the particular trigger used in its automated system to determine that a request should be rejected for multiple requests coming in too quickly, getting such a message indicates that YouTube Center sent more requests to YouTube in a given period of time than was allowed.

As YouTube is quite unlikely to consider any request rate less than "a human can reasonably produce [,..] using a [...] web browser" to be too fast; it seems clear that YouTube Center, on such occasions (and perhaps others in which YouTube simply did not flag the behavior) has, in fact, caused the user to violate the section of the Terms of Service quoted above; regardless of whether YouTube considers such violations significant; takes any action; suspends or terminates accounts; etc.

Also, it is worthy of note that YouTube does not, in general, answer the very questions that would actually answer whether YouTube Center were ever involved in an account suspension/termination; unless someone [has gotten/can get] a confirmation directly from YouTube, on the record, that such access is not a problem, there is no way to say with any certainty at all that YouTube Center was not involved in any particular situation.

As YTC is a tool intended to be used by individuals to access YouTube, while logged into their accounts, despite your indications of its lack of usefulness as a metric for suspending accounts, etc.; it is not proper for such a tool to either be in a position of ignorance in relation to (or of knowingly violating) the Terms of Service for the service that the tool is specifically intended to be used for.

Given these factors, it seems that is would be reasonable to consider the fact that YTC does not throttle requests to YouTube in a manner consistent with TOU #4 Section H as an issue that should be mitigated to avoid any risk to users that might be therefore put in violation of the TOU as a result of using YTC.

Possible mitigations that I see might be one of these, or some combination:

At a minimum: clearly warning users that the tool can and will put them in violation of the TOU; in fact, repeatedly, should they browse to multiple pages with multiple thumbnails after the errors occur. (In fact, even if the requests are honored, and data is returned, if the data is requested faster than indicated in the mentioned Section of the TOU, they still will technically be in violation of the TOU; and upon reflection it seems unwise to ever do something like this.)
Okay, thank you for your response; I will try those things. It's nice to know that there's any chance as it was very disheartening since I had never knowingly done anything that would trigger a violation; so it was just like a mystery thing that happened to me and not even a way to avoid it. I wouldn't have even posted an issue if I hadn't seen those messages and had so few ideas as to what had happened, but as you clearly have a lot more knowledge of policy, etc. I will defer to your judgement on the matter.