Closed kelek- closed 7 years ago
Check the comment here -
https://github.com/yadayada/acd_cli/commit/654871759c2e9b5a829926c8c11f9f382f2c0a3d
thank you @madyoda
I was trying to follow what happens next and next thing I see is a big 404 🙄
Yeah I was having issues last night and today as well.
acdcli sync
Traceback (most recent call last):
File "/usr/local/bin/acdcli", line 11, in <module>
sys.exit(main())
File "/usr/local/bin/acd_cli.py", line 1338, in main
acd_client = client.ACDClient(CACHE_PATH)
File "/usr/local/lib/python3.5/dist-packages/acdcli/api/client.py", line 31, in __init__
self.handler = oauth.create_handler(path)
File "/usr/local/lib/python3.5/dist-packages/acdcli/api/oauth.py", line 24, in create_handler
return AppspotOAuthHandler(path)
File "/usr/local/lib/python3.5/dist-packages/acdcli/api/oauth.py", line 144, in __init__
self.load_oauth_data()
File "/usr/local/lib/python3.5/dist-packages/acdcli/api/oauth.py", line 88, in load_oauth_data
self.get_auth_token(reload=False)
File "/usr/local/lib/python3.5/dist-packages/acdcli/api/oauth.py", line 103, in get_auth_token
self.refresh_auth_token()
File "/usr/local/lib/python3.5/dist-packages/acdcli/api/oauth.py", line 183, in refresh_auth_token
'Error refreshing authentication token: %s' % response.text)
acdcli.api.common.RequestError: RequestError: 1004, Error refreshing authentication token:
<html><head>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>503 Over Quota</title>
<style><!--
body {font-family: arial,sans-serif}
div.nav {margin-top: 1ex}
div.nav A {font-size: 10pt; font-family: arial,sans-serif}
span.nav {font-size: 10pt; font-family: arial,sans-serif; font-weight: bold}
div.nav A,span.big {font-size: 12pt; color: #0000cc}
div.nav A {font-size: 10pt; color: black}
A.l:link {color: #6f6f6f}
A.u:link {color: green}
//--></style>
</head>
<body text=#000000 bgcolor=#ffffff>
<table border=0 cellpadding=2 cellspacing=0 width=100%>
<tr><td bgcolor=#3366cc><font face=arial,sans-serif color=#ffffff><b>Error</b></td></tr>
<tr><td> </td></tr></table>
<blockquote>
<H1>Over Quota</H1>
This application is temporarily over its serving quota. Please try again later.
<p>
</blockquote>
<table width=100% cellpadding=0 cellspacing=0><tr><td bgcolor=#3366cc><img alt="" width=1 height=4></td></tr></table>
</body></html>
https://tensile-runway-92512.appspot.com/ shows: Error Over Quota This application is temporarily over its serving quota. Please try again later.
Any news on when this will be fixed?
Yadayada silence, turning off issues page, and not explaining anything, makes people abandon this so good project.
Why this silence? No fix date? I dont understand this attitude.
No news from Yadayada ? Hope amazon will unban acdcli otherwise my whole stuff is gone :(
@yadayada can you give us any more information? If there is a serious security issue with this package that cannot be resolved, it should be removed from PyPI and current users should be notified if possible.
If there is a temporary issue that can be fixed, I think everyone would appreciate more detail about the problem, and what steps need to be taken.
Amazon banned acdcli? How do you know that?
Plus, if amazon banned acdcli, that does not mean that your data is lost, neither amazon banned you.
It looks like Amazon has rate-limited the acdcli
client's ability to refresh access tokens, and the authorization webapp is showing an "over quota" error (from Google App Engine, I think).
Perhaps the authorization app was compromised somehow, started sending out lots of garbage requests, and got rate-limited by Amazon?
Unfortunately, it's not possible to even create your own Amazon security profile (the 'advanced' setup method) anymore, because you won't be able to whitelist your app for the Cloud Drive API without getting an "invitation" from AWS.
@bjmc that information is the same I (and others know). But, @yadayada did not say anything about this. I am starting to think that rclone is the only valid fix for this. Its a pitty.
Ye for the moment rclone is the only way to mount the ACD on a linux terminal :/
Well, after all the work getting acdcli working like a charm... Its a pitty to switch to rclone.
Acd was great.
My biggest problem is rsync doesnt work with rclone so i hope acdcli comes back :(
@Tangoes could you give any source for a setup of rclone in a linux w/o gui (ubuntu (16.04) preferred) Really want my plex up again. ._.
@elPerstin https://rclone.org/install/
then rclone mount your_remote: /home/you/mount
Just in case: https://github.com/ajkis/scripts/wiki/best-plex-rclone-mount-settings Follow this advices and read the full thread in rclone forums, it will help a lot.
Rclone mount is quite good for plex right now. You will only have some issues when you start playing something in the middle and not from the beginning.
@madyoda that's it? just rclone mount remote: /path
and wait until it's done?
Looks good, changes will be live after unmount/mount, or live? Biggest thing is, that it seems like it has a loooooong time, mounting a 10TB-150k-File-ACD :-/
Not to be rude, but maybe this github issue isn't the best place for user support for a different piece of software. Is there an rclone
forum that would be more appropriate?
You're maybe not totally wrong, but it seems legit to me, that we're discussing alternatives, as there is not a single word from the dev. whatever, i'll get through it. and i'm still hoping, acd_cli i coming back soon.
Well @bjmc, not to be rude, but this github ISSUE thread... Oh wait! Dev is mute, and disabled issues from his github!
I repeat, its a great piece of software, but I am moving to rOther one. At least, by now.
A mute dev isn't the best way to handle a potential security problem. Because of the way this was handled, i'd never use the software even if it was fixed or even if it wasn't this piece of software's fault.
@bgemmill I am using your fork and since you are the most recent active developer... Do you know what is going on?
Only thing that seems to be the problem is that the Amazon app that handles the tokens is no longer active.
@elPerstin
Rclone mount runs in foreground, so you either need to put a & at the end or run it in some kind of background, I.e screen, tmux, etc.
Sent from my iPhone
On 15 May 2017, at 18:06, elPerstin notifications@github.com wrote:
@madyoda that's it? just rclone mount remote: /path and wait until it's done?
Looks good, changes will be live after unmount/mount, or live? Biggest thing is, that it seems like it has a loooooong time, mounting a 10TB-150k-File-ACD :-/
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
@BabyDino, there are two problems. The one with the quota, and the one with the security breach. Are they the same? Is one caused by the other? Can we do something to help?
Just @yadayada knows.
Writing up a TL;DR. Give me 10 mins or so for everyone confused WTF is going on.
@Tangoes The quota issue is because of Google. The appspot quotas will reset every midnight PST. This morning I could successfully enter the website and download the sourcecode (/src folder). I am in CEST.
So the quota issue is just because of all the token refresh requests. But once it started working this morning, I was unable to get a new token. This is because of the Amazon app which is no longer active (The client id message).
So, is there any way to get this working without using appspot?
And the security breach is confirmed? I am sorry, issues are gone, so I don't know.
If Amazon would let individuals whitelist their own client apps for the drive API again, then that would allow you to use acdcli
without relying on the App Engine authorization app.
Unfortunately, the Amazon page currently says:
You must be invited into the Amazon Drive API service to whitelist your app for Amazon Drive or you will not be able to send any node-related requests.
If anyone knows somebody at AWS, maybe ask them if Amazon is planning to open that API up again?
This is the last word I've seen from Amazon: https://forums.developer.amazon.com/questions/36991/how-do-i-whitelist-my-new-security-profile.html
@Tangoes security breach not confirmed.
@bjmc Amazon have said the Drive API is now closed. (Meaning no new applications are accepted)
Yes. You need to create a new app in the Google developer console (source code attached). Then you need to create an Amazon app which should be whitelisted. It should work again. I didn't have the time today to get into the Amazon stuff. But maybe someone else can.
auth.zip `
Then you need to create an Amazon app which should be whitelisted.
You can't. Drive API is closed for new apps 😢
@BabyDino yeah(from what i know), but rclone's app was already approved/whitelisted so they're fine.
Writing a TL;DR for people who are unaware of what it happening / what is going on.
Note: This is all from memory as the GitHub issues were removed from the repository
Approx. 30 days ago a GitHub issue was open on the yadayada/acd_cli
GitHub repository called "Not my file". The author detailed that their acd_cli database got corrupted. They removed it, resynced and then had someone elses (!) data. Full account access, fully accessable, etc.
@yadayada (acd_cli owner/developer) recommended the author of the issue to contact Amazon support straight away, if they did or not I cannot remember.
Not much happened for another 20-25 days or so until another guy (Saren someone I believe) commented saying he had exactly the same issue, with screenshots showing that they, indeed, had full access. It was pretty scary as they showed family photos, backups, et cetera.
The other guy eventually contacted Amazon support after a bit of chatter and they got back to him promptly saying they need more detail on the issue, how they authorized to acd_cli (whether tensile-runway appspot method, etc) and other things.
Right around this time the appspot (https://tensile-runway-92512.appspot.com/) went into "error mode" (seen HERE) detailing "400 Bad Request - Unknown client_id". Now to me, this sounds like if it's "unknown" acd_cli got revoked from Amazon's Drive API, which seems to be the case with the amount of comments coming in.
A few hours later, tensile-runway went to its "Over Quota" state (picture) which it is currently in at the time of writing this. Over Quota seems like it's under load, exceeded bandwidth quota, something like that which is indeed understandable as all the acd_cli instances will be pinging this site to try and get a new Amazon auth key, being unsuccessful.
We don't know what's happening right now, @yadayada hasn't said anything on the matter and put a scary message on the top of the README.
Hope I've got this right, as it is completely from memory. If anyone has any questions I'll do my best to answer, based on memory.
RCLONE (install github forums)has been working pretty well for me so far, utilizing the rclone mount command in the format rclone mount your_remote: /home/your/mount/path
(runs in foreground, add a &
to the end to run in background).
Edit 1: Seems you can still link to issues, so here's the two (if issues ever get opened back up) re: the current sitation. #549 #559
@madyoda Ah. That's what I figured (and therefore removed my reply ;))
@madyoda I think you got it spot on. From my point of view, there are two options at the moment: 1) man in the middle attack, which points to appspot and tokens were compromised. I've been analyzing the published source, which seems ok to me (I am not a Python developer). 2) Amazon screwed up big time.
The Question for me really is, is acdcli got banned because of Security Reasons or because of API Usage. If its first i think the project here will never come up again. And this explains the reason why @yadayada not responding to anything. If last maybe Amazon said that's too much we kill you bye. But if its last we have the same problem with rclone in a few days to months.
@ChoosenEye it only makes sense that it's security reasons. If you read my tl;dr above, the acd_cli client token got taken down just after someone contacted Amazon security.
@ChoosenEye I don't think this is Amazon pulling the plug on the API. Makes no sense. They can disable the App connected to the API because of security reasons (which is what happened)
@BabyDino yes but maybe its a perfect argument to kill one of the most bandwith/storage impact applications which can't be rentable when you see the Storage/Bandwith use of most of us here^^
@ChoosenEye I can create the same amount of traffic with the desktop app ;)
@BabyDino yes you can but the really feature the mounting as a drive doesn't work with the desktop app and also the Desktop App limits your Bandwith usage dramatically and it's much easier for monitoring ^^
There isn't a desktop app for Linux.
@ChoosenEye I can see where you're coming from, but given the circumstances I believe @BabyDino has hit the nail right on the head, so to say, here.
@madyoda yes english not the best i know ^^
@ChoosenEye your English is fine! 😄
@madyoda thanks but... no not really
@bjmc True, that's why there is an API. So we can connect different applications to the drive.
Anyway, I think the problem is clear... Now we need a solution because my backups are waiting.. Anyone here with a whitelisted app?
As I didnt find any other solution to get in touch with others regarding this project (especially yadayada - your mail wasn't valid as well) I thought it might be the best idea to create a pull request (which is garbage ofc) to try and find out what happend?!