search

yadayada / acd_cli

An unmaintained command line interface and FUSE filesystem for Amazon (Cloud) Drive
Other
1.35k stars 167 forks source link

I created this pull request only to ask what happend to acd_cli's issues page?! It just vanished! #562

Closed kelek- closed 7 years ago

kelek- commented 7 years ago

As I didnt find any other solution to get in touch with others regarding this project (especially yadayada - your mail wasn't valid as well) I thought it might be the best idea to create a pull request (which is garbage ofc) to try and find out what happend?!

tombowditch commented 7 years ago

Check the comment here -

https://github.com/yadayada/acd_cli/commit/654871759c2e9b5a829926c8c11f9f382f2c0a3d

kelek- commented 7 years ago

thank you @madyoda

AiMAnsarie commented 7 years ago

I was trying to follow what happens next and next thing I see is a big 404 🙄

shadycuz commented 7 years ago

Yeah I was having issues last night and today as well.

acdcli sync
Traceback (most recent call last):
  File "/usr/local/bin/acdcli", line 11, in <module>
    sys.exit(main())
  File "/usr/local/bin/acd_cli.py", line 1338, in main
    acd_client = client.ACDClient(CACHE_PATH)
  File "/usr/local/lib/python3.5/dist-packages/acdcli/api/client.py", line 31, in __init__
    self.handler = oauth.create_handler(path)
  File "/usr/local/lib/python3.5/dist-packages/acdcli/api/oauth.py", line 24, in create_handler
    return AppspotOAuthHandler(path)
  File "/usr/local/lib/python3.5/dist-packages/acdcli/api/oauth.py", line 144, in __init__
    self.load_oauth_data()
  File "/usr/local/lib/python3.5/dist-packages/acdcli/api/oauth.py", line 88, in load_oauth_data
    self.get_auth_token(reload=False)
  File "/usr/local/lib/python3.5/dist-packages/acdcli/api/oauth.py", line 103, in get_auth_token
    self.refresh_auth_token()
  File "/usr/local/lib/python3.5/dist-packages/acdcli/api/oauth.py", line 183, in refresh_auth_token
    'Error refreshing authentication token: %s' % response.text)
acdcli.api.common.RequestError: RequestError: 1004, Error refreshing authentication token:
<html><head>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>503 Over Quota</title>
<style><!--
body {font-family: arial,sans-serif}
div.nav {margin-top: 1ex}
div.nav A {font-size: 10pt; font-family: arial,sans-serif}
span.nav {font-size: 10pt; font-family: arial,sans-serif; font-weight: bold}
div.nav A,span.big {font-size: 12pt; color: #0000cc}
div.nav A {font-size: 10pt; color: black}
A.l:link {color: #6f6f6f}
A.u:link {color: green}
//--></style>

</head>
<body text=#000000 bgcolor=#ffffff>
<table border=0 cellpadding=2 cellspacing=0 width=100%>
<tr><td bgcolor=#3366cc><font face=arial,sans-serif color=#ffffff><b>Error</b></td></tr>
<tr><td>&nbsp;</td></tr></table>
<blockquote>
<H1>Over Quota</H1>
This application is temporarily over its serving quota.  Please try again later.

<p>
</blockquote>
<table width=100% cellpadding=0 cellspacing=0><tr><td bgcolor=#3366cc><img alt="" width=1 height=4></td></tr></table>
</body></html>
tsurelad commented 7 years ago

https://tensile-runway-92512.appspot.com/ shows: Error Over Quota This application is temporarily over its serving quota. Please try again later.

ScottyCameron commented 7 years ago

Any news on when this will be fixed?

Tangoes commented 7 years ago

Yadayada silence, turning off issues page, and not explaining anything, makes people abandon this so good project.

Why this silence? No fix date? I dont understand this attitude.

ghost commented 7 years ago

No news from Yadayada ? Hope amazon will unban acdcli otherwise my whole stuff is gone :(

bjmc commented 7 years ago

@yadayada can you give us any more information? If there is a serious security issue with this package that cannot be resolved, it should be removed from PyPI and current users should be notified if possible.

If there is a temporary issue that can be fixed, I think everyone would appreciate more detail about the problem, and what steps need to be taken.

Tangoes commented 7 years ago

Amazon banned acdcli? How do you know that?

Plus, if amazon banned acdcli, that does not mean that your data is lost, neither amazon banned you.

bjmc commented 7 years ago

It looks like Amazon has rate-limited the acdcli client's ability to refresh access tokens, and the authorization webapp is showing an "over quota" error (from Google App Engine, I think).

Perhaps the authorization app was compromised somehow, started sending out lots of garbage requests, and got rate-limited by Amazon?

Unfortunately, it's not possible to even create your own Amazon security profile (the 'advanced' setup method) anymore, because you won't be able to whitelist your app for the Cloud Drive API without getting an "invitation" from AWS.

Tangoes commented 7 years ago

@bjmc that information is the same I (and others know). But, @yadayada did not say anything about this. I am starting to think that rclone is the only valid fix for this. Its a pitty.

ghost commented 7 years ago

Ye for the moment rclone is the only way to mount the ACD on a linux terminal :/

Tangoes commented 7 years ago

Well, after all the work getting acdcli working like a charm... Its a pitty to switch to rclone.

Acd was great.

ghost commented 7 years ago

My biggest problem is rsync doesnt work with rclone so i hope acdcli comes back :(

elPerstin commented 7 years ago

@Tangoes could you give any source for a setup of rclone in a linux w/o gui (ubuntu (16.04) preferred) Really want my plex up again. ._.

tombowditch commented 7 years ago

@elPerstin https://rclone.org/install/

then rclone mount your_remote: /home/you/mount

rafuz commented 7 years ago

Just in case: https://github.com/ajkis/scripts/wiki/best-plex-rclone-mount-settings Follow this advices and read the full thread in rclone forums, it will help a lot.

Rclone mount is quite good for plex right now. You will only have some issues when you start playing something in the middle and not from the beginning.

elPerstin commented 7 years ago

@madyoda that's it? just rclone mount remote: /path and wait until it's done?

Looks good, changes will be live after unmount/mount, or live? Biggest thing is, that it seems like it has a loooooong time, mounting a 10TB-150k-File-ACD :-/

bjmc commented 7 years ago

Not to be rude, but maybe this github issue isn't the best place for user support for a different piece of software. Is there an rclone forum that would be more appropriate?

elPerstin commented 7 years ago

You're maybe not totally wrong, but it seems legit to me, that we're discussing alternatives, as there is not a single word from the dev. whatever, i'll get through it. and i'm still hoping, acd_cli i coming back soon.

Tangoes commented 7 years ago

Well @bjmc, not to be rude, but this github ISSUE thread... Oh wait! Dev is mute, and disabled issues from his github!

I repeat, its a great piece of software, but I am moving to rOther one. At least, by now.

calisro commented 7 years ago

A mute dev isn't the best way to handle a potential security problem. Because of the way this was handled, i'd never use the software even if it was fixed or even if it wasn't this piece of software's fault.

BabyDino commented 7 years ago

@bgemmill I am using your fork and since you are the most recent active developer... Do you know what is going on?

Only thing that seems to be the problem is that the Amazon app that handles the tokens is no longer active.

tombowditch commented 7 years ago

@elPerstin

Rclone mount runs in foreground, so you either need to put a & at the end or run it in some kind of background, I.e screen, tmux, etc.

Sent from my iPhone

On 15 May 2017, at 18:06, elPerstin notifications@github.com wrote:

@madyoda that's it? just rclone mount remote: /path and wait until it's done?

Looks good, changes will be live after unmount/mount, or live? Biggest thing is, that it seems like it has a loooooong time, mounting a 10TB-150k-File-ACD :-/

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

Tangoes commented 7 years ago

@BabyDino, there are two problems. The one with the quota, and the one with the security breach. Are they the same? Is one caused by the other? Can we do something to help?

Just @yadayada knows.

tombowditch commented 7 years ago

Writing up a TL;DR. Give me 10 mins or so for everyone confused WTF is going on.

BabyDino commented 7 years ago

@Tangoes The quota issue is because of Google. The appspot quotas will reset every midnight PST. This morning I could successfully enter the website and download the sourcecode (/src folder). I am in CEST.

So the quota issue is just because of all the token refresh requests. But once it started working this morning, I was unable to get a new token. This is because of the Amazon app which is no longer active (The client id message).

Tangoes commented 7 years ago

So, is there any way to get this working without using appspot?

Tangoes commented 7 years ago

And the security breach is confirmed? I am sorry, issues are gone, so I don't know.

bjmc commented 7 years ago

If Amazon would let individuals whitelist their own client apps for the drive API again, then that would allow you to use acdcli without relying on the App Engine authorization app.

Unfortunately, the Amazon page currently says:

You must be invited into the Amazon Drive API service to whitelist your app for Amazon Drive or you will not be able to send any node-related requests.

If anyone knows somebody at AWS, maybe ask them if Amazon is planning to open that API up again?

This is the last word I've seen from Amazon: https://forums.developer.amazon.com/questions/36991/how-do-i-whitelist-my-new-security-profile.html

tombowditch commented 7 years ago

@Tangoes security breach not confirmed.

@bjmc Amazon have said the Drive API is now closed. (Meaning no new applications are accepted)

BabyDino commented 7 years ago

Yes. You need to create a new app in the Google developer console (source code attached). Then you need to create an Amazon app which should be whitelisted. It should work again. I didn't have the time today to get into the Amazon stuff. But maybe someone else can.

auth.zip `

tombowditch commented 7 years ago

Then you need to create an Amazon app which should be whitelisted.

You can't. Drive API is closed for new apps 😢

tombowditch commented 7 years ago

@BabyDino yeah(from what i know), but rclone's app was already approved/whitelisted so they're fine.

tombowditch commented 7 years ago

Writing a TL;DR for people who are unaware of what it happening / what is going on.

Note: This is all from memory as the GitHub issues were removed from the repository

What is happening?

Approx. 30 days ago a GitHub issue was open on the yadayada/acd_cli GitHub repository called "Not my file". The author detailed that their acd_cli database got corrupted. They removed it, resynced and then had someone elses (!) data. Full account access, fully accessable, etc.

@yadayada (acd_cli owner/developer) recommended the author of the issue to contact Amazon support straight away, if they did or not I cannot remember.

Not much happened for another 20-25 days or so until another guy (Saren someone I believe) commented saying he had exactly the same issue, with screenshots showing that they, indeed, had full access. It was pretty scary as they showed family photos, backups, et cetera.

The other guy eventually contacted Amazon support after a bit of chatter and they got back to him promptly saying they need more detail on the issue, how they authorized to acd_cli (whether tensile-runway appspot method, etc) and other things.

Right around this time the appspot (https://tensile-runway-92512.appspot.com/) went into "error mode" (seen HERE) detailing "400 Bad Request - Unknown client_id". Now to me, this sounds like if it's "unknown" acd_cli got revoked from Amazon's Drive API, which seems to be the case with the amount of comments coming in.

A few hours later, tensile-runway went to its "Over Quota" state (picture) which it is currently in at the time of writing this. Over Quota seems like it's under load, exceeded bandwidth quota, something like that which is indeed understandable as all the acd_cli instances will be pinging this site to try and get a new Amazon auth key, being unsuccessful.

We don't know what's happening right now, @yadayada hasn't said anything on the matter and put a scary message on the top of the README.

Hope I've got this right, as it is completely from memory. If anyone has any questions I'll do my best to answer, based on memory.

Alternatives

RCLONE (install github forums)has been working pretty well for me so far, utilizing the rclone mount command in the format rclone mount your_remote: /home/your/mount/path (runs in foreground, add a & to the end to run in background).

Edits

Edit 1: Seems you can still link to issues, so here's the two (if issues ever get opened back up) re: the current sitation. #549 #559

BabyDino commented 7 years ago

@madyoda Ah. That's what I figured (and therefore removed my reply ;))

BabyDino commented 7 years ago

@madyoda I think you got it spot on. From my point of view, there are two options at the moment: 1) man in the middle attack, which points to appspot and tokens were compromised. I've been analyzing the published source, which seems ok to me (I am not a Python developer). 2) Amazon screwed up big time.

ChoosenEye commented 7 years ago

The Question for me really is, is acdcli got banned because of Security Reasons or because of API Usage. If its first i think the project here will never come up again. And this explains the reason why @yadayada not responding to anything. If last maybe Amazon said that's too much we kill you bye. But if its last we have the same problem with rclone in a few days to months.

tombowditch commented 7 years ago

@ChoosenEye it only makes sense that it's security reasons. If you read my tl;dr above, the acd_cli client token got taken down just after someone contacted Amazon security.

BabyDino commented 7 years ago

@ChoosenEye I don't think this is Amazon pulling the plug on the API. Makes no sense. They can disable the App connected to the API because of security reasons (which is what happened)

ChoosenEye commented 7 years ago

@BabyDino yes but maybe its a perfect argument to kill one of the most bandwith/storage impact applications which can't be rentable when you see the Storage/Bandwith use of most of us here^^

BabyDino commented 7 years ago

@ChoosenEye I can create the same amount of traffic with the desktop app ;)

ChoosenEye commented 7 years ago

@BabyDino yes you can but the really feature the mounting as a drive doesn't work with the desktop app and also the Desktop App limits your Bandwith usage dramatically and it's much easier for monitoring ^^

bjmc commented 7 years ago

There isn't a desktop app for Linux.

tombowditch commented 7 years ago

@ChoosenEye I can see where you're coming from, but given the circumstances I believe @BabyDino has hit the nail right on the head, so to say, here.

ChoosenEye commented 7 years ago

@madyoda yes english not the best i know ^^

tombowditch commented 7 years ago

@ChoosenEye your English is fine! 😄

ChoosenEye commented 7 years ago

@madyoda thanks but... no not really

BabyDino commented 7 years ago

@bjmc True, that's why there is an API. So we can connect different applications to the drive.

Anyway, I think the problem is clear... Now we need a solution because my backups are waiting.. Anyone here with a whitelisted app?