Rate Exceeded

[nemymongus]

I am getting the following when trying to do a sync. I have tried delete-everything and reinstalling with no gain. Both computers that I had set up are broken but the message is slightly different on the one that I didn't reinstall on and it has the 429 error code. Did acd_cli get shut down again?

17-05-30 21:37:42.718 [CRITICAL] [acdcli.api.client] - Error getting endpoint data. Response: {"message":"Rate exceeded"} Traceback (most recent call last): File "/usr/local/bin/acd_cli", line 9, in load_entry_point('acdcli==0.3.2', 'console_scripts', 'acd_cli')() File "/usr/local/bin/acd_cli.py", line 1657, in main acd_client = client.ACDClient(CACHE_PATH, SETTINGS_PATH) File "/usr/local/lib/python3.5/dist-packages/acdcli/api/client.py", line 46, in init self._load_endpoints() File "/usr/local/lib/python3.5/dist-packages/acdcli/api/client.py", line 63, in _load_endpoints self._endpoint_data = self._get_endpoints() File "/usr/local/lib/python3.5/dist-packages/acdcli/api/client.py", line 80, in _get_endpoints raise Exception Exception

[ChrisRomp]

Ditto:

Traceback (most recent call last):
  File "/usr/local/bin/acd_cli", line 9, in <module>
    load_entry_point('acdcli==0.3.2', 'console_scripts', 'acd_cli')()
  File "/usr/local/bin/acd_cli.py", line 1688, in main
    ret = args.func(args)
  File "/usr/local/bin/acd_cli.py", line 849, in usage_action
    r = acd_client.get_account_usage()
  File "/usr/local/lib/python3.5/dist-packages/acdcli/api/account.py", line 67, in get_account_usage
    raise RequestError(r.status_code, r.text)
acdcli.api.common.RequestError: RequestError: 429, {"message":"Rate exceeded"}

[ChrisRomp]

I figured I'd try to enable my own security profile as outlined here, but then I started getting invalid_scope errors when trying to get a token under my new profile.

And now this page has the following message:

The Amazon Drive API and SDKs are currently closed to new developers. Thank you for your interest. We received an overwhelming number of API invitation requests with many innovative integration scenarios. We have ended the invitation period to focus on enabling new customer experiences with current developers.

I'm unsure if this is related, or how recent that message is, but it's crap timing. :-/

[au5ton]

Same here, same stacktrace as @nemymongus.

[au5ton]

It would probably be safe to assume that this is just another Amazon Cloud Drive quota we blew over (#550). I would wait until things settle, I doubt this is the end again, unless @yadayada went behind Amazon's back and is using a different CLIENT_ID and never had their ban lifted in the first place.

This is likely just Amazon's throttling as described in #416

[jxyzn]

Still get 429 even switching between multiple authorized apps. Perhaps something wrong on the backend.

In the response headers: x-amzn-ErrorType: TooManyRequests:http://internal.amazon.com/coral/com.amazon.clouddrive.service/

Body: {"code":null,"info":null,"logref":null,"message":"Rate exceeded"}

[chrisgavin]

Yep. I'm getting a 429 with a different app too.

[karbentur]

Works fine with Advanced Users (Security Profile) auth method. Yes, it means you have to register your own app on https://developer.amazon.com

[chrisgavin]

@karbentur, registering a security profile won't work anymore because Amazon have to whitelist it and they aren't accepting any new apps. I'm also not convinced it would help because the security profiles for several apps seem not to be working.

[karbentur]

@chrisgavin, ok, now I see. Anyway, security profiles method still works fine.

[bdeluca]

@karbentur how can you say it works fine? I have a security but I cant generate a new oauth key? I the page to generate them is gone?

[DDP74]

I have the same problem: [CRITICAL] [acdcli.api.client] - Error getting endpoint data. Response: {"message":"Rate exceeded"} Traceback (most recent call last): File "/usr/bin/acd_cli", line 11, in load_entry_point('acdcli==0.3.2', 'console_scripts', 'acd_cli')() File "/usr/bin/acd_cli.py", line 1646, in main acd_client = client.ACDClient(CACHE_PATH, SETTINGS_PATH) File "/usr/lib/python3.5/site-packages/acdcli/api/client.py", line 46, in init self._load_endpoints() File "/usr/lib/python3.5/site-packages/acdcli/api/client.py", line 69, in _load_endpoints self._endpoint_data = self._get_endpoints() File "/usr/lib/python3.5/site-packages/acdcli/api/client.py", line 80, in _get_endpoints raise Exception Exception

[ederjc]

+1

[AiMAnsarie]

+1

[ShapeShifter499]

Same here

[lloydie2009]

+1

[Kedryn]

for me, acd_cli sync give this:

acdcli.api.common.RequestError: RequestError: 429, [acd_api] no body received.

and Expandrive gives an empty folder.

Amazon is messing again with 3rd party tools

[AiMAnsarie]

My Expandrive is also not working.

[Kedryn]

From Expandrive support chat, they are getting error 429 as acd_cli but it's not an oauth error like last time

[AiMAnsarie]

That's actually a good news in a sense that it could be temporary. As Expandrive is a paid (and an expensive one) service and the developers play by the rules but even they're down.

[Kedryn]

OR, ACD decided to shut down 3rd party tools. Remeber that the 3rd party tools web page has been removed from amazon few days ago

[ChrisRomp]

@kedryn Yep that's what I'm concerned about. Guess all we can do is wait.

[AiMAnsarie]

That would be a pretty stupid move on Amazon's account because I haven't seen many people using it as a "backup/restore" only storage. Even if people do then Amazon hasn't made even a remotely useful app or web interface to perfectly utilize stored media. That's enough inconvenience for me to switch to some other cloud storage with a lot less space.

[ricardoclaus]

Gentlemen, Today I got the same problem while trying to connect to Amazon Drive. Acdcli returns the error: [Acdcli.api.client] - Error getting endpoint data. Response: {"message": "Rate exceeded"}

I emailed Amazon, where I quickly responded. I wondered how to use Amazon Drive on Linux. The sad news is that Amazon Drive supports only connection via web browser. Acdcli and rclone client apps are not supported. That's why we're not getting access to Amazon Drive.

Here is the answer portion of the email:

Hello,

I understand your concern about using third party clients and apps to access Amazon Drive on Linux. Unfortunately, third party applications such as, acd_cli and Rclone client access has been revoked and they no longer supports Amazon Drive access. I'm sorry for any inconvenience this might have caused to you. I'd suggest you to please access Amazon Drive using website on Linux. Here's the link:

https://www.amazon.com/clouddrive

[chrisgavin]

Oh dear. Hopefully this is just the support person being incompetent and getting confused over the recent OAuth issues. If they really have banned all third party tools intentionally then ACD is completely useless.

On the plus side, maybe this is the motivation needed to reverse engineer the "official" client.

[thenoahcomputer]

@ricardoclaus I think they are repeating old news to you (I hope). That's one of the the 3 things people seemed to be told by first level support when calling about this last time. Given that other apps are also not working and all this has happened several times before, I would treat this as a likely Amazon problem at least for today.

Or maybe too many people started syncing once acd_cli was reauthorized by Amazon and we have collectively exceeded Amazon's daily quota for acd_cli

[talisto]

ODrive is still working, so they haven't banned all the 3rd party clients (yet).

[Kedryn]

Expandrive told me that Amazon is esperiencing selective outage. For example NetDrive is working, Arq backup is not

Il 31 mag 2017 4:16 PM, "Chris R" notifications@github.com ha scritto:

@Kedryn https://github.com/kedryn Yep that's what I'm concerned about. Guess all we can do is wait.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/yadayada/acd_cli/issues/572#issuecomment-305200293, or mute the thread https://github.com/notifications/unsubscribe-auth/AMKyUGzgSmNcryoAzwzNHvx6idSGcMdLks5r_XY_gaJpZM4NrFp9 .

[kconat]

ODrive is kind of Amazon's buddy. They where there in the first hour and Amazon used their auto-sync feature to promote drive. At the time the official app was just manual and quite unusable. I highly doubt they would ban them that easily.

[tombowditch]

Shameless plug, but there's a discord server based on discussing future of unlimited cloud storage providers & future of acd_cli

I know last time the issues got out of hand, so there's a acd_cli channel there should anyone be interested.

[pink-mist]

The official acd_cli chat is on gitter though, as linked from the main github page: https://gitter.im/yadayada/acd_cli

[bdeluca]

I think what we are about to see is amazon is acquiring or partnering with some of the providers and the rest are dust.

[asabla]

Probably this issue is related to: https://www.arqbackup.com/blog/arq-amazon-drive-outage-resolved/

Soooo just Amazon not being the greatest at communicating what's going on i guess. Unless @yadayada maybe had some sort of NDA on him until this issue (at amazon not acd_cli) is resolved?

[a5m0]

Interesting to see that arq is back... but I'm still seeing Rate exceeded messages on acd_cli.

[chrisgavin]

I think lots of apps have been banned for having public client secrets like rclone was. Arq seemed to have been banned mistakenly for this reason but access was then restored.

acd cli has probably been banned for using another app's security profile. I don't think it's coming back :(

[Guymer]

Same here (using acd_cli 0.3.2, api 0.9.3) - has anybody been able to get round this yet?

17-06-04 15:17:23.009 [CRITICAL] [acdcli.api.client] - Error getting endpoint data. Response: {"message":"Rate exceeded"}
Traceback (most recent call last):
  File "/home/fff/.local/bin/acdcli", line 11, in <module>
    load_entry_point('acdcli==0.3.2', 'console_scripts', 'acdcli')()
  File "/home/fff/.local/bin/acd_cli.py", line 1657, in main
    acd_client = client.ACDClient(CACHE_PATH, SETTINGS_PATH)
  File "/home/fff/.local/lib/python3.5/site-packages/acdcli/api/client.py", line 46, in __init__
    self._load_endpoints()
  File "/home/fff/.local/lib/python3.5/site-packages/acdcli/api/client.py", line 63, in _load_endpoints
    self._endpoint_data = self._get_endpoints()
  File "/home/fff/.local/lib/python3.5/site-packages/acdcli/api/client.py", line 80, in _get_endpoints
    raise Exception
Exception

[AlexCzar]

I have the same problem. I too wrote to Amz support for confirmation. If it is confirmed that acd won't work anymore, I'll have to switch to a different service, as ACD is completely unusable for me without acd_cli. Will probably switch to SpiderOak

[lloydie2009]

The thing with SoiderOak is they only provide plans upto 5TB and is costly compared to ACD.

Does anyone have any suggestions other than GoogleDrive as an alternative. I'm looking to store about 20TB total.

[AlexCzar]

@lloydie2009 they've updated their plans, they have unlimited now and others > 5TB, yes, it's costly, unlimited is $150 per year, I think. But ACD is indeed absolutely unusable for me without acd_cli. @yadayada can you please confirm that this outage is permanent and we won't see acd_cli working again due to the latest Amazon policy change?

[yadayada]

@AlexCzar I can neither confirm nor deny that the old/new security profile was banned temporarily or permanently, as I have not received any kind of notification or response from the Amazon Drive team.

I have applied to bump the new Appspot app to production limits, but I'm fairly certain that won't happen. It uses a spare security profile of mine.

You might want to give @chrisgavin's fork a go, see https://github.com/chrisgavin/cheeky_acd_cli.

BTW, I cancelled my renewal:

[AiMAnsarie]

@yadayada Which Cloud Storage are you switching to next?

[d3athh]

@yadayada

Thank you for all your hard work with acd_cli, been a big fan/user for a long time! 72+TBs. Had to make a github account, as I don't socialize...

Anyway, I just re-created a new oauth.json token with https://drivesink.appspot.com/config And pasted it ~/.cache/acd_cli/

It's sloppy but it's working fine for the past couple days 👍

-lurker d3ath

[danyderigon]

@d3athh Thank you very much ! It works very well ;)

[cyberea]

RIP

[AlexCzar]

Wow! acd_cli works iwth drivesink. Now I'm totally confused, I've already canceled my subscription to ACD it will expire in August. But with this method working I just don't know... will it still be working in 3 months? in 6? @yadayada in light of this will you still be abandoning acd_cli?

[Linwood-F]

I don't mean to hijack this, but I gave up on ACD_CLI and tried a commercial tool I already had, which may be of interest (I have no connection except as a customer). Goodsync is primarily a windows/mac product and commercial, not open source (about $30 for a license). They also offer a free Linux command line version, and they support ACD with both. The catch on the command line version is that it is poorly documented if used alone, the simple and recommended way is to build a job with windows or mac, then export the job and run it on linux (possibly editing the XML-like definition file). This is more arcane than it may sound as the file format is ill defined as well as ill documented, but the product itself is quite solid. The sync aspect is both 2 way sync and also varieties of one-way sync, e.g. backup. It will also do delete-on-copy one-way sync, e.g. move/archive type. I spent a day or so banging at the syntax and managed to convert my ACD_CLI scripts to use Goodsync's linux server instead. So far, and a few gig and maybe 3000 files later, it is uploading without error (well, without substantial error, it gets a few errors, but properly saves the file to retry). So for $30 there's an alternative of sorts that being commercial may not (MAY not) end up being killed by Amazon.

I liked and used ACD_CLI a long time, sorry to see it not going well, and hope it comes back. I just couldn't wait much longer hoping.

[blitz313]

@Linwood-F were you able to get it to work on a FUSE mount by any chance? I use my ACD instance for both backups and supplementary storage between multiple servers, so I need both command line and mountable.

[Linwood-F]

I did not try anything else; I used it just to archive jpg files off from a security system to ACD, so I was copying a whole folder at a time with ACD_CLI, which worked nicely with a sync program, I just had it do a 1-way sync of a folder (a quick call to SED to replace the folder names in the job file let me change which folders for which runs). Sorry, no help there. And this alternative does take some effort to get going; their software is nice in a windows/mac GUI, but the command line is pretty arcane -- it's well documented in one sense, but does not give you the syntax that the various cloud providers need, so getting syntax right is a challenge, though it's pretty easy to overcome by doing it in the gui then exporting the job and editing to change windows to linux paths.

[blitz313]

@Linwood-F thanks. They have 30-Day trial, so I'll give that a go this weekend. Thanks for the Windows tip, will try yoinking the syntax from my Windows machine and pouring over to the Linux. See what kinda luck I have. Thanks again

[simcop2387]

Looks like goodsync might work well enough under wine to get the job xml at least, https://appdb.winehq.org/objectManager.php?sClass=application&iId=11981

[Linwood-F]

Export the job though. You can also take the whole file (if you can figure out which one, I still am not sure) from windows, but if you export it, the file contains the account information as well as folders, connections, etc. There's a lot of repeated junk in there, but if you set up a job to move 1 folder, with one-way sync from windows to ACD, and set any other options you want (like move vs copy, threads, probably turn off propagate deletions) then name.tix file will be all you should need, then run the program on linux wiith

gsync sync jobname /job-file=jobfilename.tix

it should work, though first edit to change windows paths to linux paths. The online manual documents most of the contents, though gives short shrift to the syntax, so a working file first is useful, the prune and change and try as you go. Do take care as being a (possibly) 2 way sync program it absolutely can wipe out the "from" side if you are not careful.