Degoo and Google API

[forrie]

Would anyone have direct contact with someone at Degoo; at least, at the business level.

I ask (party in ignorance): wouldn't they benefit from allowing authenticated access to one's storage via the Google API, where various tools could be used at that level? For example, I use "rclone" for interacting with Google Drive and S3; it's fast, efficient.

Pardon if this is the wrong place to ask this, I didn't see a mailing list.

[bernd-wechner]

I chat with one of their product managers on occasion at Degoo. And am keen for them to provide a file hash in the metadata for files. Alas, direct access to the Google File Store is never likely to happen for a couple of reasons:

1. Degoo have a corporate account and they provide keyed URLS for upload and download. These URLs contain a key encrypted version of the URL itself, that you need a private key to create, a private key that rests with Degoo and is not shared,nor legally I suspect shareable, it authenticates such URLs as theirs and not some hackers. To wit, to get URLs to interact with the Google File Store, needs a request sent to Degoo first.

2. They are not committed to using the Google File Store per se. They neither advertise that they use it not have they always used it. They, I suspect, have a sensible desire to remain agile in this space and reactive, migrating their customers data if need be to another backend.

3. Degoo have been trying to grow a customer base aggressively and establish an identity in the market. They have provided very affordable lifetime cloud storage deal and are thereby winning (for want of a better word) a demography of customers who have a strong interest in their ongoing viability (so they get value out of their lifetime deal, which is, realistically for the shorter of, their own lives, or Degoo's). Alas their business focus at present is the phone app space as a media backup tool, competing with Google Photos in a way, hoping to differentiate themselves sufficiently to win paying customers. They have a Web app as well, but no desktop app (they had one but decomissioned it - a clear signal that maintaining it cost more than it promised to earn). Which all part of the reason I've invested some time into reverse engineering their Web API and implementing a CLI interface to manage nightly backups.

There's no mailing list. This is just private repo by some dude (me) who was hacking up a Degoo CLI. It's not even a published Python package yet ;-). But it's evolving and drawn a few PRs which is nice.

[jneuhaus]

When I first paid for Degoo, I used the desktop app, which was terrible. They dumped that, and I was hoping there would be an API so tools like rclone could be used instead, but alas I believe they are firmly focused on mobile device storage for all the reasons you outline. One big indication of this are the limitations of their Web-based App. It limits the number of files that can be uploaded, so it's dead in the water for a backup solution. Your project is great, I never realized reverse engineering could be so fun! Now, let's hope they don't dramatically change their internals an brick all your code.

[bernd-wechner]

How does it limit the number of files? I'm curious. I might have said similar, but a different way, that is, it's a wholly impractical inrerface for backing up a drive. I did on experiment once, trying to upload the root dir, but of course this just brings the Javascript app to its knees and nothing fruitful happens in any meaningful time frame. Hence the desire to script the interaction. But are you are of an explicit limit to the number of files?

Not that it matters, just a curio.

To be honest I just split all the API code out to: https://github.com/bernd-wechner/Degoo/blob/master/degoo/API.py

and it only implemented the rudimentary functions their API clearly supports that can be found by sniffing the web requests. If they change it's not great (and they have changed and will change I am sure) then this is the bit that simply needs adapting. To be honest if they stick with GraphQL then the exercise here upskills me - and anyone else who wants to - in fluid GraphQL sniffing anyhow, it's not that hard just a bit time consuming so a few evening work to save samples of each interaction then study them and extract the meaningful ones and implement them in Python ;-).

[jneuhaus]

The limit I encountered (app.degoo.com/files) was triggered when I chose the 'Upload folder' button and selected a folder with over 100,000 files and folders. Degoo stopped uploading with a Dialog window saying something like, "You cannot upload more than 5,000 files." I wonder if that exception condition would be triggered by your python-driven scripts?

[bernd-wechner]

Good to know! Easy to test, if you have said folder to try it with. But, my guess is, we won't encounter it. I suspect this is probably a limit imposed by the JS front end not the GraphQL backend. Worth testing though.

[forrie]

Thanks for clarifying.

I paid for (got sucked into?) their lifetime 10TB deal; but my experience has been extremely frustrating (to be polite). I don't need to hash through all the missing features and problems with the service, as you're well aware :)

They design the service to be easy to upload to, but awkward and difficult to download from. That's by design, I'm pretty sure.

I got royally pissed off when they sent me a message indicating my account was going to be terminated for copyright infringement. It had to do with the vast number of MP3s I have that I legally own. They backed off. From there, I downloaded copies off my material and have been using other services (ie: S3).

I am not using it at this point, as I have no reliable way to interact with it that's useful to me. Sad, but true.

Seeing that I still have it and, presumably will for the foreseeable future, my next option is to at least mirror some encrypted data (ie: Cryptomator containers) that can be updated or stored there perpetually -- and will be free from their meddling bots (so much for privacy and top secret blah blah).

I'm glad this tool is being developed. I expect many people in my circumstances will appreciate this. It's too bad that Degoo doesn't have the resources to provide something similar -- even customer service. I can't even find their physical address published anywhere -- but they are investing in advertising campaigns through Stack Social, etc., so I'm really puzzled at what their future map is (if they even have one) for their business.

I tried working with this code early on (this project) but wasn't able to get anything working -- I figured I would lurk and wait for things to get developed :)

Sorry for the rant, but this is the premise of my inquiry. I had thought "if" I could work with the Google API that would simplify things -- alas I was wrong there.

[bernd-wechner]

No worries. I was rather disappointed by their software too, hence worked on this ;-).

I think though now, us 10TB lifetime dealers should stick together a team to try and ensure Degoo stays viable and we can keep our 10TB of backup storage. The more the merrier.

[forrie]

Has there been any hint that they are planning to improve their code or improve the interface. Surely they can afford to pay programmers to help them out and I cannot imagine they're not unaware of these problems.

In my scenario, encrypted containers have literally thousands of files -- one problem I see is not being able to "sync" content (ie: rsync or rclone, replicate only changes) -- with this tool on the CLI, I imagine it's an all or nothing scenario, which wouldn't scale (that could be accounted for by local scripting). Uploading a huge 450GB container file there may not be efficient, either -- there's no way to unarchive it on-site. Breaking things down into smaller containers may be necessary.

I guess I can store all my memes on Degoo, at least LOL

[bernd-wechner]

Interestingly I recall reading and have notes somewhere, that the Google Cloud Store does support (in fact I think any back end does nowadays) restartable uploads and downloads. That is the all or nothing days are mostly if not entirely behind us. That said this CLI would need to do something in order to exploit that and it's on the list: https://github.com/bernd-wechner/Degoo/issues/20

The largest files I expect to move are probably around the GB magnitude and with internet speeds today that's fast enough that transmission interruption is a low risk and restarting form scratch not a huge overhead. But of course if you're needing to transfer much larger files then https://github.com/bernd-wechner/Degoo/issues/20 is what we'll need for sure!

[bernd-wechner]

I think this issue can now close, and any further conversation move to Discussions which github now supports.

[arkarphyo]

Degoo

[buzzy]

@forrie If you have no plans of using your 10TB account, would you mind giving it to me? :)

[ShawnZDesign]

I've been thinking about buying the Degoo LT deal. What about using with the UI Vision RPA open-source software? (formerly known as Kantu)

[bernd-wechner]

I've been thinking about buying the Degoo LT deal. What about using with the UI Vision RPA open-source software? (formerly known as Kantu)

I'm wondering to what end? The two major concerns here seem to be:

1. fixing the login issues https://github.com/bernd-wechner/Degoo/issues/42
2. A FUSE interface https://github.com/bernd-wechner/Degoo/issues/38

Both of which @buzzy has been fixing in a go implementation, and I'm guessing a few of us can't wait to see that (I can't). As to this python implementation, the first remains a technical hurdle due to current Python shortcomings in JA3 fingerprint mimicry.

[cbertelli]

I see there is a nodejs implementation here: https://github.com/levnikort/degoo-api-js and a fuse driver here: https://github.com/MDKPredator/degoo_drive Maybe the 403 Error is only due to a lack of instructions. Anyway I couldn't connect to Degoo with your driver and PyFuse3 cannot be built on Mac.

[bernd-wechner]

Thanks for those links, this has fallen into the background against my many other commitments alas.

1. The JS lib is just a lib, so for use in a building a JS app or page, but interesting.
2. MDKPredator has done a great job it seems of progressing the work started here by adding Fuse support - a shame it doens't work on the Mac which is a *nix variant. I've not tried it.
3. The 403 has a known cause, there are no lacking instructions per se (well I mean there is nothing but a lack of instructions from Degoo, they have not been forthcoming with any, all our work is reliant on reverse engineering). The reason is that they implemented JA3 fingerprinting which for the moment cannot be spoofed in Python, though there are avenues to explore (there's a curl build that has JA3 spoofing support and curl may mainline that faster than Python does JA3 spoofing and if that can be used to do the login and get the session tokens it could be packaged with this to make for a fairly seemless experience. But alas, it's all work, for which, at present, I lack the time.

You can however still work pretty well with these utilities if you login using your browser and extract the credential (a token and an API key) from the network transactions you can see on the browser debugger). You can save those manually into the credetials file and be rolling. It's a lot of hassle just to get logged in ;-). But alas until we can spoof the JA3 fingerprint of a major browser we're stuck. Degoo have blocked us.

It is however only a mater of time before we can. For starters @buzzy has been working on a Go implementation in part because Go does support JA3 spoofing. In time Python will gain support for it too.

[Schaken]

Hello. I develop video games and apps using Unity. I am posting this here because this is API related. I cannot seem to find any API documentation. I see where you have developed many good endpoints, like exploring the cloud, download and upload, but how do we make use of it? I do not use Python, I use strictly C#.

Excuse me if I missed it, just a link to point me in the right direction would be awesome!