search

perkeep / gphotos-cdp

This program uses the Chrome DevTools Protocol to drive a Chrome session that downloads your photos stored in Google Photos.
Apache License 2.0
651 stars 36 forks source link

Couldn't Sign-In #1

Closed daneroo closed 4 years ago

daneroo commented 4 years ago

This is the message I get after authenticating with a google account (from the chromedp opened window):

Sign in with a supported browser To help protect your account, Google doesn’t let you sign in from some browsers. Google might > stop sign-ins from browsers that:

  • Don’t support JavaScript or have Javascript turned off.
  • Have unsecure or unsupported extensions added.
  • Use automation testing frameworks.
  • Are embedded in a different application.

Is it possible to bypass this behavior?

I am on MacOS, with an up-to-date Chrome (Version 78.0.3904.108)

mpl commented 4 years ago

Well, are you sure that the "automation testing framework" is the reason? I'm asking because my setup is similar to yours and I've never seen that. And I don't recall having set anything special in chrome to allow that.

I'm wondering: maybe they disallow automation when the account does not have 2 factor auth enabled? Have you enabled it?

daneroo commented 4 years ago

Thanks for chiming in. I am not sure that the cause is "Using automation testing frameworks", but It seems that this condition is being triggered by the authentication flow itself (from Google's end).

I did see some references to similar issues (StackOverflow, Selenium related things) and they seem to be rather recent, but I did not find any useful hints...

I do have 2-factor auth enabled, and I also tried it with alternative 2nd factors (Text Message/Authenticator/Google Prompt). Unfortunately to no avail. I also tried logging out from any other active sessions in regular Chrome. Finally, I revoked all "Trusted Devices" (i.e. devices that do not require 2nd factor) in my Google Account Security Settings. One interesting fact is that the error message comes after the password entry, but before the 2nd-factor flow is triggered. I.e. the 2nd factor is never actually invoked.

The last thing I tried was to point the Session.profileDir to my Default Chrome profile

&Session{
    profileDir: "/Users/MYUSERNAME/Library/Application Support/Google/Chrome/Default",
   ...

This was just hoping that the profile may somehow already be authenticated. That did not work either.

That was a useful suggestion though, so thanks. I will continue to try to find other paths or hints,..

If you have any other ideas, I would love to help figure this out. This is a really useful project.

mpl commented 4 years ago

Thanks for chiming in. I am not sure that the cause is "Using automation testing frameworks", but It seems that this condition is being triggered by the authentication flow itself (from Google's end).

I did see some references to similar issues (StackOverflow, Selenium related things) and they seem to be rather recent, but I did not find any useful hints...

I do have 2-factor auth enabled, and I also tried it with alternative 2nd factors (Text Message/Authenticator/Google Prompt). Unfortunately to no avail. I also tried logging out from any other active sessions in regular Chrome. Finally, I revoked all "Trusted Devices" (i.e. devices that do not require 2nd factor) in my Google Account Security Settings. One interesting fact is that the error message comes after the password entry, but before the 2nd-factor flow is triggered. I.e. the 2nd factor is never actually invoked.

The last thing I tried was to point the Session.profileDir to my Default Chrome profile

&Session{
    profileDir: "/Users/MYUSERNAME/Library/Application Support/Google/Chrome/Default",
   ...

This was just hoping that the profile may somehow already be authenticated. That did not work either.

Yes that should work, but I think you just just went too far in the path; i.e. the profile dir is the parent dir of that path. If you strip "Default" it should work. However, I wouldn't advise directly messing around in your actual profile dir without at least making a backup first. Even better, I would suggest a safer way:

That was a useful suggestion though, so thanks. I will continue to try to find other paths or hints,..

If you have any other ideas, I would love to help figure this out. This is a really useful project.

What about trying with an identical setup (same machine, chrome, etc) but a different google account? That would help us figure out whether it's a problem with your profile/account.

daneroo commented 4 years ago

Trying alternative accounts was a great idea.

So, there is something strange going on with my main account.

You can close the issue if you like, I'll keep poking at it until I find the solution..

P.S.: I have other things I'm looking at, and I'd like to help. Are Github Issues the best way to chat about this? (Or would you prefer twitter/gopher slack,...)

mpl commented 4 years ago

Understood. Feel free to report here when you find out what was the problem please.

regarding your question: if it's an actual issue, yes open one here. Otherwise if you want to discuss something technical about it, I think for now the best thing would be to use the Perkeep mailing list.

daneroo commented 4 years ago

@mpl . As reported above, I have been having authentication issues. This only affected one of my google accounts. Several others were tried and all of them worked correctly.

I also tried something similar using puppeteer which is using Chromium, from Node.js and found the same issue; all accounts except one was working.

Idea: Create a Chrome profile (that is authenticated) by manually invoking Chrome, and then re-use it for gphotos-cdp.

The following invocation of Chrome was a sufficient workaround to create a profile (manually) that could then be re-used with gphotos-cdp's default options for the problematic account:

/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \
  --no-first-run \
  --password-store=basic \
  --use-mock-keychain \
  --user-data-dir=/var/folders/NN/XXYYZZ/T/gphotos-cdp \
  https://photos.google.com/

where /var/folders/NN/XXYYZZ/T/gphotos-cdp is your profile directory, i.e. --user-data-dir=... or Session.profileDir.

For reference, this is how chromedp / (gphotos-cdp) starts Chrome on MacOS for me, where the problematic option for authentication is most-likely --enable-automation:

/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \
  --enable-automation \
  --use-mock-keychain \
  --no-default-browser-check \
  --enable-features=NetworkService,NetworkServiceInProcess \
  --disable-dev-shm-usage \
  --disable-popup-blocking \
  --disable-sync \
  --no-first-run \
  --disable-hang-monitor \
  --disable-ipc-flooding-protection \
  --metrics-recording-only \
  --safebrowsing-disable-auto-update \
  --disable-features=site-per-process,TranslateUI,BlinkGenPropertyTrees \
  --disable-renderer-backgrounding \
  --force-color-profile=srgb \
  --user-data-dir=/var/folders/NN/XXYYZZ/T/gphotos-cdp \
  --disable-background-networking \
  --disable-background-timer-throttling \
  --disable-backgrounding-occluded-windows \
  --disable-extensions \
  --disable-breakpad \
  --disable-client-side-phishing-detection \
  --disable-default-apps \
  --disable-prompt-on-repost \
  --password-store=basic \
  --remote-debugging-port=0 \
  about:blank
Ramblurr commented 4 years ago

I am experiencing the same behavior with my google account, however I am on Fedora Linux.

Using a normal chrome invocation and copying the data from ~/.config/google-chrome to /tmp/gphotos-cdp did not work for me.

The following invocation of Chrome was a sufficient workaround to create a profile (manually) that could then be re-used with gphotos-cdp's default options for the problematic account:

Seeding /tmp/gphotos-cdp using @daneroo 's invocation, logging in, exiting, and then running gphotos-cdp worked for me!

FrankReh commented 3 years ago

Genius! Thank you for this app and thank you for this issue and @daneroo 's invocation showing how to get around the authentication trouble. I'm running this from a Mac and experienced the same. The manual script let me login and authenticate, two-factor no less, and then a call to this excellent tool with -dev passed authentication and pulled them down.

florianmski commented 3 years ago

Just to re-iterate on how to make this work as it took me a bit even with the previous comments:

  1. go run main.go and take note of the session dir 
    Session Dir: /var/folders/FOO/BAR/T/gphotos-cdpSOMENUMBER
  2. run following and log in 
    /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \
--no-first-run \
--password-store=basic \
--use-mock-keychain \
--user-data-dir= /var/folders/FOO/BAR/T/gphotos-cdp \
https://photos.google.com/
  3. go run main.go --dev in order to re-use the same user-data-dir instead of generating a new one
madisonlurker commented 3 years ago

@florianmski does this work with chromium too. I am on linux and not having any luck. Do you have any step by step instructions.

florianmski commented 3 years ago

@madisonlurker Sorry not really, this instructions are for a mac user. I would guess it would not look too different in term of command line parameters for linux but you'd probably need to use chromium instead of /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \

kerneis commented 3 years ago

The instructions worked for me with chromium on Linux (Debian testing), using ./gphotos-cdp -dev -headless -v as the first and third step, and chromium --no-first-run --password-store=basic --use-mock-keychain --user-data-dir=/tmp/gphotos-cdp https://photos.google.com/ as the second step. I built with CGO_ENABLED=0 go build.

nomeata commented 1 year ago

Should this be reopened, given that it still affects user (see e.g. #18)?