Incorrect usage of http proxy causes some gam commands to fail

[bpoliakoff]

I know that using a proxy with gamadv-xtd3 isn't recommended but still expected to basically work (https://github.com/taers232c/GAMADV-XTD3/wiki/HTTPS-Proxy).

I've encountered a situation on a Debian Bullseye (11.6) client, using the latest released version of gamadv-xtd3 (6.51.02) where many commands work via a proxy, but some don't.

With the environment variables http_proxy and https_proxy set, the command "gam info user USERNAME" works properly.

But with the same variables set the command "gam user USERNAME check serviceaccount" fails with the following error:

Traceback (most recent call last):
  File "__init__.py", line 67922, in ProcessGAMCommand
  File "__init__.py", line 11373, in checkServiceAccount
  File "__init__.py", line 8663, in getLocalGoogleTimeOffset
  File "httplib2/__init__.py", line 1720, in request
  File "httplib2/__init__.py", line 1440, in _request
  File "httplib2/__init__.py", line 1362, in _conn_request
  File "httplib2/__init__.py", line 1045, in connect
  File "httplib2/socks.py", line 504, in connect
  File "httplib2/socks.py", line 465, in __negotiatehttp
httplib2.socks.HTTPError: (403, b'Proxy Error')

By checking my apache proxy server's logs, and verifying with a tcpdump packet capture, I can see that in the latter case gamadv-xtd3 is attempting to talk to a Google API endpoint via http (port 80), incorrectly using the http CONNECT verb. CONNECT is only used to create a tunnel, via a proxy, to an https protected service.

Here's the apache error log for the request (redacting client IP address): [Tue Mar 14 15:36:04.193180 2023] [proxy:error] [pid 27277:tid 140215851153152] [client 10.x.x.x:52600] AH00898: Connect to remote machine blocked returned by admin.googleapis.com:80

And here's the problematic request from my packet capture (redacting client and proxy server IP addresses):

307 0.328381 10.x.x.x 10.x.x.y HTTP CONNECT admin.googleapis.com:80 HTTP/1.1

When the same computer uses gamadv-xtd3 to talk directly to Google, the "check serviceaccount" command runs without a hitch.

I'm guessing one of several things could be happening here:

1) gamadv-xtd3 should be talking to admin.googleapis.com via https on tcp/443 (as the service is probably available via https) 2) or gamadv-xtd3 needs to be told to make the request to the proxy via the http "GET" verb for this API endpoint, if that particular endpoint really is only available via http rather than https 3) or the python library that gamadv-xtd3 uses to generate its requests has a bug with regard proxy usage

I was able to get things working by having my network admin make an exception and set up a one-to-one IP NAT for my server, so I'm set, but it feels like this issue should be fixed.

Ben

PS Thanks to the author and all contributors, this software is invaluable!

[taers232c]

Ben,

Send me a Meet/Zoom invitation and I'll help.

Ross

[taers232c]

Ben,

6.51.04 now connects via https

Ross

[bpoliakoff]

Fantastic, I just tried 6.51.04 and it seems to resolve the issue! Thanks for the insanely speedy turnaround! No need for a Meet/Zoom invite now, I assume, right?

[taers232c]

It would be helpful to me if we could.

Thanks

On Wed, Mar 15, 2023 at 12:08 PM Ben Poliakoff @.***> wrote:

Fantastic, I just tried 6.51.04 and it seems to resolve the issue! Thanks for the insanely speedy turnaround! No need for a Meet/Zoom invite now, I assume, right?

— Reply to this email directly, view it on GitHub https://github.com/taers232c/GAMADV-XTD3/issues/344#issuecomment-1470630895, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCTYL5YKJ34JN4A2OOEJSTW4IHRVANCNFSM6AAAAAAV4HD3VI . You are receiving this because you commented.Message ID: @.***>

-- Ross Scroggs @.***

[taers232c]

You said some commands work, other don't; what are they?

Thanks

[bpoliakoff]

I'm free now - 3pm Pacific today. https://meet.google.com/tsy-tyrq-igw Let me know if another time would be better.

[taers232c]

Ben,

I'm also in PST, talk to you at 3.

Ross

On Wed, Mar 15, 2023 at 1:25 PM Ben Poliakoff @.***> wrote:

I'm free now - 3pm Pacific today. https://meet.google.com/tsy-tyrq-igw Let me know if another time would be better.

— Reply to this email directly, view it on GitHub https://github.com/taers232c/GAMADV-XTD3/issues/344#issuecomment-1470791025, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCTYLZRJNYCZ22TBEV3BC3W4IQSBANCNFSM6AAAAAAV4HD3VI . You are receiving this because you commented.Message ID: @.***>

-- Ross Scroggs @.***

[taers232c]

Ben,

From Jay:

If local time is well off, it breaks https because the server certificate will be seen as too old or new and thus invalid. Http doesn't have that issue.

Jay

[taers232c]

More from Jay,

Note also though that we CAN NOT actually trust HTTP because anyone could spoof the server response. So the logic should be something like:

• Check HTTP server time and temporarily use it so that we're hopefully close
• Check HTTPS server time so we can validate we're really talking to Google servers and our time is accurate.

Feel free to send a PR if you have suggestions on improving that logic

[taers232c]

Try 6.51.05

[bpoliakoff]

But of course you can't actually use http for most of these API calls. Presumably the tiny minority of API endpoints that support http also support https. I'll try 6.51.05 tomorrow. But some of this feels like premature optimization, given that it would be entirely reasonable to abort if https is not available.

[bpoliakoff]

Just tested with the very latest release (GAMADV-XTD3 6.51.07), works great via the proxy!

[taers232c]

Good deal.

On Fri, Mar 17, 2023 at 11:02 AM Ben Poliakoff @.***> wrote:

Just tested with the very latest release (GAMADV-XTD3 6.51.07), works great via the proxy!

— Reply to this email directly, view it on GitHub https://github.com/taers232c/GAMADV-XTD3/issues/344#issuecomment-1474214367, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCTYL5PVV77CRZUYZDCAJLW4SRKZANCNFSM6AAAAAAV4HD3VI . You are receiving this because you commented.Message ID: @.***>

-- Ross Scroggs @.***

[jay-eleven]

Solved, closing...