Can't make analytics calls work through a proxy server

[machinehum]

• OS: Debian GNU/Linux 12 (bookworm)
• PHP version: 8.2.15
• Package name and version: analytics-data 0.9.4, analytics-admin 0.10.0

I'm trying to retrieve Google Analytics 4 data with the google cloud PHP libraries and having a devil of a time with proxy settings (server is behind an outbound proxy). I'll start by saying that if I set the https_proxy env var, then everything works as expected, so my config is working otherwise. For various reasons that's not ideal and I'd rather deal with it in code, and that seems to be supported via things like passing an authHttpHandler to CredentialsWrapper, or rest transport configs, but I've tried a hundred permutations and can't make it work. I also can't find solid docs.

Here's one example (slightly simplified):

$proxyHttpClient =  new \GuzzleHttp\Client([
            'proxy'      => 'http://foo.com:3128',
            'exceptions' => false,
            'base_uri'   => 'https://www.googleapis.com'
        ]);

        $proxyHttpHandler = HttpHandlerFactory::build($proxyHttpClient);

        $credentialWrapper =  CredentialsWrapper::build(
            'keyFile' => $this->getClientConfiguration(),
            'authHttpHandler' => $proxyHttpHandler
        );

        $adminClient = new AnalyticsAdminServiceClient(['credentials' => $credentialWrapper, 'transportConfig' => ['rest' => ['httpHandler' => $proxyHttpHandler]]]);

        $adminClient->getProperty($propertyId);

This code times out at the proxy, at the first step of trying to reach oauth.googleapis.com. If I set https_proxy env var, it works. When I do some debug inspecting in CredentialsWrapper, it seems to be getting the correct arguments. I inspected a bunch of things at various points of GapicClientTrait::buildClientOptions and ::setClientOptions, and it seemed right. I tried a few permutations of the client constructor arguments, for example using credentialsConfig instead of constructing the CredentialsWrapper. No difference.

This feels hard to call a "bug" because there isn't real clear documentation the right way to do this, but I'm hoping to get some help here. In case it's useful this is what the credentialWrapper looks like at the end of GapicClientTrait:: setClientOptions:

credentialsWrapper:
Google\ApiCore\CredentialsWrapper::__set_state(array(
   'credentialsFetcher' =>
  Google\Auth\FetchAuthTokenCache::__set_state(array(
     'fetcher' =>
    Google\Auth\Credentials\UserRefreshCredentials::__set_state(array(
       'auth' =>
      Google\Auth\OAuth2::__set_state(array(
         'authorizationUri' => NULL,
         'tokenCredentialUri' =>
        GuzzleHttp\Psr7\Uri::__set_state(array(
           'scheme' => 'https',
           'userInfo' => '',
           'host' => 'oauth2.googleapis.com',
           'port' => NULL,
           'path' => '/token',
           'query' => '',
           'fragment' => '',
           'composedComponents' => NULL,
        )),
         'redirectUri' => NULL,
         'clientId' => 'XXXXXXXXX.apps.googleusercontent.com',
         'clientSecret' => 'XXXX-XXXXX-XXXXXXX',
         'username' => NULL,
         'password' => NULL,
         'scope' => NULL,
         'state' => NULL,
         'code' => NULL,
         'issuer' => NULL,
         'audience' => NULL,
         'sub' => NULL,
         'expiry' => 3600,
         'signingKey' => NULL,
         'signingKeyId' => NULL,
         'signingAlgorithm' => NULL,
         'refreshToken' => 'XXXXXXXXXXXX',
         'accessToken' => NULL,
         'idToken' => NULL,
         'grantedScope' => NULL,
         'expiresIn' => NULL,
         'expiresAt' => NULL,
         'issuedAt' => NULL,
         'grantType' => NULL,
         'extensionParams' =>
        array (
        ),
         'additionalClaims' =>
        array (
        ),
         'codeVerifier' => NULL,
         'resource' => NULL,
         'subjectTokenFetcher' => NULL,
         'subjectTokenType' => NULL,
         'actorToken' => NULL,
         'actorTokenType' => NULL,
         'issuedTokenType' => NULL,
      )),
       'quotaProject' => NULL,
    )),
     'eagerRefreshThresholdSeconds' => 10,
     'maxKeyLength' => 64,
     'cacheConfig' =>
    array (
      'lifetime' => 1500,
      'prefix' => '',
    ),
     'cache' =>
    Google\Auth\Cache\MemoryCacheItemPool::__set_state(array(
       'items' => NULL,
       'deferredItems' => NULL,
    )),
  )),
   'authHttpHandler' =>
  Auth\HttpHandler\Guzzle7HttpHandler::__set_state(array(
     'client' =>
    GuzzleHttp\Client::__set_state(array(
       'config' =>
      array (
        'proxy' => 'http://foo.com:3128',
        'exceptions' => false,
        'base_uri' =>
        GuzzleHttp\Psr7\Uri::__set_state(array(
           'scheme' => 'https',
           'userInfo' => '',
           'host' => 'www.googleapis.com',
           'port' => NULL,
           'path' => '',
           'query' => '',
           'fragment' => '',
           'composedComponents' => NULL,
        )),
        'handler' =>
        GuzzleHttp\HandlerStack::__set_state(array(
           'handler' =>
          \Closure::__set_state(array(
          )),
           'stack' =>
          array (
            0 =>
            array (
              0 =>
              \Closure::__set_state(array(
              )),
              1 => 'http_errors',
            ),
            1 =>
            array (
              0 =>
              \Closure::__set_state(array(
              )),
              1 => 'allow_redirects',
            ),
            2 =>
            array (
              0 =>
              \Closure::__set_state(array(
              )),
              1 => 'cookies',
            ),
            3 =>
            array (
              0 =>
              \Closure::__set_state(array(
              )),
              1 => 'prepare_body',
            ),
          ),
           'cached' => NULL,
        )),
        'allow_redirects' =>
        array (
          'max' => 5,
          'protocols' =>
          array (
            0 => 'http',
            1 => 'https',
          ),
          'strict' => false,
          'referer' => false,
          'track_redirects' => false,
        ),
        'http_errors' => true,
        'decode_content' => true,
        'verify' => true,
        'cookies' => false,
        'idn_conversion' => false,
        'headers' =>
        array (
          'User-Agent' => 'GuzzleHttp/7',
        ),
      ),
    )),
  )),
))

[machinehum]

@bshaffer Any thoughts?

[bshaffer]

@Hectorhammett (who is on rotation this week)

[bshaffer]

Just by skimming this, I know when I've used proxies in the past I've had to set "verify" to "false" in the guzzle client constructor.

[bshaffer]

I would suggest working directly with the guzzle client, and even making an HTTP call to our APIs directly with the guzzle client, instead of using the GAPIC client (you can authenticate using these instructions). Because if you can get that to work, then your above code should work as well. That will at least help you isolate the issue, to know if the problem is in your Guzzle configuration or your Google Client configuration.

Looking at your code, everything looks fine, so I'm guessing you need to configure something differently in guzzle. If that's not the case, we can help you debug from there. Good luck!

[machinehum]

I'm working on doing more tests as you described, but to be clearer on the initial issue: Nothing I do in the Guzzle configurations has any effect whatsoever. The configured Guzzle object shows up in e.g. GapicClientTrait debug as I shared before, but as far as I can tell that client is being completely ignored and a default one used when it comes time to actually make the OAuth requests. So either injecting it in the manner I did isn't actually supported, or there's a bug in the library that's causing it to be ignored. Can you clarify whether that method of injecting a Guzzle client should actually work?

[bshaffer]

I would pass in the client like this:

use Google\Auth\HttpHandler\HttpHandlerFactory;
use Google\Analytics\Admin\V1beta\Client\AnalyticsAdminServiceClient;

$proxyHttpClient =  new \GuzzleHttp\Client([
    'proxy'      => 'http://foo.com:3128',
    'exceptions' => false,
    'base_uri'   => 'https://www.googleapis.com'
]);

$proxyHttpHandler = HttpHandlerFactory::build($proxyHttpClient);

$adminClient = new AnalyticsAdminServiceClient([
    // pass the HTTP handler in to the "credentialsConfig" option
    'credentialsConfig' => ['authHttpHandler' => $proxyHttpHandler],
    // ensure we are using "rest" transport (gRPC is the default when the grpc.so extension is enabled)
    'transport' => 'rest', 
    // set the http handler on the RestTransport
    'transportConfig' => ['rest' => ['httpHandler' => $proxyHttpHandler]]
]);

[machinehum]

Testing this way, it seems to work. I'm going to keep testing to narrow down where the issue in the original code is.

One note: using the created Guzzle client for authHttpHandler worked, but trying to use it in the rest transport config caused a fatal error in RestTransport (with or without a proxy set in the Guzzle client). Here's my test code:

`use Google\Auth\ApplicationDefaultCredentials; use GuzzleHttp\Client; use GuzzleHttp\HandlerStack; use Google\Auth\HttpHandler\HttpHandlerFactory; use Google\Analytics\Admin\V1beta\Client\AnalyticsAdminServiceClient; use Google\Analytics\Admin\V1beta\GetPropertyRequest; use Google\Analytics\Admin\V1beta\Property; use Google\ApiCore\ApiException;

// specify the path to your application credentials putenv('GOOGLE_APPLICATION_CREDENTIALS=credentials.json');

// define the scopes for your API call $scopes = ['https://www.googleapis.com/auth/drive.readonly','https://www.googleapis.com/auth/analytics.edit', 'https://www.googleapis.com/auth/analytics.manage.users', 'https://www.googleapis.com/auth/analytics.manage.users.readonly', 'https://www.googleapis.com/auth/analytics.readonly'];

// create middleware $middleware = ApplicationDefaultCredentials::getMiddleware($scopes); $stack = HandlerStack::create(); $stack->push($middleware);

// create the HTTP client $client = new Client([ // 'proxy' => 'http://foo.com', 'verify' => false, 'exceptions' => false, 'handler' => $stack, 'base_uri' => 'https://www.googleapis.com', 'auth' => 'google_auth' // authorize all requests ]);

// make the request $response = $client->get('drive/v2/files');

// show the result! print_r((string) $response->getBody());

$proxyHttpHandler = HttpHandlerFactory::build($client);

$adminClient = new AnalyticsAdminServiceClient([ // pass the HTTP handler in to the "credentialsConfig" option 'credentialsConfig' => ['authHttpHandler' => $proxyHttpHandler], // ensure we are using "rest" transport (gRPC is the default when the grpc.so extension is enabled) 'transport' => 'rest', // set the http handler on the RestTransport 'transportConfig' => ['rest' => ['httpHandler' => $proxyHttpHandler]] ]);

$request = (new GetPropertyRequest()) ->setName('properties/9999999999999');

$response = $adminClient->getProperty($request); print_r((string) $response->getBody());`

The drive call works as expected. The AnalyticsAdminServiceClient call issues this fatal:

`Fatal error: Uncaught Error: Call to undefined method GuzzleHttp\Psr7\Response::then() in /vendor/google/gax/src/Transport/RestTransport.php:123 Stack trace:

0 /vendor/google/gax/src/GapicClientTrait.php(641): Google\ApiCore\Transport\RestTransport->startUnaryCall(Object(Google\ApiCore\Call), Array)

1 /vendor/google/gax/src/Middleware/CredentialsWrapperMiddleware.php(58): Google\Analytics\Admin\V1beta\Client\AnalyticsAdminServiceClient->Google\ApiCore{closure}(Object(Google\ApiCore\Call), Array)

2 /vendor/google/gax/src/Middleware/FixedHeaderMiddleware.php(68): Google\ApiCore\Middleware\CredentialsWrapperMiddleware->__invoke(Object(Google\ApiCore\Call), Array)

3 /vendor/google/gax/src/Middleware/RetryMiddleware.php(92): Google\ApiCore\Middleware\FixedHeaderMiddleware->__invoke(Object(Google\ApiCore\Call), Array)

4 /vendor/google/gax/src/Middleware/RequestAutoPopulationMiddleware.php(73): Google\ApiCore\Middleware\RetryMiddleware->__invoke(Object(Google\ApiCore\Call), Array)

5 /vendor/google/gax/src/Middleware/OptionsFilterMiddleware.php(61): Google\ApiCore\Middleware\RequestAutoPopulationMiddleware->__invoke(Object(Google\ApiCore\Call), Array)

6 /vendor/google/gax/src/GapicClientTrait.php(606): Google\ApiCore\Middleware\OptionsFilterMiddleware->__invoke(Object(Google\ApiCore\Call), Array)

7 /vendor/google/gax/src/GapicClientTrait.php(552): Google\Analytics\Admin\V1beta\Client\AnalyticsAdminServiceClient->startCall('GetProperty', 'Google\Analytic...', Array, Object(Google\Analytics\Admin\V1beta\GetPropertyRequest), 0, 'google.analytic...')

8 /vendor/google/cloud/AnalyticsAdmin/src/V1beta/Client/AnalyticsAdminServiceClient.php(1452): Google\Analytics\Admin\V1beta\Client\AnalyticsAdminServiceClient->startApiCall('GetProperty', Object(Google\Analytics\Admin\V1beta\GetPropertyRequest), Array)

9 /test.php(56): Google\Analytics\Admin\V1beta\Client\AnalyticsAdminServiceClient->getProperty(Object(Google\Analytics\Admin\V1beta\GetPropertyRequest))

10 {main}

thrown in /vendor/google/gax/src/Transport/RestTransport.php on line 123`

[bshaffer]

That error is because it's expecting a Promise but it's getting a Response. That can be fixed by calling async on the HttpHandler instead of just passing in the handler, e.g. passing in this to the transport config instead of $httpHandler:

[$httpHandler, 'async']

[Hectorhammett]

Any updates @machinehum ? Did the suggested solution worked?

[Hectorhammett]

Closed due inactivities! If you need more assistance please let us know!