Api does not properly reuse or destroy cURL handles when making private requests

[sudofox]

I was running into issues with placing orders and other things and realized that I'm ending up with many, many open connections to GDAX -- the curl handle itself is staying open after private requests. This only seems to occur for private (requiring API auth) requests.

<?php

// Demo - requires lsof to show us our open conns from PHP

require 'vendor/autoload.php'; // gives us our mrteye\Gdax stuff

$apiConfig = (object) array(
  'key' => 'GDAX_API_KEY_HERE',
  'secret' => 'GDAX_API_SECRET_HERE',
  'pass' => 'GDAX_API_PASS_HERE',

  'api_url' => 'https://api-public.sandbox.gdax.com',
  'time_url' => 'https://api-public.sandbox.gdax.com/time'
);

use mrteye\Gdax\Api as Api;
use mrteye\Gdax\Auth as Auth;
use mrteye\Gdax\Exception as Exception;

$products = false;

$apiAuth = new Auth(
        $apiConfig->key,
        $apiConfig->secret,
        $apiConfig->pass,
        $apiConfig->time_url
);

// Get the GDAX API and start making calls.
$gdaxPrivate = new Api($apiConfig->api_url, $apiAuth);

echo "[INFO] Api has been instantiated.\n";
echo "[INFO] ". exec("lsof -i|grep php|grep ESTABLISHED|wc -l") . " open connections to GDAX.\n";

while (true) {

  // A private request.

  echo "[INFO] Executing cancelAllOrders (a _privateRequest)\n";
  $gdaxPrivate->cancelAllOrders($param = [
      'product_id' => 'BTC-USD'
  ]);
  echo "[INFO] ". exec("lsof -i|grep php|grep ESTABLISHED|wc -l") . " open connections to GDAX.\n";

  // A public request.

  echo "[INFO] Executing getProducts (a _publicRequest)\n";
  $gdaxPrivate->getProducts();
  echo "[INFO] ". exec("lsof -i|grep php|grep ESTABLISHED|wc -l") . " open connections to GDAX.\n";

}

$ php example.php
[INFO] Api has been instantiated.
[INFO] 0 open connections to GDAX.
[INFO] Executing cancelAllOrders (a _privateRequest)
[INFO] 2 open connections to GDAX.
[INFO] Executing getProducts (a _publicRequest)
[INFO] 2 open connections to GDAX.
[INFO] Executing cancelAllOrders (a _privateRequest)
[INFO] 3 open connections to GDAX.
[INFO] Executing getProducts (a _publicRequest)
[INFO] 3 open connections to GDAX.
[INFO] Executing cancelAllOrders (a _privateRequest)
[INFO] 4 open connections to GDAX.
[INFO] Executing getProducts (a _publicRequest)
[INFO] 4 open connections to GDAX.
[INFO] Executing cancelAllOrders (a _privateRequest)
[INFO] 5 open connections to GDAX.
[INFO] Executing getProducts (a _publicRequest)
[INFO] 5 open connections to GDAX.
[INFO] Executing cancelAllOrders (a _privateRequest)
[INFO] 6 open connections to GDAX.
[INFO] Executing getProducts (a _publicRequest)
[INFO] 6 open connections to GDAX.
[INFO] Executing cancelAllOrders (a _privateRequest)
[INFO] 7 open connections to GDAX.
[INFO] Executing getProducts (a _publicRequest)
[INFO] 7 open connections to GDAX.
[INFO] Executing cancelAllOrders (a _privateRequest)
[INFO] 8 open connections to GDAX.
[INFO] Executing getProducts (a _publicRequest)
[INFO] 8 open connections to GDAX.
[INFO] Executing cancelAllOrders (a _privateRequest)
[INFO] 9 open connections to GDAX.
[INFO] Executing getProducts (a _publicRequest)
[INFO] 9 open connections to GDAX.
[INFO] Executing cancelAllOrders (a _privateRequest)
[INFO] 10 open connections to GDAX.

Oddly, we are left with 2 open connections after the first private API call.

Over time, the number of open connections to GDAX increase manyfold -- I've had it hit near a thousand -- before things stop working due to having too many open connections.

[sudofox]

Hey!

Wondering if you've been able to reproduce the issue? I've done some poking around myself; while I haven't had much time yet, I'm thinking about stripping down your class into the core components extending the Curl class (AppCurl) and the _call function to see what may be causing the difference.

I do see you calling resetConnection each time _call() runs. Perhaps the method type or extra headers is causing something different to occur?

Would be great to see this resolved.

[mrteye]

Hey, I think this issue is prematurely closed... I have not merged anything in yet, and I thought it would only close after I merged.

I did find the problem. The private calls all make use of the GDAX's timer api which uses a separate Curl connection. I was not closing this connection. It now keeps one connection open for all Auth calls.

Thank you very much for the well documented issue. I have created a test case with your examples. I should be submitting that soon also.

[mrteye]

Thanks again for your feedback. Everything is updated. The latest version is 1.2.0.

[sudofox]

Thank you very much! Very much appreciated, and I'm glad to help.