Same old Refresh token issue

[paqio]

Been reading old issues, but at least for me the problem is still here.

Could not connect to Exact: Could not acquire or refresh tokens [http 401]

  $config = LaravelExactOnline::loadConfig() == null ? new \App\Exact() : LaravelExactOnline::loadConfig();

            $connection = new \Picqer\Financials\Exact\Connection();

            $connection->setRedirectUrl(route('exact.callback'));
            $connection->setExactClientId(config('laravel-exact-online.exact_client_id'));
            $connection->setExactClientSecret(config('laravel-exact-online.exact_client_secret'));
            $connection->setBaseUrl('https://start.exactonline.' . config('laravel-exact-online.exact_country_code'));

            if (config('laravel-exact-online.exact_division') !== '') {
                $connection->setDivision(config('laravel-exact-online.exact_division'));
            }

            if (isset($config->authorisationCode)) {
                $connection->setAuthorizationCode($config->authorisationCode);
            }

            if (isset($config->accessToken)) {
                $connection->setAccessToken(unserialize($config->accessToken));
            }

            if (isset($config->refreshToken)) {
                $connection->setRefreshToken($config->refreshToken);
            }

            if (isset($config->tokenExpires)) {
                $connection->setTokenExpires($config->tokenExpires);
            }

            try {
                if (isset($config->authorisationCode)) {
                    $connection->connect();
                }
            } catch (\GuzzleHttp\Exception\RequestException $e) {
                $connection->setAccessToken(null);
                $connection->setRefreshToken(null);
                $connection->connect();
            } catch (\Exception $e) {
                throw new \Exception('Could not connect to Exact: ' . $e->getMessage());
            }

            $config->accessToken = serialize($connection->getAccessToken());
            $config->refreshToken = $connection->getRefreshToken();
            $config->tokenExpires = $connection->getTokenExpires();

            LaravelExactOnline::storeConfig($config);

            return $connection;

[paqio]

Tried also the tokenUpdateCallback , couldn't make it work. Callback was not called

[notfalsedev]

I'm having the same issue. If i'm correct Exact has changed the token expire time to 10 minutes.

I think the issue is in the tokenHasExpired function in the Connection class.

private function tokenHasExpired()
{
    if (empty($this->tokenExpires)) {
        return true;
    }

    return $this->tokenExpires <= time() + 10;
}

In this case the token is always expiring before the time has being past. I think this could be switched, let the token expire before the time. Like this:

private function tokenHasExpired()
{
    if (empty($this->tokenExpires)) {
        return true;
    }

    return ($this->tokenExpires - 60) < time();
}

The token wil "expire" 1 minute before the actual expire time, and the token will be refreshed. So the next requests will have a valid token.

[notfalsedev]

Created a pull request for this #387 Hope this fixes the issue.

[websmurf]

Same issue for me, also with LaravelExactOnline. Will check out the latest master branch to test your commit @artisaninweb

[paqio]

It works for me use this in the Provider:

 if (isset($config->tokenExpires)) {
                $connection->setTokenExpires($config->tokenExpires);
            }
            $connection->setTokenUpdateCallback('\App\Exact::tokenUpdateCallback');

            try {
                $connection->connect();
            } catch (\Exception $e) {
                throw new \Exception('Could not connect to Exact: ' . $e->getMessage());
            }
 return $connection;

Artisans solution doesn't look right

[eaibizz]

Using the setTokenUpdateCallback to a callable function which saves your config does work indeed. Don't forget to set the data from the config before calling the connect()-function again. And don't be so stupid like me to "cache" the connection as long as the refresh token is valid as it's impossible to refresh your token when it has already expired. It should be renewed just before being expired, which this client handles automatically.

[websmurf]

I've created a pull request for the Laravel package to implement this method. That should resolve the issue (it does for me so far).

[whvandervelde]

The implementation in this lib seems to cause confusion/trouble for more people. I was running into this issue as well and already used a pre-expiry period of 60 seconds to make sure access_token is valid when trying API calls.

To explain, after acquiring an authorization_code and swapping it for an access_token and refresh_token, AFAIK using only the access_token should be enough to use the API because its supposed to be a bearer token (with normally a short limited lifespan).

However when trying to access the API with this lib it would still return unauthorized 401, that is until I added the authorization_code as well. That just seems wrong because the authorization_code is exchanged for an access_token and after that should not matter anymore.

Skipping through the code, the confusion seems to come from the connect() function that expects either refresh_token or authorization_code to be set, otherwise it is assumed to need Authentication. See line 164 and 377 on https://github.com/picqer/exact-php-client/blob/master/src/Picqer/Financials/Exact/Connection.php

I understand the lib wants to help by doing things automagically, but requiring setting a refresh_token or authorization_code doesn't make sense to me. When you know you have a valid access_token already and/or are managing tokens etc yourself this is confusing to say the least and makes the lib less flexible to use.

Maybe the above also helps somebody in case they missed setting either refresh token or authorization_code as well as an access_token .. or maybe somebody can help me understand.

[stephangroen]

It might be nice to change this in a future major release. I agree that it shouldn't be required to set a refresh token, but changing this would impact the lib too much in terms of expected functionality.

[FawadNL]

I have the same issue above. First time token was generated but it is not generated automatically after expire callback

[HMDag]

Someone found a solution for this issue?

[FawadNL]

Hi Guys, I have found a working solution for this for once and for all, please read my comment in my PR #417

[JelleSolvari]

I've created a pull request for the Laravel package to implement this method. That should resolve the issue (it does for me so far).

Where can I find the Laravel package?

[ovvessem]

@joostsolvari you can find it here https://github.com/websmurf/laravel-exact-online. Keep in mind that this is forked from an abandoned project.

[websmurf]

@ovvessem @joostsolvari I've published a few updates on the package (and created a few releases as well in order to avoid having to use dev-master)

[DePalmo]

Where to get this latest release? I checked, latest tagged is 3.24.0 and I am using that one ... and I am again having Could not acquire or refresh tokens [http 401] despite following everything for past year! It looks like the code works from a few days and up to few weeks but then randomly hitting this error.

Is it possible that the refresh token expires too and that's the reason why I am seeing this error? Should I create a CRON job that would check the validity of the keys more periodically and update if required?

[websmurf]

@DePalmo I've run into this quite a bit as well; what I've noticed (we monitor all exceptions and errors through Sentry) is that it almost always happend just after an exception occurred on another part of the logic flow. So I think the callback doesn't get called properly if an exception is thrown somewhere, resulting in a not updated refresh_token

[DePalmo]

That sounds logical why it could happen, but in our case, the ExactOnline sync is completely separated from other workflow, so there are no exceptions expected. I am too logging all errors, but none up till now correlates (with datetime stamps, at least) with this error.

So that's why I got the idea that it might happen that the refresh token expires too and we're getting back this error. Unfortunately it seems that there's no info on how long the refresh token is valid.

[FawadNL]

@DePalmo you are right in that probably, cause when I created cronjob, the issue would no longer occur. Note; it only works if you also use the customization I did in connection.php

417

This is my cronjob for a custom wordpress integration:

<?php
$wp_load = substr( dirname( __FILE__ ), 0, strpos( dirname( __FILE__ ), 'wp-content' ) ) . 'wp-load.php';

if ( ! empty( $wp_load ) && file_exists( $wp_load ) ) {
        require_once $wp_load;
        //cron things
} else {
        die('Could not load WordPress');
}

global $connection;
if(get_option('exactonline2_reset') == "No") {  

    $connection = connect();
    print_r($connection);

}

[alexjeen]

Hi all,

Regarding this, please use a mutex that makes sure the refresh token can not be acquired with multiple threads:

    $connection->setAcquireAccessTokenLockCallback(function() use ($model) {
        return \Yii::$app->mutex->acquire('credential_refresh_' . $model->credential_id, 60);
    });
    $connection->setAcquireAccessTokenUnlockCallback(function() use ($model) {
        return \Yii::$app->mutex->release('credential_refresh_' . $model->credential_id);
    });

The library supports this, but you just need to use a mutex that is local to your framework. Most of the time this issue arises when one thread refreshes a token, and the other shortly there after tries to use it.

[DePalmo]

Thank you for pointing out these functions! Did do some testing but would appreciate some clarifications.

1. I noticed that setAcquireAccessTokenLockCallback is called each time the connect code runs. Is that correct?
2. setAcquireAccessTokenUnlockCallback is called just before the connect code finishes?
3. Inside setAcquireAccessTokenLockCallback, should I call the database and load the token if it was refreshed and then store it back? I am looking into this: https://github.com/picqer/exact-php-client/pull/381

[eaibizz]

Hi, I'm also still suffering from issues with the refresh token being old when refreshing.

In the past few weeks I've debugged between other projects for other customers the cron for the one customer we've built a link to Exact with. Nevertheless from oktober 2019 until mid of august this year the cron has been working correctly, but in the last month we're continuesly suffering from the refresh token not being able to refresh the access token because it's too old.

The reaction from the support desk was:

Oplossing - Mogelijke oplossing van uw vraag Onze handling van de refresh tokens is verbeterd waardoor het hergebruiken van een token refresh token nog scherper wordt gecontroleerd. We krijgen veel meldingen binnen van mensen die de picqer library gebruiken en bij (tot nu toe) alle gevallen is naar voren gekomen dat er in een hele korte tijd (in dezelfde seconde), meerdere calls worden gedaan voor het opvragen van een nieuw token. Graag controleren dat uw applicatie inderdaad maar 1 keer een call doet voor het opvragen van een nieuw access token en niet per ongeluk meerdere waardoor de tokens uit de vorige call weer ongeldige worden verklaard.

When I debug Connection.php by sending a mail to myself when:

• acquireAccessToken
• createRequest
• parseResponse are being called, I only once get a mail with the request for access tokens (directly after the initial authorization request has been sent to acquire an authorization code).

When the cron is set to run every 3 minutes (it takes 1 minute to complete), I get 3 successfull responses and the fourth one is being denied for having a refresh token which is too old. In the mean time I only get mails with the CRUD-requests, but never a request for a new token as the code is reusing the same client for the tokens not being expired (yet). Only after the 10 minutes of token validity have been expired, the next request is failing in acquireAccessToken with the BadResponseException.

What I've already tried without any success:

• Updated this package to its latest version
• Implemented the acquireAccessTokenLockCallback en ...UnlockCallback and I see those are successfully called.
• Implementing a check for a valid access token in my own code to not recalling the connect() method again, but reusing the existing client within the time frame of 10 minutes
• Called the acquireAccessToken endpoint manually after having received the first access and refresh token, but even then I get a "Refresh token is too old" message

For your information: Our cronjob is normally running every 20 minutes and uses this command to prevent the cron being started multiple times within the same timeframe: /usr/bin/flock -n /tmp/clientname.lock nice -n 15 /home/clientname/domains/clientname.ext/public_html/cronjobfilename.ext cronjob run 1> /dev/null --

What is really frustrating is that every time I want the cronjob to succeed, I need to clear all the saved tokens from the database within 10 minutes before the cron is starting, authenticating myself for a new authorization code and waiting for the cron having completed. This I'm doing at the moment 2 times a day manually for the customer to have its sales orders submitted to Exact on a daily base. There doesn't seem to be another way at the moment, besides having a solution to fix the annoying refresh token too old issue, so everything will be running automatically again.

Do you have any clue what to do next, cause I'm really running out of options 😞

[alexjeen]

Hi there, same for you eabizz, implement the locks around the connect function, and it works fine. We have hunderds of different companies setup with Exact connections, and we never encounter this issue.

@DePalmo

1. I noticed that setAcquireAccessTokenLockCallback is called each time the connect code runs. Is that correct?

Yes, see sample code below.

2. setAcquireAccessTokenUnlockCallback is called just before the connect code finishes?

The callback in that function is called, to acquire a lock. The function itself is not called, but internally it uses this as a way to continue.

3. Inside setAcquireAccessTokenLockCallback, should I call the database and load the token if it was refreshed and then store it back? I am looking into this: #381

No, you should use a mutex of sorts, to make sure there is only one thread calling it at the same time. We use a database mutex for Yii2 because our application is distributed. The mutex should exist on all the applications that are using the token. So if you are running a cron, and a web app, they should use the same mutex so that if a customer tries to find something in the web app at the same the time the cron is running, only one thread refreshes the token. We use the Mutex from Yii2 as described here: https://www.yiiframework.com/doc/api/2.0/yii-mutex-mutex We also specify a timeout of 60 seconds, that way the other application can always continue with a slight delay, as the other thread is refreshing the token.

This is the sample code how we have implemented it on our side:

$connection = new Connection();
$connection->setRedirectUrl(\Yii::$app->params['exact_'.strtolower($model->meta_tenant_country).'_redirect_url']);
$connection->setExactClientId(\Yii::$app->params['exact_'.strtolower($model->meta_tenant_country).'_client_id']);
$connection->setExactClientSecret(\Yii::$app->params['exact_'.strtolower($model->meta_tenant_country).'_client_secret']);
$connection->setBaseUrl('https://start.exactonline.' . strtolower($model->meta_tenant_country));
$connection->setAcquireAccessTokenLockCallback(function() use ($model) {
    return \Yii::$app->mutex->acquire('credential_refresh_' . $model->credential_id, 60);
});
$connection->setAcquireAccessTokenUnlockCallback(function() use ($model) {
    return \Yii::$app->mutex->release('credential_refresh_' . $model->credential_id);
});

if ($code) { // Retrieves authorizationcode from database
    $connection->setAuthorizationCode($code);
}

if ($model->getAuthValue('exact_access')) { // Retrieves accesstoken from database
    $connection->setAccessToken(json_decode($model->getAuthValue('exact_access'), true));
}

if ($model->getAuthValue('exact_refresh')) { // Retrieves refreshtoken from database
    $connection->setRefreshToken($model->getAuthValue('exact_refresh'));
}

if ($model->getAuthValue('exact_expires')) { // Retrieved the expires from database
    $connection->setTokenExpires($model->getAuthValue('exact_expires'));
}

try {
    if ($model->getAuthValue('exact_access') && $redirect) {
        // note that this always redirects
        $connection->connect();
    }
} catch (\Exception $e) {
    throw new \Exception('Could not connect to Exact: '.$e->getMessage());
}

Read up on what a Mutex is here: https://stackoverflow.com/questions/34524/what-is-a-mutex

[alexjeen]

Also please note, that the current implementation at Exact is that access tokens and refresh tokens are now user bound. That means if person A makes a connection in system A with app A, and also makes a connection in system B with app A, one of those apps will get refresh app tokens.

If you have multiple applications that use the same user account, either use a new client_id or actually use two user accounts.

This wasn't the case before.

[alexjeen]

And more.. stuff since I've bumped my head into this multiple times, the access token has to be valid until the END of the request at Exact, we use a trick here and refresh the token a couple of minutes earlier. That way the token is always valid until the end of the request even if it takes 5 minutes:

    $model->setAuthValue('exact_access', json_encode($connection->getAccessToken()));
    $model->setAuthValue('exact_refresh', $connection->getRefreshToken());
    // expire 5 min earlier
    $model->setAuthValue('exact_expires', $connection->getTokenExpires() - (5*60));
    $model->setAuthValue('exact_code', $code);
    $model->save();

[eaibizz]

And more.. stuff since I've bumped my head into this multiple times, the access token has to be valid until the END of the request at Exact, we use a trick here and refresh the token a couple of minutes earlier. That way the token is always valid until the end of the request even if it takes 5 minutes:

    $model->setAuthValue('exact_access', json_encode($connection->getAccessToken()));
    $model->setAuthValue('exact_refresh', $connection->getRefreshToken());
    // expire 5 min earlier
    $model->setAuthValue('exact_expires', $connection->getTokenExpires() - (5*60));
    $model->setAuthValue('exact_code', $code);
    $model->save();

Thanks, I'm going to try this, since the (mutex) lock was already implemented and we've excluded that multiple applications or threads were running at the same time.

[DePalmo]

@alexjeen Thank you for your thorough explanation and time you took to write it all! Now I have better understanding how these two functions should be used and will follow your advice. And I just hope that this will solve this never-ending problem.

Fingers crossed!

[woutersamaey]

@alexjeen I was carefully reading your implementation, but was wondering if there should be an exception thrown if the lock is already set. How does this code actually prevent anything?

This code returns booleans and there are no exceptions.

$connection = new Connection(); ... $connection->setAcquireAccessTokenLockCallback(function() use ($model) { return \Yii::$app->mutex->acquire('credentialrefresh' . $model->credential_id, 60); }); $connection->setAcquireAccessTokenUnlockCallback(function() use ($model) { return \Yii::$app->mutex->release('credentialrefresh' . $model->credential_id); });

In Picquer, the callbacks are run at the beginning and the end, but I can't see where the concurrent request is in fact stopped or prevented from running? There is also no method like "isLocked()" ? Shouldn't there be?

If I throw an exception from the acquireAccessTokenLockCallback, the exception will pass the finally block and get unlocked straight away, so that's no good.

[woutersamaey]

Okay, I think I misunderstood. The lock method actually halts execution until the lock is freed, then everything continues normally. You will still refresh multiple times, but at least it won't happen at the same time.

That's the whole idea right?

[DePalmo]

From what I understood, that is correct. When the acquireAccessTokenLockCallback is called the first time, it sets a "lock" and when called the second time:

• if no lock, code continues
• if lock found, second code execution is "paused" until the lock from first execution is released

Since I am on core PHP, I was not able to use above code because it's from Yii framework and PHP v7 no longer includes Mutex class (and couldn't find a suitable replacement), I created my own solution by using PhpFastCache (which was already integrated into my code, you can find a sample using flock on SO):

function mutex_lock($cache) {
    // loop for 90 seconds, waiting for the lock to be released, but only if lock is already in place
    for ($i = 0; $i < 90 && $cache->has('exact-locked'); $i++) {
        sleep(1);
    }
    // if no lock found, set it
    if (!$cache->set('exact-locked', true, 30)) {
        trigger_error('Mutex: unable to get a lock for new request!');
    }
    return true;
}

function mutex_unlock($cache) {
    // clear the lock
    return $cache->delete('exact-lock');
}

And then I use above like this:

$connection->setAcquireAccessTokenLockCallback(function () use ($cache) {
    return mutex_lock($cache);
});

$connection->setAcquireAccessTokenUnlockCallback(function () use ($cache) {
    return mutex_unlock($cache);
});

[woutersamaey]

@DePalmo the Yii framework just runs 2 MySQL queries. If you want, you can run them without a framework.

Locking

SELECT GET_LOCK(%s, %u);

Result should be string "1".

Unlocking

SELECT RELEASE_LOCK(%s);

No result

[DePalmo]

@woutersamaey Thank you for pointing this out! Since I already have a working solution (for now!), I am sure this will be helpful to future readers.

[brunogoossens]

After investigating the problem with postman, I noticed that the first time you get an access and refresh token from the authorization code request. You need to refresh the token instead of using the access token until it expires. After doing so, I could use the access token en refresh tokens without problems. The example code does not refresh the token after the authorization request witch results in the old refresh token used error after 10 minutes.

This was my solution. I divided the connection into 3 parts:

// called by the authorize route
public function authorizeConnection($host)
{
    $connection = new Connection();
    $connection->setBaseUrl($this->systemConfigService->get('ExactOnline.config.baseURL'));
    $connection->setExactClientId($this->systemConfigService->get('ExactOnline.config.clientID'));
    $connection->setExactClientSecret($this->systemConfigService->get('ExactOnline.config.clientSecret'));
    $connection->setRedirectUrl($host . '/exact-online/oauth/callback');
    $connection->connect();
}

// called by the callback route
public function initConnection($host, $code = false)
{
    // When getting the access token and refresh token for the first time, you need to refresh the token instead of using the access token. Else the refresh token does not work!
    $connection = new Connection();
    $connection->setBaseUrl($this->systemConfigService->get('ExactOnline.config.baseURL'));
    $connection->setExactClientId($this->systemConfigService->get('ExactOnline.config.clientID'));
    $connection->setExactClientSecret($this->systemConfigService->get('ExactOnline.config.clientSecret'));
    $connection->setRedirectUrl($host . '/exact-online/oauth/callback');
    $connection->setAuthorizationCode($code);
    $connection->connect();
    $connection->setAccessToken(null);
    $connection->connect();
    $this->systemConfigService->set('ExactOnline.config.refreshToken', $connection->getRefreshToken());
    $this->systemConfigService->set('ExactOnline.config.accessToken', $connection->getAccessToken());
    $this->systemConfigService->set('ExactOnline.config.tokenExpires', $connection->getTokenExpires());
}

// called when talking to exact
public function getConnection($host = false) : Connection
{
    $connection = new Connection();
    $connection->setExactClientId($this->systemConfigService->get('ExactOnline.config.clientID'));
    $connection->setExactClientSecret($this->systemConfigService->get('ExactOnline.config.clientSecret'));
    $connection->setBaseUrl($this->systemConfigService->get('ExactOnline.config.baseURL'));
    $connection->setAccessToken($this->systemConfigService->get('ExactOnline.config.accessToken'));
    $connection->setRefreshToken($this->systemConfigService->get('ExactOnline.config.refreshToken'));
    $connection->setTokenExpires($this->systemConfigService->get('ExactOnline.config.tokenExpires'));

    $logger = $this->logger;
    $systemConfigService = $this->systemConfigService;
    $connection->setTokenUpdateCallback(function ($connection) use ($logger, $systemConfigService) {
        $systemConfigService->set('ExactOnline.config.refreshToken', $connection->getRefreshToken());
        $systemConfigService->set('ExactOnline.config.accessToken', $connection->getAccessToken());
        $systemConfigService->set('ExactOnline.config.tokenExpires', $connection->getTokenExpires());
    });

    $connection->connect();

    return $connection;
}

[rutgergrasgroen]

@FawadNL How can i use your PR in my composer, so i can use you code in my project?

[FawadNL]

@rutgergrasgroen you can use my fork: https://github.com/FawadNL/exact-php-client But also ensure to apply to above solution of Mutex as well for concurrency. And you might still need to create a cronjob, in our case it doesn't work without cronjob.

[FawadNL]

@stephangroen Maybe its a good idea to mention the use of mutex in the readme as well

[rutgergrasgroen]

@FawadNL Ok thanks,i will look into that. What code do i need to trigger in the cronjob? And how often?

[eaibizz]

After investigating the problem with postman, I noticed that the first time you get an access and refresh token from the authorization code request. You need to refresh the token instead of using the access token until it expires. After doing so, I could use the access token en refresh tokens without problems. The example code does not refresh the token after the authorization request witch results in the old refresh token used error after 10 minutes.

This was my solution. I divided the connection into 3 parts:

// called by the authorize route
public function authorizeConnection($host)
{
    $connection = new Connection();
    $connection->setBaseUrl($this->systemConfigService->get('ExactOnline.config.baseURL'));
    $connection->setExactClientId($this->systemConfigService->get('ExactOnline.config.clientID'));
    $connection->setExactClientSecret($this->systemConfigService->get('ExactOnline.config.clientSecret'));
    $connection->setRedirectUrl($host . '/exact-online/oauth/callback');
    $connection->connect();
}

// called by the callback route
public function initConnection($host, $code = false)
{
    // When getting the access token and refresh token for the first time, you need to refresh the token instead of using the access token. Else the refresh token does not work!
    $connection = new Connection();
    $connection->setBaseUrl($this->systemConfigService->get('ExactOnline.config.baseURL'));
    $connection->setExactClientId($this->systemConfigService->get('ExactOnline.config.clientID'));
    $connection->setExactClientSecret($this->systemConfigService->get('ExactOnline.config.clientSecret'));
    $connection->setRedirectUrl($host . '/exact-online/oauth/callback');
    $connection->setAuthorizationCode($code);
    $connection->connect();
    $connection->setAccessToken(null);
    $connection->connect();
    $this->systemConfigService->set('ExactOnline.config.refreshToken', $connection->getRefreshToken());
    $this->systemConfigService->set('ExactOnline.config.accessToken', $connection->getAccessToken());
    $this->systemConfigService->set('ExactOnline.config.tokenExpires', $connection->getTokenExpires());
}

// called when talking to exact
public function getConnection($host = false) : Connection
{
    $connection = new Connection();
    $connection->setExactClientId($this->systemConfigService->get('ExactOnline.config.clientID'));
    $connection->setExactClientSecret($this->systemConfigService->get('ExactOnline.config.clientSecret'));
    $connection->setBaseUrl($this->systemConfigService->get('ExactOnline.config.baseURL'));
    $connection->setAccessToken($this->systemConfigService->get('ExactOnline.config.accessToken'));
    $connection->setRefreshToken($this->systemConfigService->get('ExactOnline.config.refreshToken'));
    $connection->setTokenExpires($this->systemConfigService->get('ExactOnline.config.tokenExpires'));

    $logger = $this->logger;
    $systemConfigService = $this->systemConfigService;
    $connection->setTokenUpdateCallback(function ($connection) use ($logger, $systemConfigService) {
        $systemConfigService->set('ExactOnline.config.refreshToken', $connection->getRefreshToken());
        $systemConfigService->set('ExactOnline.config.accessToken', $connection->getAccessToken());
        $systemConfigService->set('ExactOnline.config.tokenExpires', $connection->getTokenExpires());
    });

    $connection->connect();

    return $connection;
}

Big thanks to @brunogoossens who shared the final trick which made the Exact connection work again, so I don't have to manually reconnect anymore 3 times a day to be sure all data is being sent. I've done what he said: dividing the connect method into the authorize(), initConnection() and connection() method where the initConnection()-function does the double connection with and without access token and now the refresh token is being refreshed every time the cronjob is running.

[rutgergrasgroen]

@eaibizz Could you show me the complete code of your cronjob?

[eaibizz]

@rutgergrasgroen Unfortunately I can't show you the complete code as it isn't an open source project and there are way too many different classes involved to dump all of the code here in a post. Regarding the 3 stages of connection, I've used the same structure as Bruno did in his post combined with the acquireAccessToken(Un)lockCallbacks from earlier posts for which I didn't use a custom library but a simple file_put_contents() in combination with file_exists() and unlink().

The steps of the authorization (triggered within the GUI) are:

• User pushes a button called "Link to Exact"
• User is being redirected to Exact Online by the authorizeConnection() function
• User logs in at Exact Online
• User is being redirected to the response url of our application with query parameter: ?code=XXX
• We save the authorization code into our database config
• When successfully saved the authorization code into our database, we directly call the initConnection() function to acquire and refresh the tokens
• Now the access token and refresh token are being saved into our database config
• If saved correctly, we redirect the user to a success page

The steps of the cronjob are (in the case of SalesInvoices for example):

• Connect to Exact
• Get sales invoices from our database which haven't been sent to Exact yet
• Map our sales invoices from the database to Picqer\Exact class, with the following subtasks as example:
  • Retrieve customer accounts by email address and create custom account if not existing, else use GUID of existing account
  • Retrieve existing products by unique id and create new product if not existing, else use GUID of product
  • ...etc...
• Send mapped sales invoices with Picqer\Exact library to Exact
• Save in our database which sales invoices has been sent successfully, so they won't be retrieved in the next run of the cronjob

With every connection type (authorization, initConnection and regularConnection ) I set:

• RedirectURL => response url
• ExactClientId
• ExactClientSecret
• TokenUpdateCallback => callable function
• AcquireAccessTokenLockCallback => callable function
• AcquireAccessTokenUnlockCallback => callable function

In the authorization connection type I set (from our database config):

• Division id (if available)

In the initConnection connection type I set (from our database config):

• Authorization code
• Division id (if available)

In the regularConnection connection type I set (from our database config):

• Access token
• Refresh token
• Expires in
• Division id (if available)

In the tokenUpdateCallback I save into our database config:

• Access token
• Refresh token
• Expiration date

In the acquireAccessTokenLockCallback I check if a file (accesstoken.lock) exists in a custom directory, if not I write one with as contents the date at which it has been created via file_put_contents().

In the acquireAccessTokenUnlockCallback I check if the file (accesstoken.lock) exists the custom directory. If it does, I do a unlink() to remove the file.

To debug the cron and to generate output for later debugging, I call the log() function for every step that I take to see the output:

/**
   * Log message
   *
   * @param string $message Message
   *
   * @return void
   */
  protected function log($message)
  {
      echo '[' . date("Y-m-d H:i:s") . '] ' . $message . '<br />' . PHP_EOL;
  }

For example:

$this->log("Start sequential connection with Exact");

With an output similar to this:

[2020-11-04 11:40:02] Start exporting sales invoices
[2020-11-04 11:40:02] Connect to ExactOnline
[2020-11-04 11:40:02] Start sequential connection with Exact
[2020-11-04 11:40:02] Creating the access token lockfile
[2020-11-04 11:40:02] - The access token lock has been successfully created
[2020-11-04 11:40:03] Save acquired tokens
[2020-11-04 11:40:03] - Access token: [VALUE OF THE ACCESS TOKEN]
[2020-11-04 11:40:03] - Refresh token: [VALUE OF THE REFRESH TOKEN]
[2020-11-04 11:40:03] - Expiration date: 2020-11-04 11:50:03
[2020-11-04 11:40:03] Unlocking the access token lockfile
[2020-11-04 11:40:03] - The access token lock has been successfully removed
[2020-11-04 11:40:03] Get sales invoices to export
[2020-11-04 11:40:03] 5 sales invoices exported
[2020-11-04 11:40:03] Save response in database

If you have further questions about our implementation, feel free to send me a DM. It can be in Dutch if you'd like.

[FawadNL]

@eaibizz thank you for sharing your code. However, when a second user connects to exact, the GET/POSTS calls throws an error: "Wrong Division" Even though the correct division id is used for that second user. Any ideas why this is happening?

[eaibizz]

@FawadNL Did you already debug Connection.php of this package to see if you really get the same division ID back for that second user which you pass to the connection object? In our case we've only added support for one single division ID, so we won't run against this issue.

Could you verify that you don't create a new connection for user 1 and your code caches the connection so user 2 is reusing the same connection?

[alexjeen]

Hello,

I made a PR for this as well, as for us, in some cases it was not working (timing issue): https://github.com/picqer/exact-php-client/pull/449

In our case, if we would have three conversions at the same time, 2/3 would not get the new "refresh" token. With this PR that is fixed. You just need to implement that together with the mutex locks above:

$connection->setTokenRefreshCallback(function ($connection) use ($model) {
    // we use Yii2 ActiveRecord, so we call refresh() to update the model with the latest token information from the db
    $model->refresh();
    // we then set these new tokens and their expiry date into the class, so it will not refresh again 
    $connection->setAccessToken(json_decode($model->getAuthValue('exact_access'), true));
    $connection->setRefreshToken($model->getAuthValue('exact_refresh'));
    $connection->setTokenExpires($model->getAuthValue('exact_expires'));
});

With this callback, we can post 50/60 files at the same time to Exact without issues 👍

Please note, that you would only really need this, if multiple threads are using the same access token and refresh token. With this callback you can prevent that one of those threads gets out of sync and tries to refresh with an old token.

Have a great sinterklaas!

[stephangroen]

Thanks all, looking at both solutions. #449 seems a bit more complete, I have heard more people running into issues with multiple threads. I'll need to update the README for the solution that is merged, so it'll be clear to new users how to implements the callbacks.

[FawadNL]

@eaibizz the token was indeed not user-bound as it failed to refresh for that specific user. Have implemented the solution of @alexjeen and it resolved the issue. Thank you both for the help! Really saved the day :)

[stephangroen]

Thanks all! #449 has been merged and is available in v3.26.0.