[BUG] ERROR PutLogEvents, InvalidSequenceTokenException

[mohamednagy]

Describe the bug The package has been integrated with laravel 7.0 to sync logs to cloudWatch but i'm getting this error sometimes

InvalidSequenceTokenException (client): The given sequenceToken is invalid. The next expected sequenceToken is: 49610385040921553485418540785990629652624902462896384722 - {"__type":"InvalidSequenceTokenException","expectedSequenceToken":"49610385040921553485418540785990629652624902462896384722","message":"The given sequenceToken is invalid. The next expected sequenceToken is: 49610385040921553485418540785990629652624902462896384722"}

Expected behavior PutLogEvents should be executed smoothly .

Please tell about your environment:

• PHP Version: 7.3
• Operating system (linux AWS ECS with Fargate type):
• Application mode:
  • [ ] Web app
  • [ ] CLI app
  • [x] Daemon worker

[khoa002]

We have this same problem and I think the reason being is that we have 10 Laravel jobs running concurrently on different AWS EC2 instances, and they all use the same log stream names. I fixed the issue (maybe temporarily?) by changing the log stream name for those jobs to include the AWS instance ID (or hostname if not available), and the PID of the supervisor process.

class CloudWatchLoggerFactory
{
    public function __invoke(array $config)
    {
        $instanceId = @file_get_contents("http://instance-data/latest/meta-data/instance-id"); // AWS Instance ID
        if (empty($instanceId)) {
            $instanceId = gethostname(); // use hostname if AWS instance ID is not available
        }
        $instanceId .= '_' . getmypid(); // the PID of the current worker

        $sdkParams = $config['sdk'];
        $tags = $config['tags'] ?? [];
        $name = $config['name'] ?? 'cloudwatch';
        $groupName = $config['group'] ?? env('AWS_CLOUDWATCH_LOGGING_GROUP_NAME', config('app.name') . '-' . config('app.env'));
        $streamName = $config['stream'] ?? env('AWS_CLOUDWATCH_LOGGING_DEFAULT_STREAM_NAME', 'cloudwatch-log-all');
        $streamName .= '_' . $instanceId;
        $retentionDays = $config['retention'] ?? 14;
        $client = new CloudWatchLogsClient($sdkParams);
        $handler = new CloudWatch($client, $groupName, $streamName, $retentionDays, 10000, $tags);
        $logger = new Logger($name);
        $logger->pushHandler($handler);
        return $logger;
    }
}

I can still search/query all the streams under the same log group in AWS Cloudwatch, so this is works for now. Hope this helps someone else using the same stack.

[maxbanton]

Hi @mohamednagy! It's limited on AWS side, since library uses AWS SDK, which uses HTTP API. Problem may be in concurrent requests. Try to use solution proposed by @khoa002.