kainxspirits / laravel-pubsub-queue

A Laravel queue driver for Google PubSub.
MIT License
44 stars 36 forks source link

Double base64encode #69

Open Sergio-Ben opened 8 months ago

Sergio-Ben commented 8 months ago

Hey there. While using this package, I noticed my payload that running through google pub/sub is encoded twice. The problem is in the PubSubQueue class in pushRaw function

Screenshot 2024-02-08 at 13 56 22

$publish = ['data' => base64_encode($payload)];

On this line we are encoding our payload, and then in

$topic->publish($publish);

We are actually executing code from Google/cloud-pubsub library, which is doing exactly the same, so the payload is encoded twice at the stage of sending it to the pub/sub.

The same if for pulling messages from subscriptions, Google package doing the base64_decode by itself.

Just wondering what was the initial purpose of encoding payload inside this package ?

kainxspirits commented 8 months ago

Hello @Sergio-Ben

I actually don't use this package and I don't remember why I made that choice at the time. But initially, the topic auto-creation was not part of this package and I probably did assume that the "encoding" option was set to false. With the topic auto-creation, I believe all the messages are now automatically encoded to base64, so I agree with you it can be kind of inefficient now and it might be a good idea to disable this behavior.

I'm open to a well-tested fix in PR.

Sergio-Ben commented 8 months ago

Hey @kainxspirits

This actually has nothing to do with topic auto-creation. The message encoding during publishing to the topic was introduced with the initial release starting 0.1.0 tag. For the REST connection, the package will set encode flag to true

if ($connectionType === 'grpc') { $this->connection = new Grpc($this->configureAuthentication($config)); $this->encode = false; } else { $this->connection = new Rest($this->configureAuthentication($config)); $this->encode = true; }

in https://github.com/googleapis/google-cloud-php-pubsub/blob/v0.1.0/PubSubClient.php#L126

and then in https://github.com/googleapis/google-cloud-php-pubsub/blob/v0.1.0/Topic.php#L463

The same flag is used to check if the message should be encoded or not.

But yes, i'll try to find some spare time and create some PR, so you can review it :)