dashbitco / broadway_cloud_pub_sub

A Broadway producer for Google Cloud Pub/Sub
Apache License 2.0
70 stars 24 forks source link

Catches {:error, :invalid, integer()} case when receiving messages #84

Closed jekku closed 2 years ago

jekku commented 2 years ago

What's here?

This PR is for the current version of the library.

In the current version of this library (v0.7.1), when receive_messages/3 is invoked, we sometimes receive this kind of error tuple with 3 members:

{:error, :invalid, integer}

The stacktrace from the exception thrown from the library looks like this:

FunctionClauseError: no function clause matching in BroadwayCloudPubSub.GoogleApiClient.handle_response/2
  File "lib/broadway_cloud_pub_sub/google_api_client.ex", line 146, in BroadwayCloudPubSub.GoogleApiClient.handle_response/2
  File "lib/broadway_cloud_pub_sub/google_api_client.ex", line 116, in BroadwayCloudPubSub.GoogleApiClient.receive_messages/3
  File "lib/broadway_cloud_pub_sub/producer.ex", line 240, in BroadwayCloudPubSub.Producer.handle_receive_messages/1
  File "lib/broadway/topology/producer_stage.ex", line 229, in Broadway.Topology.ProducerStage.handle_info/2
  File "lib/gen_stage.ex", line 2117, in GenStage.noreply_callback/3

What we've checked and tried

jekku commented 2 years ago

Commenting some snaps with the whole stacktrace from the platform that stores our exceptions just in case it helps

image
josevalim commented 2 years ago

Thank you! It seems this bug needs to be patched upstream instead, by whatever is actually returning the invalid tuple, no? :)

jekku commented 2 years ago

That seems to be reasonable. Further justified by the move to Finch as well. Just wondering currently when we'll might get a release and some docs on the newer version? :)

Closing this now.

josevalim commented 2 years ago

Please do try it out and let us know it works, that will certainly get us closer to a new release. Thank you!