search

dashbitco / broadway_cloud_pub_sub

A Broadway producer for Google Cloud Pub/Sub
Apache License 2.0
70 stars 24 forks source link

Attributes don't seem to be passing through into metadata #35

Closed Adzz closed 5 years ago

Adzz commented 5 years ago

When I add a message to the pub sub queue, I can add attributes:

Screenshot 2019-09-09 at 18 59 59

But this is the message that I get through to handle_message/3

%Broadway.Message{
  acknowledger: {BroadwayCloudPubSub.GoogleApiClient,
   {BroadwayCloudPubSub.GoogleApiClient, #Reference<0.377512039.1466957828.121278>}, "REDACTED"},
  batch_key: :default,
  batch_mode: :bulk,
  batcher: :default,
  data: "MESSAGE",
  metadata: %{},
  status: :ok
}

I would expect that the attributes appear in the metadata field, but I could be wrong on that assumption...

mcrumm commented 5 years ago

@Adzz are you testing with your forked branch? Your pattern match for a string "data" param is then trying to take atom keys for the other fields. I'm assuming this is why you're not seeing any of the metadata, not just attributes. (See https://github.com/plataformatec/broadway_cloud_pub_sub/commit/faded044449b984fd7f34dc451df826201e38fea#diff-aba84c6fefc1deabd03d66f245fc000bR148)

Can you please post at least partial output from running GoogleApi.PubSub.V1.Api.Projects.pubsub_projects_subscriptions_pull/5 for your subscription?

Related to #34 , the pull request should return a %GoogleApi.PubSub.V1.Model.PullResponse{} whose receivedMessages field should contain a list of %GoogleApi.PubSub.V1.Model.PubsubMessage{} structs.

If this is not the output you're receiving, I would file an issue on the API client repo: https://github.com/googleapis/elixir-google-api/

Adzz commented 5 years ago

@mcrumm Oh of course, how stupid sorry! Thanks I'll close this issue in favour of the other one, which is the root cause here. Sorry for wasting your time on that

mcrumm commented 5 years ago

No worries! I just want to make sure we're tracking down the right issue.