Add examples for creating push subscriptions and updating subscriptions

[foresmac]

In which file did you encounter the issue?

https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/pubsub/cloud-client/subscriber.py

Did you change the file? If so, how?

No.

Describe the issue

There are no examples in Python for creating push subscriptions (it's totally possible to do, but I had to figure how to do so by reading the code and trial and error in a REPL), nor for updating said subscriptions. I tried figuring out how to update a push subscription's push_endpoint, but I couldn't sort out how to correctly make either a FieldMask or a dict with similar structure. I tried looking here to figure out what a FieldMask is supposed to look like, but it's autogenerated and looks like gibberish to me. Comment says to look in google/cloud/pubsub_v1/proto/pubsub.proto file that doesn't appear to be there.

When in doubt, more examples are better than just one, but one would be infinitely better than none.

[foresmac]

@andrewsg Looks like there's an example of creating a push subscription added recently; thank you! Where is pubsub_v1.types.PushConfig documented? I can't seem to find it.

Also, any chance there will be an example of updating a subscription?

[foresmac]

This might be what I'm looking for, but it seems to suggest that the only thing that can be changed is the push_endpoint: https://googlecloudplatform.github.io/google-cloud-python/latest/pubsub/types.html#google.cloud.pubsub_v1.types.PushConfig

How can I change, for example, the retention duration?

It seems as though there is an example somewhere of updating the ack deadline (though this doesn't show up in the documentation with the other examples).

[andrewsg]

Hi @foresmac, I looked into this and it looks like retention duration is not configurable for push subscriptions right now.

Other subscription attributes like the topic and ack_deadline_seconds can only be configured on subscription creation; the sub has to be deleted and recreated to change it.

I believe PushConfig (only) can be modified on a live subscription by using an API call: https://cloud.google.com/pubsub/docs/reference/rpc/google.pubsub.v1#google.pubsub.v1.ModifyPushConfigRequest -- so, it looks like you're correct that only the PushConfig attributes are changeable.

If this doesn't meet your needs, please let me know a bit about your use case in a reply and I'll keep track of that.

[foresmac]

Well, most specifically, if you have a "bad" message in a topic, and the subscription is set up as push, it'll just keep hitting your endpoint over and over again multiple times per second for seven days—the default retention duration. Now, I know the message is bad, and I've set up ways to prevent the bad message from getting sent to the topic in the future. But I don't want to "fix" the endpoint to deal with this malformed message, and since there's no way whatsoever to monitor topics and remove problematic messages, the only "good" solution is to configure the retention duration to be a more reasonably smaller time interval.

Another possible solution is to engineer every endpoint to basically be a huge try/except block, and do something (log an error, I guess) with the malformed message. Even then, if something somehow causes an uncaught exception, I don't want PubSub to just keep banging away at an endpoint that practically speaking is never going to work. Since it seems unlikely that PubSub is going to radically alter the way it works (an exponential backoff on retries? a management console like AppEngine's task queue?), I'd like to set a more reasonable retention duration, since if I'm not going to change an endpoint in a day, I'm certainly not going to change it in seven.

[foresmac]

@andrewsg Does that make sense?

[andrewsg]

Yes, it does! Thank you for the feedback. I will send this along to our product manager.

[andrewsg]

Hi @foresmac, update here, my product manager says that the ability to customize the retention period for push subscriptions is in alpha right now. If you would like to participate, reach out at cloud-pubsub@google.com.