GoogleCloudPlatform / functions-framework-python

FaaS (Function as a service) framework for writing portable Python functions
https://pypi.org/p/functions-framework/
Apache License 2.0
878 stars 118 forks source link

enhance framework to support pull subscriptions in addition to push subscriptions with pubsub #310

Open MikeMoore63 opened 9 months ago

MikeMoore63 commented 9 months ago

I have need for functions with push endpoints that MUST be internal network so simply not supported. Event sources like pubsub only support push subscriptions to run services when using (vpc perimiters)[ https://cloud.google.com/vpc-service-controls/docs/supported-products#table_pubsub] but this limits to constraints of these services i.e. ram or cpu for example and to work that fits n the time of push subscription (10 minutes). Others i suspect have same issue. So I need it to be abe to use with pull subscriptions to give me a framework I can move code between run and pull as constraints get removed/changed.

Appreciate an enhancement. I would expect

  1. A new flag pubsub_subscription being added for a pull subscription name functionality only operates when the flag is set.
  2. A second paramter max_msgs is used for flow control of the pull i.e. how many messages allocated per function default 5
  3. A third paramter time_out to be set for lease duration default 60 seconds (like push subscription time out)
  4. A boolean retry_function (same as pub sub push retry) basically means nak on failure of function or ack default True

If pubsub_subcription is set set a background thread to process pull is set up and port for server randomised that as it pulls each message it posts as if a push subscription to the function framework so a loopback within process effectively. If success 200 is returned the message is acked if not 200 if retry true its nacked and if retry false its acked. If not on windows signal alrm can be used to emulate the time out f a push subscription and is treated as a failure.

I do have a fork with something like this working if example would be helpful or a contribution.