spring-attic / spring-cloud-gcp

Integration for Google Cloud Platform APIs with Spring
Apache License 2.0
704 stars 694 forks source link

Can I run 'Spring Cloud Stream with google pub/sub binder' as a Google Cloud Function? #2620

Closed amirmv2006 closed 3 years ago

amirmv2006 commented 3 years ago

I don't think this is a bug, so I'm just asking this as a general question!

I don't believe I'd be able to run a cloud stream application with pub/sub binder as Google Cloud Function, right? If so, is that intentional or is there another way to do that?

I want to start my tiny application as server-less, but I also want to be able to easily migrate to a "server-full" solution if I have enough users. Using Spring Cloud Stream I'll be able to support high load, but unfortunately I can't seem to run a Spring Cloud Stream application as server-less.

dzou commented 3 years ago

@amirmv2006 -- If you want to use Google Cloud Function, I think you would have to first start with using Spring Cloud Function (and using the GCP adapter with it): https://github.com/spring-cloud/spring-cloud-function

Basically this will let you write a serverless function using Spring framework, and thereby letting you use our pub/sub autoconfigurations via adding our starter dependencies.

Once you have that, you can follow the sample instructions for deploying it. We have a simple Spring Cloud Function sample app deployed to GCF here that you can check out: https://github.com/spring-cloud/spring-cloud-function/tree/master/spring-cloud-function-samples/function-sample-gcp-http

amirmv2006 commented 3 years ago

@dzou thanks for the fast reply!

I did have a look into the samples before, but unfortunately they include functions that are too simple! One receives a http request and returns a http response, and the other one would just "sink" a pub/sub message. What I'm looking for is

That's something that is not as easy with just spring functions, so that's why I was thinking about a cloud stream solution.

P.S. looking at https://github.com/spring-cloud/spring-cloud-function/blob/master/spring-cloud-function-adapters/spring-cloud-function-adapter-gcp/src/main/java/org/springframework/cloud/function/adapter/gcp/FunctionInvoker.java shows that what I explained might not be easy as the code for GCP Functions will just "drop" return of a RawBackgroundFunction and will just write the return of a HttpFunction function as http response.

dzou commented 3 years ago

Oh I see. Sounds like in that case you will have to combine concepts from different samples to achieve what you want.

To receive a http request and publish a pub/sub message and then return a http response

I think you can start off with code in the spring cloud function GCP sample but also add in the Pub/Sub starter that we offer (check out the Pub/Sub sample)

Then you will probably have to write code for your Spring Cloud Function like:

@SpringBootApplication
public class CloudFunctionMain {

    public static void main(String[] args) {
        SpringApplication.run(CloudFunctionMain.class, args);
    }

    @Bean
    public Function<String, String> function(PubSubTemplate template) {
                // This code isn't precisely accurate, but this is the idea
                // "value" is the input to the spring cloud function; maybe the message you publish is derived from this.
        return value -> template.publishMessage(TOPIC_NAME, "your_message")
    }
}

You would add our Pub/Sub starter and gain access to the Pub/Sub autoconfigurations, inject the PubSubTemplate into the code, then call it from the Spring Cloud Function.

Or receive a pub/sub message and publish another message on another topic.

This part you would need to use a Google Cloud "Background function". See this. Background functions are triggered when a message is delivered to a Pub/Sub topic. Then in your background function, do the same as described above, inject a pub/sub template into your code then publish a message to another topic of your choosing.

amirmv2006 commented 3 years ago

Good to know, although I think this would not be ideal for us to "couple" our code to the infrastructure which in this case is the GCP pub/sub medium. I think there's potential in cloud streams to help with decoupling this, but that's probably tomorrow's problem. Today I will just use the template as you suggested (in a separate function to hopefully get rid off in the future :sweat_smile:)

Thanks for your time :+1:

dzou commented 3 years ago

Ah gotcha I see. Yeah regarding Spring Cloud Function + Spring Cloud Stream, I think this use-case is possible. I say start with the simple approach for now and slowly iterate on it. Think it will become easier as you go.