[FEATURE] I'd like a Pipedream workflow to remain "warm" so that all events are processed immediately, without an initial "cold start" penalty

[Paktas]

Agree. Always-on environments could be part of commercial plans. This is the reason why we are still doubtfull of moving all comms to Pipedream.

[danthegoodman1]

This would be great for people who are using pipedream as a serious backbone to something.

[dylburger]

Before we implement this feature, I wanted to share our recommended approach for keeping your workflows warm. You can keep any number of workflows warm using this technique: the cron workflow can trigger any number of other workflows.

[rodenp]

Consistent response time is important to me and it would be great if i did not have to use any workarounds to keep the connection warm. The beauty of pipedream lies in being able to create any workflow exactly tailored to your needs. This is awesome. However i have recently come across this issue of cold starts, since my workflow is currently not being triggered often enough. My use case involves customers landing on a page where some details are input, then the pipedream workflow is triggered and finally a response page is rendered. The response time from triggering the workflow to the final response page being rendered varies wildly from a second through to 5+ seconds which is way too long for a customer to wait. Now i understand why, it appears the response times are lower when the workflow is still warm. However i cannot predict when the workflow will be triggered. There will often be periods of inactivity that could span into many hours. Now that i know this I am wondering why many other people have not raised this as an issue. I would have thought that this would impact a lot of people.

[pocaloso]

Agree to make always-on environment be part of the paid plan. Or, an even better solution - can we have an additional on/off switch in the workflow settings to keep it warm? By using suggested 5 mins period to invoke a workflow, in one month we will require 12 24 30 = 8640 invocations. We should double it to 17280 becuase of the nature of send and receive. Multiply by invocation unit price is what we should pay for turnning on the switch. Of course, a reasonable discount should be applied.

[rodenp]

I am finding more and more that i need workflows that have consistent response times regardless of how often they are triggered. Since a lot of the workflows result in a web page being displayed response time is a major factor. Especially in the early stages when there is little traffic. I am finding that this is becoming a deal breaker for me fully adopting and committing to pipedream. I do not know of other workflow systems that have this warming up concept. It doesn't make sense in todays world. Maybe for an entry level free plan this would make sense. I would like to see a paid plan where workflows are always warm and i don't have to think about triggering dummy flows.

[cristiandarie-inki]

Same here. We use such a workflow to display data on demand for users, and having them to wait for about 10 seconds for the page to load, as it happens right now, is not ideal. The exact same functionality takes about 1 second to execute with Integromat (make.com), however we'd prefer to keep this part in Pipedream if the execution time would be fixed.

[osseonews]

You should run some workflows on the V8 engine with Cloudflare Workers??, similar to how NextJs is doing it now with their Edge functions. So workflows that don't need to load immediately can stay the same on the current backend, but those that should load immediately (no cold start) should run off V8, e.g. cloudflare workers.

[osseonews]

Please implement Edge functions, like Cloudflare workers. See how NextJS is implementing it. This is the only thing missing in Pipedream really. The cold start issue is a major pain as we end up having to develop functions in Cloudflare as opposed to doing everytihng in Pipedream.

[nebbles]

Adding detail for the use-case in my situation: we use Pipedream for responding to Slack events. Since our workflow can go cold, it means it often errors due to Slack's strict requirement to use trigger_id within 3 seconds. From the user experience using our Slackbot, it means there can be no response at all to a slash command (but if the user retries it would then work fine).

Based on the 'cooling rate' of Pipedream workflows, a workflow would need a cron trigger (in addition to an HTTP endpoint) to invoke it every 5 minutes. This equates to 288 invocations/day (up to 8,928 extra invocations a month) which is 89% of the free tier quota, but also 44.6% of the paid tier quota. That's pretty hefty just to keep one workflow warm. (We like pipedream, so we have more than one workflow!).

Hoping this is something the Pipedream team can offer soon, even if it is only (quite rightly) a feature reserved for the paid tiers!

[dylburger]

@nebbles Thanks for the comment. Just to confirm, when we ship this feature, you'd also pay to keep the worker warm. Based on the manner this works in AWS, it may end up costing more than the current invocation-based cost of warming you outlined. But it should also keep the workflow warm 100% of the time, so you're paying more to ensure reduced latency.

We'll keep y'all updated and let you know the details of the implementation so you can use it and provide more feedback.

[osseonews]

@dylburger I think you need to build the functions that require no cold starts on something like Cloudflare which is the V8 engine. What's proposed is not a great solution. See what Vercel has done with Edge. They built this on Cloudflare, so it should be possible to use your interface to call functions on Cloudflare. From my experience, Cloudflare workers are simply incredible and a great adjunct to Pipedream. We use both. But, Pipedream has an amazing interface which would be helpful in also managing Cloudflare workers.

[dylburger]

@osseonews Unfortunately Cloudflare and Vercel limit what you can do in that environment much more than Pipedream. See the Vercel Edge limitations and the Workers limits docs. They aren't deploying containers in the same manner we are. The extra functionality on Pipedream requires different deployment model.

We are always working to optimize this, so this initial feature to keep a given workflow warm isn't the end of our iteration :) .

[osseonews]

@dylburger Yeah, that's somewhat true, but Cloudflare workers alone are incredibly inexpensive and ridiculously fast. What I think would be a killer product, though, is a better interface for building/running Cloudflare Workers. I would think Pipedream could provide that type of service. But, I really wouldn't know given that I know 0 about how your product is actually built. It would be an adjunct to your current platform that could come at an extra cost. I know for sure, I would pay for some better interface for cf workers, that would work like Pipedream. Anyway, just an idea for you.

[nebbles]

Just to confirm, when we ship this feature, you'd also pay to keep the worker warm. Based on the manner this works in AWS, it may end up costing more than the current invocation-based cost of warming you outlined. But it should also keep the workflow warm 100% of the time, so you're paying more to ensure reduced latency.

@dylburger I think on the most latency-sensitive stuff, it would make sense to pay (even if more). I hadn't realised that it could cost more than the cost of keeping workflows warm with scheduled invocations, but I guess that probably makes sense. If the cost were more, then it would be specifically selected for particular workloads, rather than a fleet-wide attempt to improve workflow UX.

[osseonews]

Curious if there has been work on this at all? Cold start ups, and having to constantly warm functions, are very "old school" at this point. Most other platforms have already solved this by offering functionality on V8 engine. Isn't there a way Pipedream can eliminate this issue? It would greatly increase the use cases for the platform and increase your revenue. Slow start up of workflows is a huge issue in many applications.

[dylburger]

@osseonews Yes, we’re testing something internally that should address this issue. Like I mentioned before, the v8 runtime is a substantially different architecture, and comes with different limits, so it’s not appropriate for our use case. But we absolutely want to address this!

[osseonews]

Great, thanks! Please post when you are ready for us to test.

[dannyroosevelt]

We shipped this feature! You can read more here: https://pipedream.com/blog/eliminate-cold-starts-with-warm-workers/