Cronjob Support

[timsuchanek]

It would be awesome to be able to define a cronjob to call the deployed app periodically.

Serverless is supporting this feature too, would like to see it in up.

I'm using serverless for my app now, would like to try up too!

[tj]

Possibly! it could be a bit error-prone, if you didn't add some kind of secret/auth then anyone could hit that same end-point and trigger potentially expensive work. That's my main worry there. Lambda's are private by default instead of public, so that helps .

[JohannesKaufmann]

I thought about the following api:

{
  "name": "app",
  "schedules": {
    "key": "rate(2 hours)"
  }
}

Up would then call the endpoint /__UP__/key and would not allow any requests from the outside to __UP__/*. This way you can have an endpoint that responds to the cron job but its not accessible to the outside. The posibility that a developer uses this route already is quite low.

But if a developer is using it locally (up start) it should let requests through in order to test them but it should print a warning in the console.

It would be also nice to check that the cron or rate is valid since AWS has some limits (regarding the question mark) and maybe even print the "human readable" version to the console.

Maybe even allow the use of an array of cronjobs for one key?

[tj]

Hmm maybe:

{
  "schedules": [
    {
      "rate": "5m",
      "method": "POST",
      "url": "/whatever", 
      "header": {
        "X-Secret": "whatever"
      },
      "body": "string or object for json?"
    }
  ]
}

We'd need the option to define the request (body as well). I haven't decided yet if event related stuff is out of Up's scope or not, but I'm interested in hearing about use-cases for this feature if anyone has a few!

[kaihendry]

The use case I'd need is to run a job at a specific time point in the future. Wonder if this can be somehow realised.

[tj]

@kaihendry does it help if it's scheduled HTTP vs schedule Lambda?

I think for now, at least until this project supersedes apex(1), it might be out of scope. I can't think of any huge wins for it being HTTP, if anything it's more work because Lambda funcs are already hidden to the public.

[kaihendry]

Sorry, I still need to figure out "scheduled lambda". It's just that the job is the same between the http and the lambda. Guess there is no harm in me duplicating code. Just use golang modules!

My use case is slightly esoteric. I'm basically telling my API to do things in the future.

[tj]

If / when Up supports "regular" Lambda functions I think it'll be a good to thing to add. I might combine apex(1) and up(1) eventually, but I'm not on-edge about scheduled requests making sense for Up since any customer could hit those urls as well.

[mariokostelac]

@tj is a grand vision for up to be serverless app management or just serverless http app management? Because mine is to use up for everything 😁. If crons can be supported without http, I think it'd be good to support them on the simplest possible way (I think aws does support it with triggers). That would definitely push up closer towards superseding apex.

[tj]

Long-term I'd definitely like to replace apex(1), fewer tools would be nice! The FaaS part is a bit easier in comparison so at least that part should go quickly once I get there haha.

[mariokostelac]

@tj what's the easiest way find what you need help with?

As well, since there is a free and pro version, how do I know that I am not creating something I will have to pay for later on? And how is tension between "I think it should be free" vs "pro has this feature for $15" usually resolved in similar business models?

[tj]

@mariokostelac I haven't seen many people try to monetize OSS in general really, so it's new to me as well. As far as contributions go, people have to accept that the bottom line is that this project wouldn't exist without $$ (which I have 0 of right now haha), I can't create things for free unfortunately.

So it's really up to the contributors, totally understand if they don't want to contribute but hey!

[mariokostelac]

What about first question? :)

@tj what's the easiest way find what you need help with?

[tj]

Ah sorry was just editing haha. I'll try and tag those as "good first issue" but there aren't many for now. They're all pretty much open. I'll try and remember to assign myself if I'm actively working on it

[ethangclark]

@timsuchanek @tj I just realized that Apex Ping can cover my use-case for this feature, since I don't need invocations to occur at exact times... It would probably work for a lot of other users as well.

So maybe this is more in the scope of that project.

Edit: actually, merging Up and Ping into "Apex Services" or whatever would be even more baller than merging Up and Apex... Maybe you want to keep your products completely separated, but these two are very complimentary, and could be marketed as such

[benseitz]

Any new thoughts on including some sort of scheduled events for up? I think a lot of architectures would depend on that.

[tj]

@benseitz probably not if/when FaaS support is addded, since I think FaaS makes much more sense for event processing than HTTP or cron style stuff. It could work with hitting some specified HTTP end-point but then that code is not private and anyone outside can trigger it as well (without some auth, which is awkward)

@8balloon most of my stuff going forward will use the same billing/team "engine", but Ping yeah will unfortunately probably remain separate haha, at least until I can rewrite it.

[benseitz]

@tj how would you suggest to tackle such a problem? 😬

[tj]

@benseitz apex(1) and Serverless are definitely better suited for it framework-wise at the moment, I guess it'll come down to if Up should feature-creep into FaaS territory or not. I'd definitely prefer to use one tool, but it becomes a bit less elegant as well so I have a hard time deciding there haha.

Based on the feedbacks far it seems like people would like to see Up have some FaaS features as well.

[benseitz]

@tj what do you think about plugins? mup for example has plugins for different environments (like mup-aws-beanstalk). I could see up with various plugins for different environments and/or additional functionality.

PS: I like up so much, I'm willing to create a "secret" endpoint that will be public. Still I would need a service, that calls that on a regular basis.

[tj]

@benseitz yeah plugins could be pretty sweet, although since Go is sort of less flexible that way it might be a bit tricky to distribute them in a way that isn't annoying.

I have #453 for plugin related ideas, definitely something I want to explore, especially for cases like RDS where it would be a really leaky abstraction to try and provide a nicer config, there are just so many parameters, but it would still be nice to have something.

I'll give the scheduling some more thought too, since Up provides the proxy I think we could maybe automatically add a little security layer so people can't hit it at random.

[jsalonen]

Any follow up on this issue?

Having exactly the same request: very simple cron / scheduled execution of a URL. I don't mind setting up a "secret URL" either if that's what is required.

Things I've already tried:

• Hacky node-cron run in the same process as express (surprise, surprise, it dies)

Things I'm next considering:

• Setting up a separate scheduler project with Serverless + cron triggering a secret URL
• Moving back to Serverless

So yeah, would appreciate if anyone has come up with a decent workaround.

(And yes, have a pro subscription :))

[kaihendry]

I've been using Cloudwatch Events at work, though the AWS limits are quite low. This would typically invoke an Apex lambda to do your bidding.

[tj]

@jsalonen not yet, I think if it were to make its way into Up, Up would have to replace apex/apex, which is definitely a possibility but I need to think on that some more. I'm not a huge fan of using HTTP for scheduled stuff, Lambda + Scheduled events is a bit more elegant since it's private, but it might be nice if Up supports regular FaaS as well as other CloudFormation resources.

[kulor]

To wade in on this as I was looking for scheduling in Up, I've been using Zappa for scheduling and they have a nice config and invocation pattern which makes for dreamy deployment and few moving parts in your infrastructure: https://github.com/Miserlou/Zappa#scheduling. I think it'd be ace to adopt a similar approach for Up.