RaythaHQ / raytha

Raytha is a powerful CMS with an easy-to-use interface and fast performance. It offers custom content types, a template engine, and various access controls. It supports multiple storage providers and an automatically generated REST API. Upgrade your development workflow with Raytha.
MIT License
155 stars 30 forks source link

Build in `BackgroundJob` infrastructure #79

Closed apexdodge closed 1 year ago

apexdodge commented 1 year ago

We need to be able run background jobs or long running jobs such as full data exports / data imports / web hooks, and anything else that may be best served running outside the main web request.

Neither Quartz, nor Hangfire are MIT license. So we may have to roll our own.

apexdodge commented 1 year ago

Proof of concept for creating our own mini background task infrastructure.

Pretty much just followed this tutorial 1-1 https://learn.microsoft.com/en-us/aspnet/core/fundamentals/host/hosted-services?view=aspnetcore-7.0&tabs=visual-studio

This will primarily be used for exports to CSV to start out with, but lays the ground work for future "fire and forget" longer running tasks, like data imports, web hooks, cloning content types + copying data, file storage clean up, etc, etc.

Known limitations: this implementation is using the in-memory queue and so it comes with all the shortcomings associated with that. But I think this is not a big deal for the types of tasks we'll be performing out of the box. For those that need a more intense background job processor, consider adding Hangfire to your fork.

apexdodge commented 1 year ago

Need to add a table to store the status of the background task and result.

apexdodge commented 1 year ago

Really good progress here.

Remaining task is to clean up the code and then move on to #80

apexdodge commented 1 year ago

Export to csv in background task now works, but there is an issue with the url to the file since it runs in a background service. Cannot use HttpContextAccessor. Need a proper way to generate the absolute url.

apexdodge commented 1 year ago

Released