Recreate equivalent of RQ Dashboard/Monitor inside BTU

[brian-pond]

Situation

Python RQ does not ship with a GUI. To fully understand the RQ status (including errors :astonished: ), there are only a few options:

1. Inspect the Redis database using CLI tools:

   • redis-cli is a possibility. However, a lot of RQ data is pickled. It cannot be deciphered by a human.
   • The built-in rq CLI is too simplistic. It has a few commands. But nothing that gives administrators the insight and depth needed to troubleshoot.

2. Install a third-party GUI tool, that knows how to read RQ data from Redis.

   • Examples are rq-dashboard and rqmonitor
   • See also: https://python-rq.org/docs/monitoring/

I've used both of these GUI tools. They are nice. However, they introduce headaches:

• More web software to install and maintain. Sure, Docker might help isolate it. But that assumes you have Docker on your Production web server. And also assumes you're familiar and comfortable with configuring and maintaining Docker.
• Another subdomain to register.
• Another website to add to your Reverse Proxy.
• More usernames and passwords to remember (you don't want strange humans examining/editing your RQ)

Suggestion

Even without BTU, Frappe Framework relies heavily on Python RQ. It's a very important component of the full stack:

• Every call to frappe.enqueue needs it.
• Scheduled Job Types need it.

So. Why should we install more software, when Frappe is already a web server with python-rq installed? Instead, why not just make a new Web Page that queries the Python RQ, and shows the data?

Just like what RQ Dashboard and RQ Monitor are doing. But using Frappe instead of Flask.

Goals

In the BTU App, create various Web Pages that do exactly the same thing as RQ Dashboard and RQ Monitor.

• Show the Jobs that are in the Redis database, by status: Finished, Queued, Error, Started, etc.
• Show all the details about 1 Job, including the pickled results, standard output and standard error, etc.
• Create a button to delete a Job.
• Create a button to delete all Failed Jobs.
• Create a button to purge the Queues.

[brian-pond]

Update: With the release of ERPNext v15, there is now a new virtual DocType "RQ Job". This probably fulfills 80% of my requirements above?