laravel / horizon

Dashboard and code-driven configuration for Laravel queues.
https://laravel.com/docs/horizon
MIT License
3.84k stars 645 forks source link

Very slow loading of the job list page #1360

Closed OleksandrWebLab closed 8 months ago

OleksandrWebLab commented 8 months ago

Horizon Version

5.21.1

Laravel Version

10.25.1

PHP Version

8.2.11

Redis Driver

PhpRedis

Redis Version

5.0.7

Database Driver & Version

MySQL 5.7

Description

I noticed that the job list page started to load very slowly.

A quick analysis showed that this is due to the size of the objects that are stored in the queue.

The screenshot shows an example of how page data update requests are performed. It can be seen that the first request was for 50 records of 50 megabytes in size and took 46 seconds to load. All other requests are still in the process of loading and are executed asynchronously, regardless of whether the previous result has been loaded or not.

image

I think the solution is to remove unnecessary data from the query result before returning the result and should be like this:

<?php

namespace Laravel\Horizon\Http\Controllers;

use Illuminate\Http\Request;
use Laravel\Horizon\Contracts\JobRepository;

class PendingJobsController extends Controller
{
    //...
    public function index(Request $request)
    {
        $jobs = $this->jobs->getPending($request->query('starting_at', -1))->map(function ($job) {
            $job->payload = json_decode($job->payload);
            unset($job->payload->data); // <-- removing unnecessary data from the result

            return $job;
        })->values();

        return [
            'jobs' => $jobs,
            'total' => $this->jobs->countPending(),
        ];
    }
    //...
}

Steps To Reproduce

Add a large object to the job, with many relations, collections, etc.

github-actions[bot] commented 8 months ago

Thank you for reporting this issue!

As Laravel is an open source project, we rely on the community to help us diagnose and fix issues as it is not possible to research and fix every issue reported to us via GitHub.

If possible, please make a pull request fixing the issue you have described, along with corresponding tests. All pull requests are promptly reviewed by the Laravel team.

Thank you!

driesvints commented 8 months ago

Since we have only one report of this I'm going to close this one. We'd appreciate a PR if anything can be done, thanks.