brendt
commented
5 years ago Are you using a sync queue driver or horizon? You should use the latter one.
Closed francoism90 closed 4 years ago
brendt
commented
5 years ago Are you using a sync queue driver or horizon? You should use the latter one.
francoism90
commented
5 years ago @brendt I'm using Horizon as queue driver and using auto as preset.
I found out that multiple jobs of the same Media-model are spawn when it has an invalid character in the filename and FFMpeg cannot probe the file, resulting in a loop. When the invalid char(s) have been fixed/removed the regenerate progress runs fine and the loop doesn't occur.
Is it true the PerformConversions job doesn't have any throttle? It would be great to lower the number of max. allowed conversions that happen in a specific time period, preventing the jobs from stalling. Or is this possible already? :)
brendt
commented
5 years ago We probably could make improvements in this area. But we need to take into account that Laravel's throttling only works with Redis (https://laravel.com/docs/5.8/queues#rate-limiting), I don't think it'll be straight forward to add.
I found out that multiple jobs of the same Media-model are spawn when it has an invalid character in the filename and FFMpeg cannot probe the file, resulting in a loop.
Are you able to constantly reproduce this? Are you willing to submit a PR with a failing test?
francoism90
commented
5 years ago It would be great if a job (e.g. Spatie\MediaLibrary\Jobs\PerformConversions) could be controlled by the user's queue driver - adding Horizon features to set job rate limiting, max. simultaneously procs for jobs, etc. Or is this already possible? :)
I'll try to reproduce this and submit a PR.
francoism90
commented
5 years ago @brendt I think I have solved the throttle issue:
<?php
namespace App\Support\MediaLibrary\Jobs;
use Illuminate\Bus\Queueable;
use Spatie\MediaLibrary\Models\Media;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Spatie\MediaLibrary\FileManipulator;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Support\Facades\Redis;
use Spatie\MediaLibrary\Conversion\ConversionCollection;
class PerformConversions implements ShouldQueue
{
use InteractsWithQueue, SerializesModels, Queueable;
/** @var \Spatie\MediaLibrary\Conversion\ConversionCollection */
protected $conversions;
/** @var \Spatie\MediaLibrary\Models\Media */
protected $media;
/** @var bool */
protected $onlyMissing;
public function __construct(ConversionCollection $conversions, Media $media, $onlyMissing = false)
{
$this->conversions = $conversions;
$this->media = $media;
$this->onlyMissing = $onlyMissing;
}
/**
* Determine the time at which the job should timeout.
*
* @return \DateTime
*/
public function retryUntil()
{
return now()->addHour(1);
}
/**
* @return mixed
*/
public function handle()
{
Redis::funnel('medialibrary:conversions')->limit(1)->then(function () {
app(FileManipulator::class)->performConversions($this->conversions, $this->media, $this->onlyMissing);
return true;
}, function () {
return $this->release(10);
});
}
}/*
* Here you can override the class names of the jobs used by this package. Make sure
* your custom jobs extend the ones provided by the package.
*/
'jobs' => [
'perform_conversions' => App\Support\MediaLibrary\Jobs\PerformConversions::class,
'generate_responsive_images' => Spatie\MediaLibrary\Jobs\GenerateResponsiveImages::class,
],
spatie-bot
commented
4 years ago Dear contributor,
because this issue seems to be inactive for quite some time now, I've automatically closed it. If you feel this issue deserves some attention from my human colleagues feel free to reopen it.
martindrapeau
commented
2 years ago @francoism90 in which file can I override the perform_conversions class? I want to specify a number of tries for this job. Maybe there is an easier way to do that than overriding the class?
/* * Here you can override the class names of the jobs used by this package. Make sure * your custom jobs extend the ones provided by the package. */ 'jobs' => [ 'perform_conversions' => App\Support\MediaLibrary\Jobs\PerformConversions::class, 'generate_responsive_images' => Spatie\MediaLibrary\Jobs\GenerateResponsiveImages::class, ],
francoism90
commented
2 years ago @martindrapeau I'm doing something different nowadays. I simple listen for MediaHasBeenAdded/MediaHasBeenUpdated and do my own logic (conversions, jobs, services).
martindrapeau
commented
2 years ago Ok. I ended up setting a number of tries in the Horizon supervisor. Merci François. --Martin
On Tue, Mar 22, 2022 at 3:24 AM François M. @.***> wrote:
@martindrapeau https://github.com/martindrapeau I'm doing something different nowadays. I simple listen for MediaHasBeenAdded/ MediaHasBeenUpdated and do my own logic (conversions, jobs, services).
— Reply to this email directly, view it on GitHub https://github.com/spatie/laravel-medialibrary/issues/1538#issuecomment-1074822595, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAZNXEBFXD543UUBKNUD5LVBFYRFANCNFSM4IPGS3DA . You are receiving this because you were mentioned.Message ID: @.***>
When using the following command to regenerate the following 7 models:
And something goes wrong, e..g.:
This happens:
It seems to duplicate like crazy and it's filling up the drive space. I've tried multiple locations (even non cache), but I'm unable to address this issue.
Do you have an idea?
Thanks.