SpartnerNL / Laravel-Excel

🚀 Supercharged Excel exports and imports in Laravel
https://laravel-excel.com
MIT License
12.29k stars 1.92k forks source link

[QUESTION]: Running queue stopped automatically when trying to export big data from queue. Is this normal or I am missing to update my env var? #2779

Closed DeepakAryal closed 3 years ago

DeepakAryal commented 4 years ago

I have to export big data in excel. So I am trying to queue query so that i can store exported file. but when i run queue to export excel, It stopped without completing the task.

In controller: (new LeadExcelExporter($options, $user))->queue('exportedLeads.xlsx', 'local')->allOnQueue('exports');

In exporter: public function query() { $ExampleRepository = resolve(ExampleRepository::class);

    return $ExampleRepository->createBaseBuilder($this->options);
}
ghost commented 4 years ago

Thanks for submitting the ticket. Unfortunately the information you provided is incomplete. We need to know which version you use and how to reproduce it. Please include code examples. Before we can pick it up, please check (https://github.com/Maatwebsite/Laravel-Excel/blob/3.1/.github/ISSUE_TEMPLATE.md) and add the missing information. To make processing of this ticket a lot easier, please make sure to check (https://laravel-excel.maatwebsite.nl/3.1/getting-started/contributing.html) and double-check if you have filled in the issue template correctly. This will allow us to pick up your ticket more efficiently. Issues that follow the guidelines correctly will get priority over other issues.

patrickbrouwers commented 4 years ago

I would need a bit more info like stacktrace

DeepakAryal commented 4 years ago

actually i am able to solve the issue but taking so much time to download 90k data from a queue job. is there any way to make export more faster?

patrickbrouwers commented 4 years ago

I would need to see some more info. A lot of things impact the speed: server, concerns, config, php setting, ...

skillcraft-io commented 4 years ago

I have some similar issues, I have pushed an export job onto the queue, to query the Database, build out the xlsx, store it on the server and email the user when its done...

Everything works fantastic with small number of records... however, whenever the datasets grows the queues continuously timeout, even though the datasets not that big (well under 20K).. the only error is the fact the queue timed-out/ran long.

I push these exports onto a dedicated queue worker with no timeout and 1gb memory limit.. still takes forever....

@DeepakAryal - how did you resolve/what was the issue?

DeepakAryal commented 3 years ago

@zhxscript I decided to use this package: https://github.com/rap2hpoutre/fast-excel if record to export is more than 20k

stale[bot] commented 3 years ago

This bug report has been automatically closed because it has not had recent activity. If this is still an active bug, please comment to reopen. Thank you for your contributions.

tharusha-fernando commented 9 months ago

Anyone got this fixed?? My queue stops after few iterations . when i start again it stops again after few iterations 2024-01-25 17:29:40 Maatwebsite\Excel\Jobs\AppendQueryToSheet ......................................................... RUNNING
2024-01-25 17:29:48 Maatwebsite\Excel\Jobs\AppendQueryToSheet ................................................. 7,535.96ms DONE
2024-01-25 17:29:48 Maatwebsite\Excel\Jobs\AppendQueryToSheet ......................................................... RUNNING
2024-01-25 17:29:56 Maatwebsite\Excel\Jobs\AppendQueryToSheet ................................................. 8,500.61ms DONE
2024-01-25 17:29:56 Maatwebsite\Excel\Jobs\AppendQueryToSheet ......................................................... RUNNING
2024-01-25 17:30:06 Maatwebsite\Excel\Jobs\AppendQueryToSheet ................................................. 9,450.56ms DONE
2024-01-25 17:30:06 Maatwebsite\Excel\Jobs\AppendQueryToSheet ......................................................... RUNNING
2024-01-25 17:30:15 Maatwebsite\Excel\Jobs\AppendQueryToSheet ................................................. 9,440.69ms DONE
2024-01-25 17:30:27 Maatwebsite\Excel\Jobs\AppendQueryToSheet ......................................................... RUNNING
2024-01-25 17:30:38 Maatwebsite\Excel\Jobs\AppendQueryToSheet ................................................ 10,980.05ms DONE
2024-01-25 17:30:49 Maatwebsite\Excel\Jobs\AppendQueryToSheet ................................................ 10,816.74ms DONE
PS C:\Users\Admin\Desktop\DBEETA\SchoolProject\start\schoolproject> php artisan queue:work 2024-01-25 17:31:03 Maatwebsite\Excel\Jobs\AppendQueryToSheet ................................................ 11,139.52ms DONE

ahmadyousefdev commented 9 months ago

@tharusha-fernando Same here, after around 20 minutes or so using queue:work the process stops. no logs, no anything.