undkonsorten / taskqueue

Manges async tasks given from other typo3 extensions
1 stars 2 forks source link

Allowed memory size of nnnn bytes exhausted #52 #8

Open LinkPool2 opened 11 months ago

LinkPool2 commented 11 months ago

Hi, trying to send the command "abgeschlossene löschen" in TYPO3 BE I'll get;

[24-Sep-2023 10:42:31] WARNING: [pool typo3-11] child 3446167 said into stderr: "NOTICE: PHP message: PHP Fatal error: Allowed memory size of nnnn bytes exhausted (tried to allocate 20480 bytes) in typo3_src/typo3_src-11.5.30/typo3/sysext/extbase/Classes/DomainObject/AbstractDomainObject.php on line 218"

https://wwww.host.de/typo3/module/tools/TaskqueueTaskqueue?
token=53645615038777ecd70e161a5a6bae5b1f2fdb80
&tx_taskqueue_tools_taskqueuetaskqueue%5Baction%5D=deleteFinished
&tx_taskqueue_tools_taskqueuetaskqueue%5Bcontroller%5D=Task

At this moment our Taskq contains 102.480 tasks

    /**
     * action delete finished tasks
     *
     * @throws StopActionException
     * @throws UnsupportedRequestTypeException
     * @throws IllegalObjectTypeException
     */
    public function deleteFinishedAction(): void
    {
        $tasks = $this->taskRepository->findFinished();
        $this->addFlashMessageForDeletion($tasks);
        foreach ($tasks as $task) {
            $this->taskRepository->remove($task);
        }
        $this->redirect('list');
    }

Maybe it would be better so create & call a taskRepository->deleteFinished() Method. Something like this - untested:


use TYPO3\CMS\Core\Database\ConnectionPool;

    /**
     * delete all finished tasks, returns the count of the deleted tasks
     * @return int
     */
    public function deleteFinished(): int
    {
        $queryBuilder = GeneralUtility::makeInstance(ConnectionPool::class)->getQueryBuilderForTable('tx_taskqueue_domain_model_task');
        $affectedRows = $queryBuilder
            ->delete('tx_taskqueue_domain_model_task')
            ->where(
                $queryBuilder->expr()->eq('status', TaskInterface::FINISHED)
            )
            ->executeStatement();
        return $affectedRows;
    }
Starkmann commented 10 months ago

We could do something like this, but would loose the ability to use @Cascade("remove") inside the TaskModel.

You have tow usecases/implementations here:

  1. Very slim task model with only ids.
  2. Big tasks with objects and relations.

1 is very fast and works for endless tasks 2 is rather slow and suitable for only a few tasks that have many object relations and @Cascade("remove")

We have to keep in mind here that in injection in model is no longer possible under typo3 12.

So we might do this as a feature toggle and the user can decide which usecase they need.

See https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/Configuration/FeatureToggles.html