search

pimcore / data-importer

This extension adds a comprehensive import functionality to Pimcore Datahub.
Other
38 stars 58 forks source link

[Bug]: The "kernel" service is synthetic error when processing parallel #420

Open IronSean opened 2 months ago

IronSean commented 2 months ago

Expected behavior

datahub:data-importer:process-queue-parallel will properly process rows, or throw errors when there is a logic error

Actual behavior

Occasionally rows fail with the following error: Error processing element: IMPORT,DATA,HERE The "kernel" service is synthetic, it needs to be set at boot time before it can be used.

This seems like a possible race condition as it seems to happen more when the server is under load by other tasks, but I might be imagining that.

Steps to reproduce

Inconsistent. Run lots of 10k, 100k, 1m row datahub imports with multiple parallel workers (8 in my case on 4 cores) while the server is under other workloads.

Sometimes there is no error, sometimes a few rows, and sometimes the whole import fails.

github-actions[bot] commented 1 month ago

Thanks a lot for reporting the issue. We did not consider the issue as "Pimcore:Priority", "Pimcore:ToDo" or "Pimcore:Backlog", so we're not going to work on that anytime soon. Please create a pull request to fix the issue if this is a bug report. We'll then review it as quickly as possible. If you're interested in contributing a feature, please contact us first here before creating a pull request. We'll then decide whether we'd accept it or not. Thanks for your understanding.

abdelrhmanyosry commented 1 month ago

I have the same issue. image