BlackbitDigitalCommerce / pimcore-data-director

Import Bundle for Pimcore
16 stars 3 forks source link

Very slow import #187

Open betterapp opened 5 days ago

betterapp commented 5 days ago

Why complex dataports running so slow. We have dataport that should import about 35k products with some attributes, bricks data and so on.

image

betterapp commented 5 days ago

How is it possible that canceled dataport show time progress - it is running or what ?

https://github.com/user-attachments/assets/f7554d0d-b2d0-4bc8-a603-6160fc55b373

betterapp commented 4 days ago

image

Terrible slow.

betterapp commented 4 days ago

@BlackbitDevs

I checked log and see something strange.

./php.log:[2024-11-08T09:44:38.997926+01:00] pimcore.ERROR: could not resolve field "DriveTrains~BackGearType" in class Product [] []
./php.log:[2024-11-08T09:44:42.292088+01:00] pimcore.ERROR: could not resolve field "ListPrice1" in class Product [] []
./php.log:[2024-11-08T09:44:42.292161+01:00] pimcore.ERROR: could not resolve field "ListPrice2" in class Product [] []
./php.log:[2024-11-08T09:44:42.292188+01:00] pimcore.ERROR: could not resolve field "ListPrice3" in class Product [] []
  1. We do not have such fields in class: ListPrice1, ListPrice2, ListPrice3, DriveTrains~BackGearType image image

Those fields names was changed in class definition and dataport was changed also.

  1. DriveTrains~BackGearType - why this name have ~ ??

  2. There is no such string ListPrice1 in any table in database image

Our settings: image

Our mapping: image

So question is why DataDirector try to do anything with those not existed fields ? Is old mapping is cached or what ?

betterapp commented 4 days ago

Another problem

@BlackbitDevs

This request: image

With such payload: image

Took almost 20 seconds:

image

betterapp commented 4 days ago

We use:

"name": "blackbit/data-director", "version": "3.7.32",