search

BlackbitDigitalCommerce / pimcore-data-director

Import Bundle for Pimcore
16 stars 3 forks source link

Data not saved into database when using DD #40

Closed betterapp closed 2 years ago

betterapp commented 2 years ago

I have strange behaviour.

I have a class named Product which have FuturePrices field - it is FieldCollection (FututrePrice).

Product class have id=2 Table name for FuturePrice is: object_collection_FuturePrice_2

When I import data (direct from admin panel or via API call) I see that data are not saved into database table. But when I open product in pimcore admin - I see data are there.

My DD configuration for this field looks like this:

return [
    'FuturePrice' => [
        [
            'FromDate' => $params['rawItemData']['ValidFrom']['value'],
            'NorthPrice' => $params['rawItemData']['NorthPrice']['value'],
            'SouthPrice' => $params['rawItemData']['SouthPrice']['value'],
            'BasePrice' => $params['rawItemData']['BasePrice']['value'],
        ]
    ]
];

XML:

<?xml version="1.0" encoding="UTF-8"?>
<PIMPrices>
    <Prices>
        <ArticleNumber>1234567890123</ArticleNumber>
        <SouthPrice>123.00</SouthPrice>
        <NorthPrice>123.00</NorthPrice>
        <ValidFrom>07.07.2022</ValidFrom>
    </Prices>
</PIMPrices>

logs:

[INFO] Importing object /test1
[INFO] Value for field ArticleNumber: 1234567890123
[INFO] Value for field FuturePrices: {
    "FuturePrice": [
        {
            "FromDate": "07.07.2022",
            "NorthPrice": "123.00",
            "SouthPrice": "123.00",
            "BasePrice": ""
        }
    ]
}
[INFO] Not adding field collection item because item 0 is equal
[INFO] Reason for saving: Publish state changed
[INFO] /test1 queued for saving
[INFO] Successfully saved new unpublished version for #30734 /test1

Is it a bug in DataDirector or schould I look for somewhere else.

BlackbitDevs commented 2 years ago

I actually thought this bug only got introduced in 3.1 - but obviously it was also present in 3.0. I just backported the fix to 3.0 - can you please try if this still happens with 3.0.11?

betterapp commented 2 years ago

Does not work in 3.0.11 :(

BlackbitDevs commented 2 years ago

This means you still do not get the message Successfully saved #30734 /test1? Can you please post the complete log for processing of this raw data item? Does this import use compatibility mode or not (see dataport > Settings > Advanced settings)?

betterapp commented 2 years ago 
Logs for Rawdata Processing (complete) (Dataport Id: 58, Resource: /home/daniel/Web/bohus-pimcore/web/var/assets/Files/prices.xml, language: en, Started at: 2022-07-11 09:46:00)
[INFO] Importing raw item 217745
[INFO] Key column (ArticleNumber) callback return: 1234567890123
[DEBUG] Fetching item from database
[INFO] Importing object /test1
[INFO] Value for field ArticleNumber: 1234567890123
[INFO] Value for field FuturePrices: {
    "FuturePrice": [
        {
            "FromDate": "07.07.2022",
            "NorthPrice": "123.00",
            "SouthPrice": "123.00",
            "BasePrice": ""
        }
    ]
}
[INFO] /test1 queued for saving
[INFO] Successfully saved new unpublished version for #30734 /test1

When I open object I see data: obraz

But under mysql I see it is empty obraz

It use Compatibility mode

betterapp commented 2 years ago

But If i change it to not use Compatibility mode the result is the same.

But strange if for me that it says:

Successfully saved new unpublished version

The object is published so why it say new unpublished version ?

betterapp commented 2 years ago

Is that mean we still got bug in DD ??

BlackbitDevs commented 2 years ago

Successfully saved new unpublished version The object is published so why it say new unpublished version ?

Unpublshied versions do not refer to the publish state of the object. An unpublished version is a version which may be published later. It reflects the behaviour when you click "Only save new version" in Pimcore backend.

Is that mean we still got bug in DD ??

Seams so, will take another look...

betterapp commented 2 years ago

Is it possible that fields changed by API is not saved in database at all - not only fields collection ? Because now i tried update one Product's field and I see that it has old value in database. But when I open the product in Admin Panel it show new one.

BlackbitDevs commented 2 years ago

Yes, the cause is that the object is just not getting saved to database at all - it is not about field collections. Could you check if the error also exists in 3.1.x-dev?

betterapp commented 2 years ago

we can not check with dev. we need released version. because it is production

betterapp commented 2 years ago

Hi. I have a problem with calculator class implementend for some object's fields. When I save object in PimCore admin - everything is ok.

When I try to save object by DD I got error:

[ERROR] Object #30734 could not be saved. Reverted changes. Error: Error: Call to undefined method Pimcore\Model\Element\ElementDescriptor::getCurrency() in /home/daniel/Web/pimcore/src/AppBundle/Calculators/ProfitDegreeSouth.php:33 Stack trace:

In calculator I have code like this: obraz

And the DD log print out:

Logs for Rawdata Processing (Dataport Id: 58, Resource: /home/daniel/Web/bohus-pimcore/web/var/assets/Files/prices.xml, language: en, Started at: 2022-07-13 08:36:33 by admin) [INFO] --- Importing next raw data item [INFO] Key column (ArticleNumber) callback return: 1234567890123 [DEBUG] Fetching item from database [INFO] Importing object /test1 [INFO] Value for field ArticleNumber: 1234567890123 [WARNING] Skipped Currency:path:Currency:isoCode: because: Invalid data in fieldBasePriceCurrency` [type: manyToOneRelation] [INFO] Value for field FuturePrices: { "FuturePrice": [ { "FromDate": "1657144800", "NorthPrice": 123, "SouthPrice": 123, "NetPurchasePrice": null, "BasePrice": null, "BasePriceCurrency": [], "BundlePrice": null } ] } [INFO] /test1 queued for saving [WARNING] Unexpected output during run: string(1) "1" string(32) "Pimcore\Model\DataObject\Product" string(1) "2" string(33) "Pimcore\Model\DataObject\Supplier" string(1) "3" string(33) "Pimcore\Model\DataObject\Currency" string(1) "1" string(32) "Pimcore\Model\DataObject\Product" string(1) "2" string(39) "Pimcore\Model\Element\ElementDescriptor"

[ERROR] Object #30734 could not be saved. Reverted changes. Error: Error: Call to undefined method Pimcore\Model\Element\ElementDescriptor::getCurrency() in /home/daniel/Web/bohus-pimcore/src/AppBundle/Calculators/ProfitDegreeSouth.php:33 Stack trace:

0 [internal function]: AppBundle\Calculators\ProfitDegreeSouth->compute()

1 /home/daniel/Web/pimcore/vendor/pimcore/pimcore/models/DataObject/Service.php(1605): call_user_func()

2 /home/daniel/Web/pimcore/var/classes/DataObject/Product.php(2780): Pimcore\Model\DataObject\Service::getCalculatedFieldValue()

3 /home/daniel/Web/pimcore/src/PriceHistoryBundle/EventListener/PriceHistoryListener.php(107): Pimcore\Model\DataObject\Product->getProfitSouth()

4 /home/daniel/Web/pimcore/src/PriceHistoryBundle/EventListener/PriceHistoryListener.php(44): PriceHistoryBundle\EventListener\PriceHistoryListener->isPriceUpdated()

5 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/EventDispatcher/Debug/WrappedListener.php(126): PriceHistoryBundle\EventListener\PriceHistoryListener->onPreUpdate()

6 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/EventDispatcher/EventDispatcher.php(264): Symfony\Component\EventDispatcher\Debug\WrappedListener->__invoke()

7 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/EventDispatcher/EventDispatcher.php(239): Symfony\Component\EventDispatcher\EventDispatcher->doDispatch()

8 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/EventDispatcher/EventDispatcher.php(73): Symfony\Component\EventDispatcher\EventDispatcher->callListeners()

9 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/EventDispatcher/Debug/TraceableEventDispatcher.php(168): Symfony\Component\EventDispatcher\EventDispatcher->dispatch()

10 /home/daniel/Web/data-director/lib/Pim/EventDispatcher.php(30): Symfony\Component\EventDispatcher\Debug\TraceableEventDispatcher->dispatch()

11 /home/daniel/Web/data-director/lib/Pim/Item/Importer.php(5294): Blackbit\DataDirectorBundle\lib\Pim\EventDispatcher->dispatch()

12 /home/daniel/Web/data-director/lib/Pim/Item/Importer.php(5048): Blackbit\DataDirectorBundle\lib\Pim\Item\Importer->saveConcrete()

13 /home/daniel/Web/data-director/lib/Pim/Item/Importer.php(1130): Blackbit\DataDirectorBundle\lib\Pim\Item\Importer->saveObject()

14 /home/daniel/Web/data-director/lib/Pim/Item/Importmanager.php(560): Blackbit\DataDirectorBundle\lib\Pim\Item\Importer->import()

15 /home/daniel/Web/data-director/Command/ImportPimCommand.php(263): Blackbit\DataDirectorBundle\lib\Pim\Item\Importmanager->importDataport()

16 /home/daniel/Web/data-director/Command/ImportPimCommand.php(175): Blackbit\DataDirectorBundle\Command\ImportPimCommand::import()

17 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/Console/Command/Command.php(255): Blackbit\DataDirectorBundle\Command\ImportPimCommand->execute()

18 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php(1027): Symfony\Component\Console\Command\Command->run()

19 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Console/Application.php(97): Symfony\Component\Console\Application->doRunCommand()

20 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php(273): Symfony\Bundle\FrameworkBundle\Console\Application->doRunCommand()

21 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Console/Application.php(83): Symfony\Component\Console\Application->doRun()

22 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php(149): Symfony\Bundle\FrameworkBundle\Console\Application->doRun()

23 /home/daniel/Web/pimcore/bin/console(36): Symfony\Component\Console\Application->run()

24 {main} `

I do not know why compute method from calculator class have Pimcore\Model\Element\ElementDescriptorclass under $object->getSupplier() where Pimcore\Model\DataObject\Supplier should be. When I save object via Admin panel there is no Pimcore\Model\Element\ElementDescriptor

I did not have this problem in 3.0.11.

BlackbitDevs commented 2 years ago

Just scanned through the core because the ElementDescriptor has never crossed my way. But I found https://github.com/pimcore/pimcore/blob/5a7729d37b353ae13271a2447311efe43ba33382/models/Element/Recyclebin/Item.php#L342-L363 - here for recycle bin items the relations get converted to ElementDescriptor objects - but I do not see anything similar when a recycle bin objects get restored in https://github.com/pimcore/pimcore/blob/5a7729d37b353ae13271a2447311efe43ba33382/models/Element/Recyclebin/Item.php#L389 - I think it has nothing to do with the Data Director but with Pimcore itself. Object loading in Pimcore backend is not done with the getter methods, so I suspect that when using the getter method the ElementDescriptor does not get resolved while with Pimcore backend loading it gets resolved.

Have opened an issue at Pimcore https://github.com/pimcore/pimcore/issues/12638

betterapp commented 2 years ago

I understand. But this error was not in 2.8.x DD as I remember thats why I thought it is a DD problem ;)

BlackbitDevs commented 2 years ago

Will check again, where the ElementDescriptors get resolved. Perhaps something is also missing in the Data Director...

In your case, does it happen with all objects or only with some? Because when only with some, may it be that those have been restored rom recycle bin?

betterapp commented 2 years ago

it was not restored from recycle bin. But I checked it and 2.8.8 - OK - saved in database (no ElementDescriptor problem) 3.0.6 - OK - saved in database (no ElementDescriptor problem) 3.0.10 - NOT OK - not saved in database (no ElementDescription problem) 3.0.11 - NOT OK - not saved in database (no ElementDescription problem) 3.1 - own calculator class enabled: NOT OK - not saved in database, ElementDescription problem 3.1 - own calculator class disabled: OK saved in database (no ElementDescriptor problem)

The calculator class was disabled like this: obraz

BlackbitDevs commented 2 years ago

Can you please output get_class($object) in a version where it does not work (e.g. 3.0.11)?

betterapp commented 2 years ago

In version 3.0.11 there is no problem with ElementDescription but with not saving data to database. Problem with ElementDescriptor is in 3.1.x-dev

betterapp commented 2 years ago

You got the output earier

obraz

string(1) "1" string(32) "Pimcore\Model\DataObject\Product" string(1) "2" string(33) "Pimcore\Model\DataObject\Supplier" string(1) "3" string(33) "Pimcore\Model\DataObject\Currency" string(1) "1" string(32) "Pimcore\Model\DataObject\Product" string(1) "2" string(39) "Pimcore\Model\Element\ElementDescriptor"

BlackbitDevs commented 2 years ago

The CalculatedValue Valculator gets called twice (we can see this from your output). Your stack trace in https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/40#issuecomment-1182833671 shows only first call. The error only occurs on the second call. Can you also post the stack trace of this one?

betterapp commented 2 years ago

@BlackbitNeueMedien

We use this on production.

So the problem here is: 2.8.8 is OK but have problem with import data that unique value is used 3.0.6 is OK but does not have value under __source for XLS files 3.0.10-11 does not save objects data into database 3.1.x - save object into database but for calculated field we got ElementDescriptor problem

The best option for us is to use 3.0.6 but we need to have __source data for XLS file.

Can You prepare patch that will fix it for 3.0.6 and then we can use the patch with cweagans/composer-patches package ?

betterapp commented 2 years ago

This stack trace is for second call:

[ERROR] Object #30734 could not be saved. Reverted changes. Error: Error: Call to undefined method Pimcore\Model\Element\ElementDescriptor::getCurrency() in /home/daniel/Web/bohus-pimcore/src/AppBundle/Calculators/ProfitDegreeSouth.php:33 Stack trace:

0 [internal function]: AppBundle\Calculators\ProfitDegreeSouth->compute()

https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/1 /home/daniel/Web/pimcore/vendor/pimcore/pimcore/models/DataObject/Service.php(1605): call_user_func() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/2 /home/daniel/Web/pimcore/var/classes/DataObject/Product.php(2780): Pimcore\Model\DataObject\Service::getCalculatedFieldValue() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/3 /home/daniel/Web/pimcore/src/PriceHistoryBundle/EventListener/PriceHistoryListener.php(107): Pimcore\Model\DataObject\Product->getProfitSouth() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/4 /home/daniel/Web/pimcore/src/PriceHistoryBundle/EventListener/PriceHistoryListener.php(44): PriceHistoryBundle\EventListener\PriceHistoryListener->isPriceUpdated() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/5 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/EventDispatcher/Debug/WrappedListener.php(126): PriceHistoryBundle\EventListener\PriceHistoryListener->onPreUpdate() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/6 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/EventDispatcher/EventDispatcher.php(264): Symfony\Component\EventDispatcher\Debug\WrappedListener->__invoke() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/7 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/EventDispatcher/EventDispatcher.php(239): Symfony\Component\EventDispatcher\EventDispatcher->doDispatch() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/8 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/EventDispatcher/EventDispatcher.php(73): Symfony\Component\EventDispatcher\EventDispatcher->callListeners() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/9 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/EventDispatcher/Debug/TraceableEventDispatcher.php(168): Symfony\Component\EventDispatcher\EventDispatcher->dispatch() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/10 /home/daniel/Web/data-director/lib/Pim/EventDispatcher.php(30): Symfony\Component\EventDispatcher\Debug\TraceableEventDispatcher->dispatch() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/11 /home/daniel/Web/data-director/lib/Pim/Item/Importer.php(5294): Blackbit\DataDirectorBundle\lib\Pim\EventDispatcher->dispatch() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/12 /home/daniel/Web/data-director/lib/Pim/Item/Importer.php(5048): Blackbit\DataDirectorBundle\lib\Pim\Item\Importer->saveConcrete() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/13 /home/daniel/Web/data-director/lib/Pim/Item/Importer.php(1130): Blackbit\DataDirectorBundle\lib\Pim\Item\Importer->saveObject() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/14 /home/daniel/Web/data-director/lib/Pim/Item/Importmanager.php(560): Blackbit\DataDirectorBundle\lib\Pim\Item\Importer->import() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/15 /home/daniel/Web/data-director/Command/ImportPimCommand.php(263): Blackbit\DataDirectorBundle\lib\Pim\Item\Importmanager->importDataport() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/16 /home/daniel/Web/data-director/Command/ImportPimCommand.php(175): Blackbit\DataDirectorBundle\Command\ImportPimCommand::import() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/17 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/Console/Command/Command.php(255): Blackbit\DataDirectorBundle\Command\ImportPimCommand->execute() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/18 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php(1027): Symfony\Component\Console\Command\Command->run() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/19 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Console/Application.php(97): Symfony\Component\Console\Application->doRunCommand() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/20 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php(273): Symfony\Bundle\FrameworkBundle\Console\Application->doRunCommand() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/21 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Console/Application.php(83): Symfony\Component\Console\Application->doRun() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/22 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php(149): Symfony\Bundle\FrameworkBundle\Console\Application->doRun() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/23 /home/daniel/Web/pimcore/bin/console(36): Symfony\Component\Console\Application->run() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/24 {main} `[ERROR] Object #30734 could not be saved. Reverted changes. Error: Error: Call to undefined method Pimcore\Model\Element\ElementDescriptor::getCurrency() in /home/daniel/Web/bohus-pimcore/src/AppBundle/Calculators/ProfitDegreeSouth.php:33 Stack trace:

0 [internal function]: AppBundle\Calculators\ProfitDegreeSouth->compute()

https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/1 /home/daniel/Web/pimcore/vendor/pimcore/pimcore/models/DataObject/Service.php(1605): call_user_func() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/2 /home/daniel/Web/pimcore/var/classes/DataObject/Product.php(2780): Pimcore\Model\DataObject\Service::getCalculatedFieldValue() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/3 /home/daniel/Web/pimcore/src/PriceHistoryBundle/EventListener/PriceHistoryListener.php(107): Pimcore\Model\DataObject\Product->getProfitSouth() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/4 /home/daniel/Web/pimcore/src/PriceHistoryBundle/EventListener/PriceHistoryListener.php(44): PriceHistoryBundle\EventListener\PriceHistoryListener->isPriceUpdated() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/5 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/EventDispatcher/Debug/WrappedListener.php(126): PriceHistoryBundle\EventListener\PriceHistoryListener->onPreUpdate() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/6 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/EventDispatcher/EventDispatcher.php(264): Symfony\Component\EventDispatcher\Debug\WrappedListener->__invoke() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/7 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/EventDispatcher/EventDispatcher.php(239): Symfony\Component\EventDispatcher\EventDispatcher->doDispatch() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/8 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/EventDispatcher/EventDispatcher.php(73): Symfony\Component\EventDispatcher\EventDispatcher->callListeners() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/9 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/EventDispatcher/Debug/TraceableEventDispatcher.php(168): Symfony\Component\EventDispatcher\EventDispatcher->dispatch() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/10 /home/daniel/Web/data-director/lib/Pim/EventDispatcher.php(30): Symfony\Component\EventDispatcher\Debug\TraceableEventDispatcher->dispatch() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/11 /home/daniel/Web/data-director/lib/Pim/Item/Importer.php(5294): Blackbit\DataDirectorBundle\lib\Pim\EventDispatcher->dispatch() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/12 /home/daniel/Web/data-director/lib/Pim/Item/Importer.php(5048): Blackbit\DataDirectorBundle\lib\Pim\Item\Importer->saveConcrete() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/13 /home/daniel/Web/data-director/lib/Pim/Item/Importer.php(1130): Blackbit\DataDirectorBundle\lib\Pim\Item\Importer->saveObject() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/14 /home/daniel/Web/data-director/lib/Pim/Item/Importmanager.php(560): Blackbit\DataDirectorBundle\lib\Pim\Item\Importer->import() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/15 /home/daniel/Web/data-director/Command/ImportPimCommand.php(263): Blackbit\DataDirectorBundle\lib\Pim\Item\Importmanager->importDataport() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/16 /home/daniel/Web/data-director/Command/ImportPimCommand.php(175): Blackbit\DataDirectorBundle\Command\ImportPimCommand::import() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/17 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/Console/Command/Command.php(255): Blackbit\DataDirectorBundle\Command\ImportPimCommand->execute() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/18 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php(1027): Symfony\Component\Console\Command\Command->run() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/19 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Console/Application.php(97): Symfony\Component\Console\Application->doRunCommand() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/20 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php(273): Symfony\Bundle\FrameworkBundle\Console\Application->doRunCommand() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/21 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Console/Application.php(83): Symfony\Component\Console\Application->doRun() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/22 /home/daniel/Web/pimcore/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php(149): Symfony\Bundle\FrameworkBundle\Console\Application->doRun() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/23 /home/daniel/Web/pimcore/bin/console(36): Symfony\Component\Console\Application->run() https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/24 {main} `

betterapp commented 2 years ago

As I remember it is called more times. In version where we do not have this problem it is about 5 times.

BlackbitDevs commented 2 years ago

Can you turn off compatibility mode? Your stack trace shows that is is currently enabled. Of course this is only a hotfix solution but as you have this in production, it might solve the issue for now. In the meantime I will try to find a sustainable solution...

betterapp commented 2 years ago

I check without compatibility mode: https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/40#issuecomment-1180078957

result is the same

betterapp commented 2 years ago

The patch for 3.0.6

3.0.6 is OK but does not have value under __source for XLS files

would be best option now for us because as I see it is a big problem.

betterapp commented 2 years ago

Can I switch to 2.8.8 from 3.0.11 without problems ?

BlackbitDevs commented 2 years ago

Of course you can downgrade but you would have to care for reverting migrations, see https://www.doctrine-project.org/projects/doctrine-migrations/en/1.8/reference/managing_migrations.html#reverting-migrations - the migrations themselves all have a down() method.

About compatibility mode: https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/40#issuecomment-1180078957 was before I applied the fix from 3.1. Especially your stack trace should look different when you use compatibility mode. Perhaps you can give it a try again and send the stack trace. Please also send the exact version which you used for the stack trace because the line numbers in https://github.com/BlackbitDigitalCommerce/pimcore-data-director/issues/40#issuecomment-1186783825 did not match 3.0.11 or current 3.1-dev.

betterapp commented 2 years ago

Hmm. DD 3.1

Looks like if I enable compatibility mode (ON) everything is OK. Strange. Probably something wrong in my testing or not saved after changing this dataport parameter. sorry.

// STACK FOR COMPATIBILITY MODE: OFF

Logs for Rawdata Processing (complete) (Dataport Id: 58, Resource: /home/daniel/Web/bohus-pimcore/web/var/assets/Files/prices.xml, language: en, Started at: 2022-07-19 07:05:44) [INFO] --- Importing next raw data item [INFO] Key column (ArticleNumber) callback return: 12345 [DEBUG] Fetching item from database [INFO] Importing object /12345 [INFO] Value for field ArticleNumber: 12345 [WARNING] Skipped Currency:path:Currency:isoCode: because: Invalid data in fieldBasePriceCurrency` [type: manyToOneRelation] [INFO] Value for field FuturePrices: { "FuturePrice": [ { "FromDate": "1657144800", "NorthPrice": 123, "SouthPrice": 123, "NetPurchasePrice": null, "BasePrice": null, "BasePriceCurrency": [], "BundlePrice": null } ] } [INFO] /12345 queued for saving [ERROR] Object #32272 could not be saved. Reverted changes. Error: Error: Call to undefined method Pimcore\Model\Element\ElementDescriptor::getCurrency() in /home/daniel/pimcore/src/AppBundle/Calculators/ProfitDegreeSouth.php:26 Stack trace:

0 [internal function]: AppBundle\Calculators\ProfitDegreeSouth->compute()

1 /home/daniel/Web/bohus-pimcore/vendor/pimcore/pimcore/models/DataObject/Service.php(1605): call_user_func()

2 /home/daniel/Web/bohus-pimcore/var/classes/DataObject/Product.php(2780): Pimcore\Model\DataObject\Service::getCalculatedFieldValue()

3 /home/daniel/Web/bohus-pimcore/src/PriceHistoryBundle/EventListener/PriceHistoryListener.php(106): Pimcore\Model\DataObject\Product->getProfitSouth()

4 /home/daniel/Web/bohus-pimcore/src/PriceHistoryBundle/EventListener/PriceHistoryListener.php(43): PriceHistoryBundle\EventListener\PriceHistoryListener->isPriceUpdated()

5 /home/daniel/Web/bohus-pimcore/vendor/symfony/symfony/src/Symfony/Component/EventDispatcher/Debug/WrappedListener.php(126): PriceHistoryBundle\EventListener\PriceHistoryListener->onPreUpdate()

6 /home/daniel/Web/bohus-pimcore/vendor/symfony/symfony/src/Symfony/Component/EventDispatcher/EventDispatcher.php(264): Symfony\Component\EventDispatcher\Debug\WrappedListener->__invoke()

7 /home/daniel/Web/bohus-pimcore/vendor/symfony/symfony/src/Symfony/Component/EventDispatcher/EventDispatcher.php(239): Symfony\Component\EventDispatcher\EventDispatcher->doDispatch()

8 /home/daniel/Web/bohus-pimcore/vendor/symfony/symfony/src/Symfony/Component/EventDispatcher/EventDispatcher.php(73): Symfony\Component\EventDispatcher\EventDispatcher->callListeners()

9 /home/daniel/Web/bohus-pimcore/vendor/symfony/symfony/src/Symfony/Component/EventDispatcher/Debug/TraceableEventDispatcher.php(168): Symfony\Component\EventDispatcher\EventDispatcher->dispatch()

10 /home/daniel/Web/data-director/lib/Pim/EventDispatcher.php(30): Symfony\Component\EventDispatcher\Debug\TraceableEventDispatcher->dispatch()

11 /home/daniel/Web/data-director/lib/Pim/Item/Importer.php(5294): Blackbit\DataDirectorBundle\lib\Pim\EventDispatcher->dispatch()

12 /home/daniel/Web/data-director/lib/Pim/Item/Importer.php(5048): Blackbit\DataDirectorBundle\lib\Pim\Item\Importer->saveConcrete()

13 /home/daniel/Web/data-director/lib/Pim/Item/Importer.php(1130): Blackbit\DataDirectorBundle\lib\Pim\Item\Importer->saveObject()

14 /home/daniel/Web/data-director/lib/Pim/Item/Importmanager.php(560): Blackbit\DataDirectorBundle\lib\Pim\Item\Importer->import()

15 /home/daniel/Web/data-director/Command/ImportCompleteCommand.php(242): Blackbit\DataDirectorBundle\lib\Pim\Item\Importmanager->importDataport()

16 /home/daniel/Web/data-director/Command/ImportCompleteCommand.php(148): Blackbit\DataDirectorBundle\Command\ImportCompleteCommand::import()

17 /home/daniel/Web/bohus-pimcore/vendor/symfony/symfony/src/Symfony/Component/Console/Command/Command.php(255): Blackbit\DataDirectorBundle\Command\ImportCompleteCommand->execute()

18 /home/daniel/Web/bohus-pimcore/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php(1027): Symfony\Component\Console\Command\Command->run()

19 /home/daniel/Web/bohus-pimcore/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Console/Application.php(97): Symfony\Component\Console\Application->doRunCommand()

20 /home/daniel/Web/bohus-pimcore/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php(273): Symfony\Bundle\FrameworkBundle\Console\Application->doRunCommand()

21 /home/daniel/Web/bohus-pimcore/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Console/Application.php(83): Symfony\Component\Console\Application->doRun()

22 /home/daniel/Web/bohus-pimcore/vendor/symfony/symfony/src/Symfony/Component/Console/Application.php(149): Symfony\Bundle\FrameworkBundle\Console\Application->doRun()

23 /home/daniel/Web/bohus-pimcore/bin/console(36): Symfony\Component\Console\Application->run()

24 {main} `

betterapp commented 2 years ago

When version 3.1 will be deployed and tagged ?

BlackbitDevs commented 2 years ago

Sorry, just found the problem. Please update to 3.0.12, this should fix the problem with only a new version being saved but not the object itself. About the ElementDescriptor thing I will check later.

betterapp commented 2 years ago

I can confirm that data are saved to database in 3.0.12 Thanks.