BlackbitDigitalCommerce / pimcore-data-director

Import Bundle for Pimcore
16 stars 3 forks source link

NFR: Optimisation for Continues Integration #61

Open kaurov opened 1 year ago

kaurov commented 1 year ago

Can you please improve DataDirector to make it safier for support:

BlackbitDevs commented 1 year ago

Thanks for your suggestions, @kaurov. We will put them on our todo list. You can speed up processing of your requested points by writing to info@blackbit.de - we will send you an offer and then you can decide which points shall be implemented and in which order.

One note about the attribute mapping hints:

Here, you can add your existing hint 'Field * is used as key field. Consider adding an index in the field definition with the 'Indexed' checkbox to improve performance' -- which user can see only if open import logs, currently. This is already visible as a mouseover hint when you move the mouse over the key icon of a key field:

Bildschirmfoto 2022-10-24 um 16 15 48

And about:

Please show not only for currenly loaded file (first) but all unique values of all XML files from folder provided as source for dataport. I mean following columns: incoming data 'Example result' 'Parsed' data after processing or PHP Exception

Currently you can select another raw data item used for preview by clicking inside the Example result column:

Bildschirmfoto 2022-10-24 um 16 17 31

Of course you do not see all possible values in one sight this way but can only manually test certain raw data items. For this reason we plan a dry-run. At the end of a dry-run you would see an overview of all the imported objects and what data changed. Sadly until now we did not find a client who was willing to pay for that...

BlackbitDevs commented 1 year ago

Please store plugin_pimdataport->sourceconfig container in dataport.json as JSON, not serialised string. The reason is -- that string is of HUGE size and developer commit it as generated without reading it, so error can be easily made.

This is an important point for comparing it in Git (or with the integrated versioning tool). Think I will implement this shortly...

BlackbitDevs commented 1 year ago

Concerning

Also, Customer provided values like following which DataDirector cannot process itself, I need to prepare in func: '- 52 F' -- space after minus is not supported '12 turns per minute' -- measurement unit cannot have spaces even is fexists in Pimcore. why?

I have enhanced quantity value recognition when provided as a string. From version 3.3 the following works: