Torq IT New Additions

[jeremygnoonan]

As discussed with @fashxp regarding a number of new features to the data importer. Had started a private extensions repo @ Torq but then decided to connect with Pimcore and see if any of it was interest to pull into the main repo.

Did my best to update the docs, so a good place to reference everything that has changed. Highlights:

• Added Symfony Expression filter to all file formats
• Added the Composed Parent and Composed Path resolvers (my favourite addition)
• Added a Composed Path Asset Loader (related to above)
• Replaced the Excel parser with Openspout. This has worked well for me. open for input on if we see risk here. MUCH more memory friendly.
• Added the option to Bulk Load excel files to queue table (just dawned on me that this could easily be done for all file formats). SUPER fast for large files, can queue 100K+ records in seconds.
• Added a loading strategy based on the value of a property
• Added a transformation to make a string safe for a Pimcore key
• Added a transformation to return a constant value (useful to assign something as a variant)
• Added a data target that allows adding an image to the end of an image gallery.
• Added a data target that allows setting a property
• Added the overwrite options from Direct into Classification Store DataTarget

Submitting this as draft as I want to do some additional testing, but also wanted to give visibility.

[github-actions[bot]]

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

[jeremygnoonan]

recheck

[herbertroth]

Can you please merge or rebase the changes from 1.x to your PR? This is mostly for using qodana to check the PR.

[jeremygnoonan]

Can you please merge or rebase the changes from 1.x to your PR? This is mostly for using qodana to check the PR.

Done

[jeremygnoonan]

@herbertroth can you point me in the correct direction to sign the CLA? I am not getting a prompt here like the main Pimcore repo.

[herbertroth]

You can sign the CLA by just posting a Pull Request Comment same as the below format.

I have read the CLA Document and I hereby sign the CLA

[herbertroth]

I have read the CLA Document and I hereby sign the CLA

[herbertroth]

recheck

[herbertroth]

@jeremygnoonan Sorry, there was a problem with the allowlist in our cla check. github-actions[bot] was not in this list. So please rebase again...

[jeremygnoonan]

I have read the CLA Document and I hereby sign the CLA

[jeremygnoonan]

@herbertroth done

[herbertroth]

@jeremygnoonan Sorry, sorry, sorry... pleas rebase again...

[alexz707]

Hi Jeremy,

First and foremost, thank you so much for your extensive contribution to our project! We truly appreciate the time and effort you've invested in enhancing the codebase with numerous features and bug fixes.

After reviewing your pull request, it appears that it contains some breaking changes, such as the addition of a method to an interface and typed properties to existing ones. While we value the improvements you've made, we've found that the PR is quite comprehensive, and handling such a large set of changes can be challenging. Given the wide range of the changes, it would be beneficial for us to work through them in smaller, more manageable pieces.

Before you embark on the task of splitting the PR, we suggest reaching out to our enterprise support team. They can provide valuable insights into which features would be best to start with and guide you through the process. Collaborating with them at this stage will help ensure a smoother review and integration process.

Once again, thank you for your dedication to improving our project. We look forward to working together to make the integration process as seamless as possible.

Best regards, Alex