search

instride-ch / pimcore-data-definitions

Data Definitions Plugin for Pimcore
Other
77 stars 58 forks source link

[Perf] remove \Wvision\Bundle\DataDefinitionsBundle\Interpreter\AssetUrlInterpreter::deduplicateAsset fuzzy #402

Open dkarlovi opened 10 months ago

dkarlovi commented 10 months ago
Q A
Bug report? no
Feature request? yes
BC Break report? no
RFC? no
Branch? master

Fuzzy as used currently doesn't scale well, with a lot of assets in a asset-heavy import it can take a second and it can get executed hundreds of times. Much better is to have a simple lookup by single value, the issue is the value can be an array (multiple URLs match the same asset).

We could support say up to 5 URLs and then do

(name = 'url1' and data = $value) OR (name = 'url2' and data = $value)...

It's not pretty, but should be much faster, without us needing to migrate the data structure to say JSON or something.

dkarlovi commented 10 months ago

We did this change inline on a project and the improvement is substantial, went from 200ms queries to 5ms queries.