search

metafacture / metafacture-fix

Work in progress towards an implementation of the Fix language for Metafacture
Apache License 2.0
6 stars 2 forks source link

`include` impacts performance #287

Open TobiasNx opened 1 year ago

TobiasNx commented 1 year ago

A fix transfomation in one file is faster and needs less storage when executed than a fix transformation that is distributed over multiple files and combined via include-functions.

blackwinter commented 1 year ago

Previous chat discussion (in German).

BTW: I would consider this a bug, but that's certainly debatable.

dr0i commented 1 year ago

Fixing this may help fixing https://github.com/hbz/lobid-resources/issues/1666.

dr0i commented 8 months ago

Don't know if this can be closed @blackwinter ? With https://github.com/metafacture/metafacture-fix/pull/327 we gained at least some performance.

blackwinter commented 8 months ago

Ultimately, @TobiasNx has to decide whether his use case has improved sufficiently. I didn't link the pull request to this issue intentionally.

The next step would need to be to look at the RecordTransformer implementation itself, which would potentially also affect the main Fix and call_macro() in addition to include().

TobiasNx commented 8 months ago

It seems that it got 30 min faster.

blackwinter commented 8 months ago

It seems that it got 30 min faster.

That's roughly 3.5 %.

TobiasNx commented 2 months ago

In my opinion the first step is done and it improved the performance. Next step would be the suggested rework of RecordTransformer