esonderegger / fecfile

a python parser for the .fec file format
https://esonderegger.github.io/fecfile/
Apache License 2.0
44 stars 13 forks source link

Cache #10

Closed jsfenfen closed 5 years ago

jsfenfen commented 5 years ago

see speedtests.py which isn't a test per se but records the run time. I added a larger test file as well.

It caches the regex lookups to a local dictionary in cache.py. Local testing found it runs about 15X faster.

To lookup using regexes each time. The lookups in gettype get run on every value.

To cache the lookups to a local dict,so that the innermost loop lookups is hitting a dictionary rather than several regex lookups.

Hope not?

Arguably a cleaner approach would be restructuring the data to support direct lookups, this is a "cheaper" approach that I could do by cutting and pasting rather than serious refactoring

esonderegger commented 5 years ago

Code looks good to me. I just did some testing locally with this branch and the speedups are indeed impressive.

Thanks for suggesting this! 👍