[ X ] Tests for the changes have been added (if applicable)
see speedtests.py which isn't a test per se but records the run time. I added a larger test file as well.
[ ] Docs have been added / updated (if applicable)
What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
It caches the regex lookups to a local dictionary in cache.py. Local testing found it runs about 15X faster.
What is the current behavior? (You can also link to an open issue here)
To lookup using regexes each time. The lookups in gettype get run on every value.
What is the new behavior (if this is a feature change)?
To cache the lookups to a local dict,so that the innermost loop lookups is hitting a dictionary rather than several regex lookups.
Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)
Hope not?
Other information:
Arguably a cleaner approach would be restructuring the data to support direct lookups, this is a "cheaper" approach that I could do by cutting and pasting rather than serious refactoring
see speedtests.py which isn't a test per se but records the run time. I added a larger test file as well.
[ ] Docs have been added / updated (if applicable)
What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
It caches the regex lookups to a local dictionary in cache.py. Local testing found it runs about 15X faster.
To lookup using regexes each time. The lookups in gettype get run on every value.
To cache the lookups to a local dict,so that the innermost loop lookups is hitting a dictionary rather than several regex lookups.
Hope not?
Arguably a cleaner approach would be restructuring the data to support direct lookups, this is a "cheaper" approach that I could do by cutting and pasting rather than serious refactoring