dynamicexpresso / DynamicExpresso

C# expressions interpreter
http://dynamic-expresso.azurewebsites.net/
MIT License
1.99k stars 374 forks source link

Parseasdelegate performance #298

Closed acquleo closed 1 year ago

acquleo commented 1 year ago

I'm running an application which is parsing 100000 expressions ready to be evaluated at runtime. The parsing of the expression using parseasdelegate takes forever. Do you know any limitations of optimization to be made in order to speed up this process? Thanks

metoule commented 1 year ago

Do you have an example of an expression that your application evaluates? It could help pinpoint any bottleneck. Can you also define what "forever" is to you? I benchmarked the parsing in the past, and it took about 10ms per expression if I remember correctly.

acquleo commented 1 year ago

I'll give more details tomorrow when I'm at work.

If it would be 10ms for each parse then 100000 expressions would be 16 minutes of parsing so it will fit my forever sentence because I've stopped the application long before. Using parse which returns a lambda is quicker but I've got an impressive memory usage. 2 GB of RAM with those 100000 expressions.

I'm trying to figure out the best compromise of memory usage and startup time.

All the expressions are like the following: {Id1} | {id2} | {id3} | {id4} Then the expression is elaborated by a regex replace to the following expression: Al("id1") | Al("id2") | Al("id3") | Al("id4")

The execution time of the regex is very quick.

Then the resulting expression is parsed by dynamic Expresso, Al is a function that takes a string in input and returns a boolean.

Leonardo

davideicardi commented 1 year ago

@acquleo

If your expressions are all like the one you provided (with different values I suppose), probably DynamicExpresso is not the best tool for the job. Because you don't really need it for this kind of scenario. Maybe you can just use a regex to get all the values and do a custom processing.

Anyway in the past I had a similar scenario where I have a lof of expressions. I solved it using a "lazy" parsing approach: I don't parse all expressions in the initialization but only the ones that I use (like a parsing "on demand"), and then cache the result.

acquleo commented 1 year ago

You're right. I'm evaluating another expression evaluator which provides enough functionalities and better performance. I'm using ncalc instead of dynamic Expresso.