Execution times

[cchudant]

Description

In depth info about execution plans This is more of a meta-issue (/roadmap) focusing everything about execution times.

Plans I have in mind:

• first audit of execution time, map possible improvement, low hanging fruits improvements, see if there are design flaws Will be using flamegraphs and run our current e2e tests with that
• Work on benchmarks
• CI integration to see regressions / improvements with each versions
• probably an auto generated page with these execution times of well known models
• Add execution info (or links to) on the readme for known models

I am not sure whether all of this is overkill or not since we're just using tract and not really touching the perf sensitive parts. We'll see.

Additional Information

None

Checklist

• [ ] This tests concerns BlindAI Client
• [x] This tests concerns BlindAI Server

[JoFrost]

"I am not sure whether all of this is overkill or not since we're just using tract and not really touching the perf sensitive parts. We'll see."

We are not only using Tract actually, and because we are executing the code inside an enclave, the performance sensitive part is affected. We cannot rely on the same way than the original library to load a model in memory, because of the enclave context we have to load the model entierly in one-go, and the memory management inside an enclave is different than on a normal application. This PR can be an opportunity to also test the performance with hardware optimizations we can use inside an enclave.