search

IITH-Compilers / IR2Vec

Implementation of IR2Vec, LLVM IR Based Scalable Program Embeddings
Other
84 stars 39 forks source link

Benchmarks time evaluation script #111

Closed m-atalla closed 5 months ago

m-atalla commented 5 months ago

Hi, as we discussed in #101 this PR adds a script to replicate the TimeTaken experiment results for the latest release of IR2Vec. Further, this script should serve as a utility to evaluate potential speed up achieved via a parallel implementation.

As for the SQLite Amalgamation benchmark adding it would be as simple as adding another IR file to the benchmarks directory; when I ran the benchmark I got an average runtime of 28725.14 and 1624.31 ms for the flow-aware and symbolic modes respectfully on an AMD Ryzen 7 5800x 3.8GHz. I didn't include the SQLite IR file here as I didn't want to pollute this PR with about 500k lines of LLVM IR :sweat_smile:.

I added the script under the src/test-suite/ directory, please let me know if any additional changes are needed.

Thank you.

svkeerthy commented 5 months ago

Hi @m-atalla,

Thanks for the PR and for sharing the benchmarking numbers! The script looks good to me.

Regarding the SQLite Amalgamation, I suggest adding the source as part of the tests. You could modify the CMake configuration to clone the source repo and convert it to .ll files. We can address this in a new PR.

Best, Venkat

m-atalla commented 5 months ago

Thanks, that's actually a great suggestion! will probably send a PR next Friday.