erikbern / ann-benchmarks

Benchmarks of approximate nearest neighbor libraries in Python
http://ann-benchmarks.com
MIT License
4.88k stars 735 forks source link

Can we run profiler and export the profiler data while running benchmarks? #533

Open Mukulareddy opened 3 months ago

Mukulareddy commented 3 months ago

Hello, I am trying to run profiler while running ann-benchmarks using the following code: `from multiprocessing import freeze_support import cProfile import pstats from ann_benchmarks.main import main import yappi

if name == "main": profiler = cProfile.Profile() profiler.enable()

freeze_support()
main()  # The function that runs the benchmark

profiler.disable()
stats = pstats.Stats(profiler).sort_stats('cumtime')
stats.print_stats(10)  # Print top 10 results
stats.dump_stats('profile_results.prof')

Then run python run.py --dataset fashion-mnist-784-euclidean --algorithm milvus-ivfflat --runs 1` I am able to get the profiler data. But i am not sure if this is the correct way to profile for Milvus. It would be great if someone who have already done profiling for Milvus database share their opinion.