Open madhu2000u opened 1 month ago
If this is triggering when trying to collect data, and the baseline data doesn't get updated, I think this is a very sensible change.
One question though I'd like input from others is if this is merged should this be an error or a warning? I can see arguments either way on this.
Baseline data must be present for all benchmarks. Its absence should be a catastrophic fail. We need to get the baseline measurement added. I have this on my todo list, but you can propose a value if you have access to the reference platform (STM32F407 @ 16MHz with FPU disabled).
Would you raise an issue to ensure this is dealt with promptly. Thanks.
Thank you @simonpcook and @jeremybennett for your review and inputs. I do not have the reference platform to propose a value. But I have raised an issue (#202) for this as you suggested.
Also, do you have any suggestions in the code addition to handle such errors or is this good to go for a merge?
Thank you!
When collecting the relative results, it compares with baseline data. Recently, Depthconv benchmark was added with this commit 07282ee but a baseline data was not added to the baseline-data/size.json.
This commit does not add a baseline data for Depthconv into baseline-data/size.json but rather adds error handling in benchmark_size.py should any new benchmarks get added but a baseline data is not included.