search

espressif / esp-idf

Espressif IoT Development Framework. Official development framework for Espressif SoCs.
Apache License 2.0
13.68k stars 7.29k forks source link

Use static analysis tools during CI (IDF-91) #145

Closed bvernoux closed 5 years ago

bvernoux commented 7 years ago

When I see all the issues (#122, #133, #142, #144) we have with some crash, buffer/stack overflow I think a good idea will be to use https://scan.coverity.com which could be very helpful on some hard to track bug especially on TCP/IP ... For information it is free for use with open source code There is just a problem as some part of the code (wifilib ...) are binary blob and those modules could be not checked ...

annejan commented 7 years ago

It's also easy to integrate with travisCI . . The main downside is the very low submission amount in their free tier https://scan.coverity.com/faq#frequency

bvernoux commented 7 years ago

It is better than nothing even worst case 2 builds per week, I suspect actual code is less than 100Klines anyway so => Up to 12 builds per week, with a maximum of 3 builds per day, for projects with fewer than 100K lines of code

igrr commented 7 years ago

Thanks for the suggestion. I have been looking at using a static analyzer, although a different one. Because in our environment CI has to happen before things are merged into master, and the code is not exposed publicly before it is merged, I was looking at using one of the standalone commercial tools.

That being said, the issues you have linked to this one are not exactly the ones which the static analyzer can find. Stack overflow checking is quite hard, especially when function pointers come into play. Issue #141 asserts with a clear message ("Untested FreeRTOS function vTaskSuspend"), which is intentional, so it's also not something a static analyzer can help with :) I'm not dismissing the usefulness of a static analyzer, just pointing out that it's scope is somewhat different.

bvernoux commented 7 years ago

Yes issue #141 or other linked issue could be probably not detected by static analyzer, it was mainly as example anyway Coverity Scan is a bit more than a simple static analyzer ...

annejan commented 7 years ago

You know you can setup coverity for your own idf-template based project? I think I'll try this if I find the time later today . .

bvernoux commented 7 years ago

I'm using it with hydrabus project for the firmware https://github.com/bvernoux/hydrafw it is why I know it is very good it has found lot of problems in fatfs too with some special case (I suspect due to fat corruption so it is special cases) which do some buffer overflow ...

PerMalmberg commented 6 years ago

@annejan Did you manage to get Coverity to run on an E-I project? I tried back in november, but never got it to work.

sqrtroot commented 3 years ago

@david-cermak Are there any plans on releasing the setup you have in the internal project IDF_ANALYSIS_UTILS? In the gitlab ci file there are several mentions of it, but it isn't available publicly. I'm especially interested in generate_report.sh for one of our projects.

david-cermak commented 3 years ago

@sqrtroot Some long term plans, we do have (some notes about the setup in this issue ), but that's not going to be in the near future. The reporting part is based on codereport package (https://pypi.org/project/codereport/) and could be shared I think (but is probably a bit obsoleted by now, as we've already switched to a different reporting platform)

sqrtroot commented 3 years ago

Thanks for the quick reply I'll look into codereport and hopefully we can see some build in analysis in the future :smile: