search

fgsect / fexm

Automated fuzzing framework
Apache License 2.0
123 stars 26 forks source link

Interested in more types of seed crawlers? #4

Closed doshyt closed 6 years ago

doshyt commented 6 years ago

Hi,

I've been trying to set up FEXM and realized that it can crawl Github and PCAPs at the moment.

Would it make sense to offer my help in developing new types of crawlers for other code hosting solutions? I.e. TSVS, Gitlab, generic git repos etc.

Is it something that would help increase project adoption?

domenukk commented 6 years ago

Hey, You are correct. The main reasoning behind crawling GitHub is the assumption, that developers include trivial examle input as test cases for their tools - which in turn will work for other targets. For networking, we default to download all example pcaps from the Wireshark Wiki right now since some larger protocols are supported there.

Both seem like good defaults, although for specific targets, a more directed crawler can yield a better performance, of course.

We welcome any ideas for other sensible crawlers. Processing repos like https://gitlab.com/explore is definitely useful. I think if we get rid of the github rate limit that way, we could already increase adoption.

bshastry commented 6 years ago

I've been trying to set up FEXM and realized that it can crawl Github and PCAPs at the moment.

I believe it can crawl GitHub for file types other than pcaps as well. Edit: I guess you are referring to the GitHub crawler (any filetype) and the dedicated pcap crawler (which I was not aware of until now, lol)

Would it make sense to offer my help in developing new types of crawlers for other code hosting solutions? I.e. TSVS, Gitlab, generic git repos etc.

Indeed. Research shows that, more diverse the seeds, more likely it is to increase test coverage and eventually find bugs. So, more support for seed crawling is definitely good. PRs welcome :smile:

Is it something that would help increase project adoption?

I believe so. My reasoning is as follows: more crawlers -> more seeds -> better seeds -> increased test coverage -> more bugs.

domenukk commented 6 years ago

Closing this issue for now. Feel free to open issues for more specific crawlers you'd like to see (or even better, PRs ;) ) Thanks