search

zeek / zeek-docs

Documentation for Zeek
https://docs.zeek.org
Other
50 stars 69 forks source link

Document weirds #271

Open JustinAzoff opened 3 months ago

JustinAzoff commented 3 months ago

Weirds are generated as strings and are not documented like other enums.

This ended up on my radar recently due to popular LLMs completely hallucinating what DNS_Conn_count_too_large means. This is because that weird and many others like it are not really documented anywhere.

For each weird we should probably have:

Also, are all weirds tested? do we have a pcap that generates each one?

timwoj commented 3 months ago

Also, are all weirds tested? do we have a pcap that generates each one?

A very naive survey of grepping the source for "Weird(", filtering the results a bit, and then searching through testing/btests/Baselines for matches shows we're not anywhere close to 100% test coverage on the weirds. This matches with the lack of test coverage on a lot of the error cases across the source in general though (see also https://github.com/zeek/zeek/issues/2283).

This ended up on my radar recently due to popular LLMs completely hallucinating what DNS_Conn_count_too_large means.

Considering we don't have test coverage for any of the DNS wierds other than the EDNS ones, this isn't surprising.