Lua Rules and Decoders - ohhh my

[jrossi]

I put Lua in the Decoders and Rules

I hate that doing anything crazy requires C code in ossec, and I thinking that is the wrong way to go. We need a simple core of C with high level interfaces to add features. I think that should be Lua as it intergrates so well with C and is highly portable. Right now I am just lua 5.2, but moving to luajit for server installs is something we can talk about for later.

Lua Decoder

<decoder name="sshd-lua">
  <parent>sshd</parent>
  <regex>xxxxxxx</regex>
  <lua context="default">
return function(T)
        local newT = {}
        newT.holder = "jeremy rossi"
        newT.dstport = "80"
        newT.dstip = "10.1.1.1"
        return newT
end
   </lua>
</decoder>

Everything in the Lua tags gets loaded at start up and run in a context="default". A lua contact is a complete lua state machine and we can support as many as we need, but data is never shared between contexts, but within a single context lua functions can interact with each other.

They key API for decoders is the following.

1. The code MUST return a function when loaded.
2. When executed a Lua Table is passed into the function for review and actions with the contents of Eventinfo details known at this point.
3. A table or nil can be returned from the fucntion
   • nil means nothing changes got to next decoder
   • table means merge the table returned with eventinfo

     Lua Rules

<!-- SSHD messages -->
<group name="syslog,sshd,">
  <rule id="5799" level="14">
    <decoded_as>sshd</decoded_as>
    <regex>xxxxx</regex>
    <lua>
counter = 0
return function(T)
        counter = counter + 1
        print(counter)
        for k,v in pairs(T) do
                print(k, " = ", v)
        end
end
</lua>
  </rule>

</group> <!-- SYSLOG, SSHD -->

All the same things from decoders apply, but more data is avaiable and i have not figured out how the results of output from lua should be used. Right now I am thinking the following:

(bool, table) as the required return from rules. leaving the follow possiable:

(true, table) -> alert, merge table into eventinfo (false, table) -> no alert, merge table into eventinfo (nil, table) -> no alert and contiune rule checking, merge table into eventinfo (true, nul) -> alert, no cahnge to eventinfo (false, nil) -> no alert, dont merge (nil, nil) -> no alert and contiune rule checking, no change to eventinfo

But this is something I want to get feedback on.

Code

I have my completely broken, but works for testing avaiable for review here: https://github.com/jrossi/ossec-hids/compare/lua-rules

here is the sample output using the above listed examples.

2014/10/13 05:59:21 ossec-testrule: INFO: Reading local decoder file.
4
2014/10/13 05:59:21 ossec-testrule: INFO: Started (pid: 23384).
ossec-testrule: Type one log per line.

**Phase 1: Completed pre-decoding.
       full event: 'Nov 17 21:41:22 localhost sshd[8060]: xxxxxxxxxxxxxxxx'
       hostname: 'localhost'
       program_name: 'sshd'
       log: 'xxxxxxxxxxxxxxxx'

**Phase 2: Completed decoding.
       decoder: 'sshd'
       dstport: '80'
       dstip: '10.1.1.1'
1
program_name     =      sshd
log      =      xxxxxxxxxxxxxxxx
location         =      stdin
dstport  =      80
dstip    =      10.1.1.1
full_log         =      Nov 17 21:41:22 localhost sshd[8060]: xxxxxxxxxxxxxxxx
hostname         =      localhost

**Phase 3: Completed filtering (rules).
       Rule id: '5799'
       Level: '14'
       Description: '(null)'
**Alert to be generated.

[jrossi]

@ossec and others, feedback? Ideas? Hate? love? mild heartburn when thinking about Dynamic Logic in rules and helping them out on the mailing list?

[cgzones]

I have to admit, i somehow dislike scripting in general. But maybe you or others can convince me ...

[jrossi]

@cgzones for OSSEC I feel we need Lua for people to start changing the internals without going to C. I feel we could get more people and more development via Lua and make the community stronger and more diverse.

Good example of something. Accumlator feature. @reyjrar spent a lot of time to add this and had to learn C again just get this done. After OSSECCON he and I think the feature can be removed and let Lua take over that role.

The other example is execd should be able to run lua scripts as active response. This will help with the fork bomb issue that some users are having. And give them more control over what they want to do and how often.

https://prezi.com/qzorrskkvksp/the-state-of-ossec/ is my presentation for OSSECCON 2014.

[awiddersheim]

I agree. There are probably a lot of user/configuration tools that can be done in lua that just don't need to be in C. I don't really muck with the rules very often so the power of your example is probably lost on me.

[jrossi]

@awiddersheim Lots more to be done with Lua - ossec-lua just got easier to use checkout #380 as it will make setting up some things really easy. With scripting means our code needs to have a clean API and useable.