search

good-tools / wiregasm

Packet Analyzer powered by Wireshark compiled for WebAssembly
GNU General Public License v2.0
59 stars 4 forks source link

Add support for Lua dissectors #2

Closed devanlai closed 1 year ago

devanlai commented 1 year ago

Overview

This pull request is my attempt to address #1, adding support for Lua-based dissectors by building and including Lua 5.2.4. It seems to work, but I suspect there are less hacky ways to incorporate Lua into the library.

I've done some cleanup to remove some unnecessary patches now that I have a working baseline.

Technical details

Building Lua

I added Makefile rules to download and build Lua 5.2.4 following a similar process to the other emscripten-ified libraries. The Lua build system is surprisingly simple, using only a plain Makefile without using autoconf or anything similar. From looking at the emscripten build process used by ysugimoto/webassembly-lua, it seems like setting CC=emcc -s WASM=1 is sufficient.

Future note: mainline Wireshark has some changes to fetch and build Lua as a CMake external project. Maybe updating to that version of Wireshark will eventually obviate the need to externally build Lua.

Linking lrexlib

I'm not sure if it's a side-effect of everything getting built as a static library instead of a shared library in some cases, but before I patched it, I was running into linker errors where wiregasm couldn't find luaopen_rex_pcre2. That's from lrexlib, which is built as a static library (even normally) and provides a regex library wrapper that's used by wslua. wslua is incorporated into epan as an object library, so epan picks up the dependency on lrexlib when Lua support is enabled.

Normally I would expect a private dependency of a static library to be incorporated seamlessly, but clearly something broke down. I ended up adding a patch to export lrexlib.a so that wiregasm can link against it directly, but I suspect there's a better way to do it, or at least a less messy implementation of this patch.

Future enhancements

Plugins directory

Right now I'm still using the patched /nonexistent/plugins directory for plugins, but it seems like we could probably give it some friendlier name. But it works fine as it is now and I'm not particular about the color of this bikeshed.

Lua Plugin reloading

Wireshark has the capability to reload all lua plugins at runtime, which is handy during development. In wiregasm, adding a function trigger that would be nice to enable importing lua dissectors after initialization, though it should be possible to accomplish this already just by destroying the wiregasm instance and recreating it. I've dug into this in the past to troubleshoot it, but I don't recall exactly where it lives and I would have to look it up again.

dehydr8 commented 1 year ago

Hi there, I was going to suggest the same things you've already incorporated:

  1. There was no need to build the lua target statically, i.e. no patches required
  2. Wireshark would pick up lua and configure LUA_FOUND, LUA_INCLUDE_DIRS, LUA_LIBRARIES, LUA_INCLUDE_DIR and LUA_VERSION_NUM automatically using FindLua
  3. Exporting lrexlib is still necessary as the linker complains about luaopen_rex_pcre2 during link time

At the time of building wiregasm, I had no intention of using the PLUGIN_DIR which was set to be under /nonexistent, with lua support added, we should change it as you suggested.

For loading plugins dynamically, I'll try looking for a hook and expose it in the lib, for now, plugins are loaded when epan_init gets called.

I'm attaching a patch that adds the following functionality:

  1. Adds a test case for a simple lua dissector, checks it against a sample pcap file
  2. Updates PLUGIN_DIR to /plugins and EXTCAP_DIR to /extcap
  3. Adds a new BeforeInitCallback to the Wiregasm wrapper to perform actions before the library gets initialized. This is used to set up dissectors in the /plugins directory before init() gets called
  4. Adding mkdirTree and mkdir definitions to the EmscriptenFileSystem interface

Please review it and add it to your PR at your will, or if you want, I can push them.

Thanks

dehydr8 commented 1 year ago

On a side note, the repository currently uses semantic-release style commit messages and the build process determines versions and creates change log based on them. I'd suggest following the same style of commit messages.

Thanks

dehydr8 commented 1 year ago

Hi Devan, I managed to add the patch that I attached above to your PR, I've also modified the following:

  1. /plugins and /uploads directories are now created when the runtime is initialized
  2. Added reloading support using wslua_reload_plugins, the header had to be exported so I've added a patch with the name 0008-export-wslua-headers.patch
  3. Added tests for reloading lua plugins with a sample dissector and pcap file

If you can, please test it and let me know if it works for your use-case.

Thanks

devanlai commented 1 year ago

Wow, thanks for working so quickly to expose the extra hooks!

I've tested your changes and confirmed that they work for reloading the lua plugins to add new ones or replace existing ones after initialization. I've also rebased my changes to flatten the fixup changes and hopefully bring the commit messages in line with the semantic release conventions.

github-actions[bot] commented 1 year ago

:tada: This PR is included in version 1.3.0 :tada:

The release is available on:

Your semantic-release bot :package::rocket: