an implementation in Rust of Timeline logging from SnabbSwitch

[Geal]

Timeline

Description

The Snabb Switch project, written in Lua, has a lot of innovative concepts, and one I really like is the Timeline logging.

It separates the data you log from its formatting:

• each entry in the log contains a high resolution timestamp, up to four 64 bit values, and an index referencing the formatting string
• the log contains an array of formatting strings

When you read the log, you load the formatting strings, and for each log entry, you look up the corresponding formatting string, and you use it to print the log line.

Why is it a nice idea?

It's a cool pattern for logging a lot of data quickly without using much storage, and that project will touch on many Rust concepts. Be careful though, this project can be a lot of work. But you'll get through it!

Skills you'll learn

• formatting in Rust
• binary formats (serialization and deserialization)
• procedural macros
• command line applications

Resources to get started

Here is a probable plan for how you could write that project:

• use a procedural macro to interpret calls to something like timeline!(logger, "values: [ {} | {} | {} | {} ]", val1, val2, val3, val4);
  • your macro should store the format string somewhere in its memory and define an index for it
  • the code will be replaced by logger.log(42, val1, val2, val3, val4); (with 42 the index you just defined)
• use another procedural macro to transform timeline_strings!() into an array of format strings
• write some helper functions to use the array of format strings to initialize the log file
• write a command line application to open that log file and display it

Mentor(s)

• @geal

[laurentgir]

Hi Geal,

As a Rust beginner, I'd like to try giving an implementation in Rust of Timeline logging from SnabbSwitch.

I'm trying to wrap my head around the procedural macro needed to parse the calls. The term is a bit confusing. Appart from the main documentation, I found this thread really interesting and I guess now that you thought about the "function-like" form of procedural macro (not the "custom derive" nor "attribute-like" form) . Is that what you had in mind ?

And also, what is the logger struct in timeline!(logger, "values: [ {} | {} | {} | {} ]", val1, val2, val3, val4); ? Is it an instance of the timeline struct such as tl in local proc = timeline.define(tl, 'example', 'trace', "processed $tcp, $udp, and $other")

Thanks for your help