datalust / seq-tickets

Issues, design discussions and feature roadmap for the Seq log server
https://datalust.co/seq
96 stars 5 forks source link

Optimize @MessageTemplate search with @EventType index #2181

Closed Suchiman closed 4 months ago

Suchiman commented 4 months ago

If someone searches for

@MessageTemplate = 'Hello {Name}'

that could be trivially optimized to

@EventType = ToEventType('Hello {Name}')

which then performs a lot faster due to the new indexing capabilities.

nblumhardt commented 4 months ago

Thanks for the suggestion! Unfortunately, @EventType isn't strictly determined by @MessageTemplate - data sources can specify event types explicitly (though most rely on the default hash of the message template).

We're just about to ship 2024.3, which will allow an index to be created on @MessageTemplate if you wish. The UI only allows regular properties to be indexed, but running:

seqcli expressionindex create -e "@MessageTemplate"

will have the effect you want (with a similar storage cost to the @EventType index). Let me know if it sounds like this will help.

Best regards, Nick

Suchiman commented 4 months ago

data sources can specify event types explicitly

TIL

with a similar storage cost to the @EventType index

I guess in that case i'll live with having to remember to use the second trick :) My main usecase here has been figuring out if a log message i'm seeing in code has ever happened, so it's not a frequent thing to do but it hugely benefits from the index, being able to confirm in a few seconds that this message never happened in the past year (or only did a handful of times).

Thanks for the quick response as always Nick!