mozilla-services / heka

DEPRECATED: Data collection and processing made easy.
http://hekad.readthedocs.org/
Other
3.4k stars 531 forks source link

How to define a variable to distinguish the source logfile #1975

Open ruiaylin opened 7 years ago

ruiaylin commented 7 years ago

like this :

I have two nginx instance running on port 80 , 81 .

i do not know how to config this to support for multi instance .

michaelgibson commented 7 years ago

I'm not 100% sure what you are asking, may need some clarification.

I'm assuming you are asking how to collect from multiple log files and how to differentiate between them once in Heka? Assuming you have some way of uniquely identifying each of your log streams, the logstreamer input has a config field "differentiator" When defined it will set the "Logger" header in the message.

[accesslogs]
type = "LogstreamerInput"
log_directory = "/var/log/nginx"
file_match = '(?P<Port>[^/]+)-access\.log'
differentiator = ["nginx.", "Port", ".access"]

[LogOutput]
message_matcher = "Logger == '<Port Value>'"
encoder = "RstEncoder"

[RstEncoder]

This allows you to distinguish between different incoming log streams.