Add support for Go's multi-line error logs

[otisg]

e.g.

panic: runtime error: index out of range

goroutine 1 [running]:
github.com/sematext/foo/beef.(*upbuilder).InitUprobes(0xc4202d3680)
    /home/johndoe/Sematext/dev-st/foo/src/github.com/sematext/foo/beef/uprobe_builder.go:94 +0x18
main.main.func1(0xc42045e000, 0x19b6180, 0x0, 0x0)
    /home/johndoe/Sematext/dev-st/foo/src/github.com/sematext/foo/cmd/foo/main.go:196 +0xc77
github.com/sematext/foo/vendor/github.com/spf13/cobra.(*Command).execute(0xc42045e000, 0xc4200a8170, 0x0, 0x0, 0xc42045e000, 0xc4200a8170)
    /home/johndoe/Sematext/dev-st/foo/src/github.com/sematext/foo/vendor/github.com/spf13/cobra/command.go:766 +0x2c1
github.com/sematext/foo/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc42045e000, 0x1, 0x100027b, 0x1)
    /home/johndoe/Sematext/dev-st/foo/src/github.com/sematext/foo/vendor/github.com/spf13/cobra/command.go:852 +0x30a
github.com/sematext/foo/vendor/github.com/spf13/cobra.(*Command).Execute(0xc42045e000, 0xc42045a000, 0xc42045e000)
    /home/johndoe/Sematext/dev-st/foo/src/github.com/sematext/foo/vendor/github.com/spf13/cobra/command.go:800 +0x2b
main.main()
    /home/johndoe/Sematext/dev-st/foo/src/github.com/sematext/foo/cmd/foo/main.go:307 +0x51c

[megastef]

I'm afraid there is no general to attack this kind of logs. GO can use various logging frameworks and the parser needs to know how regular log lines start.

For the output of Sematext agent (some are written in GO and using Logrus) we use this patterns (and especially blockStart property for multi-line detection):

- # sematext/agent logs 
  sourceName: !!js/regexp /sematext\/agent/
  blockStart: !!js/regexp /^panic\:|(^INFO|^ERRO|^WARN|^FAT|^TRAC|^DEB)|^time=|^\S{2,}/
  match:
    - type: sematext_agent_golang
      regex: !!js/regexp /time=(\S+)\slevel=(\S+?)\smsg="(.+?)"\ssource="(.+?)"/i
      fields: [ts, severity, message, source]
      dateFormat: YY-MM-DDTHH:mm:ss
    - type: sematext_agent_golang
      regex: !!js/regexp /([A-Z]+)\[(.+?)\]\s(.*)/i
      fields: [severity, ts, message]
      dateFormat: YY-MM-DDTHH:mm:ss

And work well for logs, which have Logrus plain format or Key-Value format.

INFO[2019-01-16T18:47:08Z] loading configuration from /opt/spm/properties/st-agent.yml 
ERRO[2019-01-16T18:47:08Z] unable to load configuration file: open /opt/spm/properties/st-agent.yml: no such file or directory 
ERRO[2019-01-16T18:47:08Z] open /opt/spm/properties/agent.properties: no such file or directory 
ERRO[2019-01-16T18:47:08Z] unable to set watch on file /opt/spm/properties 
time="2019-01-16T18:47:08Z" level=info msg="starting st-agent with pid [1]" source="agent/main.go:142"

panic: runtime error: index out of range

goroutine 1 [running]:
github.com/sematext/foo/beef.(*upbuilder).InitUprobes(0xc4202d3680)
    /home/johndoe/Sematext/dev-st/foo/src/github.com/sematext/foo/beef/uprobe_builder.go:94 +0x18
main.main.func1(0xc42045e000, 0x19b6180, 0x0, 0x0)
    /home/johndoe/Sematext/dev-st/foo/src/github.com/sematext/foo/cmd/foo/main.go:196 +0xc77
github.com/sematext/foo/vendor/github.com/spf13/cobra.(*Command).execute(0xc42045e000, 0xc4200a8170, 0x0, 0x0, 0xc42045e000, 0xc4200a8170)
    /home/johndoe/Sematext/dev-st/foo/src/github.com/sematext/foo/vendor/github.com/spf13/cobra/command.go:766 +0x2c1
github.com/sematext/foo/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc42045e000, 0x1, 0x100027b, 0x1)
    /home/johndoe/Sematext/dev-st/foo/src/github.com/sematext/foo/vendor/github.com/spf13/cobra/command.go:852 +0x30a
github.com/sematext/foo/vendor/github.com/spf13/cobra.(*Command).Execute(0xc42045e000, 0xc42045a000, 0xc42045e000)
    /home/johndoe/Sematext/dev-st/foo/src/github.com/sematext/foo/vendor/github.com/spf13/cobra/command.go:800 +0x2b
main.main()
    /home/johndoe/Sematext/dev-st/foo/src/github.com/sematext/foo/cmd/foo/main.go:307 +0x51c