splunk / splunk-connect-for-syslog

Splunk Connect for Syslog
Apache License 2.0
152 stars 107 forks source link

Trying to fix timezone using getting - unexpected LL_IDENTIFIER, expecting '}' #2261

Closed chipzzz closed 9 months ago

chipzzz commented 10 months ago

Was the issue replicated by support? No What is the sc4s version ? 3.4.0 Is there a pcap available? no Is the issue related to the environment of the customer or Software related issue? no Is it related to Data loss, please explain ? Protocol? Hardware specs? no Last chance index/Fallback index? present Is the issue related to local customization? yes Do we have all the default indexes created? yes Describe the bug Trying to fix timzene issues using https://splunk.github.io/splunk-connect-for-syslog/main/troubleshooting/troubleshoot_resources/#fix-timezone

My working config:

                    block parser app-parser-splunk() {
                      channel {
                        rewrite {
                          r_set_splunk_dest_default(
                            index("%(index)s")
                            sourcetype("%(default_sourcetype)s")
                            vendor("%(name)s")
                            product("%(name)s")
                            template("t_msg_only")
                          );
                        };
                        %(sourcetype_filters)s
                      };
                    };
                    application app-parser-splunk[sc4s-syslog] {
                        parser { app-parser-splunk(); };
                    };
                    application app-fallback-json[sc4s-syslog] {
                      filter {
                        "${PROGRAM}" eq ""
                        and
                        message("{" type(string) flags(prefix));
                      };
                      parser { app-fallback-json(); };
                    };

I'm adding:

fix-time-zone("EST5EDT");

and

                    application app-dest-rewrite-fix_tz_something-d_fmt_hec_default[sc4s-lp-dest-format-d_hec_fmt] {
                        filter {
                            match('cisco' value('fields.sc4s_vendor') type(string));
                        };    
                        parser { app-dest-rewrite-fix_tz_something-d_fmt_hec_default(); };   
                    };

FInal Config:

                    block parser app-parser-splunk() {
                      channel {
                        rewrite {

                         fix-time-zone("EST5EDT");

                          r_set_splunk_dest_default(
                            index("%(index)s")
                            sourcetype("%(default_sourcetype)s")
                            vendor("%(name)s")
                            product("%(name)s")
                            template("t_msg_only")
                          );
                        };

                        %(sourcetype_filters)s

                      };
                    };

                    application app-dest-rewrite-fix_tz_something-d_fmt_hec_default[sc4s-lp-dest-format-d_hec_fmt] {
                        filter {
                            match('cisco' value('fields.sc4s_vendor') type(string));
                        };    
                        parser { app-dest-rewrite-fix_tz_something-d_fmt_hec_default(); };   
                    };

                    application app-parser-splunk[sc4s-syslog] {
                        parser { app-parser-splunk(); };
                    };

                    application app-fallback-json[sc4s-syslog] {
                      filter {
                        "${PROGRAM}" eq ""
                        and
                        message("{" type(string) flags(prefix));
                      };
                      parser { app-fallback-json(); };
                    };

Getting Error:

Error parsing parser expression, syntax error, unexpected LL_IDENTIFIER, expecting '}' in parser generator app-parser:12:15-12:66:
7       #Start Application app-dest-rewrite-fix_tz_something-d_fmt_hec_default
8       channel {
9           filter { 
10              match('cisco' value('fields.sc4s_vendor') type(string));
11           };
12---->     parser {  app-dest-rewrite-fix_tz_something-d_fmt_hec_default();  };
12---->               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
13          rewrite {
14             set-tag('.app.app-dest-rewrite-fix_tz_something-d_fmt_hec_default');
15             set('app-dest-rewrite-fix_tz_something-d_fmt_hec_default' value('.app.name'));
16          };
17          flags(final);

Included from root generator lp_dest_alts_global:20:17-20:65:
15                  app-parser(topic(sc4s-lp-dest-select-drop-d_hec_fmt));
16              };
17          } else {
18              if {
19                  parser {
20---->                 app-parser(topic(sc4s-lp-dest-format-d_hec_fmt));
20---->                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
21                  };
22              } elif {
23                  parser {
24                      app-parser(topic(sc4s-lp-dest-format-hec_fmt));
25                  };

Included from /etc/syslog-ng/conf.d/log_paths/2/lp_dest_alts_global/plugin.conf:4:1-4:22:

Included from /etc/syslog-ng/syslog-ng.conf:64:1-64:1:

Error parsing parser expression, syntax error, unexpected LL_IDENTIFIER, expecting '}' in parser generator app-parser:12:15-12:66:
7       #Start Application app-dest-rewrite-fix_tz_something-d_fmt_hec_default
8       channel {
9           filter { 
10              match('cisco' value('fields.sc4s_vendor') type(string));
11           };
12---->     parser {  app-dest-rewrite-fix_tz_something-d_fmt_hec_default();  };
12---->               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
13          rewrite {
14             set-tag('.app.app-dest-rewrite-fix_tz_something-d_fmt_hec_default');
15             set('app-dest-rewrite-fix_tz_something-d_fmt_hec_default' value('.app.name'));
16          };
17          flags(final);

Included from root generator lp_dest_alts_global:20:17-20:65:
15                  app-parser(topic(sc4s-lp-dest-select-drop-d_hec_fmt));
16              };
17          } else {
18              if {
19                  parser {
20---->                 app-parser(topic(sc4s-lp-dest-format-d_hec_fmt));
20---->                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
21                  };
22              } elif {
23                  parser {
24                      app-parser(topic(sc4s-lp-dest-format-hec_fmt));
25                  };

Included from /etc/syslog-ng/conf.d/log_paths/2/lp_dest_alts_global/plugin.conf:4:1-4:22:

Included from /etc/syslog-ng/syslog-ng.conf:64:1-64:1:

syslog-ng checking config
sc4s version=3.4.0
starting goss
Error parsing parser expression, syntax error, unexpected LL_IDENTIFIER, expecting '}' in parser generator app-parser:12:15-12:66:
7       #Start Application app-dest-rewrite-fix_tz_something-d_fmt_hec_default
8       channel {
9           filter { 
10              match('cisco' value('fields.sc4s_vendor') type(string));
11           };
12---->     parser {  app-dest-rewrite-fix_tz_something-d_fmt_hec_default();  };
12---->               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
13          rewrite {
14             set-tag('.app.app-dest-rewrite-fix_tz_something-d_fmt_hec_default');
15             set('app-dest-rewrite-fix_tz_something-d_fmt_hec_default' value('.app.name'));
16          };
17          flags(final);

Included from root generator lp_dest_alts_global:20:17-20:65:
15                  app-parser(topic(sc4s-lp-dest-select-drop-d_hec_fmt));
16              };
17          } else {
18              if {
19                  parser {
20---->                 app-parser(topic(sc4s-lp-dest-format-d_hec_fmt));
20---->                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
21                  };
22              } elif {
23                  parser {
24                      app-parser(topic(sc4s-lp-dest-format-hec_fmt));
25                  };

Included from /etc/syslog-ng/conf.d/log_paths/2/lp_dest_alts_global/plugin.conf:4:1-4:22:

Included from /etc/syslog-ng/syslog-ng.conf:64:1-64:1:
chipzzz commented 9 months ago

The problem points to this section https://github.com/splunk/splunk-connect-for-syslog/blob/ffe639d033765a51f7b9842301c960250b71267f/package/etc/conf.d/log_paths/2/lp_dest_alts_global/plugin.jinja#L23-L65

ikheifets-splunk commented 9 months ago

@chipzzz can you please attach zip-archive with all your custom (local) configs? We need it to reproduce your case. You can send me on email: ikheifets@splunk.com

chipzzz commented 9 months ago

We can close this I was able to set this up with a separate block parser

ikheifets-splunk commented 9 months ago

Okay @chipzzz , if it's resolved then I closing that