splunk / splunk-connect-for-syslog

Splunk Connect for Syslog
Apache License 2.0
148 stars 108 forks source link

Support Broadcom SMIME syslog. #2436

Closed evslacker closed 1 month ago

evslacker commented 2 months ago

What is the sc4s version ? 2.49.8 Is there a pcap available? Yes, i can email it to the personal who will be working on this request. What the vendor name? Broadcom What's the product name? SMIME Feature Request description: SMIME is not a part of current product supported by sc4s, we have got a request to ingest SMIME syslog to splunk cloud. Should it support TCP or UDP? It support both TCP and UDP over port 514 Do you want to have it for local usage or prepare a github PR? Whichever suits best

ikheifets-splunk commented 2 months ago

Please send me pcapf file on email ikheifets@splunk.com Also please update your instance from 2.x to 3.x, because after this PR you should be ready to update :)

evslacker commented 1 month ago

A team mate of mine must have shared the pcap with you, let me know if that is sufficient to start things

ikheifets-splunk commented 1 month ago

Thanks @evslacker , I've got your pcap we will start work with that

evslacker commented 1 month ago

Hi @ikheifets-splunk Just a follow up on this, is there any update on this request.

ikheifets-splunk commented 1 month ago

Hello, @evslacker ! It's seems logs that you provided is pgp server logs. I worry that pgp servers logs might be wrongly identified as Broadcom SMIME by this reason I proposing you use user-defined parser, that you will use and we wouldn't release it.

What you need to do:

  1. Go to this directory /opt/sc4s/local/config/app-parsers
  2. In this directory create file app-syslog-pgp.conf with such content:
    
    block parser app-syslog-broadcom-smime() {
    channel {
        rewrite {
            r_set_splunk_dest_default(
                index('main')
                sourcetype('broadcom:smime')
                vendor("broadcom")
                product("smime")
            );
        };
    };
    };

application app-syslog-broadcom-smime[sc4s-syslog-pgm] { filter { program('pgp/' type(string) flags(prefix)); };
parser { app-syslog-broadcom-smime(); }; };


3. Restart SC4S
4. Check that your user-defined parser mounted  inside container, check this directory inside your container
 `/etc/syslog-ng/conf.d/local/config/app_parsers`

 P.S @evslacker please let me know is it working for you
evslacker commented 1 month ago

Hey @ikheifets-splunk apologies if this created a confusion, as i passed on the information which I received from the application Team.

i have created the parser and restarted the sc4s as well.

But I've couple of queries.

1- i was not able to find /etc/syslog-ng/conf.d/local/config/app_parser

no directory for syslog-ng found.

2- post applying the filters, logs have started coming to index=main, but seems like we are getting SMTP connection logs in the logs but not the Remote TLS Certificate data AND LDAP syslogs as seen in the pcap.

3- Do Sc4s auto ingests the logs as per the verbosity or it takes any default verbosity.

4- To parse data do we always have to do it manually from UI, or we can do it via parser?

5- How to check if we are dropping any syslog or not.

ikheifets-splunk commented 1 month ago

Hello, @evslacker ! You asked lots of question, I think it would be easy to answer it in-person. Let's schedule the call, please send me invite on ikheifets@splunk.com 27 May, I will be available on 14:00-20:00 CET

evslacker commented 1 month ago

@ikheifets-splunk thank you for the slot.

was able to get the answers later, as i updated the git comment just after config.(less patience. :p

thank you for the help

ikheifets-splunk commented 1 month ago

@evslacker, see you closed this issue, but I don't understand why. Hope my solution https://github.com/splunk/splunk-connect-for-syslog/issues/2436#issuecomment-2121100211 helped you. In general it should works correctly

evslacker commented 1 month ago

Hey @ikheifets-splunk

the parser you provided worked correctly with no issues, so i thought closing the case, in case of any issues will open a case or issue.

thank you