search

ChrisYounger / git_for_splunk

Simple versioning for your Splunk environment
https://splunkbase.splunk.com/app/4182
Apache License 2.0
37 stars 10 forks source link

Request to add git show to the indexed logs #7

Closed dougkamm closed 3 years ago

dougkamm commented 4 years ago

Could you please add the print of: git --git-dir=/opt/splunk/etc/.git show

to the index logs Then would have what was changed within the changed_files.

ChrisYounger commented 4 years ago

Hi Dougkamm,

Thanks for this suggestion. I didn't originally add this feature becuase I was expecting that most people would be pushing to a remote repository (e.g. bitbucket, gitlab, github, gogs etc) and that this system will provide a much nicer git diffing experience.

If you really want this feature, it would be best to add it yourself. In this file: https://github.com/ChrisYounger/git_for_splunk/blob/master/bin/input_module_gitforsplunk.py add the following at line 99:

ret_code, ret_output = gitcmd(["git", "show"], my_env, event_data)

I think that should do it (you dont need the --git-dir becuase that is already set as an environment variable)

dougkamm commented 4 years ago

Awesome thanks. What I’m struggling with is I’d like to have your app on all our machines and the search heads & indexers are in a DMZ with port 22 blocked so I’m not sure how to push to a remote repo on those so I’m going to try an all in Splunk solution 1st. Thanks soo much for the code. I’ll give that a try as soon as I can.

From: Chris Younger notifications@github.com Sent: Thursday, October 29, 2020 6:23 AM To: ChrisYounger/git_for_splunk git_for_splunk@noreply.github.com Cc: Kamm, Doug R Doug.Kamm@charter.com; Author author@noreply.github.com Subject: [EXTERNAL] Re: [ChrisYounger/git_for_splunk] Request to add git show to the indexed logs (#7)

CAUTION: The e-mail below is from an external source. Please exercise caution before opening attachments, clicking links, or following guidance.

Hi Dougkamm,

Thanks for this suggestion. I didn't originally add this feature becuase I was expecting that most people would be pushing to a remote repository (e.g. bitbucket, gitlab, github, gogs etc) and that this system will provide a much nicer git diffing experience.

If you really want this feature, it would be best to add it yourself. In this file: https://github.com/ChrisYounger/git_for_splunk/blob/master/bin/input_module_gitforsplunk.py add the following at line 99:

ret_code, ret_output = gitcmd(["git", "show"], my_env, event_data)

I think that should do it

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/ChrisYounger/git_for_splunk/issues/7#issuecomment-718685719, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AMOXBWTCWGTG5SVE6T6AWRDSNFGCVANCNFSM4TBAYNTQ. E-MAIL CONFIDENTIALITY NOTICE: The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.

dougkamm commented 3 years ago

Hi,

After using this for a month we really like the results; however, are seeing truncating. I’m hesitant to increase the truncation in fear of using too much of our license. Would it be possible to filter the below code to only .conf or .meta files and exclude csv etc? or at least filter to just apps we manage and ignore system apps?

Thanks,

From: Kamm, Doug R Sent: Thursday, October 29, 2020 8:13 AM To: 'ChrisYounger/git_for_splunk' reply@reply.github.com Subject: RE: [EXTERNAL] Re: [ChrisYounger/git_for_splunk] Request to add git show to the indexed logs (#7)

Awesome thanks. What I’m struggling with is I’d like to have your app on all our machines and the search heads & indexers are in a DMZ with port 22 blocked so I’m not sure how to push to a remote repo on those so I’m going to try an all in Splunk solution 1st. Thanks soo much for the code. I’ll give that a try as soon as I can.

From: Chris Younger notifications@github.com<mailto:notifications@github.com> Sent: Thursday, October 29, 2020 6:23 AM To: ChrisYounger/git_for_splunk git_for_splunk@noreply.github.com<mailto:git_for_splunk@noreply.github.com> Cc: Kamm, Doug R Doug.Kamm@charter.com<mailto:Doug.Kamm@charter.com>; Author author@noreply.github.com<mailto:author@noreply.github.com> Subject: [EXTERNAL] Re: [ChrisYounger/git_for_splunk] Request to add git show to the indexed logs (#7)

CAUTION: The e-mail below is from an external source. Please exercise caution before opening attachments, clicking links, or following guidance.

Hi Dougkamm,

Thanks for this suggestion. I didn't originally add this feature becuase I was expecting that most people would be pushing to a remote repository (e.g. bitbucket, gitlab, github, gogs etc) and that this system will provide a much nicer git diffing experience.

If you really want this feature, it would be best to add it yourself. In this file: https://github.com/ChrisYounger/git_for_splunk/blob/master/bin/input_module_gitforsplunk.py add the following at line 99:

ret_code, ret_output = gitcmd(["git", "show"], my_env, event_data)

I think that should do it

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/ChrisYounger/git_for_splunk/issues/7#issuecomment-718685719, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AMOXBWTCWGTG5SVE6T6AWRDSNFGCVANCNFSM4TBAYNTQ. E-MAIL CONFIDENTIALITY NOTICE: The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.

ChrisYounger commented 3 years ago

Hi mate. three options:

  1. use .gitignore to only track changes to apps and files that you care about. this is standard git feature and not park of my app.
  2. increase the TRUNCATE in props.conf (but obviously as you mentioned you might end up with some very large commits that use license. however these shouldn't be happening regularly).
  3. (dubious) if you log to _internal, it wont hit license, but it will use storage.

Hope this helps

dougkamm commented 3 years ago

I’ve updated the following to our gitignore:

/lookups/ !apps/(apps we use)/lookups/.csv etc/auth/ etc/passwd .pyc .log users//history/* login-info.cfg local.meta ui-prefs.conf telemetry.conf

If this is still too much data, I’ll try increasing our truncate and decrease the index internal to only a few times a day instead of every hour.

Thanks,

From: Chris Younger notifications@github.com Sent: Tuesday, November 24, 2020 5:14 PM To: ChrisYounger/git_for_splunk git_for_splunk@noreply.github.com Cc: Kamm, Doug R Doug.Kamm@charter.com; Author author@noreply.github.com Subject: [EXTERNAL] Re: [ChrisYounger/git_for_splunk] Request to add git show to the indexed logs (#7)

CAUTION: The e-mail below is from an external source. Please exercise caution before opening attachments, clicking links, or following guidance.

Hi mate. three options:

  1. use .gitignore to only track changes to apps and files that you care about. this is standard git feature and not park of my app.
  2. increase the TRUNCATE in props.conf (but obviously as you mentioned you might end up with some very large commits that use license. however these shouldn't be happening regularly).
  3. (dubious) if you log to _internal, it wont hit license, but it will use storage.

Hope this helps

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/ChrisYounger/git_for_splunk/issues/7#issuecomment-733286895, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AMOXBWTRZAIHFGMR4QTNJ5TSRQ45HANCNFSM4TBAYNTQ. E-MAIL CONFIDENTIALITY NOTICE: The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.