search

da667 / Autosnort

Repo for autosnort scripts.
MIT License
157 stars 89 forks source link

Autosnort-Ubuntu/AVATAR project - pulledpork #50

Closed 0x7fff9 closed 7 years ago

0x7fff9 commented 7 years ago

Hi all, running into some issues here. Everything is working good system wise, I can get events populated on splunk and it seems to be working good. I am experiencing an issue when I try to update the pulled pork via /usr/src/pulledpork/pulledpork.pl

I get the following:

root@ips001:/opt/splunkforwarder/etc/apps/TA-unified2/default# /usr/src/pulledpork/pulledpork.pl -c /usr/src/pulledpork/etc/pulledpork.conf

    https://github.com/shirkdog/pulledpork
      _____ ____
     `----,\    )
      `--==\\  /    PulledPork v0.7.3 - Making signature updates great again!
       `--==\\/
     .-~~~~-.Y|\\_  Copyright (C) 2009-2016 JJ Cummings
  @_/        /  66\_  cummingsj@gmail.com
    |    \   \   _(")
     \   /-| ||'--'  Rules give me wings!
      \_\  \_\\
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Checking latest MD5 for snortrules-snapshot-2990.tar.gz....
    Error 400 when fetching https://www.snort.org/reg-rules/snortrules-snapshot-2990.tar.gz.md5 at /usr/src/pulledpork/pulledpork.pl line 534.
    main::md5file("MY_OINK_CODE_WAS_HERE", "snortrules-snapshot-2990.tar.gz", "/tmp/", "https://www.snort.org/reg-rules/") called at /usr/src/pulledpork/pulledpork.pl line 2007
root@ips001:/opt/splunkforwarder/etc/apps/TA-unified2/default#

I've tried to fetch the file manually and I can only get 404s. The oink code is correct.

Also, because of this, I think, but please correct me if I'm wrong, I see splunk like the attached screenshot instead of being able to see "signature.msg"

Events:

image

By this one I can see snort is forwarding properly because those are events of test attacks.

if I filter properly the output goes empty:

image

No "signature.msg" table.

If anyone could help would be GREAT!!

instance details: Ubuntu 16.04.1 LTS VM on vBox. 4GB RAM 2 CPU AVATAR PDF followed religiously :)

Thanks! cheers.

da667 commented 7 years ago

Try running pulledpork again, this time with the "-vv" flag. This runs it in extra verbose mode. I think i know tge problem, but run it again in verbose mode so i can be sure.

On Jan 16, 2017 6:20 PM, "Guido Galego" notifications@github.com wrote:

Hi all, running into some issues here. Everything is working good system wise, I can get events populated on splunk and it seems to be working good. I am experiencing and issue when I try to update to pulled pork via /usr/src/pulledpork/pulledpork.pl

I get the following:

root@ips001:/opt/splunkforwarder/etc/apps/TA-unified2/default# /usr/src/pulledpork/pulledpork.pl -c /usr/src/pulledpork/etc/pulledpork.conf

https://github.com/shirkdog/pulledpork

`----,\    )
 `--==\\  /    PulledPork v0.7.3 - Making signature updates great again!
  `--==\\/
.-~~~~-.Y|\\_  Copyright (C) 2009-2016 JJ Cummings

@/ / 66_ cummingsj@gmail.com | \ \ (") \ /-| ||'--' Rules give me wings! _\ _\



Checking latest MD5 for snortrules-snapshot-2990.tar.gz....
   Error 400 when fetching https://www.snort.org/reg-rules/snortrules-snapshot-2990.tar.gz.md5 at /usr/src/pulledpork/pulledpork.pl line 534.
   main::md5file("MY_OIK_CODE_WAS_HERE", "snortrules-snapshot-2990.tar.gz", "/tmp/", "https://www.snort.org/reg-rules/") called at /usr/src/pulledpork/pulledpork.pl line 2007
root@ips001:/opt/splunkforwarder/etc/apps/TA-unified2/default#

I've tried to fetch the file manually and I can only get 404s.
The oink code is correct.

Also, because of this, I think, but please correct em if I'm wrong, I see
splunk like the attached screenshot instead of being able to see
"signature.msg"

Events:
[image: image]
<https://cloud.githubusercontent.com/assets/24276463/22002149/d4b6c96a-dc49-11e6-9993-55875ff915bf.png>
By this one I can see snort is forwarding properly because those are
events of test attacks.

if I filter properly the output goes empty:
[image: image]
<https://cloud.githubusercontent.com/assets/24276463/22002183/0b7304b4-dc4a-11e6-89d4-0f8e96b1b7ac.png>
No "signature.msg" table.

If anyone could help would be GREAT!!

instance details:
Ubuntu 16.04.1 LTS VM on vBox.
4GB RAM 2 CPU
AVATAR PDF followed religiously :)

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<https://github.com/da667/Autosnort/issues/50>, or mute the thread
<https://github.com/notifications/unsubscribe-auth/ACFvfwlf-MDhbd4tkJJ8vyU0yo4pm8zHks5rS_tOgaJpZM4LlEmB>
.
0x7fff9 commented 7 years ago

Hi!!

root@ips001:~# /usr/src/pulledpork/pulledpork.pl -vv -c /usr/src/pulledpork/etc/pulledpork.conf

    https://github.com/shirkdog/pulledpork
      _____ ____
     `----,\    )
      `--==\\  /    PulledPork v0.7.3 - Making signature updates great again!
       `--==\\/
     .-~~~~-.Y|\\_  Copyright (C) 2009-2016 JJ Cummings
  @_/        /  66\_  cummingsj@gmail.com
    |    \   \   _(")
     \   /-| ||'--'  Rules give me wings!
      \_\  \_\\
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Config File Variable Debug /usr/src/pulledpork/etc/pulledpork.conf
    sorule_path = /opt/snort/snort_dynamicrules/
    temp_path = /tmp
    sid_msg_version = 1
    sid_msg = /opt/snort/etc/sid-msg.map
    rule_path = /opt/snort/rules/snort.rules
    distro = Ubuntu-12-04
    rule_url = ARRAY(0x2db60b0)
    local_rules = /opt/snort/rules/local.rules
    sid_changelog = /var/log/sid_changes.log
    IPRVersion = /opt/snort/rules/iplists
    snort_path = /opt/snort/bin/snort
    config_path = /opt/snort/etc/snort.conf
    ips_policy = security
    black_list = /opt/snort/rules/black_list.rules
    version = 0.7.3
    ignore = deleted.rules,experimental.rules,local.rules
    snort_version = 2.9.9.0
MISC (CLI and Autovar) Variable Debug:
    arch Def is: x86-64
    Operating System is: linux
    CA Certificate File is: OS Default
    Config Path is: /usr/src/pulledpork/etc/pulledpork.conf
    Distro Def is: Ubuntu-12-04
    security policy specified
    local.rules path is: /opt/snort/rules/local.rules
    Rules file is: /opt/snort/rules/snort.rules
    sid changes will be logged to: /var/log/sid_changes.log
    sid-msg.map Output Path is: /opt/snort/etc/sid-msg.map
    Snort Version is: 2.9.9.0
    Snort Config File: /opt/snort/etc/snort.conf
    Snort Path is: /opt/snort/bin/snort
    SO Output Path is: /opt/snort/snort_dynamicrules/
    Will process SO rules
    Extra Verbose Flag is Set
    Verbose Flag is Set
    File(s) to ignore = deleted.rules,experimental.rules,local.rules
    Base URL is: https://www.snort.org/reg-rules/|snortrules-snapshot.tar.gz|MY-OINK_CODE_WAS_HERE https://www.snort.org/reg-rules/|opensource.gz|MY-OINK_CODE_WAS_HERE https://snort.org/downloads/community/|community-rules.tar.gz|Community http://talosintel.com/feeds/ip-filter.blf|IPBLACKLIST|open

MY HTTPS PROXY = http://10.100.100.254:3128

MY HTTP PROXY = http://10.100.100.254:3128
Checking latest MD5 for snortrules-snapshot-2990.tar.gz....
    Fetching md5sum for: snortrules-snapshot-2990.tar.gz.md5
** CONNECT https://www.snort.org/reg-rules/snortrules-snapshot-2990.tar.gz.md5/MY-OINK_CODE_WAS_HERE ==> 400 Bad Request
    Error 400 when fetching https://www.snort.org/reg-rules/snortrules-snapshot-2990.tar.gz.md5 at /usr/src/pulledpork/pulledpork.pl line 534.
    main::md5file("MY-OINK_CODE_WAS_HERE", "snortrules-snapshot-2990.tar.gz", "/tmp/", "https://www.snort.org/reg-rules/") called at /usr/src/pulledpork/pulledpork.pl line 2007

is ==> 400 Bad Request the issue?

cheers

da667 commented 7 years ago

Okay, so the CONNECT method is tge problem. In the latest copy of tge avatar PDF, (1-11-17) look at page 402 and 403. Specifically, you need to set the environment variables http_proxy, HTTP_PROXY, https_proxy, and HTTPS_PROXY. Pulledpork thinks that 172.16.1.1:3128 is an https proxy, and it is not, so we need to explicitly set proxy variables before running the rule update script.

On Jan 17, 2017 1:34 PM, "Guido Galego" notifications@github.com wrote:

Hi!!

root@ips001:~# /usr/src/pulledpork/pulledpork.pl -vv -c /usr/src/pulledpork/etc/pulledpork.conf

https://github.com/shirkdog/pulledpork
  _____ ____
 `----,\    )
  `--==\\  /    PulledPork v0.7.3 - Making signature updates great again!
   `--==\\/
 .-~~~~-.Y|\\_  Copyright (C) 2009-2016 JJ Cummings

@/ / 66_ cummingsj@gmail.com | \ \ (") \ /-| ||'--' Rules give me wings! _\ _\



Config File Variable Debug /usr/src/pulledpork/etc/pulledpork.conf
  sorule_path = /opt/snort/snort_dynamicrules/
  temp_path = /tmp
  sid_msg_version = 1
  sid_msg = /opt/snort/etc/sid-msg.map
  rule_path = /opt/snort/rules/snort.rules
  distro = Ubuntu-12-04
  rule_url = ARRAY(0x2db60b0)
  local_rules = /opt/snort/rules/local.rules
  sid_changelog = /var/log/sid_changes.log
  IPRVersion = /opt/snort/rules/iplists
  snort_path = /opt/snort/bin/snort
  config_path = /opt/snort/etc/snort.conf
  ips_policy = security
  black_list = /opt/snort/rules/black_list.rules
  version = 0.7.3
  ignore = deleted.rules,experimental.rules,local.rules
  snort_version = 2.9.9.0
MISC (CLI and Autovar) Variable Debug:
  arch Def is: x86-64
  Operating System is: linux
  CA Certificate File is: OS Default
  Config Path is: /usr/src/pulledpork/etc/pulledpork.conf
  Distro Def is: Ubuntu-12-04
  security policy specified
  local.rules path is: /opt/snort/rules/local.rules
  Rules file is: /opt/snort/rules/snort.rules
  sid changes will be logged to: /var/log/sid_changes.log
  sid-msg.map Output Path is: /opt/snort/etc/sid-msg.map
  Snort Version is: 2.9.9.0
  Snort Config File: /opt/snort/etc/snort.conf
  Snort Path is: /opt/snort/bin/snort
  SO Output Path is: /opt/snort/snort_dynamicrules/
  Will process SO rules
  Extra Verbose Flag is Set
  Verbose Flag is Set
  File(s) to ignore = deleted.rules,experimental.rules,local.rules
  Base URL is: https://www.snort.org/reg-rules/|snortrules-snapshot.tar.gz|MY-OINK_CODE_WAS_HERE https://www.snort.org/reg-rules/|opensource.gz|MY-OINK_CODE_WAS_HERE https://snort.org/downloads/community/|community-rules.tar.gz|Community http://talosintel.com/feeds/ip-filter.blf|IPBLACKLIST|open

MY HTTPS PROXY = http://10.100.100.254:3128

MY HTTP PROXY = http://10.100.100.254:3128
Checking latest MD5 for snortrules-snapshot-2990.tar.gz....
  Fetching md5sum for: snortrules-snapshot-2990.tar.gz.md5
** CONNECT https://www.snort.org/reg-rules/snortrules-snapshot-2990.tar.gz.md5/MY-OINK_CODE_WAS_HERE ==> 400 Bad Request
  Error 400 when fetching https://www.snort.org/reg-rules/snortrules-snapshot-2990.tar.gz.md5 at /usr/src/pulledpork/pulledpork.pl line 534.
  main::md5file("MY-OINK_CODE_WAS_HERE", "snortrules-snapshot-2990.tar.gz", "/tmp/", "https://www.snort.org/reg-rules/") called at /usr/src/pulledpork/pulledpork.pl line 2007```

is `==> 400 Bad Request` the issue?

cheers

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<https://github.com/da667/Autosnort/issues/50#issuecomment-273257251>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ACFvf2PzPf8sQIKY55jXa2_HSOgBMmLGks5rTQm_gaJpZM4LlEmB>
.
0x7fff9 commented 7 years ago

Fly Piggy Fly!

THANKS :D

Still I can't output stuff on splunk properly.

image

Any idea sir?

Edit: not really understanding it, all the data is here just the table signature.msg seems not to be present.

image

cheers.

0x7fff9 commented 7 years ago

Hello!! nothing like understand what is it that one is trying to do!! so yeah, it took me a while but I finally figured that I had to extract the field "msg" from the logs that got into splunk and after that, yeah! I can table them!!

Thank you because I now understand a little more about this setup!! cheers.