amiracle / homemonitor

Splunk app for home | monitor >
25 stars 5 forks source link

Sorry for the noob question #16

Open dwertheimer opened 5 years ago

dwertheimer commented 5 years ago

I am sure I am missing something here. I am trying to set up Splunk and HomeMonitor for the first time. Got it all installed but don't see any traffic. I see bandwidth monitor but that's all the data I am getting. I installed Splunk on OSX. It is running and there is no firewall. I put the Mac's local address (192.168.1.41) in the N66U's Remote Log Server. I have rebooted the router. Still see no data coming in. I checked that there is a data input for UDP on 514, and there is. I set the source type to Manual > asus with no source name override. What might I be missing?

amiracle commented 5 years ago

No worries on the question, let's see if we can get it resolved. Whenever you run Splunk on Linux (Mac OSX etc.) AND you need to listen to a port below 1028, then you have to run Splunk as root. Once you set Splunk to run as root, then it should work. For your Mac, you'll type the following: mac$ cd $SPLUNK_HOME mac$ sudo bin/splunk enable boot-start mac$ sudo bin/splunk start

You're going to change directory to your Splunk home directory. Then, execute the sudo bin/splunk enable boot-start which will automatically start Splunk on boot as root. Finally, you're going to start splunk, again with elevated privileges so that it can listen to port 514 (syslog).

yabasha3 commented 5 years ago

Hello,

I am using Home monitor 4.5.1 with Mac OSX 10.13.1. I followed every single step and reinstalled Splunk few times but I cannot get any data to come through splunk from my router. I have placed my Splunk instance as the syslog server on my Asus router. RT-AC68P. I am on the root account and so the fire wall is open. I also checked to see if my laptop has the port open and it shows so.

Not sure what could have been wrong. By default it should be routed through port 514.

yabasha3 commented 5 years ago

image image

yabasha3 commented 5 years ago

input files for default and local.... Please help.

yabasha3 commented 5 years ago

I have tried the port 3659 as the UDP port on the Confs but that did not help or mad a difference.

dwertheimer commented 5 years ago

Yabasha3, I don't think this port forwarding setting is doing what you think it's doing. Port forwarding typically routes incoming packets hitting your router from the outside world (WAN) on a certain port and redirects them to another port on the inside network (LAN). So this instruction above will send packets sent from outside your network port 3659 to your computer.

BTW, I was never able to get my OSX (10.14) computer to see the packets coming from my router using the standard syslog port. I had to change the port my router used for sending them, and it worked fine. The only problem is there's no place in the GUI to do it. So I had to telnet into my router.

Something like this: telnet 192.168.1.1

nvram set log_port= nvram commit reboot

Got the idea here: https://sourcebox.be/blog/2017/05/31/send-asus-rt-ac68u-logs-to-papertrail/

On Wed, Jan 9, 2019 at 2:06 PM yabasha3 notifications@github.com wrote:

My router is saying that It is sending syslog to to my laptop IP via port 3659.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

yabasha3 commented 5 years ago

Yabasha3, I don't think this port forwarding setting is doing what you think it's doing. Port forwarding typically routes incoming packets hitting your router from the outside world (WAN) on a certain port and redirects them to another port on the inside network (LAN). So this instruction above will send packets sent from outside your network port 3659 to your computer. BTW, I was never able to get my OSX (10.14) computer to see the packets coming from my router using the standard syslog port. I had to change the port my router used for sending them, and it worked fine. The only problem is there's no place in the GUI to do it. So I had to telnet into my router. Something like this: telnet 192.168.1.1 nvram set log_port= nvram commit reboot Got the idea here: https://sourcebox.be/blog/2017/05/31/send-asus-rt-ac68u-logs-to-papertrail/ On Wed, Jan 9, 2019 at 2:06 PM yabasha3 @.***> wrote: My router is saying that It is sending syslog to to my laptop IP via port 3659. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

Hi dwertheimer,

Thank you for your quick response. So that I have gotten this starlight

  1. remove my laptop if from the router's Log server box.
  2. Telnet into my router to change the syslog port to which port number?
  3. So after changing the port, how will my laptop communicate with my router to obtain these syslog?

Thanks, yabasha3

yabasha3 commented 5 years ago

Lets say I chose port 3869 so it will be something like this?

nvram set log_port=3869 nvram commit reboot

dwertheimer commented 5 years ago

yes, that's correct. if you set your log server to the IP of your laptop, then the router will automatically send the logs to your laptop on port

  1. but splunk needs to be configured to open that port and be listening for it.

On Thu, Jan 10, 2019 at 7:52 AM yabasha3 notifications@github.com wrote:

Lets say I chose port 3869 so it will be something like this?

nvram set log_port=3869 nvram commit reboot

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/amiracle/homemonitor/issues/16#issuecomment-453145418, or mute the thread https://github.com/notifications/unsubscribe-auth/AIiPVGypHgKU87CWDa8iw8FAFc_ODNM4ks5vB2G4gaJpZM4ZevBG .

yabasha3 commented 5 years ago

And how do you configure splunk to listen to this port ? I️ve done everything but to allow splunk to listen to this new defined syslog port

gawainXX commented 4 years ago

Greetings, I've been running into this issue.. I've tried pretty much everything listed above and then some to forward syslog traffic to my CentOS Splunk VM on 514 to no avail. I however have absolutely no issue using another port however Home Monitor seems to be hard configured to use port 514 in it's dashboards.. Is there a location that I can change this from 514 to another port such as 7001 and have HomeMonitor populate correctly? I'd rather not spin up a Windows VM if I can help it.

yabasha3 commented 4 years ago

Hey,

You need to login into the router/switch or whatever you getting your syslogs from and change the port there. Once done reconfigure the ido on splunk it receive the logsys via that new configured port.

Thanks, Mohammed E.

On Nov 27, 2019, at 10:55 PM, gawainXX notifications@github.com wrote:

 Greetings, I've been running into this issue.. I've tried pretty much everything listed above and then some to forward syslog traffic to my CentOS Splunk VM to no avail. I however have absolutely no issue using another port however Home Monitor seems to be hard configured to use port 514 in it's dashboards.. Is there a location that I can change this from 514 to another port such as 7001 and have HomeMonitor populate correctly?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

gawainXX commented 4 years ago

I've done that in pfsense. I've not really had any issue using a port above 1000 but anything below that and I can't get it to work even if I'm running splunk with root privlage, home monitor however seems to ignore. I'm guessing it has to be port 514.

On Wed, Nov 27, 2019, 9:29 PM yabasha3 notifications@github.com wrote:

Hey,

You need to login into the router/switch or whatever you getting your syslogs from and change the port there. Once done reconfigure the ido on splunk it receive the logsys via that new configured port.

Thanks, Mohammed E.

On Nov 27, 2019, at 10:55 PM, gawainXX notifications@github.com wrote:

 Greetings, I've been running into this issue.. I've tried pretty much everything listed above and then some to forward syslog traffic to my CentOS Splunk VM to no avail. I however have absolutely no issue using another port however Home Monitor seems to be hard configured to use port 514 in it's dashboards.. Is there a location that I can change this from 514 to another port such as 7001 and have HomeMonitor populate correctly?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/amiracle/homemonitor/issues/16?email_source=notifications&email_token=AMP7SJDXRORPXHK54UP5M43QV5JMVA5CNFSM4GL26BDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFLPSMQ#issuecomment-559348018, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMP7SJATEIYS4HOSWL7NLN3QV5JMVANCNFSM4GL26BDA .

yabasha3 commented 4 years ago

That’s an issue with home monitor app. It worked for me when I️ put The ido to anything above 3000. 514 will only work for a Windows splunk Sever only.

Thanks, Mohammed E.

On Nov 28, 2019, at 2:01 AM, gawainXX notifications@github.com wrote:

I've done that in pfsense. I've not really had any issue using a port above 1000 but anything below that and I can't get it to work even if I'm running splunk with root privlage, home monitor however seems to ignore. I'm guessing it has to be port 514.

On Wed, Nov 27, 2019, 9:29 PM yabasha3 notifications@github.com wrote:

Hey,

You need to login into the router/switch or whatever you getting your syslogs from and change the port there. Once done reconfigure the ido on splunk it receive the logsys via that new configured port.

Thanks, Mohammed E.

On Nov 27, 2019, at 10:55 PM, gawainXX notifications@github.com wrote:

 Greetings, I've been running into this issue.. I've tried pretty much everything listed above and then some to forward syslog traffic to my CentOS Splunk VM to no avail. I however have absolutely no issue using another port however Home Monitor seems to be hard configured to use port 514 in it's dashboards.. Is there a location that I can change this from 514 to another port such as 7001 and have HomeMonitor populate correctly?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/amiracle/homemonitor/issues/16?email_source=notifications&email_token=AMP7SJDXRORPXHK54UP5M43QV5JMVA5CNFSM4GL26BDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFLPSMQ#issuecomment-559348018, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMP7SJATEIYS4HOSWL7NLN3QV5JMVANCNFSM4GL26BDA .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

gawainXX commented 4 years ago

Does home monitor work for you when using a different port # then 514? I'm new to splunk and will need to look up the terminology.

On Wed, Nov 27, 2019, 11:11 PM yabasha3 notifications@github.com wrote:

That’s an issue with home monitor app. It worked for me when I️ put The ido to anything above 3000. 514 will only work for a Windows splunk Sever only.

Thanks, Mohammed E.

On Nov 28, 2019, at 2:01 AM, gawainXX notifications@github.com wrote:

I've done that in pfsense. I've not really had any issue using a port above 1000 but anything below that and I can't get it to work even if I'm running splunk with root privlage, home monitor however seems to ignore. I'm guessing it has to be port 514.

On Wed, Nov 27, 2019, 9:29 PM yabasha3 notifications@github.com wrote:

Hey,

You need to login into the router/switch or whatever you getting your syslogs from and change the port there. Once done reconfigure the ido on splunk it receive the logsys via that new configured port.

Thanks, Mohammed E.

On Nov 27, 2019, at 10:55 PM, gawainXX notifications@github.com wrote:

 Greetings, I've been running into this issue.. I've tried pretty much everything listed above and then some to forward syslog traffic to my CentOS Splunk VM to no avail. I however have absolutely no issue using another port however Home Monitor seems to be hard configured to use port 514 in it's dashboards.. Is there a location that I can change this from 514 to another port such as 7001 and have HomeMonitor populate correctly?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/amiracle/homemonitor/issues/16?email_source=notifications&email_token=AMP7SJDXRORPXHK54UP5M43QV5JMVA5CNFSM4GL26BDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFLPSMQ#issuecomment-559348018 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AMP7SJATEIYS4HOSWL7NLN3QV5JMVANCNFSM4GL26BDA

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/amiracle/homemonitor/issues/16?email_source=notifications&email_token=AMP7SJGBAQZUJTTNNT4BV7LQV5VLBA5CNFSM4GL26BDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFLVBUQ#issuecomment-559370450, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMP7SJAKJLG4P3SDIVH3JULQV5VLBANCNFSM4GL26BDA .

yabasha3 commented 4 years ago

Yes , after I️ switched it to a different port on the router to send logsys on a different port.

Thanks, Mohammed E.

On Nov 28, 2019, at 2:54 AM, gawainXX notifications@github.com wrote:

Does home monitor work for you when using a different port # then 514? I'm new to splunk and will need to look up the terminology.

On Wed, Nov 27, 2019, 11:11 PM yabasha3 notifications@github.com wrote:

That’s an issue with home monitor app. It worked for me when I️ put The ido to anything above 3000. 514 will only work for a Windows splunk Sever only.

Thanks, Mohammed E.

On Nov 28, 2019, at 2:01 AM, gawainXX notifications@github.com wrote:

I've done that in pfsense. I've not really had any issue using a port above 1000 but anything below that and I can't get it to work even if I'm running splunk with root privlage, home monitor however seems to ignore. I'm guessing it has to be port 514.

On Wed, Nov 27, 2019, 9:29 PM yabasha3 notifications@github.com wrote:

Hey,

You need to login into the router/switch or whatever you getting your syslogs from and change the port there. Once done reconfigure the ido on splunk it receive the logsys via that new configured port.

Thanks, Mohammed E.

On Nov 27, 2019, at 10:55 PM, gawainXX notifications@github.com wrote:

 Greetings, I've been running into this issue.. I've tried pretty much everything listed above and then some to forward syslog traffic to my CentOS Splunk VM to no avail. I however have absolutely no issue using another port however Home Monitor seems to be hard configured to use port 514 in it's dashboards.. Is there a location that I can change this from 514 to another port such as 7001 and have HomeMonitor populate correctly?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/amiracle/homemonitor/issues/16?email_source=notifications&email_token=AMP7SJDXRORPXHK54UP5M43QV5JMVA5CNFSM4GL26BDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFLPSMQ#issuecomment-559348018 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AMP7SJATEIYS4HOSWL7NLN3QV5JMVANCNFSM4GL26BDA

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/amiracle/homemonitor/issues/16?email_source=notifications&email_token=AMP7SJGBAQZUJTTNNT4BV7LQV5VLBA5CNFSM4GL26BDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFLVBUQ#issuecomment-559370450, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMP7SJAKJLG4P3SDIVH3JULQV5VLBANCNFSM4GL26BDA .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

amiracle commented 4 years ago

I will look to make an update where the syslog port can be whatever port you want. Typically the splunk server needs to run as root for ports under 1024. If you want to change the listening port, you can do so on the inputs page or the inputs.conf.

gawainXX commented 4 years ago

Greetings, Hare are some screenshots of my config. Currently have the Syslog data from pfsense coming in on UDP 7001 which I confirmed is populating the homemonitor index although home monitor itself does not appear to be picking the data up. https://imgur.com/a/h6aihir

I’ve tried running splunk as root via sudo splunk start but was unable to pull syslog data on that port, nothing would populate in the index.

Can you please advise what I’d want to do, I tried looking up the acronym IDO but wasn’t able to determine what it refers to.

Sent from Mail for Windows 10

From: Kamilo Amir Sent: Thursday, November 28, 2019 7:55 AM To: amiracle/homemonitor Cc: gawainXX; Comment Subject: Re: [amiracle/homemonitor] Sorry for the noob question (#16)

I will look to make an update where the syslog port can be whatever port you want. Typically the splunk server needs to run as root for ports under 1024. If you want to change the listening port, you can do so on the inputs page or the inputs.conf. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

gawainXX commented 4 years ago

I've done a bit of digging around, it seems that it "should" be working on port 7001 as I can't find any hard references to Port 514 in the XML. I've also changed the Data Input sourcetype to pfsense which the queries seem to expect.

The Firewall does not seem to be providing fields that homemonitor expects. Did the data outputted by pfsense perhaps change between when you wrote the queries and pfsense 2.4.X?

I've tried dumbing the queries down a bit, such as "index=HomeMonitor sourcetype=pfsense direction=in" but nothing appears in the search. This leads me to thinking that either the logging format changed, I'm also seeing that the source type in my logs have subvalues such as sourcetype="pfsense:filterlog"

Sorry, still a hatchling in terms of experience with splunk.

::EDIT I think I may have figured it out and have been correcting the fields. E.G. for blocked traffic I've been changing the values sourcetype=$sourcetype$ Action2=BLOCK to Sourcetype="$sourcetype$" action="blocked"