search

lephisto / pfsense-analytics

Pfsense Analytics w/ Graylog, Elasticsearch, InfluxDB and Grafana fully dockerized for Firewall and DPI..
GNU General Public License v3.0
157 stars 56 forks source link

How can I change influxdb port number #34

Closed cypherstream closed 3 years ago

cypherstream commented 4 years ago

After fixing the maxmind link, I now stumble across this, and it looks like becuase I already have infludb running on the machine port 8086 for things like my cable modem signal statistics and also all of my ubiquiti unifi data that I am already running in grafana - locally on the machine already.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib/python3/dist-packages/compose/service.py", line 625, in start_container container.start() File "/usr/lib/python3/dist-packages/compose/container.py", line 241, in start return self.client.start(self.id, *options) File "/usr/lib/python3/dist-packages/docker/utils/decorators.py", line 19, in wrapped return f(self, resource_id, args, kwargs) File "/usr/lib/python3/dist-packages/docker/api/container.py", line 1095, in start self._raise_for_status(res) File "/usr/lib/python3/dist-packages/docker/api/client.py", line 263, in _raise_for_status raise create_api_error_from_http_exception(e) File "/usr/lib/python3/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception raise cls(e, response=response, explanation=explanation) docker.errors.APIError: 500 Server Error: Internal Server Error ("b'driver failed programming external connectivity on endpoint pfanalytics_influxdb_1 (b9e0960f863784eb35a07ee1b0976e43ac6c407b060c5be250f698e38ea16e27): Error starting userland proxy: listen tcp 0.0.0.0:8086: bind: address already in use'")**

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/bin/docker-compose", line 11, in load_entry_point('docker-compose==1.25.0', 'console_scripts', 'docker-compose')() File "/usr/lib/python3/dist-packages/compose/cli/main.py", line 72, in main command() File "/usr/lib/python3/dist-packages/compose/cli/main.py", line 128, in perform_command handler(command, command_options) File "/usr/lib/python3/dist-packages/compose/cli/main.py", line 1107, in up to_attach = up(False) File "/usr/lib/python3/dist-packages/compose/cli/main.py", line 1088, in up return self.project.up( File "/usr/lib/python3/dist-packages/compose/project.py", line 565, in up results, errors = parallel.parallel_execute( File "/usr/lib/python3/dist-packages/compose/parallel.py", line 112, in parallel_execute raise error_to_reraise File "/usr/lib/python3/dist-packages/compose/parallel.py", line 210, in producer result = func(obj) File "/usr/lib/python3/dist-packages/compose/project.py", line 548, in do return service.execute_convergence_plan( File "/usr/lib/python3/dist-packages/compose/service.py", line 545, in execute_convergence_plan return self._execute_convergence_create( File "/usr/lib/python3/dist-packages/compose/service.py", line 460, in _execute_convergence_create containers, errors = parallel_execute( File "/usr/lib/python3/dist-packages/compose/parallel.py", line 112, in parallel_execute raise error_to_reraise File "/usr/lib/python3/dist-packages/compose/parallel.py", line 210, in producer result = func(obj) File "/usr/lib/python3/dist-packages/compose/service.py", line 465, in lambda service_name: create_and_start(self, service_name.number), File "/usr/lib/python3/dist-packages/compose/service.py", line 457, in create_and_start self.start_container(container) File "/usr/lib/python3/dist-packages/compose/service.py", line 627, in start_container if "driver failed programming external connectivity" in ex.explanation: TypeError: a bytes-like object is required, not 'str'

cypherstream commented 4 years ago

I found it in docker-compose.yml

Changed it to 8087. influxdb: image: 'influxdb:latest' env_file:

Now maybe I can work on on getting the data out of there and put into my native bare metal influxdb on 8086. I also need to work on getting all the stuff out of this grafana instance (port 3000) out to my native grafana instance on port 80.

Finally the issue I have is both influxdb and the syslog port that pfsense is to talk to are only listening on IPv6 so I cannot access them.

Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 localhost:42247 0.0.0.0: LISTEN tcp 0 0 0.0.0.0:9393 0.0.0.0: LISTEN tcp 0 0 localhost:domain 0.0.0.0: LISTEN tcp 0 0 0.0.0.0:ssh 0.0.0.0: LISTEN tcp 0 0 localhost:omniorb 0.0.0.0: LISTEN tcp 0 0 localhost:52764 localhost:8086 ESTABLISHED tcp 0 64 grafana.local:ssh 192.168.5.13:61348 ESTABLISHED tcp 0 0 grafana.local:60776 pfSense.local:8443 ESTABLISHED tcp 0 0 localhost:9393 localhost:38640 ESTABLISHED tcp 0 0 localhost:43386 localhost:9090 ESTABLISHED tcp 0 0 localhost:43394 localhost:9090 ESTABLISHED tcp 0 0 grafana.local:54870 192.168.100.1:http ESTABLISHED tcp 0 0 localhost:9393 localhost:38588 TIME_WAIT tcp 0 0 localhost:9393 localhost:38608 TIME_WAIT tcp 0 0 localhost:38640 localhost:9393 ESTABLISHED tcp 0 0 localhost:9393 localhost:38540 TIME_WAIT tcp 0 0 localhost:9393 localhost:38570 TIME_WAIT tcp6 0 0 [::]:5601 [::]: LISTEN tcp6 0 0 [::]:9090 [::]: LISTEN tcp6 0 0 [::]:9000 [::]: LISTEN tcp6 0 0 [::]:12201 [::]: LISTEN tcp6 0 0 [::]:9001 [::]: LISTEN tcp6 0 0 [::]:1514 [::]: LISTEN tcp6 0 0 [::]:9130 [::]: LISTEN tcp6 0 0 [::]:9100 [::]: LISTEN tcp6 0 0 [::]:9200 [::]: LISTEN tcp6 0 0 [::]:http [::]: LISTEN tcp6 0 0 [::]:8086 [::]: LISTEN tcp6 0 0 [::]:ssh [::]:* LISTEN tcp6 0 0 [::]:8087 [::]:* LISTEN tcp6 0 0 [::]:3000 [::]: LISTEN tcp6 0 0 127.0.0.1:9090 127.0.0.1:43386 ESTABLISHED tcp6 0 0 127.0.0.1:9090 127.0.0.1:43394 ESTABLISHED tcp6 0 0 grafana.local:8086 pfSense.local:61811 ESTABLISHED tcp6 0 0 127.0.0.1:8086 127.0.0.1:52764 ESTABLISHED tcp6 0 0 grafana.local:8086 pfSense.local:62062 ESTABLISHED udp 0 0 localhost:domain 0.0.0.0: udp6 0 0 [::]:12201 [::]: udp6 0 0 [::]:2055 [::]: udp6 0 0 [::]:5442 [::]:* udp6 0 0 [::]:1514 [::]: raw6 0 0 [::]:ipv6-icmp [::]: 7

cypherstream commented 4 years ago

I was able to recreate the ndpi database on my local already existing influxdb and create the user ntop and a password. I was able to configure ntopng as well. I just cant get pfsense to write logs to port 5442. That port does not respond on ipv4.

I exported the dashboards from your docker instance grafana at port 3000 into my existing grafana instance at port 80. My only issue is getting the communication to work to populate graylog since port 5442 is inaccessible. The grafana source for port 9200 for elasticsearch works though, probably because its all housed within the same box. See my trouble is im somewhat of a hybrid setup. I already had influxdb and grafana up for some time. I'm just using this for graylog and elasticsearch, and all of their dependencies like mongodb, java, etc...

cypherstream commented 4 years ago

It sort of resolved itself. And I also forgot to enable the geo ip plugin in Graylog. I also had a typo in the database name in ntopng. After that I have a fully working system. Please close issue.