search

riemann / riemann-dash

An HTTP dashboard for Riemann.
MIT License
248 stars 102 forks source link

riemann-dash does not allow setting host as documented. #11

Closed blt closed 11 years ago

blt commented 11 years ago

The riemann-dash documentation implies that setting config[:client][:host] will target a non-default riemann server. When this setting is left at its default I find:

riemann@riemann:~$ cat dash.config 
set :bind, "REDACTED_IP"
#config[:client][:host] = 'REDACTED_IP'

riemann@riemann:~$ riemann-dash dash.config 
== Sinatra/1.3.5 has taken the stage on 4567 for development with backup from Thin
>> Thin web server (v1.5.0 codename Knife)
>> Maximum connections set to 1024
>> Listening on REDACTED_IP:4567, CTRL+C to stop
^C>> Stopping ...

however

riemann@riemann:~$ cat dash.config 
set :bind, "REDACTED_IP"
config[:client][:host] = 'REDACTED_IP'
riemann@riemann:~$ riemann-dash dash.config 
/home/riemann/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/riemann-dash-0.2.1/lib/riemann/dash.rb:36:in `instance_eval': undefined method `[]=' for nil:NilClass (NoMethodError)
        from /home/riemann/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/riemann-dash-0.2.1/lib/riemann/dash.rb:36:in `instance_eval'
        from /home/riemann/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/riemann-dash-0.2.1/lib/riemann/dash.rb:36:in `load_config'
        from /home/riemann/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/riemann-dash-0.2.1/lib/riemann/dash.rb:21:in `load'
        from /home/riemann/.rbenv/versions/1.9.3-p327/lib/ruby/gems/1.9.1/gems/riemann-dash-0.2.1/bin/riemann-dash:6:in `<top (required)>'
        from /home/riemann/.rbenv/versions/1.9.3-p327/bin/riemann-dash:23:in `load'
        from /home/riemann/.rbenv/versions/1.9.3-p327/bin/riemann-dash:23:in `<main>'
aphyr commented 11 years ago

Good catch! This is a total lie; artifact of outdated documentation. Pretty much all configuration, including the Riemann server, happens in the browser now. My apologies!

blt commented 11 years ago

Oh, okay! How do I configure riemann-dash to hit a non-localhost riemann server, then? I have a riemann-dash running at http://riemann.troutwine.us:4567/ and a server at the same host on the default port. When I enter riemann.troutwine.us:5555 in riemann-dash's top right-hand box, nothing changes.

Is there any way to make riemann-dash default to a non-localhost server?

default50 commented 11 years ago

Oh, okay! How do I configure riemann-dash to hit a non-localhost riemann server, then? I have a riemann-dash running at http://riemann.troutwine.us:4567/ and a server at the same host on the default port. When I enter riemann.troutwine.us:5555 in riemann-dash's top right-hand box, nothing changes.

Is there any way to make riemann-dash default to a non-localhost server?

@blt I have the same config as you and it's working, but the default port for the websockets is 5556

default50 commented 11 years ago

Good catch! This is a total lie; artifact of outdated documentation. Pretty much all configuration, including the Riemann server, happens in the browser now. My apologies!

@aphyr there's another bit of info that may be useful to have on the README:

set :bind, 'localhost' # Default 0.0.0.0

I needed to secure my installation behind a reverse proxy and it took me a while to find out that it was Sinatra's configuration :)

blt commented 11 years ago

@default50 Hmm, even after correcting the port, nothing. My riemann-dash is running http://riemann.troutwine.us:4567/ and when I boot riemann server I find:

iemann@riemann:~/riemann-0.1.5$ ./bin/riemann etc/riemann.config 
INFO [2013-03-05 11:35:09,730] main - riemann.server - TCP server {:host 166.78.122.49, :port 5555, :pipeline-factory #<server$tcp_server$pipeline_factory__6862 riemann.server$tcp_server$pipeline_factory__6862@244e5056>}  online

INFO [2013-03-05 11:35:09,861] main - riemann.server - UDP server {:host 166.78.122.49, :port 5555, :max-size 16384, :pipeline-factory #<server$udp_server$pipeline_factory__6870 riemann.server$udp_server$pipeline_factory__6870@14a2da07>} online

INFO [2013-03-05 11:35:09,894] main - riemann.server - Websockets server {:host 166.78.122.49, :port 5556} online

INFO [2013-03-05 11:35:09,935] main - riemann.core - Hyperspace core online

which seems good. nmapping the server:

blt> nmap riemann.troutwine.us

Starting Nmap 6.25 ( http://nmap.org ) at 2013-03-05 09:38 PST
Nmap scan report for riemann.troutwine.us (166.78.122.49)
Host is up (0.090s latency).
Not shown: 996 closed ports
PORT     STATE    SERVICE
22/tcp   open     ssh
1719/tcp filtered h323gatestat
4567/tcp open     tram
5555/tcp open     freeciv

Nmap done: 1 IP address (1 host up) scanned in 3.98 seconds

so 5556 doesn't look open. There's no firewall or anything on this box: it's a vanilla Debian Wheezy install. Anyway, when I enter riemann.troutwine.us:5556 into the dash's server box and hit enter nothing happens, no error and no change.

Am I misunderstanding something here? My riemann server config:

; vim: filetype=clojure
(logging/init :file "riemann.log")

; Listen on the local interface over TCP (5555), UDP (5555), and websockets; (5556)
(let [host "166.78.122.49"]
  (tcp-server :host host)
  (udp-server :host host)
  (ws-server  :host host))

; Expire old events from the index every 5 seconds.
(periodically-expire 5)

; Keep events in the index for 5 minutes by default.
(let [index (default :ttl 300 (update-index (index)))]

  ; Inbound events will be passed to these streams:
  (streams

    ; Index all events immediately.
    index

    ; Calculate an overall rate of events.
    (with {:metric 1 :host nil :state "ok" :service "events/sec"}
      (rate 5 index))

    ; Log expired events.
    (expired
      (fn [event] (info "expired" event)))
))

and the dash config:

set :bind, "166.78.122.49"
aphyr commented 11 years ago

On 03/05/2013 09:43 AM, Brian L. Troutwine wrote:

INFO [2013-03-05 11:35:09,894] main - riemann.server - Websockets server {:host 166.78.122.49, :port 5556} online

An interesting discovery from last week: the :host option to Aleph (which powers the websocket server) doesn't actually do anything. I think it should bind to 0.0.0.0 no matter what :host you pass... but I wonder if it's doing some weird IPv6 thing instead. You might try netstat -nl to see if it bound correctly.

--Kyle

blt commented 11 years ago

Looks like 5556 is only binding on localhost:

riemann@riemann:~$ netstat -nl
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN     
tcp        0      0 166.78.122.49:4567      0.0.0.0:*               LISTEN     
tcp6       0      0 :::22                   :::*                    LISTEN     
tcp6       0      0 166.78.122.49:5555      :::*                    LISTEN     
tcp6       0      0 :::5556                 :::*                    LISTEN     
udp6       0      0 166.78.122.49:5555      :::*                               
Active UNIX domain sockets (only servers)
Proto RefCnt Flags       Type       State         I-Node   Path
unix  2      [ ACC ]     STREAM     LISTENING     6024     /tmp//tmux-1000/default
unix  2      [ ACC ]     STREAM     LISTENING     28689    /tmp//tmux-1001/default
unix  2      [ ACC ]     SEQPACKET  LISTENING     1869     /run/udev/control
unix  2      [ ACC ]     STREAM     LISTENING     3701     /var/run/dbus/system_bus_socket
unix  2      [ ACC ]     STREAM     LISTENING     3542     /var/run/acpid.socket
aphyr commented 11 years ago

On 03/05/2013 09:59 AM, Brian L. Troutwine wrote:

Looks like 5556 is only binding on localhost: LISTEN tcp6 0 0 :::5556 :::*

More specifically, only on ipv6. Try passing -Djava.net.preferIPv4Stack=true to riemann. :/

--Kyle

default50 commented 11 years ago

That's not localhost, it's 0.0.0.0 equivalent of IPv6 @blt.

BTW, I'm running on vanilla Wheezy too, on EC2. Of course I have opened up port 5556 on the firewall.

This is my netstat output:

netstat -puntal|grep 555
tcp6       0      0 :::5555                 :::*                    LISTEN      29954/java      
tcp6       0      0 :::5556                 :::*                    LISTEN      29954/java      
tcp6       0      0 10.x.x.x:5556     190.176.x.x:49965   ESTABLISHED 29954/java      
tcp6       0      0 10.x.x.x:5556     190.176.x.x:49972   ESTABLISHED 29954/java      
tcp6       0      0 10.x.x.x:5556     190.176.x.x:49973   ESTABLISHED 29954/java      
udp6    4722      0 :::5555                 :::*                                29954/java

Note: IP addresses hidden on purpose.

blt commented 11 years ago

@default50 Quite so; I'm less familiar with IPv6 than I should be.

@aphyr Starting up riemann server:

$ ./bin/riemann -Djava.net.preferIPv4Stack=true etc/riemann.config 
INFO [2013-03-05 12:10:13,667] main - riemann.server - TCP server {:host 166.78.122.49, :port 5555, :pipeline-factory #<server$tcp_server$pipeline_factory__6862 riemann.server$tcp_server$pipeline_factory__6862@d335207>}  online

INFO [2013-03-05 12:10:13,714] main - riemann.server - UDP server {:host 166.78.122.49, :port 5555, :max-size 16384, :pipeline-factory #<server$udp_server$pipeline_factory__6870 riemann.server$udp_server$pipeline_factory__6870@61bc59aa>} online

INFO [2013-03-05 12:10:13,758] main - riemann.server - Websockets server {:host 166.78.122.49, :port 5556} online

INFO [2013-03-05 12:10:13,806] main - riemann.core - Hyperspace core online

and the netstat output is as expected:

$ netstat -nl
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN     
tcp        0      0 166.78.122.49:4567      0.0.0.0:*               LISTEN     
tcp        0      0 166.78.122.49:5555      0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:5556            0.0.0.0:*               LISTEN     
tcp6       0      0 :::22                   :::*                    LISTEN     
udp        0      0 166.78.122.49:5555      0.0.0.0:*                          
Active UNIX domain sockets (only servers)
Proto RefCnt Flags       Type       State         I-Node   Path
unix  2      [ ACC ]     STREAM     LISTENING     6024     /tmp//tmux-1000/default
unix  2      [ ACC ]     STREAM     LISTENING     28689    /tmp//tmux-1001/default
unix  2      [ ACC ]     SEQPACKET  LISTENING     1869     /run/udev/control
unix  2      [ ACC ]     STREAM     LISTENING     3701     /var/run/dbus/system_bus_socket
unix  2      [ ACC ]     STREAM     LISTENING     3542     /var/run/acpid.socket

I still don't see 5556 as open, via nmap. I continue to be unable to attach the dashboard to the server on riemann.troutwine.us. (I am correct in believing that I need only add the server name/port into the box at the top right-hand corner and hit enter?)

aphyr commented 11 years ago

On 03/05/2013 10:16 AM, Brian L. Troutwine wrote:

LISTEN tcp 0 0 0.0.0.0:5556 0.0.0.0:*

I still don't see 5556 as open, via nmap. I continue to be unable to attach the dashboard to the server on riemann.troutwine.us. (I am correct in believing that I need only add the server name/port into the box at the top right-hand corner and hit enter?)

If nmap reports the port is closed, you're probably looking at a firewall. Might be anywhere in between your browser and the server.

--Kyle

default50 commented 11 years ago

I still don't see 5556 as open, via nmap. I continue to be unable to attach the dashboard to the server on riemann.troutwine.us. (I am correct in believing that I need only add the server name/port into the box at the top right-hand corner and hit enter?)

@blt you only need that.

nmap doesn't scan all 64k ports by defaul. Your port looks open, even from here:

nmap -p 5555,5556 riemann.troutwine.us

Starting Nmap 6.00 ( http://nmap.org ) at 2013-03-05 15:17 ART
Nmap scan report for riemann.troutwine.us (166.78.122.49)
Host is up (0.23s latency).
PORT     STATE SERVICE
5555/tcp open  freeciv
5556/tcp open  unknown

Nmap done: 1 IP address (1 host up) scanned in 1.44 seconds

I could even telnet into it! ;)

telnet riemann.troutwine.us 5556
Trying 166.78.122.49...
Connected to riemann.troutwine.us.
Escape character is '^]'.

^]

telnet> Connection closed.

Moreover, and sorry if I'm tripping on you, but I loaded your dash on my browser (port 4567) and changed the URL to riemann.troutwine.us:5556 which stopped the message about lost connections. Changed the grid query to true and saw everything empty, but didn't got an error either (maybe you aren't pushing any data into riemann yet?)

I suspect your client is misbehaving, or maybe you (as a client) are behind some outgoing firewall?

blt commented 11 years ago

Hmm... On the box itself:

root@riemann:~# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

@default50's input suggests I'm misusing nmap.

@default50 I can see from the logs someone from 190.176.144.135 connecting to the server. Please, be my guest, if that is you. This is a scrub box entirely.

It's possible that I don't understand how to make use of the dashboard, is all. Is there a video tutorial on its use?

aphyr commented 11 years ago

@default50 https://github.com/default50 I can see from the logs someone from 190.176.144.135 connecting to the server. Please, be my guest, if that is you. This is a scrub box entirely.

It's possible that I don't understand how to make use of the dashboard, is all. Is there a video tutorial on its use?

No, but I'll link to one if you make it! You should just have to type riemann.troutwine.us:5556 in the box at the top right and hit enter; if that doesn't work, I'd suspect a firewall still.

--Kyle

aphyr commented 11 years ago

On 03/05/2013 10:28 AM, Sebastian Cruz wrote:

I still don't see 5556 as open, via nmap. I continue to be unable to attach the dashboard to the server on riemann.troutwine.us. (I am correct in believing that I need only add the server name/port into the box at the top right-hand corner and hit enter?)

Oh, it might also be possible that your browser doesn't support websockets--might check the JS console for log messages!

default50 commented 11 years ago

@default50 I can see from the logs someone from 190.176.144.135 connecting to the server. Please, be my guest, if that is you. This is a scrub box entirely.

@blt that's my IP :) Hi!

Just configured your dash and saved the config, and your described usage is as it's meant to be.

Some tests that could be useful:

I'm pretty sure it's the client's problem only (browser).

blt commented 11 years ago

Hi, @default50!

I think simply it's that I don't understand how to use the dashboard. How did you, for instance, configure the dashboard? I undertand how you edited the view to read "This was changed by default50 as a test, sorry!", but not what you did to make a certain server default.

I've used latest Chrome, Safari and Firefox on two separate OS X machines, and Firefox/Chrome on a linux box from a different ISP. Surely my problem is a lack of knowledge about the dashboard's operating.

aphyr commented 11 years ago

On 03/05/2013 11:08 AM, Brian L. Troutwine wrote:

Hi, @default50 https://github.com/default50!

I think simply it's that I don't understand how to use the dashboard. How did you, for instance, configure the dashboard? I undertand how you edited the view to read "This was changed by default50 as a test, sorry!", but not what you did to make a certain server default.

I've used latest Chrome, Safari and Firefox on two separate OS X machines, and Firefox/Chrome on a linux box from a different ISP. Surely my problem is a lack of knowledge about the dashboard's operating.

The views, workspaces, and the server address you type in to the top left are all stored as JSON on the riemann-dash server. When you save the config, it saves the currently connected riemann server as well.

--Kyle

blt commented 11 years ago

When you save the config, it saves the currently connected riemann server as well.

Oh, I see! Thank you very much, @aphyr and @default50.