search

influxdata / telegraf

Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
https://influxdata.com/telegraf
MIT License
14.59k stars 5.56k forks source link

[inputs.jenkins] It seems to use the authority of Anonymous Users. #5181

Closed mcpaint closed 5 years ago

mcpaint commented 5 years ago

I created Token and used. However, it seems to use 'Anonymous Users' authority.

Relevant telegraf.conf:

[[inputs.jenkins]]
url = "http://localhost:8080"
username = "{{username}}"
password = "{{Token}}"
response_timeout = "10s"

System info:

Steps to reproduce:

  1. Set Jenkins environment
  2. Run telegraf

Expected behavior:

Have to get authority of the user.

Actual behavior:

It seems to use the authority of Anonymous Users.

Additional info:

[Include gist of relevant config, logs, etc.]

2018-12-21T09:33:00Z E! [inputs.jenkins]: Error in plugin: [/computer/api/json] 403 Forbidden
2018-12-21T09:33:00Z E! [inputs.jenkins]: Error in plugin: [/api/json] 403 Forbidden
# success
curl -X POST -u <username>:<token> http://localhost:8080/computer/api/json?pretty
{
  "_class" : "hudson.model.ComputerSet",
  "busyExecutors" : 0,
  "computer" : [
    {
      "_class" : "hudson.model.Hudson$MasterComputer",
      "actions" : [

      ],
...
glinton commented 5 years ago

Can you verify the credentials are correct? In my tests, I saw the same header being passed by both curl and telegraf.

Running telegraf with the following config

[[inputs.jenkins]]
    url = "http://localhost:8080"
    username = "user"
    password = "pass"
    response_timeout = "10s"

Server recieves the following request:

GET /computer/api/json HTTP/1.1
Host: localhost:8080
User-Agent: Go-http-client/1.1
Accept: application/json
Authorization: Basic dXNlcjpwYXNz
Accept-Encoding: gzip

Running curl

curl -u user:pass http://localhost:8080/computer/api/json Server recieves the following request:

GET /computer/api/json HTTP/1.1
Host: localhost:8080
Authorization: Basic dXNlcjpwYXNz
User-Agent: curl/7.58.0
Accept: */*
mcpaint commented 5 years ago

Yes. I can verify. Curl test result is successful. But. How can I verify correct telegraf header? Can you show me the way verify correct telegraf header?

Curl test result

$ curl -I -u user:token http://localhost:8080/computer/api/json
HTTP/1.1 200 OK
Date: Sat, 22 Dec 2018 15:09:40 GMT
X-Content-Type-Options: nosniff
X-Jenkins: 2.150.1
X-Jenkins-Session: 0b0d5ca4
Content-Type: application/json;charset=utf-8
Content-Length: 1499
Server: Jetty(9.4.z-SNAPSHOT)

Additional info:

glinton commented 5 years ago

I just ran a netcat listener on port 8080 to see what headers both curl and telegraf was using. nc -l -p 8080 (flags may be different depending on nc version)

mcpaint commented 5 years ago

I'm sorry for the late reply.

Ncat test results

Telegraf:

GET /computer/api/json HTTP/1.1
Host: localhost:8080
User-Agent: Go-http-client/1.1
Accept: application/json
Authorization: Basic TFAxMjA5Nzox
Accept-Encoding: gzip

Curl:

GET /computer/api/json HTTP/1.1
Authorization: Basic TFAxMjA5Nzox
User-Agent: curl/7.29.0
Host: localhost:8080
Accept: */*
glinton commented 5 years ago

It looks like the tokens are the same, is that error log coming from another config stanza without username/password set perhaps?

mcpaint commented 5 years ago

The '$remote_user' value of request that Telegraf sent is blank. The 'L****7' is username.

Nginx access log

curl

<domain>:<port> 4.006 <ip> - L****7 [04/Jan/2019:15:41:29 +0900] 200 "GET /computer/api/json HTTP/1.1" 5773 "-" "Go-http-client/1.1" "-"

Telegraf:jenkins

<domain>:<port> 0.003 <ip> - - [04/Jan/2019:15:41:30 +0900] 403 "GET /computer/api/json HTTP/1.1" 835 "-" "Go-http-client/1.1" "-"

Nginx access log format

    log_format main '$host:$server_port $request_time $remote_addr - $remote_user [$time_local] $status '
                    '"$request" $body_bytes_sent "$http_referer" "$http_user_agent" "$http_x_forwarded_for"';

The request that Telegraf sent is a correct request? What I find the reason is difficult because I don't know 'Go' language.

glinton commented 5 years ago

Are you using a config directory or another jenkins input defined in your current config file? As far as I can tell, telegraf is sending the correct request as I have been unable to reproduce this, which leads me to believe it's a config issue. I do want to know for sure though before dismissing this as such

LINKIWI commented 5 years ago

Was there ever a solution to this? I'm experiencing the same problem: hitting /api/json works fine with curl but not with the plugin.

The HTTP 403 returned by Jenkins seems to indicate that the authentication is successful (it would otherwise return HTTP 401 on invalid credentials). Granting "Anonymous Users" full read permissions allows the plugin to produce metrics correctly.

mcpaint commented 5 years ago

I couldn't find a solution. So, I checked the permissions checkbox as follows.

Overall - Read Agent - Disconnect Job - Read

mcpaint commented 5 years ago

Additional info:

Configure Global Security

Configure Global Security

LINKIWI commented 5 years ago

I've identified a workaround. In configuration, using

url = "http://user:pass@jenkins-host"

Instead of

url = "http://jenkins-host"
username = "user"
password = "pass"

works.

¯\_(ツ)_/¯

I've been digging through the source code of both the plugin and Jenkins itself. I suspect there's a bug in the way the plugin switches between using session cookies and HTTP basic auth credentials through multiple requests. My guess is that the initial request to / returns a response cookie that the plugin then uses instead of the Basic auth credentials on the subsequent /api/json request, but the version of Jenkins I've deployed doesn't authenticate a user through the cookie alone. Encoding the credentials in the URL will pass them directly to Go's underlying HTTP client library, which will then ship them in every outgoing request regardless of the logic in this plugin.

Anyway, this workaround is acceptable to me so I'm not particularly incentivized to continue digging for a root cause.

danielnelson commented 5 years ago

I can reproduce this with the debian package from http://pkg.jenkins.io/debian-stable. The initial connection succeeds because it includes the basic auth header but subsequent queries with only the JSESSIONID cookie are not enough to authenticate.

@kelwang You mentioned that this method was used "to prevent github auth rate limit", would it be a problem if we always sent basic auth along with the previously set JSESSIONID?

This seems to work without cycling the session using curl:

$ curl -u dbn:dbn -c cookies http://debian-stretch-jenkins:8080/ -v -o /dev/null -s
*   Trying 192.168.122.17...
* TCP_NODELAY set
* Connected to debian-stretch-jenkins (192.168.122.17) port 8080 (#0)
* Server auth using Basic with user 'dbn'
> GET / HTTP/1.1
> Host: debian-stretch-jenkins:8080
> Authorization: Basic ZGJuOmRibg==
> User-Agent: curl/7.62.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Fri, 18 Jan 2019 21:58:17 GMT
< X-Content-Type-Options: nosniff
< Expires: Thu, 01 Jan 1970 00:00:00 GMT
< Cache-Control: no-cache,no-store,must-revalidate
< X-Hudson-Theme: default
< Referrer-Policy: same-origin
< Content-Type: text/html;charset=utf-8
* Added cookie JSESSIONID.2bf6c160="node01meqi2web3nlgtha9ucsc023x34.node0" for domain debian-stretch-jenkins, path /, expire 0
< Set-Cookie: JSESSIONID.2bf6c160=node01meqi2web3nlgtha9ucsc023x34.node0;Path=/;HttpOnly
< X-Hudson: 1.395
< X-Jenkins: 2.150.2
< X-Jenkins-Session: f2c97036
< X-Frame-Options: sameorigin
< X-Instance-Identity: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAg1kD280qnnCq0aWgonSfl3gAJGahqDh2I4A17n1BmVWp+J1ImG2CqeQ+Ep6m+zvoK9cCT9KAB+t44hHB4kly6w5VbIq0Ru2GN6hiYgv8a3FEWlD1jEofhHz4mntPwNZ+x3PbfF336oZofYgxs7h+qOe7mI3OZCnpDX8heDMMzLHFzBm+9Sseqq78mB26Jfy9Xg7kq1fygt1s4WNUoB0fpmKuD5dm/Bx2McQRL/TbRNZI7lOg4Oze4gEpPkKcdagduqi8dorgzcP0Q4exK7NkdmA2qVvkT25+SREbhHZeSRW6IhMJ/XzDvmUrcAGsk5RpiTHKiiOZfH06hf7QdBFi3wIDAQAB
< Content-Length: 14880
< Server: Jetty(9.4.z-SNAPSHOT)
<
{ [6315 bytes data]
* Connection #0 to host debian-stretch-jenkins left intact
$ curl -u dbn:dbn -b cookies -c cookies http://debian-stretch-jenkins:8080/api/json -v -o /dev/null -s
*   Trying 192.168.122.17...
* TCP_NODELAY set
* Connected to debian-stretch-jenkins (192.168.122.17) port 8080 (#0)
* Server auth using Basic with user 'dbn'
> GET /api/json HTTP/1.1
> Host: debian-stretch-jenkins:8080
> Authorization: Basic ZGJuOmRibg==
> User-Agent: curl/7.62.0
> Accept: */*
> Cookie: JSESSIONID.2bf6c160=node01meqi2web3nlgtha9ucsc023x34.node0
>
< HTTP/1.1 200 OK
< Date: Fri, 18 Jan 2019 21:58:26 GMT
< X-Content-Type-Options: nosniff
< X-Jenkins: 2.150.2
< X-Jenkins-Session: f2c97036
< Content-Type: application/json;charset=utf-8
< Content-Length: 699
< Server: Jetty(9.4.z-SNAPSHOT)
<
{ [699 bytes data]
* Connection #0 to host debian-stretch-jenkins left intact
$ curl -u dbn:dbn -b cookies -c cookies http://debian-stretch-jenkins:8080/api/json -v -o /dev/null -s
*   Trying 192.168.122.17...
* TCP_NODELAY set
* Connected to debian-stretch-jenkins (192.168.122.17) port 8080 (#0)
* Server auth using Basic with user 'dbn'
> GET /api/json HTTP/1.1
> Host: debian-stretch-jenkins:8080
> Authorization: Basic ZGJuOmRibg==
> User-Agent: curl/7.62.0
> Accept: */*
> Cookie: JSESSIONID.2bf6c160=node01meqi2web3nlgtha9ucsc023x34.node0
>
< HTTP/1.1 200 OK
< Date: Fri, 18 Jan 2019 21:58:36 GMT
< X-Content-Type-Options: nosniff
< X-Jenkins: 2.150.2
< X-Jenkins-Session: f2c97036
< Content-Type: application/json;charset=utf-8
< Content-Length: 699
< Server: Jetty(9.4.z-SNAPSHOT)
<
kelwang commented 5 years ago

@danielnelson should be fine, I also feel it might be safer to make it as a configurable option.