Since in Fluent Bit, we use different mechanisms to handle HTTP requests for versions HTTP/1.x and HTTP/2, different plugins that use this interface (HTTP server) behind the scenes might have different ways to deal with requests, and recently we found some issues when mixing different configuration options.
In this issue, we will focus on testing the following input plugins only:
The latest version from Fluent Bit in GIT master needs to be tested against the following matrix of configuration options:
http2: on
http2: off
net.keepalive: on
net.keepalive: off
tls: on
tls: off
This is one example configured for Splunk on http2 on, keepalive on, tls off and with a HTTP/1.1 request (it should work), e.g:
pipeline:
inputs:
- name: splunk
port: 9883
host: 0.0.0.0
http2: on
net.keepalive: on
tls: off
outputs:
- name: stdout
match: '*'
the following Python3 code can be used to test the Splunk server:
import http.client
import json
# Define the server and port
server = 'localhost'
port = 9883
# Create a connection
conn = http.client.HTTPConnection(server, port)
# Define headers
headers = {
'Connection': 'keepalive',
'Authorization': 'Splunk secret-token',
'Content-Type': 'application/json'
}
# Define the payload
payload = {
"User": "Admin",
"password": "my_secret_password",
"Event": "Some text in the event"
}
# Convert payload to JSON
json_payload = json.dumps(payload)
# Define the number of requests
num_requests = 3
# Perform the requests
for i in range(num_requests):
conn.request("POST", "/services/collector", body=json_payload, headers=headers)
response = conn.getresponse()
print(f"Request {i+1} status: {response.status}, reason: {response.reason}")
response_data = response.read().decode()
print(f"Response {i+1} data: {response_data}")
# Close the connection
conn.close()
The code above does 3 requests within a keepalive connection, the expectation is to have the following outputs:
Fluent Bit output (stdout)
[0] splunk.0: [[1720042449.585015000, {"hec_token"=>"Splunk secret-token"}], {"User"=>"Admin", "password"=>"my_secret_password", "Event"=>"Some text in the event"}]
[1] splunk.0: [[1720042449.585317000, {"hec_token"=>"Splunk secret-token"}], {"User"=>"Admin", "password"=>"my_secret_password", "Event"=>"Some text in the event"}]
[2] splunk.0: [[1720042449.585552000, {"hec_token"=>"Splunk secret-token"}], {"User"=>"Admin", "password"=>"my_secret_password", "Event"=>"Some text in the event"}]
Python code output
Request 1 status: 200, reason: OK
Response 1 data: {"text":"Success","code":0}
Request 2 status: 200, reason: OK
Response 2 data: {"text":"Success","code":0}
Request 3 status: 200, reason: OK
Response 3 data: {"text":"Success","code":0}
Notes when testing with TLS enabled
TLS server side needs to use some certificates, we can use the ones available for testing inside Fluent Bit code repo, just doing a copy is enough:
Since in Fluent Bit, we use different mechanisms to handle HTTP requests for versions HTTP/1.x and HTTP/2, different plugins that use this interface (HTTP server) behind the scenes might have different ways to deal with requests, and recently we found some issues when mixing different configuration options.
In this issue, we will focus on testing the following input plugins only:
what needs to be tested
The latest version from Fluent Bit in GIT master needs to be tested against the following matrix of configuration options:
http2: on
http2: off
net.keepalive: on
net.keepalive: off
tls: on
tls: off
This is one example configured for Splunk on http2 on, keepalive on, tls off and with a HTTP/1.1 request (it should work), e.g:
the following Python3 code can be used to test the Splunk server:
The code above does 3 requests within a keepalive connection, the expectation is to have the following outputs:
Fluent Bit output (stdout)
Python code output
Notes when testing with TLS enabled
TLS server side needs to use some certificates, we can use the ones available for testing inside Fluent Bit code repo, just doing a copy is enough:
Here is an example of Splunk input plugin with TLS enabled: