send HTTP Host with LTM Telemtry Streaming

[leamese]

Is your feature request related to a problem? Please describe.

the LTM Telemetry streaming datalogs are not interesting at this time since the HTTP Host is not included in the logs. The HTTP URI is accessible, but not HTTP Host.

Describe the solution you'd like

add HTTP HOST to LTM request logging & response logging. maybe in the form off http_host=\"$HTTP_HOST\? Current config: create ltm profile request-log telemetry_traffic_log_profile request-log-pool telemetry request-log-protocol mds-tcp request-log-template event_source=\"request_logging\",hostname=\"$BIGIP_HOSTNAME\",client_ip=\"$CLIENT_IP\",server_ip=\"$SERVER_IP\",http_method=\"$HTTP_METHOD\",http_uri=\"$HTTP_URI\",virtual_name=\"$VIRTUAL_NAME\",event_timestamp=\"$DATE_HTTP\" request-logging enabled response-log-pool telemetry response-log-protocol mds-tcp response-log-template event_source=\"response_logging\",hostname=\"$BIGIP_HOSTNAME\",client_ip=\"$CLIENT_IP\",server_ip=\"$SERVER_IP\",http_method=\"$HTTP_METHOD\",http_uri=\"$HTTP_URI\",virtual_name=\"$VIRTUAL_NAME\",event_timestamp=\"$DATE_HTTP\",http_statcode=\"$HTTP_STATCODE\",http_status=\"$HTTP_STATUS\",response_ms=\"$RESPONSE_MSECS\"

Describe alternatives you've considered

send logs with HSL via iRule. That way we can extract the http log, and add other info we might want.

Additional context

Environment

• Telemetry Streaming Version: 1.33.0
• BIG-IP Version: 16.1.4.2

[pgouband]

Hi @leamese,

From this article: https://my.f5.com/manage/s/article/K00847516

In Extended set {Host} Value of field Host in HTTP header (introduced starting from HTTP ver. 1.0, mandatory for HTTP request header starting from ver. 1.1)

Have you tested {Host}?

[leamese]

thanks a lot. I am able to use {Host}.

For anyone who would need this, in case you are sending telemetry logs to elastic: you have to add: http_host=${Host} to the request logging profile. Request log template becomes: event_source="request_logging",hostname="$BIGIP_HOSTNAME",client_ip="$CLIENT_IP",server_ip="$SERVER_IP",http_method="$HTTP_METHOD",http_uri="$HTTP_URI",virtual_name="$VIRTUAL_NAME",event_timestamp="$DATE_HTTP",http_host="${host}" https://docs.elastic.co/integrations/f5_bigip https://github.com/elastic/integrations/blob/main/packages/f5_bigip/data_stream/log/elasticsearch/ingest_pipeline/pipeline_bigipltm.yml

[kenmaina]

Hi @leamese. Sorry to drop this here. Need some help. Please share your Telemetry_Consumer declaration for Elastic Agent to receive telemetry from Big-IP F5 (LTM). I have set mine as Generic_HTTP, F5 is posting telemetry data to the agent (confirmed with tcpdump) but there is no data being indexed on ES side.

F5 BIG-IP v15.1.6.1 F5 BIG-IP agent integration v1.13.2 ES 8.12.1 Kibana 8.12.1

"ES_Consumer": { "class": "Telemetry_Consumer", "type": "Generic_HTTP", "host": "192.168.1.1", "protocol": "http", "port": 9570, "path": "/", "method": "POST", "headers": [ { "name": "content-type", "value": "application/json" } ] }

[leamese]

if you have confirmed the posting off the data with tcpdump, it feels more like a elastic/integration issue. i think the only difference is that i'm sending with https instead off http. Other values are default values i think.

Consumer config: "My_Consumer": { "class": "Telemetry_Consumer", "type": "Generic_HTTP", "host": "10.38.15.249", "protocol": "https", "port": 9570, "path": "/", "method": "POST", "headers": [ { "name": "content-type", "value": "application/json" } ], "outputMode": "processed", "compressionType": "gzip", "customOpts": [ { "name": "keepAlive", "value": true }, { "name": "keepAliveMsecs", "value": 30000 }, { "name": "maxSockets", "value": 30 }, { "name": "maxFreeSockets", "value": 30 } ], "enable": true, "trace": false, "allowSelfSignedCert": true }

You can check the other vaues with a GET: https://f5-ip/mgmt/shared/telemetry/declare