fredx30
commented
2 years ago The following code changes work in my environment. I belive the potential danger with this change is reverse and forward compatability with different versions of the elasticsearch api.
https://github.com/google/grr/compare/master...fredx30:grr:update-elasticsearch-output-plugin
Currently i am using these changes in a fairly volatile way where i have to reimplement them on container restart. Is there any way of getting these into a mainline release? Am i better off building my own releases to a private container registry of my choosing?
Edit: Is a PR here with these changes welcome?
micrictor
max-vogler
Environment
How did you install GRR? [e.g. from release DEB, HEAD DEB, PIP package, source] Dockerfile - https://hub.docker.com/layers/grr/grrdocker/grr/v3.4.6.0/images/sha256-30f404087124417eb96420ad157c5bac403748c55eafabf1e0abfbf7c20bb0c9?context=explore
What GRR version are you running?: [e.g. 3.1.2.3] 3.4.6.0
What operating system does the GRR server run on? [e.g. Ubuntu 18.04] Ubuntu 18.04. This is running in aks (Azure kubernetes service) currently using image-name: "AKSUbuntu-1804gen2containerd-2022.06.08"
What operating system does the affected GRR client run on, if applicable? [e.g. Windows 10] Windows server 2022 (Not affected)
Elastic cluster version (Likely of relevance here) Im using a single node elasticsearch cluster deployed as image (container) )version 7.17.0 deployed with the elastic helmchart elastic/elasticsearch.
Describe the issue The outputplugin elasticsearch throws errors on usage. In this case im creating a flow to request the netstat listenting ports/processes on a windows based client. API request for the flow looks like the following (Where "grr-server-host" is a dummy replacement for a valid url).
Without the outputplugin it all works fine.
Its when i add the outputplugin i start getting issues in the grr-worker. This is shown in the grr-worker.log. This is visible both within the pod/container running
grr_workerand the admin_ui when viewing a specific flow under the navbar item "managed launched flows" -> "results" -> "1 errors" (clicking this opens a modal view of an errors list with the errors below).!OBS See Debugging process blow.
"content type" : "url encode"this is resolved by adding contenttype headers for application/json.Error logs
From
grr-worker.logAdditional context Debugging Process In my case while testing and developing a fix for this i was using a small
test.jsonlfile generated with real data extracted by adding an exception to the plugin to dump the expected body of the request. I added the lineraise Exception("CheckJsonFormat: ---%s",data)at lin 150 in the plugin. https://github.com/google/grr/blob/master/grr/server/grr_response_server/output_plugins/elasticsearch_plugin.py#L150Here is my output file:
Then using this file with curl to attempt the connections myself...
The first curl request looks as follows, where
test.jsonlis the json extracted from the request as mentioned above.This results in the error
{"error":"Content-Type header [application/x-www-form-urlencoded] is not supported","status":406}So to fix this we add content type to the request and try again.This results in the error
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"The bulk request must be terminated by a newline [\\n]"}],"type":"illegal_argument_exception","reason":"The bulk request must be terminated by a newline [\\n]"},"status":400}.To fix this we add a
\nat the end of the json body.