search

elastic / integrations

Elastic Integrations
https://www.elastic.co/integrations
Other
21 stars 436 forks source link

Events are not parsing correctly in Prisma cloud integration #8994

Open chanakagl opened 9 months ago

chanakagl commented 9 months ago

We have moved to Prisma cloud integration from Custom HTTP Endpoint Logs integration. Since we moved to official integration we can see that events are parsing with below error,

[failed eval: failed to unmarshal JSON message: invalid character '<' looking for beginning of value, Processor json with tag json_event_original in pipeline logs-prisma_cloud.alert-1.0.1 failed with message: field [original] not present as part of path [event.original]]

But we used to get the event parsed correctly with Custom HTTP Endpoint Logs integration.

Let me know if there any other information is needed.

efd6 commented 9 months ago

@chanakagl This looks like the kind of error that you would expect if the integration is pointing at a page that returns HTML rather than JSON in the body. Please ensure that you have correctly configured the integration, particularly with respect to the base URL of the API. An alternative is that there is an intervening block that is displaying an HTML page.

To investigate the exact cause (there is not enough information here for me to do this), you can turn on request tracing

request tracing option

and examine the responses in the logs (obtained by getting a diagnostic bundle; the request trace will be in the logs/cel directory in the bundle).

getting diagnostic bundles

chanakagl commented 8 months ago

Thanks. I'll check and let you know

efd6 commented 8 months ago

@chanakagl I deleted that comment because it includes information that you should not make public.

efd6 commented 8 months ago

You are getting a 405 for the request as an XML message (redacted);

<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>MethodNotAllowed</Code><Message>The specified method is not allowed against this resource.</Message><Method>POST</Method><ResourceType>OBJECT</ResourceType><RequestId>...REDACT...</RequestId><HostId>...REDACT...</HostId></Error>

You'll be able to find the literal cases of these in the logs/httpjson directory.

I have a suspicion that this may be down to an extra path separator. Please try removing the final slash in the configured URL. At the moment it is https//app. … .prismacloud.io/, try https//app. … .prismacloud.io.

chanakagl commented 8 months ago

Hi,

Still i can see the same issue after changing the configured URL to https//app. … .prismacloud.io

efd6 commented 8 months ago

I would recommend trying the endpoint with POSTMAN or cURL using the construction in the agent configuration file, for example here. This would end up looking like this (sensitive parts removed).

curl -d '{"password":"…", "username":"…"}' -H "Content-Type: application/json" -X POST https://app. … .prismacloud.io/login
chanakagl commented 8 months ago

Hi,

I tried above API call and got below respond. <?xml version="1.0" encoding="UTF-8"?>

MethodNotAllowedThe specified method is not allowed against this resource.POSTOBJECTYXF0ZN9NKCBV8B78HNleIN/U6kxaEYAqQW/VkaqLkTl6TAI1flYf1LTjws9UzEJKILGX96CoAxjmhETC5dF3mQhuGP8=[root@ip Then I tried API mentioned on Prisma official documentation doco - https://docs.prismacloud.io/en/classic/cspm-admin-guide/get-started-with-prisma-cloud/access-the-prisma-cloud-api And got below respond {"message":"invalid_credentials"}
chanakagl commented 8 months ago

Hi,

We have got new credentials from the customer and it's working fine as below,

curl --request POST 'https://api.anz.prismacloud.io/login' -H 'Content-Type: application/json' --data '{"username":"xxxxxxxxxxxxxxxxxxxxxxxxx","password":"xxxxxxxxxxxxxxxx="}' {"message":"login_successful","token":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","customerNames":[{"customerName":"xxxxxxxxxxxxxxxxxxx","prismaId":"xxxxxxxxxxxxxxxxxxx","tosAccepted":true}]}

But it's keep failing from the integration. We can see the below logs from the elastic_agent logs,

[elastic_agent.filebeat][error] Error while processing http request: failed to collect first response: failed to execute http GET: server responded with status code 401: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

401 - Unauthorized: Access is denied due to invalid credentials.

401 - Unauthorized: Access is denied due to invalid credentials.

You do not have permission to view this directory or page using the credentials that you supplied.