Bitwarden integration not working with self-hosted server

[Marcussjogren]

Hi,

I have a self-hosted bitwarden server, lets say located at https://bitwarden.mydomain.com.

I am trying to use the bitwarden integration in Elastic but it does not seem to work, because I continuously get errors in the logs of the elastic agent.

If i go to https://bitwarden.mydomain.com/api/version it gives me "2024.5.0" as response, so that seems to be the proper URL for the API.

The error that I see is this: Error while processing http request: failed to collect first response: failed to execute http GET: Get "mybitwarden.mydomain.com/api/public/events?end=2024-05-30T13%3A12%3A31.696Z&start=2024-05-29T13%3A12%3A31.696Z": oauth2: "invalid_scope"

The configuration that I have entered in the integration is:

URL: https://bitwarden.mydomain.com/api client ID: <id from "view API key">> Client password: <password from "view API key"> token URL: https://bitwarden.mydomain.com/identity/connect/token

Any ideas what might be wrong?

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[efd6]

@Marcussjogren Have you been able to connect to the API using cURL as shown in the Bitwarden documentation here?

curl -X POST \
  https://bitwarden.mydomain.com/identity/connect/token \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'grant_type=client_credentials&scope=api.organization&client_id=<ID>&client_secret=<SECRET>'

and then

curl -X GET \
  https://api.bitwarden.com/public/collections \
  -H 'Authorization: Bearer <TOKEN>'

where <TOKEN> will be given by the first API request's response.

[Marcussjogren]

@Marcussjogren Have you been able to connect to the API using cURL as shown in the Bitwarden documentation here?

curl -X POST \
  https://bitwarden.mydomain.com/identity/connect/token \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'grant_type=client_credentials&scope=api.organization&client_id=<ID>&client_secret=<SECRET>'

and then

curl -X GET \
  https://api.bitwarden.com/public/collections \
  -H 'Authorization: Bearer <TOKEN>'

where <TOKEN> will be given by the first API request's response.

Hi and thank you for your effort!

If I am running the request suggested by you and the manual, I receive the error: `{"error":"invalid_grant","ErrorModel":{"Message":"No device information provided.","Object":"error"}}

administrator@myserver:~$ curl -X POST \
  https://my-bitwarden-url/identity/connect/token \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'grant_type=client_credentials&scope=api&client_id=<ID>&client_secret=<SECRET>'
{"error":"invalid_grant","ErrorModel":{"Message":"No device information provided.","Object":"error"}}

But after googling a bit more on the error message I found out that it is working if I run the below command instead and I get an access-token along with a private key and some other information.

curl -X POST \
  https://my-bitwarden-url/identity/connect/token --data-urlencode 'deviceName=fireFox' --data-urlencode 'twoFactorToken=0' --data-urlencode 'deviceIdentifier=0' --data-urlencode 'deviceType=0'\
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'grant_type=client_credentials&scope=api&client_id=<ID>&client_secret=<SECRET>'

However, when I run the below command, I do not receive any response at all.

curl -X GET \
  https://my-bitwarden-url/api/public/collections \
  -H 'Authorization: Bearer <TOKEN>'

[Marcussjogren]

and my no response at all, I mean that it is just blank, the server connection works.

[Marcussjogren]

I've been deep diving a bit into the bitwarden setup, and I found some logs which indicates that the Bitwarden Integration is trying to use "scope=api.organization" while I would require "scope=api".

Will try to see if it is possible to change that behaviour.

This is from the /opt/bitwarden/bwdata/logs/identity/20240605.txt:

2024-06-05 15:23:03.111 +00:00 [Error] Client "<USERNAME>" is not allowed access to scope "api.organization".

[fgierlinger]

The api.organization scope is hardcode in the httpjson.yml.hbs file. I can confirm @Marcussjogren tests, that changing the scope from api.organizations to api solves the problem.

https://github.com/elastic/integrations/blob/1f780eeb453adb07244c4203e2376ab060f64d6a/packages/bitwarden/data_stream/collection/agent/stream/httpjson.yml.hbs#L18-L24

[efd6]

@Marcussjogren the value that we have used for the scopes is taken from the docs (as much as is possible — the only place I could find that was in the document that I linked to). Is there a setting in your set-up for setting the scopes? Is this something that differs between the self-hosted and cloud offerings for bitwarden? I do see that "api" is described here.

@fgierlinger Thank you for filing that issue.

[efd6]

I have sent #10097 which will allow you to set the scope. From a search of the Bitwarden docs only the two scope appear to be used, but it would be good to see what the outcome of the bitwarden/server issue is.

[Marcussjogren]

@Marcussjogren the value that we have used for the scopes is taken from the docs (as much as is possible — the only place I could find that was in the document that I linked to). Is there a setting in your set-up for setting the scopes? Is this something that differs between the self-hosted and cloud offerings for bitwarden? I do see that "api" is described here.

@fgierlinger Thank you for filing that issue.

@efd6 - Thanks for your response! No, the scope seems to be a setting from Bitwarden, where api.organisation gives you access via the organizations API-key and scope=api gives you access via your accounts API-key. Since I'm alone on my installation, I don't have an organization but only my own account.

Though - I think the Bitwarden documentation falls a bit short on this and it does not state what the difference is in terms of access etc so I cannot 100% confirm that the Integration will give us the neccessary data via "scope=api".

I will try to do some tests and see what I can get with the api-scope.

[Marcussjogren]

The api.organization scope is hardcode in the httpjson.yml.hbs file. I can confirm @Marcussjogren tests, that changing the scope from api.organizations to api solves the problem.

https://github.com/elastic/integrations/blob/1f780eeb453adb07244c4203e2376ab060f64d6a/packages/bitwarden/data_stream/collection/agent/stream/httpjson.yml.hbs#L18-L24

Big thank you to @fgierlinger for helping out so fast, but also that you raised the question with Bitwarden on the "DeviceType" attributes. Truly a great community behaviour :)

[efd6]

@Marcussjogren Before I merge the PR I've sent, I would like to understand whether there will be a case that more than one scope will be required at a time (the change I have assumes not). From a search of their docs, it seems that the "api" scope is for personal instances and the "api.organization" is for org-level instances, but it would be good to have this confirmed.

[Marcussjogren]

@Marcussjogren Before I merge the PR I've sent, I would like to understand whether there will be a case that more than one scope will be required at a time (the change I have assumes not). From a search of their docs, it seems that the "api" scope is for personal instances and the "api.organization" is for org-level instances, but it would be good to have this confirmed.

Yes - that is exactly my understanding as well. However - at the moment i receive a "403 - Forbidden" when i try to list collections or events using scope=api. I saw your merge, and as far as I could tell, if we just confirm that it works as we think it does, it will do the job!

Trying to troubleshoot why I receive the "403 - Forbidden" at the moment, will update as soon as I get anything out of it but currently I'm running short because I cannot find any logs that explains why.

It might even be that the personal API is useless, but then I don't understand why it's there at all :-)

[Marcussjogren]

I have sent my question to their support, will update once I receive response.

[Marcussjogren]

Hi guys!

Sorry for the late reply.

I received the below response from Bitwarden support. From what I can understand, there is only limited amounts of data that you can collect via the API if you ar running any of the personal licenses (Personal Premium or Family Premium).

One of the biggest concerns is that you cannot retrieve event logs but you can retrieve collections, so it will be pretty useless to use this integration unless you are using their Enterprise license in my opinion. Though, it is so expensive that it is not relevant in my case. I will on the other hand start to investigate which log files might actually be worth logging manually instead.

It might be worth to inform in the integration description that an Enterprise license is required for the integration to function. You could also add to the integration documentation that there are different URL's to be used when using cloud versus on-premise.

Cloud API: https://api.bitwarden.com or https://api.bitwarden.eu Cloud auth URL: https://identity.bitwarden.com/connect/token or https://identity.bitwarden.eu/connect/token

Self hosted API: https://my-bitwarden-url/api/ Self hosted auth URL: https://my-bitwarden-url/identity/connect/token

Hi Marcus,

 Thank you for your feedback. 

In the family account, you have access to collections but do not have access to event logs, so yes, if you purchase the family plan, there is no acces to event logs, where you can see failed login or different user actions.

[efd6]

Thankyou @Marcussjogren. So can I confirm that we can close the PR I linked above; I will send a new PR adding the suggested documentation changes.