hashicorp / boundary

Boundary enables identity-based access management for dynamic infrastructure.
https://boundaryproject.io
Other
3.84k stars 287 forks source link

Boundary Desktop Issues with 0.15.0 #4370

Closed BoswellB35 closed 1 month ago

BoswellB35 commented 7 months ago

Describe the bug I've been testing the 0.15.0 release of Boundary, and have had trouble connecting to it via multiple versions of Boundary Desktop.

I initially tried connecting to my 0.15.0 controller using Version 1.7.1 of Boundary Desktop (which uses CLI Version 0.14.2). This produced the following error after entering my Cluster URL: Screenshot 2024-02-06 at 2 14 55 PM

I then updated to Boundary Desktop Version 2.0.0 (which uses CLI Version 0.15.0). This produced the following error after entering the Cluster URL: Screenshot 2024-02-06 at 1 56 38 PM

While the first error is not too surprising given a major version change, the second is perplexing as there is no newer version of the controller for me to use.

To Reproduce

  1. Deploy a controller and worker using version 0.15.0 of Boundary
  2. Using Boundary Desktop version 1.7.1, enter a Cluster URL and attempt to connect
  3. Repeat step 2 using Boundary Desktop version 2.0.0

Expected behavior Either version of Boundary Desktop should connect to the controller running Boundary version 0.15.0 and display the usual login prompt.

Additional context I am running a Boundary deployment with a single controller and worker in docker containers on Ubuntu instances, both on version 0.15.0. I've not made any configuration changes to my previous controller or worker deployment outside of updating the version, though I did not see anything in the release notes that suggested I would need to. I am running Boundary Desktop on macOS Sonoma Version 14.3, installed via direct download. My current production deployment of Boundary uses version 0.14.2; Desktop 1.7.1 works fine with that, while Desktop 2.0.0 produces the error in the second screenshot, which I would expect in that case.

zellio commented 7 months ago

Currently seeing the same issues in my customer base. I believe this issue from the controller logs is related (largely based on timing of log event, I haven't dug into the Controller code to verify)


{
    "id": "8fZb3SIAQW",
    "source": "https://hashicorp.com/boundary/ip-10-11-67-159.ec2.internal/controller",
    "specversion": "1.0",
    "type": "error",
    "data": {
        "error": "pagination.List: pagination.list: unknown, unknown: error #0: Status: 500, Kind: \"Internal\", Error: \"output fields not found when building scope proto\"",
        "error_fields": {
            "Code": 0,
            "Msg": "",
            "Op": "pagination.List",
            "Wrapped": {
                "Code": 0,
                "Msg": "",
                "Op": "pagination.list",
                "Wrapped": {}
            }
        },
        "id": "e_j3SAdXQcIm",
        "version": "v0.1",
        "op": "handlers.ErrorHandler",
        "request_info": {
            "id": "gtraceid_m74aXX1xkERqrZdmXOv1",
            "method": "GET",
            "path": "/v1/scopes?filter=(%22%2Fitem%2Ftype%22%20%3D%3D%20%22project%22)&recursive=true&scope_id=global",
            "public_id": "at_mQrSM3rcTg",
            "client_ip": "10.11.57.158"
        },
        "info": {
            "msg": "internal error returned"
        }
    },
    "datacontentype": "application/cloudevents",
    "time": "2024-02-07T16:03:37.400150659Z"
}
zellio commented 7 months ago

Successfully reproduced this issue via the CLI by having the client issue

$ boundary scopes list -recursive
Error from controller when performing list on scopes

Error information:
  Kind:                Internal
  Message:             output fields not found when building scope proto
  Status:              500
  context:             Error from controller when performing list on scopes

The issue seems to be tired to permissions in some way. Administrators with ids=*;type=*;actions=* are not seeing the issue but people with a smaller permissions set are. Currently my testing user has "ids=*;type=*;actions=read,no-op", on one of four orgs and no permissions on the others.

AdamBouhmad commented 7 months ago

Hey folks @zellio & @BoswellB35 -- thanks for using boundary & thanks for the updates here. We're investigating this internally and will provide an update once we have one

zellio commented 7 months ago

An interim solution we have found is to grant broad no-op access to all users on all scopes in the global scope

We use ldap groups for all of our users but roughly we have ended up with three roles

admins => "ids=*;type=*;actions=*"
users =>  [
  "type=scope;actions=list",
  "ids=*;type=scope;actions=read",
]
everyone else => "ids=*;type=*;actions=no-op"
william00179 commented 7 months ago

I'm also seeing this issue with Boundary 0.15.0 with Boundary desktop 2.0.0.

I'm seeing the same 500 error as @zellio has shown above but I'm not able to resolve the issue by giving the suggested permissions.

I'm unable to get beyond the unsupported controller version so tried to give the "ids=*;type=*;actions=no-op" to u_anon in the global scope but still see the same result.

@zellio do you have any more information about how you overcame the docker desktop issue?

zellio commented 7 months ago

@william00179 I haven't tested this issue in Docker. My testing and resolution is from a production deploy on AWS ECS. For reference though I had to make sure that all users had no-op on all scopes in the global scope

macmiranda commented 6 months ago

We're having similar issues with Boundary (OSS) controller version 0.15.2 and Boundary Desktop version 2.0.0-2.0.1. See attached error message.

image

The no-op grant did not make any difference. Boundary CLI works without any issues(including the one bundled with the Boundary.app).

Would be great to have some steps to debug this @AdamBouhmad

EDIT: After updating my grant strings to use ids=* instead of id=* and adding the type=scope;actions=list,no-op to u_anon at the global scope I'm now getting the same error reported by the author of this issue. 😞

AndreZiviani commented 6 months ago

We tried everything we could think of to fix this, only thing that worked was dropping the database and recreating everything (we manage everything via Terraform), after that it is working fine

I suggest starting a local boundary in dev mode and try to replicate your environment: same auth method, same roles, similar number of scopes/projects… we did that and everything worked as expected, that’s where we decided to drop the database

I know it’s not a good solution but that’s what we did

macmiranda commented 6 months ago

So I think this is related to the recursive parameter in scopesCheck below https://github.com/hashicorp/boundary-ui/blob/0902e42cb05ecbab9bd6549eba1f705c17541541/ui/desktop/app/routes/scopes.js#L26

When recursive is true, at least one of your orgs must allow unauthenticated scopes list, otherwise the Boundary API will respond with a 500 status code, which makes the check fail and Boundary Desktop show the modal (thinking that it's actually talking to an older version of the API).

BoswellB35 commented 6 months ago

We tried everything we could think of to fix this, only thing that worked was dropping the database and recreating everything (we manage everything via Terraform), after that it is working fine

I suggest starting a local boundary in dev mode and try to replicate your environment: same auth method, same roles, similar number of scopes/projects… we did that and everything worked as expected, that’s where we decided to drop the database

I know it’s not a good solution but that’s what we did

Following up on this I tore down my test environment, database and all, and started fresh on 0.15.2. Can confirm that after I was then able to connect to that deployment via Desktop version 2.0.1.

Obviously this would be a less-than-ideal solution for my production environment, even with things under Terraform control. Hoping we can see a fix that doesn't require dropping the database.

macmiranda commented 6 months ago

Should be fixed by https://github.com/hashicorp/boundary-ui/pull/2199

wichitalineman commented 5 months ago

This issue is still occurring in 0.15.4

$ boundary scopes list -recursive Error from controller when performing list on scopes

Error information: Kind: Internal Message: output fields not found when building scope proto Status: 500 context: Error from controller when performing list on scopes

$ boundary version

Version information: Build Date: 2024-04-09T14:00:40Z Git Revision: 782dfc0dae0df96b5fe0b8c4497a2d54357b4ed8 Version Number: 0.15.4

$Permissions for u_anon:
ID: u_anon Version: 1 Name: anonymous Description: The anonymous user matches any request, whether authenticated or not Authorized Actions: add-accounts set-accounts remove-accounts no-op read update delete

@AdamBouhmad Do you have any feedback on this issue?

AdamBouhmad commented 5 months ago

@wichitalineman what version is your Boundary Controller running on? 0.15.4? If so I can check & see if we can reproduce, as this should have been fixed with https://github.com/hashicorp/boundary-ui/pull/2199

william00179 commented 5 months ago

@AdamBouhmad I can confirm I'm still seeing this issue on 0.15.4 controller and workers with the latest Boundary Desktop app also.

Let me know what I can provide so you can reproduce the issue.

wichitalineman commented 5 months ago

@AdamBouhmad Yes, running 0.15.4. We upgraded from 0.14.1.

From the controller we see these logs after anyone logs on the Boundary desktop. The same error occurs with user/pass auth or oidc:

Apr 23 15:50:46 globalops-boundary-con-2 boundary[70721]: {"id":"89PVbwsV4m","source":"https://hashicorp.com/boundary/globalops-boundary-con-2/controller","specversion":"1.0","type":"error","data":{"error":"pagination.list: unknown, unknown: error #0: Status: 500, Kind: \"Internal\", Error: \"output fields not found when building scope proto\"","error_fields":{"Code":0,"Msg":"","Op":"pagination.list","Wrapped":{"Status":500,"Inner":{"kind":"Internal","message":"output fields not found when building scope proto"}}},"id":"e_TUrYFStUhZ","version":"v0.1","op":"pagination.list","request_info":{"id":"gtraceid_qOt9RqYpRFATlABeBvi4","method":"GET","path":"/v1/scopes","public_id":"at_7JXj9GadsT","client_ip":"10.91.204.209"}},"datacontentype":"application/cloudevents","time":"2024-04-23T15:50:46.591500449Z"}
Apr 23 15:50:46 globalops-boundary-con-2 boundary[70721]: {"id":"iIL0F2jQox","source":"https://hashicorp.com/boundary/globalops-boundary-con-2/controller","specversion":"1.0","type":"error","data":{"error":"pagination.List: pagination.list: unknown, unknown: error #0: Status: 500, Kind: \"Internal\", Error: \"output fields not found when building scope proto\"","error_fields":{"Code":0,"Msg":"","Op":"pagination.List","Wrapped":{"Code":0,"Msg":"","Op":"pagination.list","Wrapped":{"Status":500,"Inner":{"kind":"Internal","message":"output fields not found when building scope proto"}}}},"id":"e_TUrYFStUhZ","version":"v0.1","op":"pagination.List","request_info":{"id":"gtraceid_qOt9RqYpRFATlABeBvi4","method":"GET","path":"/v1/scopes","public_id":"at_7JXj9GadsT","client_ip":"10.91.204.209"}},"datacontentype":"application/cloudevents","time":"2024-04-23T15:50:46.591675209Z"}

We tested Boundary Desktop 1.7.1 and 2.0.2 and both had the same issue. Please let me know what further info I can provide.

macmiranda commented 5 months ago

The fix hasn't been released yet, only merged to main

william00179 commented 5 months ago

This is no longer happening with 0.16.0.

Thanks for the fix.

rfc2119 commented 2 months ago

Hello,

Sadly, we bumped into this issue on a production upgrade from 0.14.x to 0.15.4. Since Boundary CLI versions 0.13.x to 0.16.x were working ok with 0.15.4, we thought everything is smooth since the Desktop version essentially uses the CLI behind the scene. We had to revert the database schema migration by creating new database from a snapshot. I am glad OP has opened this issue.

If I understand this correctly, controller version 0.16.x fixes this issue. As unfortunately we have many active users using Desktop v1.6.0 (CLI 0.13.0) , we cannot simply update to controller 0.16.x in production. I appreciate if there is a backport for this fix.

AdamBouhmad commented 1 month ago

Hello @rfc2119,

Sorry for the delay in response here. We have backported the fix here, and it is now available in Boundary 0.15.5. To resolve the issue, you should be able to upgrade to 0.15.5 or higher. Please see the bottom of the known-issues section in 0.15.5 for more details.

For now, since we've resolved this issue, I will mark this ticket as closed.

Thanks everyone!