search

spreadshirt / backstage-plugin-s3

An S3 Viewer plugin for Backstage! 🪣
Apache License 2.0
25 stars 3 forks source link

Version @spreadshirt/backstage-plugin-s3-viewer-backend@0.8.0 breaks the frontend #92

Closed zppinho closed 6 months ago

zppinho commented 7 months ago

Overview

The backend plugin in version 0.8.0 appears to be malfunctioning after the latest release. It can fetch the buckets as expected and shown in the logs, however, they are not being displayed in the frontend. Instead, a 404 error is displayed. This error is not encountered in version 0.7.6, the same behavior of the backend finding the buckets and not displaying occurs in a production environment.

Environment

Locally with (fedora:39) Production (Kubernetes:1.24.10 - @backstage/create-app@0.5.10)

Steps to Reproduce

app-config.yaml used

# Stub app config needed to run with yarn start
app:
  title: backstage example app
  baseUrl: http://localhost:3000

backend:
  baseUrl: http://localhost:7007
  listen:
    port: 7007

permission:
  enabled: true

s3:
  bucketLocatorMethods:
    - type: config
      platforms:
        - endpoint: http://127.0.0.1:8080
          name: test
          accessKeyId: 70VkRWd3IHDxEafKZFX9 # Has to match `CEPH_DEMO_ACCESS_KEY` when using docker compose
          secretAccessKey: v0GerzwTw0cD2Dcq4m0aGeNzQVnpyzc0zW4Mc05A # Has to match `CEPH_DEMO_SECRET_KEY` when using docker compose
          region: 'us-east-1'
  allowedBuckets:
    - platform: test
      buckets:
        - foobar

Expected

The bucket should be seen in backstage frontend

Actual

Frontend returns a message with "Request failed for buckets/grouped, 404 Not Found"

Logs

Logs from the container with the bucket

Restarting RESTful API server...
c2752236-ef0d-44ef-92c1-f4efdb3f4190
changed ownership of '/etc/ceph/ceph.client.crash.keyring' from root:root to ceph:ceph
ownership of '/var/lib/ceph/crash/posted' retained as ceph:ceph
ownership of '/var/lib/ceph/crash' retained as ceph:ceph
nohup: ignoring input and appending output to 'nohup.out'
2024-04-05 16:31:00  /opt/ceph-container/bin/entrypoint.sh: SUCCESS
exec: PID 4078: spawning ceph --cluster ceph -w
exec: Waiting 4078 to quit
  cluster:
    id:     3915a179-4e6e-4511-9f25-b60b7b658423
    health: HEALTH_OK

  services:
    mon:        1 daemons, quorum bac395a105be (age 2m)
    mgr:        bac395a105be(active, since 2m)
    mds:        1/1 daemons up
    osd:        2 osds: 2 up (since 2m), 2 in (since 2m); 1 remapped pgs
    rbd-mirror: 1 daemon active (1 hosts)
    rgw:        1 daemon active (1 hosts, 1 zones)
    rgw-nfs:    1 daemon active (1 hosts, 1 zones)

  data:
    volumes: 1/1 healthy
    pools:   9 pools, 233 pgs
    objects: 238 objects, 579 KiB
    usage:   53 MiB used, 200 GiB / 200 GiB avail
    pgs:     233 active+clean

  io:
    recovery: 37 B/s, 1 objects/s

10.0.2.100 - - [05/Apr/2024 16:32:12] "GET /buckets.html HTTP/1.1" 200 -
10.0.2.100 - - [05/Apr/2024 16:32:12] "GET /css/bootstrap.min.css HTTP/1.1" 200 -
10.0.2.100 - - [05/Apr/2024 16:32:12] "GET /css/font-awesome.min.css HTTP/1.1" 200 -
10.0.2.100 - - [05/Apr/2024 16:32:12] "GET /css/style.css HTTP/1.1" 200 -
10.0.2.100 - - [05/Apr/2024 16:32:12] "GET /css/jquery.dataTables.min.css HTTP/1.1" 200 -
10.0.2.100 - - [05/Apr/2024 16:32:12] "GET /css/validationEngine.jquery.css HTTP/1.1" 200 -
10.0.2.100 - - [05/Apr/2024 16:32:12] "GET /image/ceph-nano-logo-horizontal.svg HTTP/1.1" 200 -
10.0.2.100 - - [05/Apr/2024 16:32:12] "GET /js/lib/require.js HTTP/1.1" 200 -
10.0.2.100 - - [05/Apr/2024 16:32:12] "GET /js/lib/require.config.js HTTP/1.1" 200 -

Logs from yarn start in branch @spreadshirt/backstage-plugin-s3-viewer-backend@0.8.0

 xxx@fedora  ~/Downloads/backstage-plugin-s3  ◈ @spreadshirt/backstage-plugin-s3-viewer-backend@0.8.0  yarn start
yarn run v1.22.22
$ concurrently "yarn start:backend" "yarn start:frontend"
$ yarn workspace @spreadshirt/backstage-plugin-s3-viewer-backend start
$ yarn workspace @spreadshirt/backstage-plugin-s3-viewer start
$ LEGACY_BACKEND_START=1 backstage-cli package start
$ backstage-cli package start
[0] Build succeeded
[1] Loaded config from app-config.yaml
[1] <i> [webpack-dev-server] Project is running at:
[1] <i> [webpack-dev-server] Loopback: http://localhost:3000/, http://[::1]:3000/
[1] <i> [webpack-dev-server] Content not from webpack is served from '/home/xxx/Downloads/backstage-plugin-s3/plugins/s3-viewer/public' directory
[1] <i> [webpack-dev-server] 404s will fallback to '/index.html'
[1] <i> [webpack-dev-middleware] wait until bundle finished: /
[0] Loaded config from app-config.yaml
[0] 2024-04-05T17:01:38.269Z s3-viewer-backend info Found 1 new secrets in config that will be redacted 
[0] 2024-04-05T17:01:38.282Z s3-viewer-backend warn Generated a secret for service-to-service authentication: DEVELOPMENT USE ONLY. 
[0] 2024-04-05T17:01:38.283Z s3-viewer-backend info Initializing S3 backend 
[0] 2024-04-05T17:01:38.285Z s3-viewer-backend info Fetching S3 buckets... 
[0] 2024-04-05T17:01:38.311Z s3-viewer-backend info CORS is disabled, allowing all origins 
[0] 2024-04-05T17:01:38.325Z backstage info Listening on :7007 
[0] 2024-04-05T17:01:38.443Z s3-viewer-backend info Fetched 1 S3 buckets 
[1] webpack compiled successfully
[0] 2024-04-05T17:01:48.553Z backstage info ::ffff:127.0.0.1 - - [05/Apr/2024:17:01:48 +0000] "GET /api/s3-viewer/buckets/grouped? HTTP/1.1" 404 - "http://localhost:3000/" "Mozilla/5.0 (X11; Linux x86_64; rv:124.0) Gecko/20100101 Firefox/124.0" type=incomingRequest
[0] 2024-04-05T17:01:55.270Z backstage info ::ffff:127.0.0.1 - - [05/Apr/2024:17:01:55 +0000] "GET /api/s3-viewer/buckets/grouped? HTTP/1.1" 404 - "http://localhost:3000/" "Mozilla/5.0 (X11; Linux x86_64; rv:124.0) Gecko/20100101 Firefox/124.0" type=incomingRequest

Screenshot from 2024-04-05 17-38-20

Logs from yarn start in branch @spreadshirt/backstage-plugin-s3-viewer-backend@0.7.6

xxx@fedora  ~/Downloads/backstage-plugin-s3  ◈ @spreadshirt/backstage-plugin-s3-viewer-backend@0.7.6  yarn start                                              
yarn run v1.22.22
$ concurrently "yarn start:backend" "yarn start:frontend"
$ yarn workspace @spreadshirt/backstage-plugin-s3-viewer-backend start
$ yarn workspace @spreadshirt/backstage-plugin-s3-viewer start
$ LEGACY_BACKEND_START=1 backstage-cli package start
$ backstage-cli package start
[0] Build succeeded
[1] Loaded config from app-config.yaml
[1] <i> [webpack-dev-server] Project is running at:
[1] <i> [webpack-dev-server] Loopback: http://localhost:3000/, http://[::1]:3000/
[1] <i> [webpack-dev-server] Content not from webpack is served from '/home/xxx/Downloads/backstage-plugin-s3/plugins/s3-viewer/public' directory
[1] <i> [webpack-dev-server] 404s will fallback to '/index.html'
[1] <i> [webpack-dev-middleware] wait until bundle finished: /
[0] Loaded config from app-config.yaml
[0] 2024-04-05T16:33:30.677Z s3-viewer-backend info Found 1 new secrets in config that will be redacted 
[0] 2024-04-05T16:33:30.688Z s3-viewer-backend warn Generated a secret for service-to-service authentication: DEVELOPMENT USE ONLY. 
[0] 2024-04-05T16:33:30.689Z s3-viewer-backend info Initializing S3 backend 
[0] 2024-04-05T16:33:30.690Z s3-viewer-backend info Fetching S3 buckets... 
[0] 2024-04-05T16:33:30.704Z s3-viewer-backend info CORS is disabled, allowing all origins 
[0] 2024-04-05T16:33:30.719Z backstage info Listening on :7007 
[0] 2024-04-05T16:33:30.840Z s3-viewer-backend info Fetched 1 S3 buckets 
[1] webpack compiled successfully
[0] 2024-04-05T16:33:42.037Z backstage info ::1 - - [05/Apr/2024:16:33:42 +0000] "POST /api/permission/authorize HTTP/1.1" 200 74 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)" type=incomingRequest
[0] 2024-04-05T16:33:42.044Z backstage info ::ffff:127.0.0.1 - - [05/Apr/2024:16:33:42 +0000] "GET /api/s3/buckets/grouped? HTTP/1.1" 304 - "http://localhost:3000/" "Mozilla/5.0 (X11; Linux x86_64; rv:124.0) Gecko/20100101 Firefox/124.0" type=incomingRequest
[0] 2024-04-05T16:33:59.251Z backstage info ::1 - - [05/Apr/2024:16:33:59 +0000] "POST /api/permission/authorize HTTP/1.1" 200 74 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)" type=incomingRequest
[0] 2024-04-05T16:33:59.256Z backstage info ::ffff:127.0.0.1 - - [05/Apr/2024:16:33:59 +0000] "GET /api/s3/buckets/grouped? HTTP/1.1" 304 - "http://localhost:3000/" "Mozilla/5.0 (X11; Linux x86_64; rv:124.0) Gecko/20100101 Firefox/124.0" type=incomingRequest
^C[0] 2024-04-05T16:35:06.221Z backstage info CTRL+C pressed; exiting. 
[1] <i> [webpack-dev-server] Gracefully shutting down. To force exit, press ^C again. Please wait...

Screenshot from 2024-04-05 17-38-11

Aditional info

In version 0.7.6 in a production environment there seems to be another issue regarding permissions, could you provide a working "app-config.yaml" in a production environment.

ivangonzalezacuna commented 7 months ago

Hi. Thanks for reporting!

Can you also provide the backstage version you're using? We are already aware that the latest release of Backstage if breaking this plugin due to all the permissions setup. This is currently in progress and under-review here: https://github.com/spreadshirt/backstage-plugin-s3/pull/90

However, the error described might be related to something else. I will test it first. However, I can say we are using the latest version for the backend plugin in our backstage instance in production and the listing is working fine. The main differences I can see is that you are using the config setup while we use the radosgw-admin setup, and that the permissionMiddleware is not enabled in your example.

And regarding the issue you're facing due to the permissions issue. If you're running the latest Backstage version (1.24 or 1.25), sadly you'll need to wait until we release the changes in the PR mentioned above. If you're running an older version, I believe you need to set the middleware properly. The documentation should explain how to do that (https://github.com/spreadshirt/backstage-plugin-s3/tree/main/plugins/s3-viewer-backend#permissions-setup)

ivangonzalezacuna commented 7 months ago

Alright, I found the issue. But the problem should only be happening with the yarn start setup I think. The issue is that the standaloneServer is creating the wrong route: https://github.com/spreadshirt/backstage-plugin-s3/blob/main/plugins/s3-viewer-backend/src/service/standaloneServer.ts#L52. Instead, it should be "/s3-viewer". This is something that has already been tackled in the MR supporting the latest backstage version.

zppinho commented 7 months ago

I was using the latest version of Backstage, I believe 1.25.0

"app@link:packages/app":
  version "0.0.0"
  dependencies:
    "@backstage/app-defaults" "^1.5.3"
    "@backstage/catalog-model" "^1.4.5"
    "@backstage/cli" "^0.26.2"
    "@backstage/core-app-api" "^1.12.3"
    "@backstage/core-components" "^0.14.3"
    "@backstage/core-plugin-api" "^1.9.1"
    "@backstage/integration-react" "^1.1.25"
    "@backstage/plugin-api-docs" "^0.11.3"
    "@backstage/plugin-catalog" "^1.18.2"
    "@backstage/plugin-catalog-common" "^1.0.22"
    "@backstage/plugin-catalog-graph" "^0.4.3"
    "@backstage/plugin-catalog-import" "^0.10.9"
    "@backstage/plugin-catalog-react" "^1.11.2"
    "@backstage/plugin-github-actions" "^0.6.14"
    "@backstage/plugin-org" "^0.6.23"
    "@backstage/plugin-permission-react" "^0.4.21"
    "@backstage/plugin-scaffolder" "^1.19.2"
    "@backstage/plugin-search" "^1.4.9"
    "@backstage/plugin-search-react" "^1.7.9"
    "@backstage/plugin-tech-radar" "^0.7.2"
    "@backstage/plugin-techdocs" "^1.10.3"
    "@backstage/plugin-techdocs-module-addons-contrib" "^1.1.8"
    "@backstage/plugin-techdocs-react" "^1.2.2"
    "@backstage/plugin-user-settings" "^0.8.4"
    "@backstage/theme" "^0.5.2"
    "@material-ui/core" "^4.12.2"
    "@material-ui/icons" "^4.9.1"
    "@spreadshirt/backstage-plugin-s3-viewer" "^0.5.0"
    history "^5.0.0"
    react "^18.0.2"
    react-dom "^18.0.2"
    react-router "^6.3.0"
    react-router-dom "^6.3.0"
    react-use "^17.2.4"
zppinho commented 7 months ago

backstage created with 0.5.11 and the fix mentioned worked! thanks @ivangonzalezacuna

ivangonzalezacuna commented 7 months ago

Alright. In that case, please bump the plugin to the latest version. We have just released it.

Also, please note that there are a couple possible breaking changes that you will need to apply in order to make it work. Basically those changes are affecting the permission setup: the middleware is not needed any longer and the way to set the cookie has changed a little. If you're using the new backend system all the other changes will be completely transparent for you, but you might need some more changes if you're using the old backend setup.

backstage created with 0.5.11 and the fix mentioned worked! thanks @ivangonzalezacuna

Awesome!

zppinho commented 6 months ago

Alright, I found the issue. But the problem should only be happening with the yarn start setup I think. The issue is that the standaloneServer is creating the wrong route: https://github.com/spreadshirt/backstage-plugin-s3/blob/main/plugins/s3-viewer-backend/src/service/standaloneServer.ts#L52. Instead, it should be "/s3-viewer". This is something that has already been tackled in the MR supporting the latest backstage version.

Closing this as the solution solves the issue.