backstage / backstage

Backstage is an open framework for building developer portals
https://backstage.io/
Apache License 2.0
27.02k stars 5.63k forks source link

Techdocs generation problem with permissions enabled #24934

Open pac0rro opened 1 month ago

pac0rro commented 1 month ago

📜 Description

I noticed that, when permissions enabled, the techdocs for components don't generate properly and the page turns blank with the progress indicator showing at top.

[1] 2024-05-27T17:22:45.301Z techdocs info Unable to get metadata for 'component:default/user-local-docs-test2' with error Error: Unable to read techdocs_metadata.json at /home/user/backstage/backstage-source-workspace/backstage-source/node_modules/@backstage/plugin-techdocs-backend/static/docs/default/component/user-local-docs-test2/techdocs_metadata.json. Error: Error: ENOENT: no such file or directory, open '/home/user/backstage/backstage-source-workspace/backstage-source/node_modules/@backstage/plugin-techdocs-backend/static/docs/default/component/user-local-docs-test2/techdocs_metadata.json'; caused by Error: ENOENT: no such file or directory, open '/home/user/backstage/backstage-source-workspace/backstage-source/node_modules/@backstage/plugin-techdocs-backend/static/docs/default/component/user-local-docs-test2/techdocs_metadata.json' type=plugin

I found that backstage tries to open that file a few seconds before the file is created:

$ stat /home/user/backstage/backstage-source-workspace/backstage-source/node_modules/@backstage/plugin-techdocs-backend/static/docs/default/component/user-local-docs-test2/techdocs_metadata.json
  File: /home/user/backstage/backstage-source-workspace/backstage-source/node_modules/@backstage/plugin-techdocs-backend/static/docs/default/component/user-local-docs-test2/techdocs_metadata.json
   Size: 2260            Blocks: 8          IO Block: 4096   regular file
Device: 820h/2080d      Inode: 1175341     Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/    user)   Gid: ( 1000/    user)
Access: 2024-05-27 19:22:48.299626310 +0200
Modify: 2024-05-27 19:22:48.059626390 +0200
Change: 2024-05-27 19:22:48.059626390 +0200
Birth: 2024-05-27 19:22:48.059626390 +0200

In app-config.yaml, techdocs are configured as "local":

techdocs:
  builder: 'local' # Alternatives - 'external'
  generator:
    runIn: 'local' # Alternatives - 'docker'
  publisher:
    type: 'local' # Alternatives - 'googleGcs' or 'awsS3'. Read documentation for using alternatives.
  cache:
    readTimeout: 5000

. . .

permission:
  enabled: true
  rbac:
    policies-csv-file: ../../rbac-policy.csv
    policyFileReload: true

Enabling LOG_LEVEL=debug I only see ALLOW permission checks like this:

Line 282: [1] 2024-05-27T17:22:42.091Z permission info user:default/userX is ALLOW for permission 'docs.read' and action read type=plugin

👍 Expected behavior

Documentation should be generated and rendered.

👎 Actual Behavior with Screenshots

An empty page with progress indicator shows forever:

image

👟 Reproduction steps

  1. Enable permissions
  2. Go to component page
  3. Go to docs
  4. Generation starts for two seconds and the related white page shows

📃 Provide the context for the Bug.

No response

🖥️ Your Environment

OS:   Linux 5.15.146.1-microsoft-standard-WSL2 - linux/x64
node: v18.20.3
yarn: 1.22.22
cli:  0.26.5 (installed)
backstage:  1.25.0

Dependencies:
  @backstage/app-defaults                                          1.5.5
  @backstage/backend-app-api                                       0.7.5
  @backstage/backend-common                                        0.22.0
  @backstage/backend-defaults                                      0.2.18
  @backstage/backend-dev-utils                                     0.1.4
  @backstage/backend-openapi-utils                                 0.1.11
  @backstage/backend-plugin-api                                    0.6.18
  @backstage/backend-tasks                                         0.5.23
  @backstage/backend-test-utils                                    0.3.8
  @backstage/catalog-client                                        1.6.5
  @backstage/catalog-model                                         1.5.0
  @backstage/cli-common                                            0.1.13
  @backstage/cli-node                                              0.2.5
  @backstage/cli                                                   0.26.5
  @backstage/config-loader                                         1.8.0
  @backstage/config                                                1.2.0
  @backstage/core-app-api                                          1.12.5
  @backstage/core-compat-api                                       0.2.5
  @backstage/core-components                                       0.14.7
  @backstage/core-plugin-api                                       1.9.2
  @backstage/e2e-test-utils                                        0.1.1
  @backstage/errors                                                1.2.4
  @backstage/eslint-plugin                                         0.1.8
  @backstage/frontend-plugin-api                                   0.6.5
  @backstage/integration-aws-node                                  0.1.12
  @backstage/integration-react                                     1.1.27
  @backstage/integration                                           1.11.0
  @backstage/plugin-api-docs                                       0.11.5
  @backstage/plugin-app-backend                                    0.3.67
  @backstage/plugin-app-node                                       0.1.18
  @backstage/plugin-auth-backend-module-atlassian-provider         0.1.10
  @backstage/plugin-auth-backend-module-aws-alb-provider           0.1.10
  @backstage/plugin-auth-backend-module-azure-easyauth-provider    0.1.1
  @backstage/plugin-auth-backend-module-bitbucket-provider         0.1.1
  @backstage/plugin-auth-backend-module-cloudflare-access-provider 0.1.1
  @backstage/plugin-auth-backend-module-gcp-iap-provider           0.2.13
  @backstage/plugin-auth-backend-module-github-provider            0.1.15
  @backstage/plugin-auth-backend-module-gitlab-provider            0.1.15
  @backstage/plugin-auth-backend-module-google-provider            0.1.15
  @backstage/plugin-auth-backend-module-guest-provider             0.1.4
  @backstage/plugin-auth-backend-module-microsoft-provider         0.1.13
  @backstage/plugin-auth-backend-module-oauth2-provider            0.1.15
  @backstage/plugin-auth-backend-module-oauth2-proxy-provider      0.1.11
  @backstage/plugin-auth-backend-module-oidc-provider              0.1.9
  @backstage/plugin-auth-backend-module-okta-provider              0.0.11
  @backstage/plugin-auth-backend                                   0.22.5
  @backstage/plugin-auth-node                                      0.4.13
  @backstage/plugin-auth-react                                     0.1.2
  @backstage/plugin-catalog-backend-module-github                  0.5.7
  @backstage/plugin-catalog-backend-module-scaffolder-entity-model 0.1.16
  @backstage/plugin-catalog-backend                                1.22.0
  @backstage/plugin-catalog-common                                 1.0.23
  @backstage/plugin-catalog-graph                                  0.4.5
  @backstage/plugin-catalog-import                                 0.10.10
  @backstage/plugin-catalog-node                                   1.12.0
  @backstage/plugin-catalog-react                                  1.12.0
  @backstage/plugin-catalog                                        1.20.0
  @backstage/plugin-events-backend                                 0.3.5
  @backstage/plugin-events-node                                    0.3.4
  @backstage/plugin-github-actions                                 0.6.16
  @backstage/plugin-jenkins-backend                                0.4.5
  @backstage/plugin-jenkins-common                                 0.1.26
  @backstage/plugin-jenkins                                        0.9.10
  @backstage/plugin-org                                            0.6.25
  @backstage/plugin-permission-backend-module-allow-all-policy     0.1.15
  @backstage/plugin-permission-backend                             0.5.42
  @backstage/plugin-permission-common                              0.7.13
  @backstage/plugin-permission-node                                0.7.29
  @backstage/plugin-permission-react                               0.4.22
  @backstage/plugin-proxy-backend                                  0.4.16
  @backstage/plugin-scaffolder-backend-module-azure                0.1.10
  @backstage/plugin-scaffolder-backend-module-bitbucket-cloud      0.1.8
  @backstage/plugin-scaffolder-backend-module-bitbucket-server     0.1.8
  @backstage/plugin-scaffolder-backend-module-bitbucket            0.2.8
  @backstage/plugin-scaffolder-backend-module-gerrit               0.1.10
  @backstage/plugin-scaffolder-backend-module-gitea                0.1.8
  @backstage/plugin-scaffolder-backend-module-github               0.2.8
  @backstage/plugin-scaffolder-backend-module-gitlab               0.4.0
  @backstage/plugin-scaffolder-backend                             1.22.7
  @backstage/plugin-scaffolder-common                              1.5.2
  @backstage/plugin-scaffolder-node                                0.4.4
  @backstage/plugin-scaffolder-react                               1.8.5
  @backstage/plugin-scaffolder                                     1.20.0
  @backstage/plugin-search-backend-module-catalog                  0.1.24
  @backstage/plugin-search-backend-module-pg                       0.5.27
  @backstage/plugin-search-backend-module-techdocs                 0.1.23
  @backstage/plugin-search-backend-node                            1.2.23
  @backstage/plugin-search-backend                                 1.5.9
  @backstage/plugin-search-common                                  1.2.11
  @backstage/plugin-search-react                                   1.7.11
  @backstage/plugin-search                                         1.4.11
  @backstage/plugin-sonarqube-backend                              0.2.20
  @backstage/plugin-sonarqube-react                                0.1.16
  @backstage/plugin-sonarqube                                      0.7.17
  @backstage/plugin-tech-radar                                     0.7.4
  @backstage/plugin-techdocs-backend                               1.10.5
  @backstage/plugin-techdocs-module-addons-contrib                 1.1.10
  @backstage/plugin-techdocs-node                                  1.12.4
  @backstage/plugin-techdocs-react                                 1.2.4
  @backstage/plugin-techdocs                                       1.10.5
  @backstage/plugin-user-settings                                  0.8.6
  @backstage/release-manifests                                     0.0.11
  @backstage/repo-tools                                            0.9.0
  @backstage/test-utils                                            1.5.5
  @backstage/theme                                                 0.5.5
  @backstage/types                                                 1.1.1
  @backstage/version-bridge                                        1.0.8
Done in 1.39s.

👀 Have you spent some time to check if this bug has been raised before?

🏢 Have you read the Code of Conduct?

Are you willing to submit PR?

None

igstbagusdharmaputra commented 1 month ago

It seems like me. I had a problem reloading the page docs and get from console log response -> http://localhost:7007/api/techdocs/.backstage/auth/v1/cookie HTTP ERROR 404

vinzscam commented 1 month ago

hey could you try to upgrade Backstage to see if it fixes the issue? I think there were some issues with TechDocs in v1.25

pac0rro commented 4 weeks ago

I've upgraded to v1.27.6 but the problem persists :-(

vinzscam commented 4 weeks ago

Do you have any idea where the mentioned log permission info user:default/userX is ALLOW for permission 'docs.read' and action read type=plugin is coming from? This is not coming from any of the core plugins. Are you customizing your permission policy? If yes, is catalogEntityReadPermission allowed?

pac0rro commented 3 weeks ago

Hi, Vincenzo.

Yes, this is a menu visibility check in App.tsx:

Yes, we allow "catalog.entity.read" permissions, as you can see in this piece of configuration (janus RBAC plugin):

...
p, role:default/admin_team, catalog.entity.read, read, allow
p, role:default/admin_team, catalog.entity.refresh, update, allow
p, role:default/admin_team, catalog.entity.delete, delete, allow
p, role:default/admin_team, catalog.entity.create, create, allow
...

The strange thing is that no DENY permissions are shown in logs.

igstbagusdharmaputra commented 3 weeks ago

I've upgraded to v1.27.6 but the problem persists :-(

maybe you can create open issue in repo janus-idp https://github.com/janus-idp/backstage-plugins i guess some issue with plugin rbac

pac0rro commented 2 weeks ago

maybe you can create open issue in repo janus-idp https://github.com/janus-idp/backstage-plugins i guess some issue with plugin rbac

I opened the clone issue at https://github.com/janus-idp/backstage-plugins/issues/1819

vinzscam commented 1 week ago

are you sure the issue doesn't occur if the permission framework is disabled? Honestly I I don't think the issue is related to the janus plugin or to the permission framework.

freben commented 1 week ago

Indeed. It tries to read from /home/user/backstage/backstage-source-workspace/backstage-source/node_modules/@backstage/plugin-techdocs-backend/static, which is INSIDE the backstage repo's node_modules. The @backstage/plugin-techdocs-backend package does not publish with any static directory so this was never expected to work. Did you set up your publisher as local and using a relative directory or something like that? Somethings seems off with the settings here.

pac0rro commented 1 week ago

are you sure the issue doesn't occur if the permission framework is disabled? Honestly I I don't think the issue is related to the janus plugin or to the permission framework.

Well, yes. Is so easy to change permissions to false and then the problem shows up.

pac0rro commented 1 week ago

This is my configuration for techdocs:

techdocs:
  builder: 'local' # Alternatives - 'external'
  generator:
    runIn: 'local' # Alternatives - 'docker'
  publisher:
    type: 'local' # Alternatives - 'googleGcs' or 'awsS3'. Read documentation for using alternatives.

Probably there are several wrong things from my part.
It could be caused by the old backend system / old frontend system? What is the best way to debug this issue inside VSCode?

pac0rro commented 1 week ago

... as local and using a relative directory or something like that? Somethings seems off with the settings here.

Related to what you commented I've found this documentation:


    type: 'local'

    # Optional when techdocs.publisher.type is set to 'local'.

    local:
      # (Optional). Set this to specify where the generated documentation is stored.
      publishDirectory: '/path/to/local/directory'

Do you think setting the techdocs.publisher.local.publishDirectory to a folder could fix the issue?

I will try it and let you know.

freben commented 1 week ago

yeah that would be interesting to see

pac0rro commented 6 days ago

Yes, next week I will play with those parameters to see if there is a successful docs generations. If not, I will try to debug this issue.