Be able to load the json config file for the Homepage and TechRadar from Git (ie GitHub/GitLab) #553

Closed christophe-f closed 1 year ago

What do you want to improve?

Be able to load the json config file for the Homepage and TechRadar directly from Git without the Nodejs app.

What is the current behavior?

Currently the config files need to be deployed with a NodeJS app to be accessible from the Backstage instance.

What is the new behavior?

Should be able to get the config directly from Git.

Additional Info: The current solution was implemented because the proxy was expecting an 'application/json' response and this was not returned from Git.

It seems that it is possible by passing the query params in the fetch request instead of the proxy. For example:

app-config.yaml:

'/quickaccess':
  target: https://github.com/janus-idp/backstage-showcase/raw/main/packages/app/public/homepage/data.json

useQuickAccess.ts for GitLab:

await fetch( ${url}/raw?ref=main);

Hi @christophe-f, I was looking at this and I have a few thoughts/questions.

so do we need to have two new entries? one for Home and one for tech radar i.e.

 '/quickaccess':
       target: https://github.com/janus-idp/backstage-showcase/raw/main/packages/app/public/homepage/data.json
       headers: // optional but might be useful in some scenarios
          <header_key>: <header_value>
          ...
 '/techradar':
       target: https://raw.githubusercontent.com/janus-idp/backstage-showcase/main/packages/app/public/tech-radar/data-default.json
       headers:
          <header_key>: <header_value>
          ...
   //In future this can be extended for anything extra

So with the above approach we would be making direct fetch calls over going a proxy route which we implemented in the past with https://github.com/janus-idp/backstage-showcase/pull/351/ . So will both co-exist? or it'll just be fetch going forward and with this effort do we need to remove proxy support? or have both for now and decide later based on input from users

ideal would be to make use of the existing backed proxy for git as well so that it can serve all cases. As there could be private git as well or if someone wants to have it powered with service. Playing more with it

Update:

We can use the same proxy to support/serve raw files from GitHub as below

Signature:

proxy:
  endpoints:
    # Other Proxies
    # customize developer hub instance
    '/developer-hub':
      target: <DOMAIN_URL> # i.e https://raw.githubusercontent.com/
      pathRewrite:
        '^/api/proxy/developer-hub/tech-radar
': <path to json file> # i.e /janus-idp/backstage-showcase/main/packages/app/public/tech-radar/data-default.json
    '^/api/proxy/developer-hub': <path to json file> # i.e /janus-idp/backstage-showcase/main/packages/app/public/homepage/data.json
      changeOrigin: true
      secure: true
      # Change to "false" in case of using self hosted cluster with a self-signed certificate
      headers:
    <HEADER_KEY>: <HEADER_VALUE> # optional and can be passed as needed i.e Authorization can be passed for private GitHub repo and PRIVATE-TOKEN can be passed for private GitLab repo

Tip: for GitLab private repo

curl --header "PRIVATE-TOKEN: <YOUR_ACCESS_TOKEN>" \
"https://gitlab.com/api/v4/projects/<project_id>/repository/files/<file_name>/raw?ref=<branch>"

Working config:

proxy:
  endpoints:
    '/developer-hub':
      target: https://raw.githubusercontent.com/
      pathRewrite:
        '^/api/proxy/developer-hub/tech-radar': '/janus-idp/backstage-showcase/main/packages/app/public/tech-radar/data-default.json'
        '^/api/proxy/developer-hub': '/janus-idp/backstage-showcase/main/packages/app/public/homepage/data.json'
      changeOrigin: true
      secure: true

Backstage proxy doesn't directly serve files such as default.json and gives 400 if whole URL is passed as the target but the user can make use of pathRewrite and with this can update both home and tech-radar

@debsmita1 can you pls check this if it works as expected?

I have verified this with GitHub and GitLab both with public and private repositories. For private repositories, users would need to pass the appropriate headers with the access token

Thanks @debsmita1. Marking this ticket as closed, follow https://github.com/janus-idp/backstage-showcase/issues/553#issuecomment-1750284006 to fetch raw files from git.

janus-idp / backstage-showcase