Very complex frontend routing configuration

[JumpLink]

After much trial and error, I think I now know why I failed to test the docker image of the frontend locally.

It took me a long time to understand how the frontend docker image works. Here is a summary:

When the frontend is started locally without docker:

• A local development server is started using vite

• Within this development server, a proxy is started which forwards all requests corresponding to the regular expression ^/(api|login|upload|uploads|favicon.png) to the URL API_PROXY_URL (which is http://localhost:3001 by default), this is the router of the backend docker compose stack which is accessible from outside via the IP 3001:

  vite.config.js:

  server: {
    port: 3000,
    // Proxy API requests to the backend
    proxy: {
      ‘^/(api|login|upload|uploads|favicon.png)’: {
        target: env.API_PROXY_URL,
        changeOrigin: true,
        cookieDomainRewrite: ‘localhost’,
      },
    },
  },

  The API_PROXY_URL is http://localhost:3001 by default.

  docker-compose.yml (for local dev):

    router:
      build:
        context: ../..
        dockerfile: ./apps/backend/Dockerfile
        target: router
      volumes:
        - ./built/static:/opt/built/static:ro
      ports:
        - ${MAIN_PORT}:80

  MAIN_PORT is 3001 by default, the backend can be accessed locally via this port.

• The production stack has a different configuration for this, the production stack has another router which forwards the requests to the frontend via nginx:

  
  location ~ ^/(assets|profile|contacts|callouts|join|auth|admin|_theme|embed.js) {
  # Proxy scrapers to legacy app for metadata
  if ($http_user_agent ~* "linkedinbot|googlebot|yahoo|bingbot|baiduspider|yandex|yeti|yodaobot|gigabot|ia_archiver|facebookexternalhit|twitterbot|developers\.google\.com") {
      proxy_pass $legacy_app/share?uri=$request_uri;
  }
  
  include sec_headers;
  add_header Content-Security-Policy "frame-ancestors ${TRUSTED_ORIGINS}";
  # add_header Content-Security-Policy "default-src 'self'; font-src 'self' data: https://use.typekit.net; style-src 'self' 'unsafe-inline'; script-src 'self' https://js.stripe.com; frame-src https://js.stripe.com; connect-src 'self' https://api.stripe.com";
  
  proxy_pass $frontend_app;
  }

• The development stack does not do this and does not provide the new frontend via nginx (I have since changed this), but at least forwards the api requests to:

  location ~ ^/api/(.*)$ {
      proxy_pass $api_app/$1$is_args$args;
  }

• The vite configuration also replaces some placeholder strings, including __appUrl__ with http://localhost:3000 which is the local address of the vite webserver itself:

  vite.config.js:

  plugins.push(
    replace({
      values: {
        __appUrl__: 'http://localhost:3000',
        __apiUrl__: env.API_BASE_URL,
        __revision__: 'dev',
        __version__: 'dev',
        __appsignalKey__: env.APPSIGNAL_KEY || '',
        __captchafoxKey__: env.CAPTCHAFOX_KEY || '',
        __maptilerKey__: env.MAPTILER_KEY || '',
        __cnrMode__: env.CNR_MODE || '',
        __experimentalFeatures__: env.EXPERIMENTAL_FEATURES || '',
      },
      preventAssignment: true,
    })
  );

• Something similar is done by a script which is executed in the docker image and is applied to all built *.js files:

  docker-entrypoint.sh:

  find /usr/share/nginx/html -type f -name '*.js' -exec \
  sed -i \
    -e s#__appUrl__#$APP_BASE_URL#g \
    -e s#__apiUrl__#$API_BASE_URL#g \
    -e s#__revision__#$APP_REVISION#g \
    -e s#__version__#$APP_VERSION#g \
    -e s#__appsignalKey__#$APPSIGNAL_KEY#g \
    -e s#__captchafoxKey__#$CAPTCHAFOX_KEY#g \
    -e s#__maptilerKey__#$MAPTILER_KEY#g \
    -e s#__cnrMode__#$CNR_MODE#g \
    -e s#__experimentalFeatures__#$EXPERIMENTAL_FEATURES#g \
  {} +

• These placeholders are used in the frontend in env.ts:

  export default {
    appUrl: '__appUrl__',
    apiUrl: '__apiUrl__',
    revision: '__revision__',
    version: '__version__',
    appsignalKey: '__appsignalKey__',
    captchafoxKey: '__captchafoxKey__',
    maptilerKey: '__maptilerKey__',
    cnrMode: '__cnrMode__',
    experimentalFeatures: '__experimentalFeatures__',
  };

• This env.appUrl ( which is the host) is not used in the frontend code, but the env.appUrl (which is the path /api/1.0) to make requests to the backend API:

  export const instance = axios.create({
    baseURL: env.apiUrl,
    withCredentials: true,
    headers: {
      'X-Beabee-Version': `${env.version} / ${env.revision}`,
    },
  });

  This complex setup is used to be able to provide the frontend build via Nginx instead of a webserver like vite.

• When a new release is made, the hive-deploy-stack is checked out via a github action, a bash script is executed (this and this) in it which checks whether the versions of the backend match the version of the frontend. If this is the case, the new versions are published in the hive-deploy-stack. The portainer instances check this regularly and use the new version.

Problems

1. One problem is that the frontend does not display a meaningful error message when the backend is not available:

2. Another problem is, that the env.appUrl is not used for the requests, so I can set whatever I want here, it won't change the fact that the backend is not accessible locally since the same host is always assumed. The way the frontend is currently implemented, everything must be accessible via the same domain, which is the domain via which the frontend is accessible. This is worked around in the production stack by an Nginx configuration, just as it is the case on the devlopment stack with the proxy in vite .

All API requests use the frontend host itself, which also means that no request errors occur, everything returns 200 even though the backend is not accessible:

5. If I want to work on the frontend code to fix this problem, I can mount my local frontend build into the docker image, but then the necessary placeholders will not be replaced. So I have to build the container from scratch every time:

   frontend:
     build:
       context: ../..
       dockerfile: ./apps/frontend/Dockerfile
     restart: unless-stopped
     # Not working because docker-entrypoint.sh is not executed
     # volumes:
     #   - ../../apps/frontend/dist:/usr/share/nginx/html
     environment:
       APP_BASE_URL: ${BEABEE_AUDIENCE}
       API_BASE_URL: /api/1.0
       APP_REVISION: dev
       APP_VERSION: dev
       APPSIGNAL_KEY: ""
       CAPTCHAFOX_KEY: ""
       MAPTILER_KEY: ""
       CNR_MODE: ""
       EXPERIMENTAL_FEATURES: ""

6. The router used in the production stack has its own repository and I had not yet dealt with it, so it was completely off my radar. The development environment uses the vite proxy for that which is included in the repository, but not the production router with his nginx config for that.

7. Checking out the hive-deploy-stack, executing a script in it and then making automatic commits in it also increases the complexity and is not documented anywhere.

8. The release scripts are bash scripts, which means that another language is used.

Proposed solution

Due to this complex setup, the problems described and the differences between the development stack and the production, it was very difficult to identify the problem here. New developers are likely to have the same problems, which makes it very difficult to get started. I also asked @wpf500 how I could test the frontend docker image locally and he didn't seem to have that in mind either. Instead, I would rethink the entire setup; I don't think the development stack should be too different from the production stack.

The routers for development and for productive hosting should be merged and usable locally and productively via environment variables.

The hive-deploy-stack should also not differ too much from the development environment and could instead be added to the monorepo and also be used for local development.

I would suggest the following steps:

• Remove the old frontend as quickly as possible
• Move the static files to the new frontend
• API requests should also be possible on other domains
• The hive-deploy-stack repo will be part of the monorepo
• The beabee-router repo will be part of the monorepo
• Than the nginx.conf can and should be merged and simplified
• The problem with the environment placeholders could be solved by vite and the docker container using the same .env file for the string replacement, so vite would exchange the placeholders with the same values. This would allow the local build files to be loaded into the docker image during the development process.
• The proxy is removed from vite and no longer necessary, as the backend can also be accessed via a different host
• The API client should have meaningful and helpful error messages
• Move the API client from the Telegram bot to this monorepo, since it is already part of a monorepo, so that should be easy. Later, it could be replaced or extended with OpenAPI. It works in Deno, in the browser and in Node and has no external dependencies.
• Thanks to the monorepo, the release process can also be simplified. Instead of checking out the hive-deploy-stack repo and committing something in it, we should summarise the release process in a JavaScript or TypeScript script. This would also eliminate another language (bash). This could still be automated using via github actions (like now) and/or with tools such as release-it.
• Instead of checking whether the version numbers of the frontend and backend should match, simply always publish all packages/apps with the same version number, even if no changes have been made to a package. This way, everyone always knows that these versions are working together.

With release-it we could:

• Automatically bump to the next patch, minor or major version for all packages
• Automatically create a GitHub Release with an automatically generated changelog (just like you can with GitHub itself)
• Automatically publish the new packages to npm
• Automatically update a CHANGELOG.md
• Automatically create and publish docker images
• Much more using existing plugins
• Or by write your own plugin
• This could also be done with different GitHub Actions, but the advantage of release-it is that it is platform-independent, also works with GitLab and everything can be managed centrally rather than being scattered. release-it itself could also be triggered by a GitHub Action

Temporary solution

I have now adapted the docker-compose.yml and nginx.conf so that I can at least test the frontend docker image locally. I have checked the whole thing into the repo so that we can continue to test locally, even though this is not a clean, permanent solution and the image needs to rebuild each time when new changes to the frontend should be tested.

[wpf500]

Thanks for documenting this. As you say, it's a confusing setup!

It would be great to unify the development and production setup, I think I agree with most of the things you've said. Let's have a look at it together tomorrow.

One thing to note though: I don't think we can remove hive-deploy-stack unfortunately. Portainer checks this repo out per stack and you can't tell it to only check out certain files, so if we changed it to point at monorepo it would check out the entire repo for each client.