search

deephaven / web-client-ui

Deephaven Web Client UI
Apache License 2.0
29 stars 31 forks source link
deephaven

readme

Web Javascript packages

We're using a monorepo to manage our packages, as it becomes cumbersome to manage the overhead of multiple separate repos for each package. Using lerna to manage all of our packages in one repo simplifies the process.

codecov

Package Overview

There are many packages located in the packages directory. A few of the more important ones are:

Contributing

For details on how to contribute to this repository, please see the contributing guidelines.

Development

Getting Started

We are using node 18.x and npm 8.x. If you are using nvm, there is an .nvmrc file, so just run nvm install to get the latest 18.x/8.x versions (or set up your environment to automatically switch). Otherwise, download from the node homepage.

In order to use the UI, you must also be running a deephaven-core server on port 10000. The server provides APIs that the web-client-ui depends upon. An easy way to get started is to launch a Deephaven container from the quick start guide.

Recommended Environment

We recommend using Visual Studio Code and installing the recommended workspace extensions which VS Code will suggest when you open the repo or when you browse the extensions panel. There are a few workspace settings configured with the repo.

If using Linux, we recommend installing directly from the deb or rpm file from the VSCode website rather than through a package manager or store such as snap or flatpak. The other install methods (for example, apt on Ubuntu installs via snap) may end up in a sandboxed environment that cannot use the debug launch configs properly.

We use Chrome for development with the React and Redux extensions.

Scripts

If your DHC address is different from the default http://localhost:10000, edit .env.local in each package to point to your local DHC. This is needed for the session websocket and for things like notebooks to be proxied correctly by Vite.

VITE_PROXY_URL=http://<dhc-host>:<port>

Local Plugin Development

The plugins repo supports serving plugins locally. DHC can be configured to proxy js-pluginsrequests to the local dev server by setting VITE_JS_PLUGINS_DEV_PORT in packages/code-studio/.env.development.local.

e.g. To point to the default dev port:

VITE_JS_PLUGINS_DEV_PORT=4100

Local Vite Config

If you'd like to override the vite config for local dev, you can define a packages/code-studio/vite.config.local.ts file that extends from vite.config.ts. This file is excluded via .gitignore which makes it easy to keep local overrides in tact.

The config can be used by running:

npm run start:app -- -- -- --config=vite.config.local.ts

For example, to proxy js-plugins requests to a local server, you could use this vite.config.local.ts:

export default defineConfig((config: ConfigEnv) => {
  const baseConfig = (createBaseConfig as UserConfigFn)(config) as UserConfig;

  return {
    ...baseConfig,
    server: {
      ...baseConfig.server,
      proxy: {
        ...baseConfig.server?.proxy,
        '/js-plugins': {
          target: 'http://localhost:5173',
          changeOrigin: true,
          rewrite: path => path.replace(/^\/js-plugins/, ''),
        },
      },
    },
  };
});

Debugging from VSCode

We have a pre-defined launch config that lets you set breakpoints directly in VSCode for debugging browser code. The Launch Deephaven config will launch a new Chrome window that stores its data in your repo workspace. With this setup, you only need to install the React and Redux devtool extensions once. They will persist to future launches using the launch config.

We prefer launching a new window instead of attaching to existing windows because it provides a cleaner debug environment (only development extensions). You would also need to launch Chrome with the remote debugging flag in order to attach to an existing instance.

Linux

If you are using Linux, you will likely need to use the direct install from the VSCode website (deb or rpm file) and not through a package manager. On Ubuntu, apt installs via snap and some Linux flavors may use flatpak. Both of these sandbox the VSCode instance in such a way that the launch debug configs will likely not work.

If you are not using Chrome (e.g. Chromium), you may need to do one of the following if the launch config is not working.

  1. Alias google-chrome-stable to launch chromium-browser. The launch config by default should try to launch google-chrome-stable, so if you can launch the browser with google-chrome-stable from a terminal, it should work.

  2. Add a new configuration to VSCode and copy the launch config from settings.json. Then add the runtimeExecutable prop to point to your browser executable. VSCode unfortnuately does not merge workspace settings.launch with workspace launch.json, so if we add more launch configs you would need to copy to your .vscode/launch.json file to get the configs.

Creating a New Package

Depending on what your package is, there are a couple of different templates that may be appropriate.

Application package

A standalone application with it's own entry point. Recommend copying the embed-widget package, removing any dependencies and files not required.

Component/library package

A component/library package that can be imported into other packages. Recommend copying the components package, removing any dependencies and files not required.

Unit Tests

We use Jest for running unit tests. We use an eslint runner in Jest to lint files as tests. We also use react-testing-library and react-hooks-testing-library for testing react components and hooks.

Run npm test to start Jest in watch mode. By default, this will only run tests that have changed relative to your git origin/main. Keeping your origin/main up to date will help keep the number of tests run to a minimum.

While the tests are running, you can press p to filter by filename or test name. You can also press Shift+P to see a list of projects and disable certain projects like the eslint or stylelint.

Log messages from our @deephaven/log package are disabled by default in tests in jest.setup.ts. To change the log level, set the DH_LOG_LEVEL env variable. For example, to enable all log messages run DH_LOG_LEVEL=4 npm test.

Note that log messages from other sources such as react prop types will still be printed since they do not go through our logger.

If you want to collect coverage locally, run npm test -- --coverage

Debugging Unit Tests

Unit tests can be debugged by running jest with the --inspect-brk flag and attaching to the node process in vscode's debugger. There are 2 launch configs that make this easier:

E2E Tests

We use Playwright for end-to-end tests. We test against Chrome, Firefox, and Webkit (Safari). Snapshots from E2E tests are only run against Linux so they can be validated in CI.

You should be able to pass arguments to these commands as if you were running Playwright via CLI directly. For example, to test only table.spec.ts you could run npm run e2e -- ./tests/table.spec.ts, or to test only table.spec.ts in Firefox, you could run npm run e2e -- --project firefox ./tests/table.spec.ts. See Playwright CLI for more details.

Snapshots are used by end-to-end tests to visually verify the output. Snapshots are both OS and browser specific. Sometimes changes are made requiring snapshots to be updated. Update snapshots locally by running npm run e2e:update-snapshots.

Once you are satisfied with the snapshots and everything is passing locally, you need to use the docker image to update snapshots for CI (unless you are running the same platform as CI (Ubuntu)). Run npm run e2e:update-ci-snapshots to update the CI snapshots. The snapshots will be written to your local directories. The Linux snapshots should be committed to git (non-Linux snapshots should be automatically ignored by git).

Differences in CI vs Local Docker Environments

Note that while both the GH actions and docker configuration use Ubuntu 22.04 images, their configurations are not identical. One known difference is the available system fonts. In some cases this can cause snapshots to differ when running locally vs in CI such as when rendering unicode characters. To mitigate this, some of our e2e tests have been configured to ensure a consistent unicode font fallback.

Local

These scripts are recommended while developing your tests as they will be quicker to iterate and offer headed mode for debugging. You will need to run npx playwright install before using these scripts. You may also need to update the browsers with the same command any time the Playwright version is updated. See Playwright Browsers for more details.

You must have a web UI running on localhost:4000/ide/ (dev mode and build preview are both fine), and deephaven-core running on port 10000 with anonymous authentication for E2E tests and application mode using the ./tests/docker-scripts/data/app.d folder. For example, the command you would run from the deephaven-core repo would be (replacing the path to point to the app.d directory on your machine):

START_OPTS="-DAuthHandlers=io.deephaven.auth.AnonymousAuthenticationHandler -Ddeephaven.application.dir=/path/to/web-client-ui/tests/docker-scripts/data/app.d" ./gradlew server-jetty-app:run

See this guide for more details on anonymous authentication.

Docker

These scripts will create a production build from your current code and run the tests in a Docker image. There is no need to install any other dependencies or have any other services running as they are all contained within Docker. Since any source code changes will require a new build, these scripts are only recommended if having issues with CI or if you need to update the snapshots for CI.

Releases

All new changes (bug fixes, feature requests) are merged to main so they are always included in the latest release. We use semantic versioning for major/minor/patch releases.

We use 3 release types

Releasing a New Version

Note: Only repo admins can do this. These steps apply to main and any release/* branches.

We use lerna and Conventional Commits to automatically handle incrementing the version, generate the changelog, and create the release.

  1. Generate a GitHub Personal access token:

    • Under Repository Access, select Only select repositories and add deephaven/web-client-ui.

    • Under Repository Permissions, set Access: Read and write for Contents. This will be necessary to push your version bump and create the release.

    • Copy the token created and replace <token> with it in the next step.

  2. Checkout the branch you want to release. main or release/*

  3. Run this npm script to bump the version, update the changelog, and create a release on GitHub. If your local remote name for the upstream repo is not origin, then set the proper git-remote (e.g. upstream).

    GH_TOKEN=<token> npm run version-bump -- --git-remote origin

  4. IMPORTANT: If releasing a release/* branch, you need to reset the latest tag on GitHub. To do that, go to the releases page and find the release that is actually the latest based on the version main is on.

    • Click edit on the actual latest release

    • Check the box for Set as the latest release

    • Update release

    This ensures that an older version doesn't get marked as latest on GitHub. It will not effect npm.

After the release is created, you can go to the actions page to see the publish action being kicked off.

Creating a hotfix/LTS branch

Note: Only repo admins can do this since release/* branches are protected.

For our enterprise product, we occasinally need to make hotfixes (patch releases) to older versions that the enterprise client depends on. Since it would be unwise to update an old version of enterprise to the newest community UI components, we keep hotfix release/* branches. There are a few steps to create one.

  1. Checkout the tag for the release where the branch should fork.

  2. Create a new branch from this point called release/vX.YY. If forking from v0.31.1 then the branch is release/v0.31.

  3. Change the distTag in lerna.json to vX.YY which corresponds to the version from the branch name. This will ensure packages are published to npm under the distTag instead of the default of latest.

  4. Commit and push the branch

Once the branch is pushed to origin, new commits will require PRs into the branch. To create a patch release, refer to the Releasing a New Version section.

Updating Dependencies

Periodically dependencies should be updated such that we're using the latest and greatest.

Analyzing Bundle Size

When adding new features, it can be useful to analyze the final package size to see what's in the package. Use source-map-explorer to see what's taking up the most size in the package. First run npm run build to build a production bundle, then run npx source-map-explorer 'packages/<package-name>/build/static/js/*.js', e.g.:

npm run build
npx source-map-explorer 'packages/code-studio/build/static/js/*.js'

Browser Support

Support is best for Google Chrome and Chromium based browsers (such as Microsoft Edge based on Chromium). We also maintain compatibility with Mozilla Firefox and Apple Safari.

Officially, the following browsers and corresponding configurations are supported:

If you encounter an issue specific to a browser, check that your browser is up to date, then check issues labeled with firefox or safari for a list of known browser compatibility issues before reporting the issue.