search

highcharts / node-export-server

Highcharts Node.js export server
Other
353 stars 260 forks source link

readme

Highcharts Node.js Export Server

Convert Highcharts.JS charts into static image files.

Upgrade Notes

In most cases, v4 should serve as a drop-in replacement for v2 and v3. However, due to changes in the browser backend, various tweaks related to process handling (e.g., worker counts, and so on) may now have different effects than before.

Significant changes have been made to the API for using the server as a Node.js module. While a compatibility layer has been created to address this, it is recommended to transition to the new API described below. It is worth noting that the compatibility layer may be deprecated at some point in the future.

An important note is that the Export Server now requires Node.js v18.12.0 or a higher version.

Additionally, with the v3 release, we transitioned from HTTP to HTTPS for export.highcharts.com, so all requests sent to our public server now must use the HTTPS protocol.

Changelog

Version 4 introduces some breaking changes, mostly related to renamed options, environment variables, function names, and reordered function parameters. For further details, please refer to the changelog document provided below, under the Breaking Changes section.

The full change log for all versions can be viewed here.

What & Why

This Node.js application/service converts Highcharts.JS charts into static image files, supporting PNG, JPEG, SVG, and PDF output. The input can be either SVG or JSON-formatted chart options.

The application is versatile and can be used as a CLI (Command Line Interface), an HTTP server, or as a Node.js module.

Use Cases

The primary use case for the Export Server is scenarios requiring headless conversion of charts. Common cases of using include automatic report generation, static caching, and incorporating charts into presentations or other documents.

In addition, the HTTP mode enables you to run your own Export Server for users, reducing reliance on the public https://export.highcharts.com/ server, which has rate limitations.

The HTTP server can be run either independently, integrating with your other applications and services, or in a way that directs the export buttons on your charts to your customized server.

To implement the latter, include the following configuration in your chart options:

{
  exporting: {
    url: "<IP to the self-hosted Export Server>"
  }
}

For systems that generate automatic reports, using the Export Server as a Node.js module is a great fit - especially if your report generator is also written in Node.js. Check here for examples.

Install

First, make sure you have Node.js installed. If not, visit nodejs.org, download and install Node.js for your platform. For compatibility reasons, version 18.12.0 or higher is required.

Once Node.js is installed, proceed to install the Export Server by opening a terminal and typing:

npm install highcharts-export-server -g

or:

git clone https://github.com/highcharts/node-export-server
npm install
npm link

Depending on your Node.js installation method, you might need to create a symlink from nodejs to node. For example, on Linux:

ln -s `which nodejs` /usr/bin/node

Running

To use the Export Server, simply run the following command with the correct arguments:

highcharts-export-server <arguments>

Configuration

There are four main ways of loading configurations:

...or any combination of the four. In such cases, the options from the later step take precedence (config file -> custom JSON -> envs -> CLI arguments).

Default JSON Config

The JSON below represents the default configuration stored in the lib/schemas/config.js file. If no .env file is found (more details on the file and environment variables below), these options will be used.

The format, along with its default values, is as follows (using the recommended ordering of core and module scripts below):

{
  "puppeteer": {
    "args": []
  },
  "highcharts": {
    "version": "latest",
    "cdnURL": "https://code.highcharts.com/",
    "coreScripts": [
      "highcharts",
      "highcharts-more",
      "highcharts-3d"
    ],
    "moduleScripts": [
      "stock",
      "map",
      "gantt",
      "exporting",
      "parallel-coordinates",
      "accessibility",
      "annotations-advanced",
      "boost-canvas",
      "boost",
      "data",
      "data-tools",
      "draggable-points",
      "static-scale",
      "broken-axis",
      "heatmap",
      "tilemap",
      "tiledwebmap",
      "timeline",
      "treemap",
      "treegraph",
      "item-series",
      "drilldown",
      "histogram-bellcurve",
      "bullet",
      "funnel",
      "funnel3d",
      "geoheatmap",
      "pyramid3d",
      "networkgraph",
      "overlapping-datalabels",
      "pareto",
      "pattern-fill",
      "pictorial",
      "price-indicator",
      "sankey",
      "arc-diagram",
      "dependency-wheel",
      "series-label",
      "solid-gauge",
      "sonification",
      "stock-tools",
      "streamgraph",
      "sunburst",
      "variable-pie",
      "variwide",
      "vector",
      "venn",
      "windbarb",
      "wordcloud",
      "xrange",
      "no-data-to-display",
      "drag-panes",
      "debugger",
      "dumbbell",
      "lollipop",
      "cylinder",
      "organization",
      "dotplot",
      "marker-clusters",
      "hollowcandlestick",
      "heikinashi",
      "flowmap",
      "export-data",
      "navigator",
      "textpath"
    ],
    "indicatorScripts": [
      "indicators-all"
    ],
    "customScripts": [
      "https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.30.1/moment.min.js",
      "https://cdnjs.cloudflare.com/ajax/libs/moment-timezone/0.5.45/moment-timezone-with-data.min.js"
    ],
    "forceFetch": false,
    "cachePath": ".cache"
  },
  "export": {
    "infile": false,
    "instr": false,
    "options": false,
    "outfile": false,
    "type": "png",
    "constr": "chart",
    "height": 400,
    "width": 600,
    "scale": 1,
    "globalOptions": false,
    "themeOptions": false,
    "batch": false,
    "rasterizationTimeout": 1500
  },
  "customLogic": {
    "allowCodeExecution": false,
    "allowFileResources": false,
    "customCode": false,
    "callback": false,
    "resources": false,
    "loadConfig": false,
    "createConfig": false
  },
  "server": {
    "enable": false,
    "host": "0.0.0.0",
    "port": 7801,
    "benchmarking": false,
    "proxy": {
      "host": "",
      "port": 8080,
      "timeout": 5000
    },
    "rateLimiting": {
      "enable": false,
      "maxRequests": 10,
      "window": 1,
      "delay": 0,
      "trustProxy": false,
      "skipKey": "",
      "skipToken": ""
    },
    "ssl": {
      "enable": false,
      "force": false,
      "port": 443,
      "certPath": ""
    }
  },
  "pool": {
    "minWorkers": 4,
    "maxWorkers": 8,
    "workLimit": 40,
    "acquireTimeout": 5000,
    "createTimeout": 5000,
    "destroyTimeout": 5000,
    "idleTimeout": 30000,
    "createRetryInterval": 200,
    "reaperInterval": 1000,
    "benchmarking": false
  },
  "logging": {
    "level": 4,
    "file": "highcharts-export-server.log",
    "dest": "log/",
    "toConsole": true,
    "toFile": true
  },
  "ui": {
    "enable": false,
    "route": "/"
  },
  "other": {
    "nodeEnv": "production",
    "listenToProcessExits": true,
    "noLogo": false,
    "hardResetPage": false,
    "browserShellMode": true
  },
  "debug": {
    "enable": false,
    "headless": true,
    "devtools": false,
    "listenToConsole": false,
    "dumpio": false,
    "slowMo": 0,
    "debuggingPort": 9222
  }
}

Custom JSON Config

To load an additional JSON configuration file, use the --loadConfig <filepath> option. This JSON file can either be manually created or generated through a prompt triggered by the --createConfig option.

Environment Variables

These variables are set in your environment and take precedence over options from the lib/schemas/config.js file. They can be set in the .env file (refer to the .env.sample file). If you prefer setting these variables through the package.json, use export command on Linux/Mac OS X and set command on Windows.

Highcharts Config

Export Config

Custom Logic Config

Server Config

Server Proxy Config

Server Rate Limiting Config

Server SSL Config

Pool Config

Logging Config

UI Config

Other Config

Debugging Config

Command Line Arguments

To supply command line arguments, add them as flags when running the application: highcharts-export-server --flag1 value --flag2 value ...

Available options:

HTTP Server

Apart from using as a CLI tool, which allows you to run one command at a time, it is also possible to configure the server to accept POST requests. The simplest way to enable the server is to run the command below:

highcharts-export-server --enableServer 1

Server Test

To test if the server is running correctly, you can send a simple POST request, e.g. by using Curl:

curl -H "Content-Type: application/json" -X POST -d '{"infile":{"title": {"text": "Chart"}, "xAxis": {"categories": ["Jan", "Feb", "Mar"]}, "series": [{"data": [29.9, 71.5, 106.4]}]}}' 127.0.0.1:7801 -o chart.png

The above should result in a chart being generated and saved in a file named chart.png.

SSL

To enable SSL support, add --certPath <path to key/crt> when running the server. Note that the certificate files needs to be named as such:

HTTP Server POST Arguments

The server accepts the following arguments in a POST request body:

The server responds to application/json, multipart/form-data, and URL encoded requests.

CORS is enabled for the server.

It is recommended to run the server using pm2 unless running in a managed environment/container. Please refer to the pm2 documentation for details on how to set this up.

Available Endpoints

Switching Highcharts Version at Runtime

If the HIGHCHARTS_ADMIN_TOKEN is set, you can use the POST /change_hc_version/:newVersion route to switch the Highcharts version on the server at runtime, ie. without restarting or redeploying the application.

A sample request to change the version to 10.3.3 is as follows:

curl -H 'hc-auth: <YOUR AUTH TOKEN>' -X POST <SERVER URL>/change_hc_version/10.3.3

e.g.

curl -H 'hc-auth: 12345' -X POST 127.0.0.1:7801/change_hc_version/10.3.3

This is useful to e.g. upgrade to the latest HC version without downtime.

Node.js Module

Finally, the Export Server can also be used as a Node.js module to simplify integrations:

// Import the Highcharts Export Server module
const exporter = require('highcharts-export-server');

// Export options correspond to the available CLI/HTTP arguments described above
const options = {
  export: {
    type: 'png',
    options: {
      title: {
        text: 'My Chart'
      },
      xAxis: {
        categories: ["Jan", "Feb", "Mar", "Apr"]
      },
      series: [
        {
          type: 'line',
          data: [1, 3, 2, 4]
        },
        {
          type: 'line',
          data: [5, 3, 4, 2]
        }
      ]
    }
  }
};

// Initialize export settings with your chart's config
const exportSettings = exporter.setOptions(options);

// Must initialize exporting before being able to export charts
await exporter.initExport(exportSettings);

// Perform an export
await exporter.startExport(exportSettings, async (error, info) => {
  // The export result is now in info
  // It will be base64 encoded (info.data)

  // Kill the pool when we are done with it
  await exporter.killPool();
});

CommonJS support

This package supports both CommonJS and ES modules.

Node.js API Reference

highcharts-export-server module

Examples

Samples and tests for every mentioned export method can be found in the ./samples and ./tests folders. Detailed descriptions are available in their corresponding sections on the Wiki.

Tips, Tricks & Notes

Note about Deprecated Options

At some point during the transition process from the PhantomJS solution, certain options were deprecated. Here is a list of options that no longer work with the server based on Puppeteer:

Additionally, some options are now named differently due to the new structure and categorization. Here is a list of old names and their corresponding new names (old name -> new name):

If you depend on any of the above options, the optimal approach is to directly change the old names to the new ones in the options. However, you don't have to do it manually, as there is a utility function called mapToNewConfig that can easily transfer the old-structured options to the new format. For an example, refer to the ./samples/module/options_phantomjs.js file.

Note about Chart Size

If you need to set the height or width of the chart, it can be done in two ways:

Set it in the chart config under:

Set it in the exporting config under:

The latter is preferred, as it allows you to set separate sizing when exporting and when displaying the chart on your web page.

Like previously mentioned, there are multiple ways to set and prioritize options, and the height, width and scale are no exceptions here. The priority goes like this:

  1. Options from the export section of the provided options (CLI, JSON, etc.).
  2. The sourceHeight, sourceWidth and scale from the chart.exporting section of chart's Highcharts options.
  3. The height and width from the chart section of chart's Highcharts options.
  4. The sourceHeight, sourceWidth and scale from the chart.exporting section of chart's Highcharts global options, if provided.
  5. The height and width from the chart section of chart's Highcharts global options, if provided.
  6. If no options are found to this point, the default values will be used (height = 400, width = 600 and scale = 1).

Note about Event Listeners

The Export Server attaches event listeners to process.exit, uncaughtException and signals such as SIGINT, SIGTERM and SIGHUP. This is to make sure that there are no memory leaks or zombie processes if the application is unexpectedly terminated.

Listeners are also attached to handle uncaught exceptions. If an exception occurs, the entire pool and browser instance are terminated, and the application is shut down.

If you do not want this behavior, start the server with --listenToProcessExits 0 or --listenToProcessExits false.

Be aware though, that if you disable this and you do not take great care to manually kill the pool of resources along with a browser instance, your server will bleed memory when the app is terminated.

Note about Resources

If --resources argument is not set and a file named resources.json exists in the folder from which the CLI tool was ran, it will use the resources.json file.

Note about Worker Count & Work Limit

The Export Server utilizes a pool of workers, where each worker is a Puppeteer process (browser instance's page) responsible for the actual chart rasterization. The pool size can be set with the --minWorkers and --maxWorkers options, and should be tweaked to fit the hardware on which you are running the server.

It is recommended that you start with the default 4, and work your way up (or down if 8 is too many for your setup, and things are unstable) gradually. The tests/other/stress-test.js script can be used to test the server and expects the server to be running on port 7801.

Each of the workers has a maximum number of requests it can handle before it restarts itself to keep everything responsive. This number is 40 by default, and can be tweaked with --workLimit. As with --minWorkers and --maxWorkers, this number should also be tweaked to fit your use case. Also, the --acquireTimeout option is worth to mention as well, in case there would be problems with acquiring resources. It is set in miliseconds with 5000 as a default value. Lastly, the --createTimeout and --destroyTimeout options are similar to the --acquireTimeout but for resource's create and destroy actions.

Usage

Injecting the Highcharts Dependency

In order to use the Export Server, Highcharts needs to be injected into the export template (see the ./templates folder for reference).

Since version 3.0.0, Highcharts is fetched in a Just-In-Time manner, making it easy to switch configurations. It is no longer required to explicitly accept the license, as in older versions. However, the Export Server still requires a valid Highcharts license to be used.

Using in Automated Deployments

Since version 3.0.0, when using in automated deployments, the configuration can be loaded either using environment variables or a JSON configuration file.

For a reference on available variables, refer to the configuration section above.

If you are using the Export Server as a dependency in your application, depending on your setup, it may be possible to set the environment variables in the package.json file as follows:

On Linux/Mac OS X:

{
  "scripts": {
    "preinstall": "export <variable1>=<value1>&&<variable2>=<value2>&&..."
  }
}

On Windows:

{
  "scripts": {
    "preinstall": "set <variable1>=<value1>&&<variable2>=<value2>&&..."
  }
}

Library Fetches

When fetching the built Highcharts library, the default behaviour is to fetch them from code.highcharts.com.

Installing Fonts

Does your Linux server not have Arial or Calibri? Puppeteer uses the system installed fonts to render pages. Therefore the Highcharts Export Server requires fonts to be properly installed on the system in order to use them to render charts.

Note that the default font-family config in Highcharts is "Lucida Grande", "Lucida Sans Unicode", Verdana, Arial, Helvetica, sans-serif".

Fonts are installed differently depending on your system. Please follow the below guides for font installation on most common systems.

Mac OS X

Install your desired fonts with the Font Book app, or place it in /Library/Fonts/ (system) or ~/Library/Fonts/ (user).

Linux

Copy or move the TTF file to the /usr/share/fonts/truetype (may require sudo privileges):

mkdir -p /usr/share/fonts/truetype
cp yourFont.ttf /usr/share/fonts/truetype/
fc-cache -fv

Windows

Copy or move the TTF file to C:\Windows\Fonts\:

copy yourFont.ttf C:\Windows\Fonts\yourFont.ttf

Google fonts

If you need Google Fonts in your custom installation, they can be had here: https://github.com/google/fonts.

Download them, and follow the above instructions for your OS.

Debug Mode

Version 4.0.0 introduced a new mode that allows debugging the Puppeteer browser instance. This is particularly useful when setting up a custom server. It helps to delve into the implementation, observe how things work, and analyze and resolve potential problems.

Launching

Setting the --enableDebug to true passes all debug options to the puppeteer.launch() function on startup. Together with the --headless option set to false, it launches the browser in a headful state providing a full version of the browser with a graphical user interface (GUI). While this serves as the minimal configuration to simply display the browser, Puppeteer offers additional options. Here is the full list:

Debugging

There are two main ways to debug code:

The npm run start:debug script from the package.json allows debugging code using both methods simultaneously. In this setup, client-side code is accessible from the devTools of a specific Puppeteer browser's page, while server-side code can be debugged from the devTools of chrome://inspect/.

For more details, refer to the Puppeteer debugging guide.

Additional Notes

Performance Notice

In cases of batch exports, using the HTTP server is faster than the CLI. This is due to the overhead of starting Puppeteer for each job when using the CLI.

So it is better to write a bash script that starts the server and then performs a set of POSTS to it through e.g. Curl if not wanting to host the Export Server as a service.

Alternatively, you can use the --batch switch if the output format is the same for each of the input files to process:

highcharts-export-server --batch "infile1.json=outfile1.png;infile2.json=outfile2.png;..."

Other switches can be combined with this switch.

System Requirements

The system requirements largely depend on your use case.

The application is largely CPU and memory bound, so for heavy-traffic situations, it needs a fairly beefy server. It is recommended that the server has at least 1GB of memory regardless of traffic, and more than one core.

License

MIT. Note that a valid Highcharts License is also required to do exports.