search

nut-tree / nut.js

Native UI testing / controlling with node
https://nutjs.dev
2.13k stars 116 forks source link
automation desktop-automation electron native node test-automation testing testing-tools ui

readme

nut.js (Native UI Toolkit)

GitHub Actions
Master Create tagged release
Develop Create snapshot release

SonarCloud badge SonarCloud Coverage

Console - Developer Tool of the Week

Please visit

nutjs.dev

for detailed documentation and tutorials

Most importantly,

please read this


About

logo

nut.js is a cross-platform native UI automation / testing tool.

It allows for native UI interactions via keyboard and / or mouse, but additionally gives you the possibility to navigate the screen based on image matching.

The Price of Open Source

If you came here after I removed public packages from npm as announced in 'the blog post', please be aware that

nut.js is developed with community in mind.

A huge "Thank you!" goes out to all sponsors who make open source a bit more sustainable!

Demo

Check out this demo video to get a first impression of what nut.js is capable of.

nut.js demo video

Tutorials

Please consult the project website at nutjs.dev for in-depth tutorials

API Docs

nut.js provides public API documentation auto-generated by TypeDoc.

Community

Feel free to join our Discord community

Modules

This list gives an overview on currently implemented and planned functionality. It's work in progress and will undergo constant modification.

Clipboard

Keyboard

Mouse

Window

Screen

(*) Requires an additional provider package, visit nutjs.dev for more info

Integration

Sample

The following snippet shows a valid nut.js example (using multiple addons):

"use strict";

const {
  mouse,
  screen,
  singleWord,
  sleep,
  useConsoleLogger,
  ConsoleLogLevel,
  straightTo,
  centerOf,
  Button,
  getActiveWindow,
} = require("@nut-tree/nut-js");
const {
  preloadLanguages,
  Language,
  LanguageModelType,
  configure,
} = require("@nut-tree/plugin-ocr");

configure({ languageModelType: LanguageModelType.BEST });

useConsoleLogger({ logLevel: ConsoleLogLevel.DEBUG });

screen.config.autoHighlight = true;
screen.config.ocrConfidence = 0.8;

function activeWindowRegion() {
  return getActiveWindow().then((activeWindow) => activeWindow.region);
}

(async () => {
  await preloadLanguages([Language.English], [LanguageModelType.BEST]);
  await sleep(5000);
  const result = await screen.find(singleWord("@nut-tree/nut-js"));
  await mouse.move(straightTo(centerOf(result)));
  await mouse.click(Button.LEFT);
  await screen.waitFor(singleWord("Native"), 15000, 1000, {
    providerData: { partialMatch: true },
  });
  const content = await screen.read({ searchRegion: activeWindowRegion() });
  console.log(content);
})();

Installation

Prerequisites

This section lists runtime requirements for nut.js on the respective target platform.

Windows

In case you're running Windows 10 N and want to use ImageFinder plugins, please make sure to have the Media Feature Pack installed.

macOS

On macOS, Xcode command line tools are required. You can install them by running

xcode-select --install

Permissions:

nut.js requires the executing application, e.g. your terminal, to be given both Accessibility and Screen Recording permissions.

Starting with release 2.3.0, nut.js will check for and request these permissions automatically:

Popup requesting screen recording permissions

It will also give you a subtle hint in case permissions are lacking:

-

Accessibility: ##### WARNING! The application running this script is not a trusted process! Please visit https://github.com/nut-tree/nut.js#macos #####

Attention:

Prior to release 2.3.0 you'll have to grant these permissions manually.

Settings -> Security & Privacy -> Privacy tab -> Accessibility -> Add...

For example, if you want to execute your node script in e.g. iTerm2, you'd have to add iTerm.app to the list. When running your script from a built-in terminal in e.g. VSCode or IntelliJ, you'd have to add the respective IDE.

accessibility permissions screen

Linux

Depending on your distribution, Linux setups may differ.

In general, nut.js requires

Installation on *buntu distributions:

sudo apt-get install libxtst-dev

Setups on other distributions might differ.

Attention:

At the moment nut.js only supports X11.

Wayland is NOT supported!

On e.g. Ubuntu you can switch to XWayland on your login screen as a workaround.

Install nut.js

Open Source

The core functionality of nut.js is open source and available on GitHub.

To build nut.js from source you'll have to build native dependencies first.

Pre-built packages

Pre-built packages are available for subscription plans.

Check out the pricing page for more information.

Once you subscribed to a plan, you'll receive a token which you can use to install the respective package, check out the registry access tutorial for reference.

With everything set up, running

npm i @nut-tree/nut-js

or

yarn add @nut-tree/nut-js

will install nut.js and its required dependencies.

Snapshot releases

nut.js also provides snapshot releases which allows to test upcoming features.

Running

npm i @nut-tree/nut-js@next

or

yarn add @nut-tree/nut-js@next

will install the most recent development release of nut.js.

Attention: While snapshot releases are great to work with upcoming features before a new stable release, it is still a snapshot release. Please bear in mind that things might change and / or break on snapshot releases, so it is not recommended using them in production.