search

jpaulm / jsfbp

FBP implementation written using JavaScript and node-fibers
MIT License
120 stars 23 forks source link

readme

npm version Build Status Known Vulnerabilities

jsfbp

JavaScript Implementation of FBP - no longer supported. Please take a copy by Oct. 1, 2021 if you need one!

General web site on Flow-Based Programming: https://jpaulm.github.io/fbp/ .

"Classical" FBP "green thread" implementation written in JavaScript, using Node-Fibers - https://github.com/laverdet/node-fibers.

JSFBP takes advantage of JavaScript's concept of functions as first-degree objects to allow applications to be built using "green threads". JSFBP makes use of an internal "Future Events Queue" which supports the green threads, and provides quite good performance (see below) - the JavaScript events queue is only used for JavaScript asynchronous functions, as before.

JSFBP is no longer supported, as Node-Fibers is being discontinued (I assume early 2021). Here is the relevant quote from https://github.com/laverdet/node-fibers :

"The author of this project recommends you avoid its use if possible. The original version of this module targeted nodejs v0.1.x in early 2011 when JavaScript on the server looked a lot different. Since then async/await, Promises, and Generators were standardized and the ecosystem as a whole has moved in that direction."

Installing Fibers

As suggested in https://github.com/laverdet/node-fibers/issues/new , make sure your version of nodejs is an even one.

Go into command mode, and enter npm install fibers.

If this command has trouble finding Python, install Python 2.7.10, then run npm --add-python-to-path='true' --debug install --global windows-build-tools. Don't know if this is still necessary!

General

To run test cases, position in your command shell to GitHub/jsfbp and type in node examples/fbptestxx, where xx is the test case number.

Test cases so far:

jsfbp

fbptest11

"Update" networks

The following diagram shows update and update_c in one diagram using the DrawFBP Enclosure function - this is not really a valid DrawFBP diagram, so no port names are shown:

update_combined

Here is update_c by itself, with component and port names marked in - it contains all the information needed to generate a running JSFBP network (the file and report icons do not generate any code):

update_c

WebSockets

fbptestws

Some of these have tracing set on, depending on what testing was being done when they were promoted!

These tests (except for fbptestws) can be run sequentially by running fbptests.bat.

Components

API

For application developers

Networks can be generated programmatically or by loading in an FBP file.

Programmatically

  1. Get access to JSFBP: var fbp = require('fbp')
  2. Create a new network: var network = new fbp.Network();
  3. Define your network:
    • Add processes: network.defProc(...) Note: when several processes use the same component, defProc takes the process name as a second argument.
    • Connect output ports to input ports: network.connect(...)
    • Specify IIPs: network.initialize(...)
  4. Create a new runtime: var fiberRuntime = new fbp.FiberRuntime();
  5. Run it!
network.run(fiberRuntime, {trace: true/false}, function success() {
    console.log("Finished!");
  });

Via an FBP file

  1. Generate an .fbp file that complies with the specification under parsefbp.
  2. Get access to JSFBP: var fbp = require('fbp')
  3. Load the contents of the .fbp file into a String: fs.readFile(__dirname + '/network.fbp' ...);
  4. Create a new network: var network = new fbp.Network.createFromGraph(fileContents); If you're using components that are local to your application, use a second parameter giving the directory that contains your components.
  5. Create a new runtime: var fiberRuntime = new fbp.FiberRuntime();

  6. Run it!

    network.run(fiberRuntime, {trace: true/false}, function success() {
console.log("Finished!");
});

    Activating trace can be desired in debugging scenarios.

Useful methods

For component developers

Component headers: 'use strict';

In most cases you do not need to require() any JSFBP-related scripts or libraries as a component developer. Everything you need is injected into the component's function as its context this (the process object) and as a parameter (the runtime object). Some utility functions are stored in core/utils.js. Import them if you really need them. You should generally refrain from accessing runtime-related code (e.g. Fibers) to ensure the greatest compatibility.

Component services

Install & Run

We use node-fibers which is known to work with Node.js 10.16.0 (as of 25.07.2019).

  1. Install Node.js

  2. Clone or download this project

  3. Execute npm install

    If you get an MSB4019 or similar error messages involving utf-8-validate and bufferutil (some dependencies deep down the dependency tree), you can just ignore them, given the optional nature of these components' compilation.

  4. Run node examples/fbptestxx.js, where fbptestxx is any of the tests listed above. If tracing is desired, change the value of the trace variable at the bottom of fbptestxx.js to true.

  5. All these tests can be run sequentially by running examples/fbptests.bat, or by running examples/fbptests.sh under bash.

Important - BitDefender Antivirus 2016 anti-ransomware feature seems to interfere with git- we suggest you leave it turned off while working with git.

Full install

If you wish to eliminate the errors mentioned in point #3 under Install, you will need to install Python 2.x and Visual Studio Express for Desktop 2013. This doesn't seem to guarantee an error-free npm install, however. Still jsfbp works fine, even with these errors.

  1. Install Node.js
  2. Install Python 2.x
  3. Install Visual Studio Express for Desktop 2013 (click on http://go.microsoft.com/fwlink/?LinkId=532500&clcid=0x409 )
  4. Clone or download this project
  5. Open a new shell (The shell should not have been opened from before the Visual Studio installation because then the PATH and other environment variables are not yet updated.)
  6. Optionally prepend Python 2.x to your PATH if you haven't already done so
    • e.g. SET PATH=C:\path\to\python2-directory\;%PATH%
  7. Execute npm install
  8. Run node examples/fbptestxx.js, where fbptestxx is any of the tests listed above. If tracing is desired, change the value of the trace variable at the bottom of fbptestxx.js to true.
  9. Install requires the following npm packages: parsefbp, fibers, mocha, chai, lodash and mocha-fibers - you may have to do npm installs for some or all of these.
  10. All these tests can be run sequentially by running examples/fbptests.bat, or by running examples/fbptests.sh under bash.

Important - BitDefender Antivirus 2016 anti-ransomware feature seems to interfere with git- we suggest you leave it turned off while working with git.

Testing with Mocha

The folder called test contains a number of Mocha tests.

  1. Run npm test to execute a series of tests (all the fbptestxx.js tests in sequence).
  2. Alternatively, you can directly execute node ./node_modules/mocha/bin/mocha --recursive --require test/test_helper.js in case you need to adjust the path to Node's binary or pass further parameters to Mocha.

Testing Sample HTTP Server

Run node examples/httpserver/fbphttpserver.js, which is a simple HTTP server which is similar to the one in the sample at: http://blog.modulus.io/build-your-first-http-server-in-nodejs

NOTE: The HTTP server components are currently all custom components, based on the components used in the simple web socket chat server described below.

Testing Simple Web Socket Chat Server

Run node examples/websocketchat/fbptestwschat.js, which is a simple web socket chat server which responds to any request by broadcasting it to all connected clients. It is similar to the chat sample at: http://socket.io/get-started/chat/ except for serving the client HTML.

examples/websocketchat/index.html is intended as a simple chat client for testing with fbptestwschat.js. If Firefox doesn't work for you, Chrome and Safari will work.

Just enter any string into the input field, and click on Send, and it will broadcast it to all clients that are connected.

Click on the Stop WS button, and the network will come down.

Tracing

Here is a sample section of the trace output for fbptest08.js:

recvr recv OK: externally to the processes. These black box processes can be rec
onnected endlessly
data: externally to the processes. These black box processes can be reconnected
endlessly
recvr IP dropped with: externally to the processes. These black box processes ca
n be reconnected endlessly
recvr recv from recvr.IN
Yield/return: state of future events queue:
- reverse2 - status: ACTIVE
---
---
reverse2 send OK
reverse2 IP dropped with:  si PBF .yllanretni degnahc eb ot gnivah tuohtiw snoit
acilppa tnereffid mrof ot
reverse2 recv from reverse2.IN
reverse2 recv OK: .detneiro-tnenopmoc yllarutan suht
reverse2 send to reverse2.OUT: thus naturally component-oriented.
Yield/return: state of future events queue:
- recvr - status: ACTIVE
---
---
recvr recv OK: to form different applications without having to be changed inter
nally. FBP is

Performance

The volume test case (fbptestvl) with 100,000,000 IPs running through three processes took 164 seconds, on my machine which has 4 AMD Phenom(tm) II X4 925 processors.

Since there are two connections, giving a total of 200,000,000 send/receive pairs, this works out to approx. 0.82 microsecs per send/receive pair. Of course, as it is JavaScript, this test only uses 1 core intensively, although there is some matching activity on the other cores (why...?!)