search

joostvangils / BPMN_RPA

Robotic Process Automation in Windows and Linux by using Diagrams.net BPMN diagrams.
GNU General Public License v3.0
40 stars 8 forks source link

readme

BPMN_RPA

Robotic Process Automation in Windows and Linux by using BPMN diagrams. Create flows in BPMN notation and execute them in Windows and Linux. This package contains 2 types of flow-engines:

  1. The Workflow Engine: runs a flow from beginning to end.
  2. The CheckList Engine: runs a flow step by step and saves the state of the flow after each step. This allows you to resume the flow at any time!

With this Framework you can draw Business Process Model Notation based Diagrams and run those diagrams with a WorkflowEngine. You can run flows that were made with one of the following programs:

There is no need for installing BPMN-RPA Studio, DrawIO or Visio to run the flows. Installing one of these applications is only needed for creating the flows.

Content:

Quick start

First start

The first time you will try to run a Flow, you will be asked to enter the path of your install directory. If you are using Windows, the path of the installation directory will be saved in the registry (path saved in registry key 'HKEY_CURRENT_USER\Software\BPMN_RPA\dbPath') and is used to create a SQLite database for logging purposes, called 'Orchestrator.db'. The WorkflowEngine must also know where your python.exe is located. You will be asked to enter the full path to the python.exe file (including the '.exe' extension). Again, if you are using Windows this path will be saved in registry key 'HKEY_CURRENT_USER\Software\BPMN_RPA\PythonPath'.

For Linux users a "settings" file together with the orchestrator database will be created (the settings file will be created in /etc/BPMN_RPA_settings). When you want to use MsSql server with a trusted connection instead of the automatically available SQLite database, then install MsSqlServer on the local machine and manually create a database called "Orchestrator". When running your first flow, all tables will be created.

Recognized Shapes

For the Workflow engine to recognize the flow, you must use the recommended shape attributes with the following Shapes:

Tasks

You can use Tasks to call Python scripts. For the WorkflowEngine to recognize the Tasks, each Task has to contain certain attributes to make this possible.

GateWays
Sequence flow arrow

Variables

The % sign is used as brackets around a Variable. For example, "%name%" is the Variable 'name'. When you use %name% as an input, the Action will use the value that has previously been stored in that Variable, so you should have an earlier Action that assigned a value to %name% as an output. By assigning output values to Variables, and then using them as input in later steps, you can pass information through a Workflow.

You can store any type of information into a variable, like:

System variables

System variables are pre-defined variables that provide information that can be used in flow attributes. The System variables content is set by the WorkflowEngine automatically and cannot be changed. All system variables begin with a double underscore between precent signs. Available variables:

Loops

You can create loops by using exclusive gateways. An exclusive gateway should always have two sequence flow arrows: one with the label "True" and the other with the label "False". The actual true/false decision isn't made in the exclusive gateway itself, but in the last Task before the exclusive gateway. A loop is started by a Task that is calling a Python script that returns a list. The task is recognized as the start of the loop by adding/using the attribute 'Loopcounter'. The loopcounter number is the starting point for the loop (for returning the n-th element of the list). The task before the Exclusive Gateway should be the 'More loop items?' Task. You can find this Task in the predefined Shapes. This Task should have two attributes: 'Function' with value 'loop_items_check' will call the loop_items_check() function in the WorkflowEngine object, and 'Loop_variable' with the variable name to loop as value. The loop_items_check() function will return True or False, which will be used by the WorkflowEngine to decide which Sequence Flow Arrow to follow.

Special loop variable options

You can get the value of the loopvariable counter by using the '.counter' attribute of the loopvariable (p.e.: %test.counter%). To get the whole list that is looped, use the '.object' attribute of the loopvariable (p.e.: '%test.object%') or just the variable name (like '%test%').

An example:

Explanation:

  1. The loop starts with the 'Loop list' Task. This The function 'returnlist' is called in the module 'hello_world.py'. There is no path specified for the module and the module name ends with '.py', so the path to the module will be 'current directory\Scripts\hello_world.py'. This script returns a List with the elements ["this", "is", "a", "test"] and stores it in the variable named '%test%'. The attribute 'Loopcounter' is the important indication that this Task will be the start of a loop. The number in this field will be the start for the loop (p.e.: setting 'Loopcounter' to 1 results in loping the list from the second element in the list).
  2. The MessageBox function is called ('current directory\Scripts\MessageBox.py'). The title will be "test", and the message will be a word from the list in confirmity with the 'Loopcounter' number ('%test%').


  3. The 'More loop items?' Task checks if the List in the variable '%test%' has any items left to loop. If so, then it returns True, otherwise it will return False. If it returns True, the 'Loopcounter' is raised by 1. The function is called within the WorkflowEngine class (no 'Module'or 'Class' specified).
  4. The Exclusive Gateway is deciding which Sequence Flow Arrow to follow. If the loop is still ongoing, the 'Loop List' Task will be called again and the next element in the list will be returned.

Retrieving information

In order to retrieve a specific item of a list, you must use the following format (notation): %VariableName[ItemNumber]%. The “ItemNumber” should be 0 for the first item of the list, 1 for the second and so on. For example, if you have a list that is stored in the variable %MyList% and contains 10 items, you can retrieve the first item with: %MyList[0]% and the last item with %MyList[9]%. For data tables, you must use the following notation: %VariableName[RowNumber][ColumnNumber]%.

If you would like to retrieve an attribute of a stored object or dictionary in a variable, then you must use the %VariableName.attributeName% notation. Just use the %VariableName% notation to retreive the full object or dictionary.

Instantiate a Class and use in Flow

You can instantiate a Python class by using ony these attributes (leave the 'Function' attribute blank or delete it):

This instantiates the class and saves the instance in the variable %test%. You can call any function of the class object by use of these attributes in Tasks following the instantiation Task (leave the 'Module' attribute blank or delete it):

Passing input to the WorkflowEngine

You can pass input to the WorkflowEngine by using the 'input_parameter' argument. Please note that it is only possible to pass a single object to the WorkflowEngine. Wrap all your inputs into a single object (like: dictionary or custom object) to pass multiple values to the WorkflowEngine.

myObject = ['this could be', 'any', 'type', 'of', 'object']
engine = WorkflowEngine(input_parameter=myObject)

Call the internal 'get_input_parameter' function to retrieve this input value and assign it to a variable name for later use in your flow:

When starting a Workflow from the commandline, you may use the 'As_dictionary' option in the 'Get input parameter' Shape of your Flow with the value 'True' to convert the string input to a dictionary object. P.e.: 

c:\> python BPMN_RPA_Starter.py test.xml "{\"key1\": \"value1\",\"key2\": \"value2\"}"

Logging

The WorkflowEngine logs all executed steps in a SQLite database, called 'Orchestrator.db'. This database is located in the install directory. If the install directory is unknown when starting the WorkflowEngine, the WorkflowEngine will ask you for the folder. This path then will be saved in the registry and the Orchestrator database will be created in that folder.

End a flow

The ending of a flow will also be logged in the Orchestrator database. When ending a flow, the output of the last executed step will also be the output of the entire flow, unless the flow is ended with an exitcode.

End flow with exitcode

If you wish to end your flow with an exitcode (0 for OK and -1 for not OK) then you can call one of the internal functions of the WorkflowEngine:

Step by step flow execution

As from BPMN-RPA version 4.x.x and above, you can perform step-by-step flow execution in your code. An example:

from BPMN_RPA.WorkflowEngine import WorkflowEngine
engine = WorkflowEngine()
doc = engine.open("c:\\test.flw")
steps = engine.get_flow(doc)
nextstep = steps[0]
while nextstep is not None:
    q = input("Execute next step? (y/n)")
    if q.lower()!="y":
        break
    print("\n")
    result =  engine.run_flow(nextstep,True )
    nextstep = engine.get_next_step(nextstep, steps, result)
    print(f"Output of this step: {result}")

PlugIn

BPMN-RPA has a Drawio plugin for checking your flows. You can download it here: PlugIn

Example

Start a flow from the commandline:

  1. Open a command prompt
  2. Enter: 
    c:\> python BPMN_RPA_Starter.py test.xml

Start a flow in code:

from BPMN_RPA.WorkflowEngine import WorkflowEngine
engine = WorkflowEngine()
doc = engine.open("test.xml")
steps = engine.get_flow(doc)
engine.run_flow(steps)

Databases

BPMN-RPA uses a SQLite database by default that is automatically generated. If you want to use MsSql server or PostgreSQL server instead, then install MsSqlServer or PostgreSQL on the host machine and manually create a database called "Orchestrator". The WorkflowEngine has a 'use_sql_server' and 'use_postgresql' parameter in the constructor. Set the parameter to True to use MsSql server or PostgreSQL server instead of the default SQLite database. When using either use_sql_server or use_postgresql, you can also specify the 'connection_string' parameter:

engine = WorkflowEngine(use_postgresql=True, connection_string="dbname=orchestrator host=my.postgres.server user=myusername password=mypassword")

Default the following connection strings are used:

If you want to use a different connection string, then you must specify the 'connection_string' parameter. The 'connection_string' parameter doesn't need to be specified when using SQLite. When running your first flow, all tables will be created in the 'orchestrator' database.

The CheckList Engine

The CheckListEngine runs any flow like the WorkflowEngine. The only difference is that the CheckListEngine will save the entire state of the whole flow in a separate file after each step. This allows you to resume the flow from the last saved state. This is very useful when you have a long-running flow with waiting periods between steps (like p.e. an onboarding flow for new employees). It is likely that you want to create several CheckList instances of the original flow, so that you can run several flows in parallel. This is possible by using the 'full_path_save_as' parameter in the CheckListEngine constructor. This parameter is optional and will default to the name of the original flow (without the .flw extension). If you want to run several flows in parallel, then you must use different instance names (i.e. 'full_path_save_as') for each flow. The CheckListEngine will create a separate file for each flow instance.

To start a new instance of a flow, just use the CheckListEngine constructor:

from BPMN_RPA.CheckListEngine import CheckListEngine

# Start a new instance of the flow 'example.flow' and create an instance file called 'example_instance1'
chkLst = ChecklistEngine(flow_name="\\my_flows\\example.flw", full_path_save_as="\\instances\\example_instance_1")
# This will load the flow and start the first step. The state of the flow will be saved in the file '\instances\example_instance1'.

# You can run the flow instance by calling the 'run_flow' function. This function has 2 optional parameters:
# 1. ask_permission: if set to True, the user will be asked for permission to execute the next step. Default is False.
# 2. msgbox: only applicable if ask_permission is set to True. If msgbox is True, a messagebox will be shown to the user. If False, the user will be asked to enter Yes (y) or No (n) in the console. Default is True.
# When the user answers 'No' to the question, the flow will be ended and the state of the flow will be saved.

chkLst.run_flow(ask_permission=True, msgbox=True)

When you want to pause the flow, just click 'No' in the MessageBox or press 'n' in the console. The state of the flow will be saved in the instance file, together with a flow diagram (.png file with the same name as the instance file). The current step will be highlighted in green in the flow diagram. Example:

To resume a flow at any time you like from a saved state, use the CheckListEngine constructor and call the 'resume_flow' function:

from BPMN_RPA.CheckListEngine import CheckListEngine

# Resume the flow instance 'example_instance1'
chkLst = ChecklistEngine()
chkLst.resume_flow("\\instances\\example_instance_1", ask_permission=True, msgbox=True)