Benchmarking of RC v3.2.0

[iharsuvorau]

Event logs:

• BPIC_2012_W_contained (train, test)
• BPIC_2017_W_contained (train, test)
• ConsultaDataMining201618 (train, test)
• Governmental_Agency (train, test)
• poc_processmining (train, test)
• Production (train, test)

General configuration template which is modified for each log, optimisation metric, and resource profile discovery type:

version: 2
common:
  log_path: assets/train_Production.xes
  test_log_path: assets/test_Production.xes
  exec_mode: optimizer
  log_ids:
    case: case_id
    activity: Activity
    resource: Resource
    start_time: start_time
    end_time: end_time
  repetitions: 5
  simulation: true
  evaluation_metrics: 
    - dl
    - circadian_emd
    - absolute_hourly_emd
    - cycle_time_emd
preprocessing:
  multitasking: false
structure:
  max_evaluations: 40
  mining_algorithm: sm3
  concurrency:
    - 0.0
    - 1.0
  epsilon:
    - 0.0
    - 1.0
  eta:
    - 0.0
    - 1.0
  gateway_probabilities:
    - equiprobable
    - discovery
  or_rep:
    - true
    - false
  and_prior:
    - true
    - false
calendars:
  max_evaluations: 40
  resource_profiles:
    discovery_type: undifferentiated
    granularity: 
      - 15
      - 60
    confidence:
      - 0.5
      - 0.85
    support:
      - 0.01 
      - 0.3
    participation: 0.4

[iharsuvorau]

Hi @david-chapela. Here're the results of benchmarking, https://owncloud.ut.ee/owncloud/s/z9MYgtALwkLJKY8. In the trials folder, successful runs have folders named like this: <optimization metric>_<resource profile discovery type>_<UUID>. Unsuccessful folders are just UUID.

[iharsuvorau]

Hi @david-chapela. Here're the plots of best configurations (metric and resource discovery type) with original logs.

Benchmarking RC 3.2.0 Apromore Plots.zip