T2.3 decission support tool

[JBenda]

Adding decision support to JULEA.

New:

• trace level access which logs server backend operations.
• tool access-replay which takes a JULEA_TRACE=access trace and replays the backend operations of it
• scripts/decission-support.py runs access-replay multiple times for different configurations described in a config file
• scripts/decission-support.R converts the output of a scripts/decission-support.py to plots and tables, which can be viewed as text or webpage
• scripts/style.css and scripts/interactive.js for interactive and nice HTML page, where to put this file properly?

Further documentation can be found in doc/decission-support.md

Additional analytics possible on this data, but not yet implemented:

• Cardinality of objects access (regarding different processes)
• read/write ratio for different object categories
• analytics regarding access type clusters and frequency

Added build option julea_trace to enable tracing also for release builds

[t-erxleben]

Does it make sense to enfore JULEA_TRACE=access in julea-access-replay so the user does not have to care to set it manually? It seems like this would be the default use case for the tool. Also: are echo and access compatible? (e.g., what happens when both are specified, since they both output to stderr).

[t-erxleben]

I rewrote the doc file here. Feel free to cherry-pick and/or argue about changes I made :)

[JBenda]

Does it make sense to enfore JULEA_TRACE=access in julea-access-replay so the user does not have to care to set it manually? It seems like this would be the default use case for the tool. Also: are echo and access compatible? (e.g., what happens when both are specified, since they both output to stderr).

You could also use replays to test a configuration for its functionality. So, you do not need to run an expensive program or made up integration test, just re-run the data access. Also reduces dependencies for such cases.

Since each echo line starts with [timestamp] it would be easy to filter out these lines. For debugging reasons (e.g. check if the access records are correct) it would make sens to see both in chronologically order.

[t-erxleben]

Does it make sense to enfore JULEA_TRACE=access in julea-access-replay so the user does not have to care to set it manually? It seems like this would be the default use case for the tool. Also: are echo and access compatible? (e.g., what happens when both are specified, since they both output to stderr).

You could also use replays to test a configuration for its functionality. So, you do not need to run an expensive program or made up integration test, just re-run the data access. Also reduces dependencies for such cases.

Since each echo line starts with [timestamp] it would be easy to filter out these lines. For debugging reasons (e.g. check if the access records are correct) it would make sens to see both in chronologically order.

Alright :)

[t-erxleben]

I get warnings during the build:

➜ ninja -C bld
ninja: Entering directory `bld'
[24/176] Compiling C object libjulea.so.p/lib_core_jtrace.c.o
../lib/core/jtrace.c: In function ‘parse_backend_operation’:
../lib/core/jtrace.c:645:1: warning: function returns an aggregate [-Waggregate-return]
  645 | parse_backend_operation(const gchar* backend_operation)
      | ^~~~~~~~~~~~~~~~~~~~~~~
../lib/core/jtrace.c: In function ‘j_trace_enter’:
../lib/core/jtrace.c:753:32: warning: function call has aggregate value [-Waggregate-return]
  753 |                         type = parse_backend_operation(backend_operation);
      |                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../lib/core/jtrace.c: In function ‘j_trace_leave’:
../lib/core/jtrace.c:1075:61: warning: function call has aggregate value [-Waggregate-return]
 1075 |                                 BackendTypeOperation type = parse_backend_operation(backend_operation);
      |                                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[176/176] Linking target julea-fuse

[t-erxleben]

I worked through the examples given in the documentation and everything worked fine.

As an addition it could be nice to mention the R package dependencies in the docs (probably as a command that install them?). I ran through a few iterations of trying to execute the R script and installing missing packages.

[JBenda]

I get warnings during the build:

➜ ninja -C bld
ninja: Entering directory `bld'
[24/176] Compiling C object libjulea.so.p/lib_core_jtrace.c.o
../lib/core/jtrace.c: In function ‘parse_backend_operation’:
../lib/core/jtrace.c:645:1: warning: function returns an aggregate [-Waggregate-return]
  645 | parse_backend_operation(const gchar* backend_operation)
      | ^~~~~~~~~~~~~~~~~~~~~~~
../lib/core/jtrace.c: In function ‘j_trace_enter’:
../lib/core/jtrace.c:753:32: warning: function call has aggregate value [-Waggregate-return]
  753 |                         type = parse_backend_operation(backend_operation);
      |                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../lib/core/jtrace.c: In function ‘j_trace_leave’:
../lib/core/jtrace.c:1075:61: warning: function call has aggregate value [-Waggregate-return]
 1075 |                                 BackendTypeOperation type = parse_backend_operation(backend_operation);
      |                                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[176/176] Linking target julea-fuse

This is fixed now.