[KIEKER-1903] Define a comprehensive and stable package structure

rju commented 1 week ago

JIRA Issue: KIEKER-1903 Define a comprehensive and stable package structure Original Reporter: Reiner Jung

The analysis part of Kieker is becoming larger and larger. And it will become even bigger when contribution from other projects will be merged. Thus, it is important to come up with a more stable package structure. Currently, we have several different taxonomies to group classes. Furthermore, some package names do not follow Java guidelines (camel case packages). All these properties makes it hard for users to find and use stages.

kieker.analysis various classes
kieker.analysis.analysisComponent deprecated stuff
kieker.analysis.annotation deprecated stuff
kieker.analysis.common empty, could contain common stuff
kieker.analysis.configuration pre defined configurations. However, they are useless. Either have a tool implemented which uses this, or create an example out of it.
kieker.analysis.debug Stages used for debugging
kieker.analysis.debug.hotspotdetection I am not sure whether this is debugging stuff
kieker.analysis.display Unclear whether this is used for anything, might be deprecated
kieker.analysis.display.annotation Unclear, might be deprecated
kieker.analysis.exception Random collection of exceptions
kieker.analysis.graph Contains a graph interface (could be replaced by Guava graphs)
kieker.analysis.graph.dependency graphbuilder and stages
kieker.analysis.graph.dependency.dot Factory for dot file export
kieker.analysis.graph.dependency.vertextypes Enumeration and mapper for vertrex types
kieker.analysis.graph.export Classes facilitating graph export ???
kieker.analysis.graph.export.blueprints
kieker.analysis.graph.export.dot
kieker.analysis.graph.export.graphml
kieker.analysis.graph.flattening Graph flattener stage uses also stuff from graph.traversal
kieker.analysis.graph.impl Implementation part of graph above.
kieker.analysis.graph.mapping mapper functions for graphs
kieker.analysis.graph.traversal some graph traversal helper
kieker.analysis.graph.util Utility classes for file operations and writing
kieker.analysis.graph.util.dot Dot file writing stuff (why is this here)
kieker.analysis.graph.util.dot.attributes a collection of enumerations
kieker.analysis.plugin Old plugins used for the old pipe and filter framework.
kieker.analysis.plugin.annotation
kieker.analysis.plugin.filter
kieker.analysis.plugin.filter.flow
kieker.analysis.plugin.filter.forward
kieker.analysis.plugin.filter.forward.util
kieker.analysis.plugin.filter.record
kieker.analysis.plugin.filter.select
kieker.analysis.plugin.filter.sink
kieker.analysis.plugin.filter.visualization
kieker.analysis.plugin.reader
kieker.analysis.plugin.reader.amqp
kieker.analysis.plugin.reader.database
kieker.analysis.plugin.reader.depcompression
kieker.analysis.plugin.reader.filesystem
kieker.analysis.plugin.reader.jms
kieker.analysis.plugin.reader.jmx
kieker.analysis.plugin.reader.list
kieker.analysis.plugin.reader.namedRecordPipe
kieker.analysis.plugin.reader.newio
kieker.analysis.plugin.reader.newio.deserializer
kieker.analysis.plugin.reader.tcp
kieker.analysis.plugin.reader.timer
kieker.analysis.plugin.reader.util
kieker.analysis.repository deprecated old repository implementation
kieker.analysis.repository.annotation
kieker.analysis.signature Java signature functionality. Here we should add functionality for other languages and move specific Java stuff to signature.java
kieker.analysis.sink Various sink stages, better moved to stage.sink
kieker.analysis.sink.display More sinks for specific output purposes. Makes no sense to call them display
kieker.analysis.source generic source classes used by different reader/source stages
kieker.analysis.source.amqp for amqp
kieker.analysis.source.file for files
kieker.analysis.source.jms for jms
kieker.analysis.source.namedpipe ...
kieker.analysis.source.rest
kieker.analysis.source.rewriter Rewriter, currently only used by TCP.
kieker.analysis.source.tcp
kieker.analysis.source.time Two identical stages with different names.
kieker.analysis.stage Generic stages
kieker.analysis.stage.adaptation Stages controlling adaptation (thematic package name)
kieker.analysis.stage.adaptation.data data structures
kieker.analysis.stage.adaptation.events events generated by the stage
kieker.analysis.stage.events throughput measuring filter should be stage
kieker.analysis.stage.events.delayfilter delays events to replay them in the correct speed
*kieker.analysis.stage.events.delayfilter.components
kieker.analysis.stage.flow trace/flow event related stages, should be trace.flow (thematic package name)
kieker.analysis.stage.forward Yet more stages addressing throughput and counting (see events)
kieker.analysis.stage.general Random stage collection
kieker.analysis.stage.model Stages to create an architecture model, maybe needs a better name (thematic package name)
kieker.analysis.stage.model.data specific data structure and events
kieker.analysis.stage.select Filtering and routing stages, stages from stage could be moved here
kieker.analysis.stage.select.timestampfilter Routing based on timestamps
kieker.analysis.stage.select.timestampfilter.components Whole lot of filters used to do so.
kieker.analysis.stage.select.traceidfilter Filter by traceId
kieker.analysis.stage.select.traceidfilter.components addition filters for trace id filtering, maybe not needed
kieker.analysis.statistics Statistics stages, better moved inside stage (thematic package name)
kieker.analysis.statistics.calculating
kieker.analysis.time Various time related stages (called filters, but they are not)
kieker.analysis.time.events
kieker.analysis.trace A lot of stages used for trace analysis, should be moved to stage.trace (thematic package name)
kieker.analysis.trace.aggregation
kieker.analysis.trace.execution
kieker.analysis.trace.graph
kieker.analysis.trace.graph.dot
kieker.analysis.trace.reconstruction
kieker.analysis.trace.repository
kieker.analysis.trace.sink Should trace sinks be here? If so should other sinks be grouped accordingly?
kieker.analysis.trace.traversal
kieker.analysis.tt Must be sorted in the above structure or deleted when deprecated
kieker.analysis.util Random stuff
kieker.analysis.util.emf ??? Maybe used to handle Ecore models?
kieker.analysis.util.stage Some filtering and other stages
kieker.analysis.util.stage.trigger Event more random stuff to run stages on triggers
kieker.analysis.util.time More time stuff, could be replaced with existing Java classes
kieker.analysis.util.timeseries Time series classes
kieker.visualization Oddly named sinks for visualization (maybe obsolete)

rju commented 1 week ago

author rju -- Wed, 13 Oct 2021 14:22:53 +0200

I would propose that we group stages by topic/thematic grouping which are:

architecture recovery (ecore model)
trace analysis
behavior analysis (not yet moved to Kieker, but I will do this when the structure is right)
architecture analysis?
graph based metrics
event flow
receiving data
adaptation

Sinks are special to the various analyses. Thus, I would put them with the respective topic.

There are also technical groupings:

Architecture model (maybe we will support different ones in future)
Sinks for specific file or output formats which are either “abstract” or “generic”.

rju commented 1 week ago

author rju -- Wed, 13 Oct 2021 14:29:57 +0200

Proposal for class names:

Consumer/Processing stages should have the suffix Stage
File and network readers/listeners (including composite stages) should be suffixed Reader
Filters are only stages that remove events from the stream. They are suffixed Filter
Routing stages should have a different suffix than Filter

rju commented 1 week ago

author DaGeRe -- Thu, 17 Feb 2022 10:26:36 +0100

When we start refactoring packages, it would be nice to fix the problem that we got Execution two times: Once kieker.model.system.model.Execution, which means something like OperationExecution (which would be a nice name, since we got OperationExecutionRecord, OperationExecutionAspect etc.), and teetime.framework.Execution, which means something like AnalysisExecution.

Furthermore, kieker.tools.trace.analysis.systemModel.Execution exists, which is deprecated - if we break all third party tools using Kieker by changing the package naming, we could fully remove this.

rju commented 1 week ago

author rju -- Thu, 24 Feb 2022 09:53:19 +0100

We agreed on a new package structure for the new stages and classes. The deprecated old pipe and filter API based classes remain where they are. The new structure is as follows:

architecture architecture related classes
- trace architecture trace analysis
- recovery architecture recovery stages
- metrics architecture evaluation stages
- repository architecture model repository stages
- adaptation adaptation of an architecture
behavior user behavior stages
generic contains generically usable stages
util utility classes/stages
- time time utilities

The plan is to rearrange the package structure for the TeeTime stuff in 1.16.0 (2.0.0) and remove the old stuff afterwards in 1.17.0 (2.1.0)

rju commented 1 week ago

author rju -- Tue, 24 May 2022 14:40:20 +0200

Started moving classes into new package structure.

rju commented 1 week ago

author rju -- Tue, 24 May 2022 18:32:50 +0200

kieker.analysis.plugin.reader.newio.deserializer might be useful, but is only used with old API (deprecated, ask Holger)
kieker.analysis.generic.sink.display (deprecated)
- MethodAndComponentFlowDisplaySink not generic
- ThreadsStatusDisplaySink not generic
- MemSwapUtilizationDisplaySink not generic
- CPUUtilizationDisplaySink not generic
To move
- kieker.analysis.stage
  - flow trace collection, filtering and reconstruction → kieker.analysis.architecture.trace (yes)
  - forward forward events unchanged, but counts, events, throughput, tee filter → kieker.analysis.generic (yes)
  - general
    - array and collection to element conversion → kieker.analysis.generic (yes)
    - HandleMetadataRecordStage.java processes KiekerMetaData record → ???
    - ControlledEventReleaseStage → ???
    - ListCollectionFilter collects incoming events and stores them in a list → ??? debug
  - model Create architecture model and prepocessing stages → kieker.analysis.architecture.recovery (yes)
    - data special events (not Kieker events) for architecture model processing
  - select
    - timestampfilter filter events by time window → generic
    - traceidfilter filter events by trace id → kieker.analysis.architecture.trace
    - TypeFilter → kieker.analysis.generic
- kieker.analysis.graph basic graph model (should be replaced by google graph library as used by other parts of Kieker)
  - dependency dependency graph for visualization
    - vertextypes format independent parts
    - dot dot format code
  - export export transformations for different file formats
  - flattening graph flattener, but a strange implementation
  - impl classes for the basic graph models
  - mapping mapping function for properties ???
  - traversal Graph traversers
  - util utils for dot and other more generic stuff
- kieker.analysis.signature signature parsing and interpretation functionality (recovery)
- kieker.analysis.statistics various statistics
- kieker.analyis.time.events timing events
- kieker.analysis.util various utils
  - debug
  - emf
  - stage
  - time
  - timeseries
- kieker.visualization teetime filters, but I am not sure what they are used for

rju commented 1 week ago

author rju -- Wed, 29 Jun 2022 12:51:42 +0200

Most classes in src moved to their correct target location.

Tests not moved so far.

rju commented 1 week ago

author rju -- Thu, 30 Jun 2022 12:42:02 +0200

kieker
- analysis
  - analysisComponent = (deprecated)
  - annotation = (deprecated)
  - architecture = all about architecture
  - adaptation = controlling probes and services instrumented with Kieker
    - data = Data model for adaptation
    - events = Control events
    - dependency = graph builders
      - export.dot = DotExportConfigurationFactory
    - metrics = metrics to analyze architecture (empty)
    - recovery = recover architecture from bvore and after operation events
      - events = Specific analysis events for recovery
      - signature = Signature processing
  - repository = handling a set of models for one architecture
  - trace = Trace process and handling
    - aggregation = Event aggregration
    - execution = Processing execution events
    - flow = Processing flow events
    - graph = Generating graph representations from traces
      - dot = Dot output sinks
    - reconstruction = Trace reconstruction stages
    - sink = Writing traces to files
    - traversal = Traversing a trace
  - behavior = managing and processing user behavior (empty, will use code from iObserve)
  - exception = general exceptions (some might be obsolete, needs work)
  - generic = generic stages that can be used on a wide variety of event types or objects
    - decompression = decompression of data streams
    - graph = Generic graph interfaces used with google graph library
      - flattening = flattens Kieker graphs
      - impl = implementation of the graph classes
      - mapping = various mappers used to produce file names for graphs
      - traversal = traversing through graphs
    - sink = various uncategorized sink stages (needs work)
      - graph = specific sinks for graphs
        
        blueprints = conversion to blueprints graph
        
        dot = dot file exporter for any graph model
        
        graphml = graphml exporter for any graph model
    - source = various event sources
      - amqp
      - file
      - namedpipe
      - rest
      - rewriter
      - tcp
      - time
    - time = time related generic stages
  - plugin = (old stuff, deprecated)
  - repository = (deprecated)
  - stage = (deprecated)
  - statistics = various statistics stages
    - calculating
  - util = wide variaty of classes with different purposes
    - emf
    - stage
      - trigger
    - time
    - timeseries

rju commented 1 week ago

author rju -- Wed, 6 Jul 2022 12:39:33 +0200

Updated version

kieker-monitoring / kieker

[KIEKER-1903] Define a comprehensive and stable package structure #1936