Config yaml file has five top level properties: version, logging, system, performance and metric.
version is always 2, since its for v2 pipeline
logging is the name of the defined logging (class LoggingFoo: here the name is LoggingFoo) which are singletons (one instance per any Name).
Sys, Perf and Met are the three entry points to use the Logger.
System is for default python (import logging) logging
Performance is for logging any hardware, benchmark,..etc metric-related log (thing that are already defined).
Metric is for input related logging (output value based on inputs).
If logging has any args for instantiation, one can define as
For the three entry points, the top level after the root logger (system, metric, performance) is called the tag. Tags are used to filter out and log in any matching tag by regex.
Under tags, we have three (four in metrics) properties,
func is the function to apply to any value of interest to log.
freq is the rate to log once a matching entry is encountered.
uses: is a list to specify which logger to use, selected by logger_id
capture (metric only): key of the dict, property name to extract from the output of an operator, ran in logging middleware
system: # Python/system level logging
".*": # reges tag to capture
- func: identity # func name or path/to/file.py:func_name
freq: 1 # rate to log
uses: # which logger to use, point to the log id above
- default
metric:
".*":
- func: max
freq: 1
uses:
- list
capture: # The key, property to log. Ex rtn: Operator = AddOneOperator(...), if capture is set to value, it would record rtn.capture, if tag above is set to any regex that recognizes AddOneOperator (currently set to ".*")
- ".*" # log every key, property that the dict, class has
In other words, for each root logger, what tags are we logging. With respect to tags, what function are we applying if any, at what rate and with respect to what loggers are we using
Root logger is a function of tags, tags are a fucntion of frequency, func and loggers.
Root logger(tag(freq, func, loggers, capture))
Examples
version: 2 # v2 uses 2
loggers:
list: # Logger ID
name: ListLogger # class name or path/to/file.py:ClassName
handler: None # Handler config, default set to default logging
default:
name: PythonLogger
handler: None
system: # Python/system level logging
".*": # reges tag to capture
- func: identity # func name or path/to/file.py:func_name
freq: 1 # rate to log
uses: # which logger to use, point to the log id above
- default
performance:
cpu:
- func: identity
freq: 1
uses:
- default
metric:
".*":
- func: max
freq: 1
uses:
- list
capture: # The key, property to log. Ex rtn: Operator = AddOneOperator(...), if capture is set to value, it would record rtn.capture, if tag above is set to any regex that recognizes AddOneOperator (currently set to ".*")
- ".*"
Config for validation
Description
Config yaml file has five top level properties: version, logging, system, performance and metric. version is always 2, since its for v2 pipeline logging is the name of the defined logging (class LoggingFoo: here the name is LoggingFoo) which are singletons (one instance per any Name). Sys, Perf and Met are the three entry points to use the Logger. System is for default python (import logging) logging Performance is for logging any hardware, benchmark,..etc metric-related log (thing that are already defined). Metric is for input related logging (output value based on inputs).
If logging has any args for instantiation, one can define as
For the three entry points, the top level after the root logger (system, metric, performance) is called the tag. Tags are used to filter out and log in any matching tag by regex. Under tags, we have three (four in metrics) properties, func is the function to apply to any value of interest to log. freq is the rate to log once a matching entry is encountered. uses: is a list to specify which logger to use, selected by logger_id
capture (metric only): key of the dict, property name to extract from the output of an operator, ran in logging middleware
In other words, for each root logger, what tags are we logging. With respect to tags, what function are we applying if any, at what rate and with respect to what loggers are we using
Root logger is a function of tags, tags are a fucntion of frequency, func and loggers. Root logger(tag(freq, func, loggers, capture))
Examples
Default config without any inputs
TODO: Handlers