Butler SenseOps Stats ("Butler-SOS") is a microservice publishing operational Qlik Sense metrics to InfluxDB, Prometheus and New Relic. Add Grafana for great looking dashboards and you get real-time monitoring of what happens inside a Qlik Sense environment.
What version of Node.js are you using? Not applicable if you use the standalone version of Butler SOS.
18.5.0
What command did you use to start Butler SOS?
./butler-sos
What operating system are you using?
Debian 12
What CPU architecture are you using?
x86_64
What Qlik Sense versions are you using?
2024 Feb sp7
Describe the Bug
In my case without setting QIX performance log level to info butler-sos gets nothing.
I use LocalLogConfig.xml from dsitrib(only changed host and port)
Config:
# Track individual users opening/closing apps and starting/stopping sessions.
# Requires log appender XML file(s) to be added to Sense server(s).
userEvents:
enable: true
excludeUser: # Optional blacklist of users that should be disregarded when it comes to user events
# - directory: LAB
# userId: testuser1
# - directory: LAB
# userId: testuser2
udpServerConfig:
serverHost: 192.168.23.59 # Host/IP where user event server will listen for events from Sense
portUserActivityEvents: 9995 # Port on which user event server will listen for events from Sense
tags: # Tags are added to the data before it's stored in InfluxDB
- name: env
value: TEST
# - name: foo
# value: bar
sendToMQTT:
enable: false # Set to true if user events should be forwarded as MQTT messages
postTo: # Control when and to which MQTT topics messages are sent
everythingTopic: # Topic to which all user events are sent
enable: true
topic: qliksense/userevent
sessionStartTopic: # Topic to which "session start" events are sent
enable: true
topic: qliksense/userevent/session/start
sessionStopTopic: # Topic to which "session stop" events are sent
enable: true
topic: qliksense/userevent/session/stop
connectionOpenTopic: # Topic to which "connection open" events are sent
enable: true
topic: qliksense/userevent/connection/open
connectionCloseTopic: # Topic to which "connection close" events are sent
enable: true
topic: qliksense/userevent/connection/close
sendToInfluxdb:
enable: true # Set to true if user events should be stored in InfluxDB
sendToNewRelic:
enable: false # Should log events be sent to New Relic?
destinationAccount:
# - First NR account
# - Second NR account
scramble: true # Should user info (user directory and user ID) be scrambled before sent to NR?
# Log events are used to capture Sense warnings, errors and fatals in real time
logEvents:
udpServerConfig:
serverHost: 192.168.23.59 # Host/IP where log event server will listen for events from Sense
portLogEvents: 9994 # Port on which log event server will listen for events from Sense
tags:
- name: env
value: TEST
# - name: foo
# value: bar
source:
engine:
enable: true # Should log events from the engine service be handled?
proxy:
enable: true # Should log events from the proxy service be handled?
repository:
enable: true # Should log events from the repository service be handled?
scheduler:
enable: true # Should log events from the scheduler service be handled?
qixPerf:
enable: true # Should log events relating to QIX performance be handled?
categorise: # Take actions on log events based on their content
enable: true
rules: # Rules are used to match log events to filters
- description: Find access denied errors
logLevel: # Log events of this Log level will be matched. WARN, ERROR, FATAL. Case insensitive.
- WARN
- ERROR
action: categorise # Action to take on matched log events. Possible values are categorise, drop
category: # Category to assign to matched log events. Name/value pairs.
# Will be added to InfluxDB datapoints as tags.
- name: qs_log_category
value: access-denied
filter: # Filter used to match log events. Case sensitive.
# - type: sw # Type of filter. sw = starts with, ew = ends with, so = substring of
# value: "Access was denied for User:"
- type: so
value: was denied for User
# - description: Find AD issues
# logLevel: # Log events of this Log level will be matched. WARN, ERROR, FATAL. Case insensitive.
# - ERROR
# - WARN
# action: categorise # Action to take on matched log events. Possible values are categorise, drop
# category: # Category to assign to matched log events. Name/value pairs.
# # Will be added to InfluxDB datapoints as tags.
# - name: qs_log_category
# value: user-directory
# filter: # Filter used to match log events. Case sensitive.
# - type: sw # Type of filter. sw = starts with, ew = ends with, so = substring of
# value: Duplicate entity with userId
- description: Qlik Sense service down
logLevel: # Log events of this Log level will be matched. WARN, ERROR, FATAL. Case insensitive.
- WARN
action: categorise # Action to take on matched log events. Possible values are categorise, drop
category: # Category to assign to matched log events. Name/value pairs.
# Will be added to InfluxDB datapoints as tags.
- name: qs_log_category
value: qs-service
filter: # Filter used to match log events. Case sensitive.
- type: sw # Type of filter. sw = starts with, ew = ends with, so = substring of
value: Failed to request service alive response from
- type: so # Type of filter. sw = starts with, ew = ends with, so = substring of
value: Unable to connect to the remote server
- description: Reload task failed
logLevel: # Log events of this Log level will be matched. WARN, ERROR, FATAL. Case insensitive.
- WARN
- ERROR
action: categorise # Action to take on matched log events. Possible values are categorise, drop
category: # Category to assign to matched log events. Name/value pairs.
# Will be added to InfluxDB datapoints as tags.
- name: qs_log_category
value: reload-failed
filter: # Filter used to match log events. Case sensitive.
- type: sw # Type of filter. sw = starts with, ew = ends with, so = substring of
value: Task finished with state FinishedFail
- type: sw # Type of filter. sw = starts with, ew = ends with, so = substring of
value: Task finished with state Error
- type: ew # Type of filter. sw = starts with, ew = ends with, so = substring of
value: Reload failed in Engine. Check engine or script logs.
- type: sw # Type of filter. sw = starts with, ew = ends with, so = substring of
value: Reload sequence was not successful (Result=False, Finished=True, Aborted=False) for engine connection with handle
ruleDefault: # Default rule to use if no other rules match the log event
enable: true
category:
- name: qs_log_category
value: unknown
enginePerformanceMonitor: # Detailed app performance data extraction from log events
enable: true # Should app performance data be extracted from log events?
appNameLookup: # Should app names be looked up based on app IDs?
enable: false
trackRejectedEvents:
enable: true # Should events that are rejected by the app performance monitor be tracked?
tags: # Tags are added to the data before it's stored in InfluxDB
- name: env
value: TEST
# - name: foo
# value: bar
monitorFilter: # What objects should be monitored? Entire apps or just specific object(s) within some specific app(s)?
# Two kinds of monitoring can be done:
# 1) Monitor all apps, except those listed for exclusion. This is defined in the allApps section.
# 2) Monitor only specific apps. This is defined in the appSpecific section.
# An event will be accepted if it matches any of the rules in the allApps section OR any of the rules in the appSpecific section.
allApps:
enable: true # Should all apps be monitored?
appExclude: # What apps should be excluded from monitoring?
# If both appId and appName are specified, both must match the event's data for it to be considered a match.
# - appId: 5b817efe-472d-43ce-8a31-6cce34af7de9
# - appName: Sales forecast
# - appId: f42d6b16-8faf-45ca-a783-59f9da47db6e
# appName: Inventory analysis
objectType:
allObjectTypes: true # Should all object types be monitored?
allObjectTypesExclude: # If allObjectTypes is set to true, the object types in this array are excluded from monitoring.
# someObjectTypesInclude (below) is ignored in that case.
# - LoadModelList
# - <Unknown>
# - linechart
# - map
someObjectTypesInclude: # What object types should be included in monitoring?
# Only applicable if allObjectTypes is set to false.
# - LoadModelList
# - sheet
# - barchart
method:
allMethods: true # Should all methods be monitored?
allMethodsExclude: # If allMethods is set to true, the methods in this array are excluded from monitoring.
# someMethodsInclude (below) is ignored in that case.
# - Global::OpenApp
# - Doc::GetAppLayout
# - Doc::CreateSessionObject
someMethodsInclude: # What methods should be included in monitoring?
# Only applicable if allMethods is set to false.
# - GenericObject::GetLayout
# - GenericObject::GetHyperCubeContinuousData
appSpecific:
enable: false # Should app specific monitoring be done?
app:
- include: # What apps should be monitored?
# If both appId and appName are specified, both must match the event's data for it to be considered a match.
# - appId: d7cf16f9-6a95-462a-9ff1-a6d413326de4
# - appName: Budget 2025
# - appId: 6931136d-c234-4358-a40c-e37153aba7c9
# appName: Sales basket analysis
objectType:
allObjectTypes: true # Should all object types be monitored?
allObjectTypesExclude: # If allObjectTypes is set to true, the object types in this array are excluded from monitoring.
# someObjectTypesInclude (below) is ignored in that case.
# - table
# - map
someObjectTypesInclude: # What object types should be included in monitoring?
# Only applicable if allObjectTypes is set to false.
# - sheet
# - barchart
# - linechart
# - map
appObject:
allAppObjects: true # Should all app objects be monitored?
allAppObjectsExclude: # If allAppObjects is set to true, the app objects in this array are excluded from monitoring.
# someAppObjectsInclude (below) is ignored in that case.
# - objectId: AaBbCc
# - objectId: DdEeFf
someAppObjectsInclude: # What app objects should be included in monitoring?
# Only applicable if allAppObjects is set to false.
# - objectId: YJEpPT
method:
allMethods: true # Should all methods be monitored?
allMethodsExclude: # If allMethods is set to true, the methods in this array are excluded from monitoring.
# someMethodsInclude (below) is ignored in that case.
# - Global::OpenApp
# - Doc::GetAppLayout
# - Doc::CreateSessionObject
someMethodsInclude: # What methods should be included in monitoring?
# Only applicable if allMethods is set to false.
# - GenericObject::GetLayout
# - GenericObject::GetHyperCubeContinuousData
sendToMQTT:
enable: false # Should log events be sent as MQTT messages?
baseTopic: qliksense/logevent # What topic should log events be forwarded to?
postTo:
baseTopic: true
subsystemTopics: true # Should log events be sent to subtopics corresponding to the QSEoW subsystems where the events originated?
sendToInfluxdb:
enable: true # Should log events be stored in InfluxDB?
sendToNewRelic:
enable: false # Should log events be sent to New Relic?
destinationAccount:
# - First NR account
# - Second NR account
source:
engine:
enable: true # Should log events from the engine service be handled?
logLevel:
error: true # Should error level log events be handled by Butler SOS?
warn: true # Should warning level log events be handled by Butler SOS?
proxy:
enable: true # Should log events from the proxy service be handled?
logLevel:
error: true # Should error level log events be handled by Butler SOS?
warn: true # Should warning level log events be handled by Butler SOS?
repository:
enable: true # Should log events from the repository service be handled?
logLevel:
error: true # Should error level log events be handled by Butler SOS?
warn: true # Should warning level log events be handled by Butler SOS?
scheduler:
enable: true # Should log events from the scheduler service be handled?
logLevel:
error: true # Should error level log events be handled by Butler SOS?
warn: true # Should warning level log events be handled by Butler SOS?
What version of Butler SOS are you using?
11.0.3
What version of Node.js are you using? Not applicable if you use the standalone version of Butler SOS.
18.5.0
What command did you use to start Butler SOS?
./butler-sos
What operating system are you using?
Debian 12
What CPU architecture are you using?
x86_64
What Qlik Sense versions are you using?
2024 Feb sp7
Describe the Bug
In my case without setting QIX performance log level to info butler-sos gets nothing. I use LocalLogConfig.xml from dsitrib(only changed host and port)
Config:
Expected Behavior
No response
To Reproduce
No response