iamseth / oracledb_exporter

Prometheus Oracle database exporter.
MIT License
488 stars 236 forks source link

0.3.0rc1 broke #113 #164

Open Hanoko opened 3 years ago

Hanoko commented 3 years ago

Hi,

I have updated by docker to 0.3.0rc1 and the #113 is broken now. I tested it from the branch you created for it on 0.2.5 and it was working fine.

I think it is doing something with the SLQ from both files, because I am getting this:

oracledb_materialized_view_sec_refresh{compile_state="NEEDS_COMPILE",queue=""} 359
oracledb_materialized_view_sec_refresh{compile_state="VALID",queue=""} 4

Where the queue label comes from other file.

And getting this errors:

time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=sec_refresh,metricHelp=Total seconds refresh.,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=sec_refresh,metricHelp=Total seconds refresh.,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=sec_refresh,metricHelp=Total seconds refresh.,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=sec_refresh,metricHelp=Total seconds refresh.,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=sec_refresh,metricHelp=Total seconds refresh.,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=sec_refresh,metricHelp=Total seconds refresh.,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=sec_refresh,metricHelp=Total seconds refresh.,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=sec_refresh,metricHelp=Total seconds refresh.,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="Unable to convert current value to float (metric=sec_refresh,metricHelp=Total seconds refresh.,value=<>)" source="main.go:318"
time="2021-02-09T16:05:30Z" level=error msg="error gathering metrics: 41 error(s) occurred:\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"NEEDS_COMPILE\" > label:<name:\"queue\" value:\"\" > gauge:<value:640 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"NEEDS_COMPILE\" > label:<name:\"queue\" value:\"\" > gauge:<value:140 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"NEEDS_COMPILE\" > label:<name:\"queue\" value:\"\" > gauge:<value:2 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:13 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:4 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:59 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"NEEDS_COMPILE\" > label:<name:\"queue\" value:\"\" > gauge:<value:452 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:13 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:5 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:4 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:5 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:49 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:6 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:5 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:5 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:15 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:5 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:4 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:6 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:5 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:5 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:5 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"NEEDS_COMPILE\" > label:<name:\"queue\" value:\"\" > gauge:<value:4 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"NEEDS_COMPILE\" > label:<name:\"queue\" value:\"\" > gauge:<value:2 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"NEEDS_COMPILE\" > label:<name:\"queue\" value:\"\" > gauge:<value:1 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"NEEDS_COMPILE\" > label:<name:\"queue\" value:\"\" > gauge:<value:2 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:159 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:37 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:38 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"NEEDS_COMPILE\" > label:<name:\"queue\" value:\"\" > gauge:<value:92 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:45 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:4 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:5 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:5 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:5 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:26 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:63 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:6 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:5 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"VALID\" > label:<name:\"queue\" value:\"\" > gauge:<value:63 > } was collected before with the same name and label values\n* collected metric \"oracledb_materialized_view_sec_refresh\" { label:<name:\"compile_state\" value:\"NEEDS_COMPILE\" > label:<name:\"queue\" value:\"\" > gauge:<value:35 > } was collected before with the same name and label values\n" source="log.go:181"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"
time="2021-02-09T16:06:30Z" level=error msg="Unable to convert current value to float (metric=total,metricHelp=Total UBEs waiting,value=<>)" source="main.go:318"

This are my custom metric files:

[[metric]]
context = "e1_waiting_ube"
labels = [ "queue" ]
metricsdesc = { total = "Total UBEs waiting" }
request = "select distinct(trim(jcjobque)) queue,  SUM(CASE WHEN jcjobsts = 'W' THEN 1 ELSE 0 END)  total from svm920.f986110 group by jcjobque"

[[metric]]
context = "process_commit"
metricsdesc = { total="total sessions with high waits for event 'log file sync'." }
request = "SELECT count(*) as total FROM v$session_event A, v$session b, v$process p, svm920.f986110 c WHERE A.event='log file sync' AND b.wait_class='Commit' AND A.SID=b.SID AND A.time_waited_micro>700000000 AND b.status='ACTIVE' AND b.paddr=p.addr AND b.process=c.jcprocessid"
ignorezeroresult = true
[[metric]]
context = "materialized_view"
labels = [ "mview_name", "compile_state" ]
metricsdesc = { sec_refresh="Total seconds refresh."}
request = "select extract( day from (to_timestamp(to_char(LAST_REFRESH_END_TIME, 'yyyy-mm-dd hh24:mi:ss'), 'yyyy-mm-dd hh24:mi:ss') - to_timestamp(to_char(LAST_REFRESH_DATE, 'yyyy-mm-dd hh24:mi:ss'),'yyyy-mm-dd hh24:mi:ss')))*24*60*60 + extract( hour from (to_timestamp(to_char(LAST_REFRESH_END_TIME, 'yyyy-mm-dd hh24:mi:ss'), 'yyyy-mm-dd hh24:mi:ss') - to_timestamp(to_char(LAST_REFRESH_DATE, 'yyyy-mm-dd hh24:mi:ss'),'yyyy-mm-dd hh24:mi:ss')))*60*60 + extract( minute from (to_timestamp(to_char(LAST_REFRESH_END_TIME, 'yyyy-mm-dd hh24:mi:ss'), 'yyyy-mm-dd hh24:mi:ss') - to_timestamp(to_char(LAST_REFRESH_DATE, 'yyyy-mm-dd hh24:mi:ss'),'yyyy-mm-dd hh24:mi:ss')))*60 + extract( second from (to_timestamp(to_char(LAST_REFRESH_END_TIME, 'yyyy-mm-dd hh24:mi:ss'), 'yyyy-mm-dd hh24:mi:ss') - to_timestamp(to_char(LAST_REFRESH_DATE, 'yyyy-mm-dd hh24:mi:ss'),'yyyy-mm-dd hh24:mi:ss'))) as sec_refresh, mview_name, compile_state from dba_mviews "
ignorezeroresult = true

Thanks in advance

Yannig commented 3 years ago

Hi,

Is it possible to provide the raw results of these requests?

Regards,

Hanoko commented 3 years ago

Sure, here it is. Check the oracledb_materialized_view_sec_refresh metric how it is wrong:

# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 0
go_gc_duration_seconds{quantile="0.25"} 0
go_gc_duration_seconds{quantile="0.5"} 0
go_gc_duration_seconds{quantile="0.75"} 0
go_gc_duration_seconds{quantile="1"} 0
go_gc_duration_seconds_sum 0
go_gc_duration_seconds_count 0
# HELP go_goroutines Number of goroutines that currently exist.
# TYPE go_goroutines gauge
go_goroutines 8
# HELP go_info Information about the Go environment.
# TYPE go_info gauge
go_info{version="go1.14.9"} 1
# HELP go_memstats_alloc_bytes Number of bytes allocated and still in use.
# TYPE go_memstats_alloc_bytes gauge
go_memstats_alloc_bytes 1.69048e+06
# HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed.
# TYPE go_memstats_alloc_bytes_total counter
go_memstats_alloc_bytes_total 1.69048e+06
# HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table.
# TYPE go_memstats_buck_hash_sys_bytes gauge
go_memstats_buck_hash_sys_bytes 3705
# HELP go_memstats_frees_total Total number of frees.
# TYPE go_memstats_frees_total counter
go_memstats_frees_total 1961
# HELP go_memstats_gc_cpu_fraction The fraction of this program's available CPU time used by the GC since the program started.
# TYPE go_memstats_gc_cpu_fraction gauge
go_memstats_gc_cpu_fraction 0
# HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata.
# TYPE go_memstats_gc_sys_bytes gauge
go_memstats_gc_sys_bytes 3.436808e+06
# HELP go_memstats_heap_alloc_bytes Number of heap bytes allocated and still in use.
# TYPE go_memstats_heap_alloc_bytes gauge
go_memstats_heap_alloc_bytes 1.69048e+06
# HELP go_memstats_heap_idle_bytes Number of heap bytes waiting to be used.
# TYPE go_memstats_heap_idle_bytes gauge
go_memstats_heap_idle_bytes 6.3512576e+07
# HELP go_memstats_heap_inuse_bytes Number of heap bytes that are in use.
# TYPE go_memstats_heap_inuse_bytes gauge
go_memstats_heap_inuse_bytes 2.74432e+06
# HELP go_memstats_heap_objects Number of allocated objects.
# TYPE go_memstats_heap_objects gauge
go_memstats_heap_objects 20290
# HELP go_memstats_heap_released_bytes Number of heap bytes released to OS.
# TYPE go_memstats_heap_released_bytes gauge
go_memstats_heap_released_bytes 6.3414272e+07
# HELP go_memstats_heap_sys_bytes Number of heap bytes obtained from system.
# TYPE go_memstats_heap_sys_bytes gauge
go_memstats_heap_sys_bytes 6.6256896e+07
# HELP go_memstats_last_gc_time_seconds Number of seconds since 1970 of last garbage collection.
# TYPE go_memstats_last_gc_time_seconds gauge
go_memstats_last_gc_time_seconds 0
# HELP go_memstats_lookups_total Total number of pointer lookups.
# TYPE go_memstats_lookups_total counter
go_memstats_lookups_total 0
# HELP go_memstats_mallocs_total Total number of mallocs.
# TYPE go_memstats_mallocs_total counter
go_memstats_mallocs_total 22251
# HELP go_memstats_mcache_inuse_bytes Number of bytes in use by mcache structures.
# TYPE go_memstats_mcache_inuse_bytes gauge
go_memstats_mcache_inuse_bytes 3472
# HELP go_memstats_mcache_sys_bytes Number of bytes used for mcache structures obtained from system.
# TYPE go_memstats_mcache_sys_bytes gauge
go_memstats_mcache_sys_bytes 16384
# HELP go_memstats_mspan_inuse_bytes Number of bytes in use by mspan structures.
# TYPE go_memstats_mspan_inuse_bytes gauge
go_memstats_mspan_inuse_bytes 45560
# HELP go_memstats_mspan_sys_bytes Number of bytes used for mspan structures obtained from system.
# TYPE go_memstats_mspan_sys_bytes gauge
go_memstats_mspan_sys_bytes 49152
# HELP go_memstats_next_gc_bytes Number of heap bytes when next garbage collection will take place.
# TYPE go_memstats_next_gc_bytes gauge
go_memstats_next_gc_bytes 4.473924e+06
# HELP go_memstats_other_sys_bytes Number of bytes used for other system allocations.
# TYPE go_memstats_other_sys_bytes gauge
go_memstats_other_sys_bytes 772231
# HELP go_memstats_stack_inuse_bytes Number of bytes in use by the stack allocator.
# TYPE go_memstats_stack_inuse_bytes gauge
go_memstats_stack_inuse_bytes 851968
# HELP go_memstats_stack_sys_bytes Number of bytes obtained from system for stack allocator.
# TYPE go_memstats_stack_sys_bytes gauge
go_memstats_stack_sys_bytes 851968
# HELP go_memstats_sys_bytes Number of bytes obtained from system.
# TYPE go_memstats_sys_bytes gauge
go_memstats_sys_bytes 7.1387144e+07
# HELP go_threads Number of OS threads created.
# TYPE go_threads gauge
go_threads 16
# HELP oracledb_asm_diskgroup_free Free space available on ASM disk group.
# TYPE oracledb_asm_diskgroup_free gauge
oracledb_asm_diskgroup_free{name="DATA"} 1.94598928384e+11
oracledb_asm_diskgroup_free{name="RECO"} 2.10067521536e+11
oracledb_asm_diskgroup_free{name="REDO1"} 8.47249408e+09
oracledb_asm_diskgroup_free{name="REDO2"} 8.47249408e+09
# HELP oracledb_asm_diskgroup_total Total size of ASM disk group.
# TYPE oracledb_asm_diskgroup_total gauge
oracledb_asm_diskgroup_total{name="DATA"} 1.503209193472e+12
oracledb_asm_diskgroup_total{name="RECO"} 2.14739976192e+11
oracledb_asm_diskgroup_total{name="REDO1"} 1.0733223936e+10
oracledb_asm_diskgroup_total{name="REDO2"} 1.0733223936e+10
# HELP oracledb_class_wait_avg_time_sec Total waits in seconds.
# TYPE oracledb_class_wait_avg_time_sec gauge
oracledb_class_wait_avg_time_sec{name="Application"} 0.36
oracledb_class_wait_avg_time_sec{name="Commit"} 0.18
oracledb_class_wait_avg_time_sec{name="Concurrency"} 0.26
oracledb_class_wait_avg_time_sec{name="Configuration"} 0.02
oracledb_class_wait_avg_time_sec{name="Idle"} 0.01
oracledb_class_wait_avg_time_sec{name="Network"} 66.76
oracledb_class_wait_avg_time_sec{name="Other"} 1.75
oracledb_class_wait_avg_time_sec{name="Scheduler"} 0.01
oracledb_class_wait_avg_time_sec{name="System I/O"} 0.21
oracledb_class_wait_avg_time_sec{name="User I/O"} 3.81
# HELP oracledb_connection_total Total count of processes or sessions.
# TYPE oracledb_connection_total gauge
oracledb_connection_total{name="process"} 494
oracledb_connection_total{name="sessions"} 475
# HELP oracledb_e1_waiting_ube_total Total UBEs waiting
# TYPE oracledb_e1_waiting_ube_total gauge
oracledb_e1_waiting_ube_total{queue="CONSULT"} 0
oracledb_e1_waiting_ube_total{queue="I-ACTIVE"} 0
oracledb_e1_waiting_ube_total{queue="QAUTOMAT"} 0
oracledb_e1_waiting_ube_total{queue="QBATCH"} 0
oracledb_e1_waiting_ube_total{queue="QSGA"} 0
oracledb_e1_waiting_ube_total{queue="QSUBSYSTEM"} 0
oracledb_e1_waiting_ube_total{queue="QURGENCIAS"} 0
oracledb_e1_waiting_ube_total{queue="QWSEDI"} 0
oracledb_e1_waiting_ube_total{queue="R55EXPSGA"} 0
# HELP oracledb_event_wait_sec_time Total waits for events in seconds.
# TYPE oracledb_event_wait_sec_time gauge
oracledb_event_wait_sec_time{name="ADR block file read"} 0.36
oracledb_event_wait_sec_time{name="Backup: MML write backup piece"} 848423.59
oracledb_event_wait_sec_time{name="Data file init write"} 97.83
oracledb_event_wait_sec_time{name="control file parallel write"} 278.86
oracledb_event_wait_sec_time{name="control file sequential read"} 33036.03
oracledb_event_wait_sec_time{name="db file parallel read"} 35762.82
oracledb_event_wait_sec_time{name="db file scattered read"} 50737.17
oracledb_event_wait_sec_time{name="db file sequential read"} 335340.32
oracledb_event_wait_sec_time{name="db file single write"} 9.25
oracledb_event_wait_sec_time{name="direct path read"} 141295.23
oracledb_event_wait_sec_time{name="direct path read temp"} 9366.86
oracledb_event_wait_sec_time{name="direct path write"} 462.92
oracledb_event_wait_sec_time{name="direct path write temp"} 10914.71
oracledb_event_wait_sec_time{name="local write wait"} 104.3
oracledb_event_wait_sec_time{name="log buffer space"} 0.86
oracledb_event_wait_sec_time{name="log file switch (checkpoint incomplete)"} 219.57
oracledb_event_wait_sec_time{name="log file switch completion"} 42.05
oracledb_event_wait_sec_time{name="log file sync"} 521673.41
oracledb_event_wait_sec_time{name="log file sync: SCN ordering"} 0
oracledb_event_wait_sec_time{name="oracle thread bootstrap"} 69.44
oracledb_event_wait_sec_time{name="read by other session"} 1464.22
oracledb_event_wait_sec_time{name="row cache read"} 0
oracledb_event_wait_sec_time{name="wait for unread message on broadcast channel"} 530823.38
# HELP oracledb_exporter_last_scrape_duration_seconds Duration of the last scrape of metrics from Oracle DB.
# TYPE oracledb_exporter_last_scrape_duration_seconds gauge
oracledb_exporter_last_scrape_duration_seconds 1.292612704
# HELP oracledb_exporter_last_scrape_error Whether the last scrape of metrics from Oracle DB resulted in an error (1 for error, 0 for success).
# TYPE oracledb_exporter_last_scrape_error gauge
oracledb_exporter_last_scrape_error 0
# HELP oracledb_exporter_scrapes_total Total number of times Oracle DB was scraped for metrics.
# TYPE oracledb_exporter_scrapes_total counter
oracledb_exporter_scrapes_total 2
# HELP oracledb_fragmentation_total total tables (> 1G) with fragmentation greater than 50%.
# TYPE oracledb_fragmentation_total gauge
oracledb_fragmentation_total 0
# HELP oracledb_index_unusable_total Total indexes with status: UNUSABLE.
# TYPE oracledb_index_unusable_total gauge
oracledb_index_unusable_total 0
# HELP oracledb_locks_deadlocks_total total blocking sessions.
# TYPE oracledb_locks_deadlocks_total gauge
oracledb_locks_deadlocks_total 0
# HELP oracledb_materialized_view_sec_refresh Total seconds refresh.
# TYPE oracledb_materialized_view_sec_refresh gauge
oracledb_materialized_view_sec_refresh{compile_state="NEEDS_COMPILE",queue=""} 287
oracledb_materialized_view_sec_refresh{compile_state="VALID",queue=""} 4
# HELP oracledb_object_invalid_total Total objects with status: INVALID.
# TYPE oracledb_object_invalid_total gauge
oracledb_object_invalid_total{type="MATERIALIZED VIEW"} 11
oracledb_object_invalid_total{type="VIEW"} 1
# HELP oracledb_parameters_memory_value Total value of parameter in bytes.
# TYPE oracledb_parameters_memory_value gauge
oracledb_parameters_memory_value{name="pga_aggregate_limit"} 0
oracledb_parameters_memory_value{name="pga_aggregate_target"} 4.294967296e+09
oracledb_parameters_memory_value{name="sga_max_size"} 2.5769803776e+10
oracledb_parameters_memory_value{name="sga_min_size"} 0
oracledb_parameters_memory_value{name="sga_target"} 2.5769803776e+10
# HELP oracledb_parameters_value Total value of parameter.
# TYPE oracledb_parameters_value gauge
oracledb_parameters_value{name="process"} 1200
oracledb_parameters_value{name="sessions"} 2000
# HELP oracledb_pga_used_bytes_limit Limit value PGA
# TYPE oracledb_pga_used_bytes_limit gauge
oracledb_pga_used_bytes_limit 4.294967296e+09
# HELP oracledb_pga_used_bytes_value total PGA inuse.
# TYPE oracledb_pga_used_bytes_value gauge
oracledb_pga_used_bytes_value 3.092344832e+09
# HELP oracledb_process_commit_total total sessions with high waits for event 'log file sync'.
# TYPE oracledb_process_commit_total gauge
oracledb_process_commit_total 0
# HELP oracledb_recovery_file_bytes_total Limit size in bytes for recovery file dest.
# TYPE oracledb_recovery_file_bytes_total gauge
oracledb_recovery_file_bytes_total{name="+RECO"} 1.9327352832e+11
# HELP oracledb_recovery_file_bytes_used Used space in the recovery file dest in bytes.
# TYPE oracledb_recovery_file_bytes_used gauge
oracledb_recovery_file_bytes_used{name="+RECO"} 4.526702592e+09
# HELP oracledb_rman_last_run_total Total backups executed during the last 7 days.
# TYPE oracledb_rman_last_run_total gauge
oracledb_rman_last_run_total 0
# HELP oracledb_scheduler_job_sec_running Total time running.
# TYPE oracledb_scheduler_job_sec_running gauge
oracledb_scheduler_job_sec_running{job_name="ATT_JOB_REFRESH_GENERAL"} 780
oracledb_scheduler_job_sec_running{job_name="GATHER_PRODDTA_STALE"} 1080
oracledb_scheduler_job_sec_running{job_name="GATHER_STATS_DD920"} 0
oracledb_scheduler_job_sec_running{job_name="GATHER_STATS_OL920"} 0
oracledb_scheduler_job_sec_running{job_name="GATHER_STATS_PD920"} 120
oracledb_scheduler_job_sec_running{job_name="GATHER_STATS_PRODCTL"} 0
oracledb_scheduler_job_sec_running{job_name="GATHER_STATS_PRODDTA_F4211"} 180
oracledb_scheduler_job_sec_running{job_name="GATHER_STATS_SVM920"} 0
oracledb_scheduler_job_sec_running{job_name="GATHER_STATS_SY920"} 60
oracledb_scheduler_job_sec_running{job_name="PURGE_F45511W"} 0
oracledb_scheduler_job_sec_running{job_name="PURGE_WC"} 120
# HELP oracledb_scheduler_job_status Status JOB (4:FAILED; 3:STOPPED; 2:RUNNING; 1:SUCCEEDED;).
# TYPE oracledb_scheduler_job_status gauge
oracledb_scheduler_job_status{job_name="ATT_JOB_REFRESH_GENERAL"} 1
oracledb_scheduler_job_status{job_name="GATHER_PRODDTA_STALE"} 1
oracledb_scheduler_job_status{job_name="GATHER_STATS_DD920"} 1
oracledb_scheduler_job_status{job_name="GATHER_STATS_OL920"} 1
oracledb_scheduler_job_status{job_name="GATHER_STATS_PD920"} 1
oracledb_scheduler_job_status{job_name="GATHER_STATS_PRODCTL"} 1
oracledb_scheduler_job_status{job_name="GATHER_STATS_PRODDTA_F4211"} 1
oracledb_scheduler_job_status{job_name="GATHER_STATS_SVM920"} 1
oracledb_scheduler_job_status{job_name="GATHER_STATS_SY920"} 1
oracledb_scheduler_job_status{job_name="PURGE_F45511W"} 1
oracledb_scheduler_job_status{job_name="PURGE_WC"} 1
# HELP oracledb_session_used_temp_total total sessions using more than 1500MB in temp.
# TYPE oracledb_session_used_temp_total gauge
oracledb_session_used_temp_total 0
# HELP oracledb_switch_logfile_total Total switch logfiles in one hour.
# TYPE oracledb_switch_logfile_total gauge
oracledb_switch_logfile_total 1
# HELP oracledb_tablespaces_bytes_free Total free space in tablespace in bytes
# TYPE oracledb_tablespaces_bytes_free gauge
oracledb_tablespaces_bytes_free{tablespace="BIMIKEL"} 9.437184e+06
oracledb_tablespaces_bytes_free{tablespace="DD920I"} 5.9015168e+08
oracledb_tablespaces_bytes_free{tablespace="DD920T"} 4.01866752e+08
oracledb_tablespaces_bytes_free{tablespace="OL920I"} 8.3165184e+07
oracledb_tablespaces_bytes_free{tablespace="OL920T"} 6.12368384e+08
oracledb_tablespaces_bytes_free{tablespace="PD920I"} 2.778857472e+09
oracledb_tablespaces_bytes_free{tablespace="PD920T"} 7.025393664e+09
oracledb_tablespaces_bytes_free{tablespace="PRODCTLI"} 2.458845184e+09
oracledb_tablespaces_bytes_free{tablespace="PRODCTLT"} 1.72457984e+09
oracledb_tablespaces_bytes_free{tablespace="PRODDTAI"} 2.6695368704e+10
oracledb_tablespaces_bytes_free{tablespace="PRODDTAT"} 3.8753533952e+10
oracledb_tablespaces_bytes_free{tablespace="SVM920I"} 7.84334848e+08
oracledb_tablespaces_bytes_free{tablespace="SVM920T"} 1.419968512e+09
oracledb_tablespaces_bytes_free{tablespace="SY920I"} 2.835480576e+09
oracledb_tablespaces_bytes_free{tablespace="SY920T"} 1.3828096e+10
oracledb_tablespaces_bytes_free{tablespace="SYSAUX"} 5.026742272e+09
oracledb_tablespaces_bytes_free{tablespace="SYSTEM"} 7.26204416e+08
oracledb_tablespaces_bytes_free{tablespace="USERS"} 5.20617984e+08
# HELP oracledb_tablespaces_bytes_total Total tablespace size in bytes.
# TYPE oracledb_tablespaces_bytes_total gauge
oracledb_tablespaces_bytes_total{tablespace="BIMIKEL"} 1.048576e+07
oracledb_tablespaces_bytes_total{tablespace="DD920I"} 1.073741824e+09
oracledb_tablespaces_bytes_total{tablespace="DD920T"} 2.147483648e+09
oracledb_tablespaces_bytes_total{tablespace="OL920I"} 2.68435456e+08
oracledb_tablespaces_bytes_total{tablespace="OL920T"} 1.073741824e+09
oracledb_tablespaces_bytes_total{tablespace="PD920I"} 5.36870912e+09
oracledb_tablespaces_bytes_total{tablespace="PD920T"} 1.610612736e+10
oracledb_tablespaces_bytes_total{tablespace="PRODCTLI"} 6.442450944e+09
oracledb_tablespaces_bytes_total{tablespace="PRODCTLT"} 8.589934592e+09
oracledb_tablespaces_bytes_total{tablespace="PRODDTAI"} 3.52187318272e+11
oracledb_tablespaces_bytes_total{tablespace="PRODDTAT"} 7.77389080576e+11
oracledb_tablespaces_bytes_total{tablespace="SVM920I"} 1.073741824e+09
oracledb_tablespaces_bytes_total{tablespace="SVM920T"} 4.294967296e+09
oracledb_tablespaces_bytes_total{tablespace="SY920I"} 4.290772992e+09
oracledb_tablespaces_bytes_total{tablespace="SY920T"} 3.4359738368e+10
oracledb_tablespaces_bytes_total{tablespace="SYSAUX"} 1.5032385536e+10
oracledb_tablespaces_bytes_total{tablespace="SYSTEM"} 2.5690112e+09
oracledb_tablespaces_bytes_total{tablespace="USERS"} 5.2166656e+08
# HELP oracledb_tablespaces_bytes_used Used space in the tablespace in bytes.
# TYPE oracledb_tablespaces_bytes_used gauge
oracledb_tablespaces_bytes_used{tablespace="BIMIKEL"} 1.048576e+06
oracledb_tablespaces_bytes_used{tablespace="DD920I"} 4.83590144e+08
oracledb_tablespaces_bytes_used{tablespace="DD920T"} 1.745616896e+09
oracledb_tablespaces_bytes_used{tablespace="OL920I"} 1.85270272e+08
oracledb_tablespaces_bytes_used{tablespace="OL920T"} 4.6137344e+08
oracledb_tablespaces_bytes_used{tablespace="PD920I"} 2.589851648e+09
oracledb_tablespaces_bytes_used{tablespace="PD920T"} 9.080733696e+09
oracledb_tablespaces_bytes_used{tablespace="PRODCTLI"} 3.98360576e+09
oracledb_tablespaces_bytes_used{tablespace="PRODCTLT"} 6.865354752e+09
oracledb_tablespaces_bytes_used{tablespace="PRODDTAI"} 3.25491949568e+11
oracledb_tablespaces_bytes_used{tablespace="PRODDTAT"} 7.38635546624e+11
oracledb_tablespaces_bytes_used{tablespace="SVM920I"} 2.89406976e+08
oracledb_tablespaces_bytes_used{tablespace="SVM920T"} 2.874998784e+09
oracledb_tablespaces_bytes_used{tablespace="SY920I"} 1.455292416e+09
oracledb_tablespaces_bytes_used{tablespace="SY920T"} 2.0531642368e+10
oracledb_tablespaces_bytes_used{tablespace="SYSAUX"} 1.0005643264e+10
oracledb_tablespaces_bytes_used{tablespace="SYSTEM"} 1.842806784e+09
oracledb_tablespaces_bytes_used{tablespace="USERS"} 1.048576e+06
# HELP oracledb_tablespaces_temp_bytes_free Total free space in tablespace in bytes
# TYPE oracledb_tablespaces_temp_bytes_free gauge
oracledb_tablespaces_temp_bytes_free{tablespace_temp="TEMP"} 3.1173115904e+10
# HELP oracledb_tablespaces_temp_bytes_total Total tablespace size in bytes.
# TYPE oracledb_tablespaces_temp_bytes_total gauge
oracledb_tablespaces_temp_bytes_total{tablespace_temp="TEMP"} 3.4358689792e+10
# HELP oracledb_tablespaces_temp_bytes_used Used space in the tablespace in bytes.
# TYPE oracledb_tablespaces_temp_bytes_used gauge
oracledb_tablespaces_temp_bytes_used{tablespace_temp="TEMP"} 3.185573888e+09
# HELP oracledb_tablespaces_undo_bytes_active Total active segments in undo(bytes.)
# TYPE oracledb_tablespaces_undo_bytes_active gauge
oracledb_tablespaces_undo_bytes_active{tablespace_undo="UNDOTBS1"} 1.069023232e+09
# HELP oracledb_tablespaces_undo_bytes_total Total segments undo(bytes.)
# TYPE oracledb_tablespaces_undo_bytes_total gauge
oracledb_tablespaces_undo_bytes_total{tablespace_undo="UNDOTBS1"} 1.139539968e+09
# HELP oracledb_tablespaces_undo_bytes_total_tbs Total size of the undo tablespaces in bytes.
# TYPE oracledb_tablespaces_undo_bytes_total_tbs gauge
oracledb_tablespaces_undo_bytes_total_tbs{tablespace_undo="UNDOTBS1"} 3.171418112e+10
# HELP oracledb_up Whether the Oracle database server is up.
# TYPE oracledb_up gauge
oracledb_up 1
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 0.23
# HELP process_max_fds Maximum number of open file descriptors.
# TYPE process_max_fds gauge
process_max_fds 1.048576e+06
# HELP process_open_fds Number of open file descriptors.
# TYPE process_open_fds gauge
process_open_fds 10
# HELP process_resident_memory_bytes Resident memory size in bytes.
# TYPE process_resident_memory_bytes gauge
process_resident_memory_bytes 4.4138496e+07
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1.61363739346e+09
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 2.037149696e+09
# HELP process_virtual_memory_max_bytes Maximum amount of virtual memory available in bytes.
# TYPE process_virtual_memory_max_bytes gauge
process_virtual_memory_max_bytes -1
Yannig commented 3 years ago

Thank you, but I was talking about the raw values returned by Oracle :)

Hanoko commented 3 years ago

Sorry for the delay, here are the raw values: e1_waiting_ube: imagen process_commit: imagen materialized_view imagen

Hanoko commented 3 years ago

Hi @Yannig ,

Could you check this issue?

Thanks.

Hanoko commented 3 years ago

Hi @Yannig ,

I have been researching a bit more and I found this on the debug log:

time="2021-05-31T15:37:25+02:00" level=debug msg="Calling function ScrapeGenericValues()" source="main.go:299"
time="2021-05-31T15:37:25+02:00" level=debug msg="About to scrape metric: " source="main.go:237"
time="2021-05-31T15:37:25+02:00" level=debug msg="- Metric MetricsDesc:  map[sec_refresh:Total seconds refresh. total:Total UBEs waiting]" source="main.go:238"
time="2021-05-31T15:37:25+02:00" level=debug msg="- Metric Context:  e1_waiting_ube" source="main.go:239"
time="2021-05-31T15:37:25+02:00" level=debug msg="- Metric MetricsType:  map[]" source="main.go:240"
time="2021-05-31T15:37:25+02:00" level=debug msg="- Metric MetricsBuckets:  map[] (Ignored unless Histogram type)" source="main.go:241"
time="2021-05-31T15:37:25+02:00" level=debug msg="- Metric Labels:  [queue]" source="main.go:242"
time="2021-05-31T15:37:25+02:00" level=debug msg="- Metric FieldToAppend:  " source="main.go:243"
time="2021-05-31T15:37:25+02:00" level=debug msg="- Metric IgnoreZeroResult:  true" source="main.go:244"
time="2021-05-31T15:37:25+02:00" level=debug msg="- Metric Request:  select distinct(trim(jcjobque)) queue, SUM(CASE WHEN jcjobsts = 'W' THEN 1 ELSE 0 END) as total from svm920.f986110 group by jcjobque" source="main.go:245"
time="2021-05-31T15:37:25+02:00" level=debug msg="Calling function ScrapeGenericValues()" source="main.go:299"
time="2021-05-31T15:37:25+02:00" level=debug msg="About to scrape metric: " source="main.go:237"
time="2021-05-31T15:37:25+02:00" level=debug msg="- Metric MetricsDesc:  map[sec_refresh:Total seconds refresh. total:Total UBEs waiting]" source="main.go:238"
time="2021-05-31T15:37:25+02:00" level=debug msg="- Metric Context:  materialized_view" source="main.go:239"
time="2021-05-31T15:37:25+02:00" level=debug msg="- Metric MetricsType:  map[]" source="main.go:240"
time="2021-05-31T15:37:25+02:00" level=debug msg="- Metric MetricsBuckets:  map[] (Ignored unless Histogram type)" source="main.go:241"
time="2021-05-31T15:37:25+02:00" level=debug msg="- Metric Labels:  [queue compile_state]" source="main.go:242"
time="2021-05-31T15:37:25+02:00" level=debug msg="- Metric FieldToAppend:  " source="main.go:243"
time="2021-05-31T15:37:25+02:00" level=debug msg="- Metric IgnoreZeroResult:  true" source="main.go:244"
time="2021-05-31T15:37:25+02:00" level=debug msg="- Metric Request:  select extract( day from (to_timestamp(to_char(LAST_REFRESH_END_TIME, 'yyyy-mm-dd hh24:mi:ss'), 'yyyy-mm-dd hh24:mi:ss') - to_timestamp(to_char(LAST_REFRESH_DATE, 'yyyy-mm-dd hh24:mi:ss'),'yyyy-mm-dd hh24:mi:ss')))*24*60*60 + extract( hour from (to_timestamp(to_char(LAST_REFRESH_END_TIME, 'yyyy-mm-dd hh24:mi:ss'), 'yyyy-mm-dd hh24:mi:ss') - to_timestamp(to_char(LAST_REFRESH_DATE, 'yyyy-mm-dd hh24:mi:ss'),'yyyy-mm-dd hh24:mi:ss')))*60*60 + extract( minute from (to_timestamp(to_char(LAST_REFRESH_END_TIME, 'yyyy-mm-dd hh24:mi:ss'), 'yyyy-mm-dd hh24:mi:ss') - to_timestamp(to_char(LAST_REFRESH_DATE, 'yyyy-mm-dd hh24:mi:ss'),'yyyy-mm-dd hh24:mi:ss')))*60 + extract( second from (to_timestamp(to_char(LAST_REFRESH_END_TIME, 'yyyy-mm-dd hh24:mi:ss'), 'yyyy-mm-dd hh24:mi:ss') - to_timestamp(to_char(LAST_REFRESH_DATE, 'yyyy-mm-dd hh24:mi:ss'),'yyyy-mm-dd hh24:mi:ss'))) as sec_refresh, mview_name, compile_state from dba_mviews " source="main.go:245"
time="2021-05-31T15:37:25+02:00" level=debug msg="Calling function ScrapeGenericValues()" source="main.go:299"

The fields MetricsDesc and Labels are mixing the between the 2 metrics mentioned. Why could the exporter do that? Do you have any idea?

Best regards.

xafer337 commented 2 years ago

Hi, The multiple custom metrics are definitely not working correctly. I want to explain more precisely this problem, as it maybe wasn't clear enough:

When including more than one custom metric files in the CUSTOM_METRICS field ( #113 ), some labels of the metrics scraped get mixed out together, without any error shown.

We now have 2 files in the custom metrics: Environment="CUSTOM_METRICS=/etc/oracledb_exporter/plus_12_1.toml,/etc/oracledb_exporter/jde920.toml"

This is one of the metrics we want to collect, which is the only metric in one of the files (plus_12_1.toml). What I want you to look at are the two labels: [ 'mview_name', 'compile_state' ]

[[metric]]
context = "materialized_view"
labels = [ 'mview_name', 'compile_state' ]
metricsdesc = { sec_refresh="Total seconds refresh." }
request = '''
select extract( day from (to_timestamp(to_char(LAST_REFRESH_END_TIME, 'yyyy-mm-dd hh24:mi:ss'), 'yyyy-mm-dd hh24:mi:ss') - to_timestamp(to_char(LAST_REFRESH_DATE, 'yyyy-mm-dd hh24:mi:ss'),'yyyy-mm-dd hh24:mi:ss')))*24*60*60 + extract( hour from (to_timestamp(to_char(LAST_REFRESH_END_TIME, 'yyyy-mm-dd hh24:mi:ss'), 'yyyy-mm-dd hh24:mi:ss') - to_timestamp(to_char(LAST_REFRESH_DATE, 'yyyy-mm-dd hh24:mi:ss'),'yyyy-mm-dd hh24:mi:ss')))*60*60 + extract( minute from (to_timestamp(to_char(LAST_REFRESH_END_TIME, 'yyyy-mm-dd hh24:mi:ss'), 'yyyy-mm-dd hh24:mi:ss') - to_timestamp(to_char(LAST_REFRESH_DATE, 'yyyy-mm-dd hh24:mi:ss'),'yyyy-mm-dd hh24:mi:ss')))*60 + extract( second from (to_timestamp(to_char(LAST_REFRESH_END_TIME, 'yyyy-mm-dd hh24:mi:ss'), 'yyyy-mm-dd hh24:mi:ss') - to_timestamp(to_char(LAST_REFRESH_DATE, 'yyyy-mm-dd hh24:mi:ss'),'yyyy-mm-dd hh24:mi:ss'))) as sec_refresh, mview_name, compile_state from dba_mviews
'''

The other file (jde920.toml) has these 2 metrics. See that the label on the first query is [ "queue" ]

[[metric]]
context = "e1_waiting_ube"
labels = [ "queue" ]
metricsdesc = { total = "Total UBEs waiting" }
request = '''
select distinct(trim(jcjobque)) queue, SUM(CASE WHEN jcjobsts = 'W' THEN 1 ELSE 0 END) as total from svm920.f986110 group by jcjobque
'''
ignorezeroresult = true

[[metric]]
context = "process_commit"
metricsdesc = { total="total sessions with high waits for event 'log file sync'." }
request = '''
SELECT count(*) as total FROM v$session_event a, v$session b, v$process p, svm920.f986110 c WHERE a.event='log file sync' AND b.wait_class='Commit' AND a.SID=b.SID AND a.time_waited_micro>700000000 AND b.status='ACTIVE' AND b.paddr=p.addr AND b.process=c.jcprocessid
'''
ignorezeroresult = true

We have some default-metrics too, but aren't relevant for the matter. When launching the exporter, we correctly get back the metrics detailed on the second file (jde920.toml), but when we expected to get back a query for "materialized_view" with the 2 labels [ 'mview_name', 'compile_state' ], we receive these metrics among others:

oracledb_materialized_view_sec_refresh{compile_state="NEEDS_COMPILE",queue=""} 359 oracledb_materialized_view_sec_refresh{compile_state="VALID",queue=""} 4

As you can see, the exporter sort broke these metrics, mixing their labels. (compile_state from "plus_12_1.toml" and queue from "jde920.toml" are never together in any query)

Trying the exact same metrics and configurations in an exporter launched from the multi-custom-metrics branch gives us the correct metrics: oracledb_materialized_view_sec_refresh{mview_name="something",compile_state="something"}

I assume the problem started when the asynchronous request to database where implemented on 0.3.x version, with maybe some metrics getting mixed when they shouldn't.

I've been looking inside the code, trying to find any differences between the 0.2.5 multi-custom-metrics branch and the last 0.3.0rc1 version, but the only hypothesis I have is that one of the asynchronous requests breaking the multiple custom metrics (somehow).

I'm very interested in having this problem solved, and I'm willing to help you in anything you need.

Thanks in advance.

xafer337 commented 2 years ago

Hi @Yannig !

Have you checked this issue?

Thanks!