Enterprise Manager 12c is a great monitoring tool, with it you can monitor a wide range of target types from databases to middleware; although out-of-the-box metrics can suit your monitoring requirements they can generate a considerable amount of white noise. In order to reduce this noise first you have to identify which are the top alerts in your system; Cloud Control comes with several predefined reports that help you to dig into multiple areas of your system, there’s a report “20 Most Common Alerts” which shows you the incidence of common alerts.
In the picture above, you can clearly see that metric “Database Time Spent Waiting (%)” appears twice in my Top 3, let’s find out our metric setting for my DB targets; in order to do this we must go to a DB home page then Oracle Database -> Monitoring -> Metrics and Collection Settings.
Wait a minute! Why I’m receiving alerts if there are no thresholds setup for any of those metrics?, this behavior is clearly explained in MOS note 1500074.1 about a default warning threshold of 30% inside the database configuration. Let’s take a look to dba_threshold to confirm.
set lines 300
column METRICS_NAME format a30
column WARNING_OPERATOR format a30
column WARNING_VALUE format a30
column CRITICAL_OPERATOR format a30
column CRITICAL_VALUE format a30
SELECT METRICS_NAME,WARNING_OPERATOR ,WARNING_VALUE,CRITICAL_OPERATOR ,CRITICAL_VALUE FROM DBA_THRESHOLDS;
METRICS_NAME WARNING_OPERATOR WARNING_VALUE CRITICAL_OPERATOR CRITICAL_VALUE
———————————– —————————— —————————— —————————— ——————————
Average Users Waiting Counts GT 10 NONE
Average Users Waiting Counts GT 10 NONE
Average Users Waiting Counts GT 10 NONE
Average Users Waiting Counts GT 10 NONE
Average Users Waiting Counts GT 10 NONE
Average Users Waiting Counts GT 10 NONE
Average Users Waiting Counts GT 30 NONE
Average Users Waiting Counts GT 30 NONE
Blocked User Session Count GT 0 NONE
Current Open Cursors Count GT 1200 NONE
Database Time Spent Waiting (%) GT 30 NONE
Database Time Spent Waiting (%) GT 30 NONE
Database Time Spent Waiting (%) GT 30 NONE
Database Time Spent Waiting (%) GT 30 NONE
Database Time Spent Waiting (%) GT 30 NONE
Database Time Spent Waiting (%) GT 30 NONE
Database Time Spent Waiting (%) GT 50 NONE
Database Time Spent Waiting (%) GT 50 NONE
Logons Per Sec GE 100 NONE
Session Limit % GT 90 GT 97
Tablespace Bytes Space Usage DO NOT CHECK 0 DO_NOT_CHECK 0
Tablespace Space Usage GE 85 GE 97
22 rows selected.
There you go!, all metrics for “Database Time Spent Waiting (%)” are set to 30% or 50% values, now the trick to disable these metrics is to set them to a different value like 99%; this will override the default value as follows:
Let’s look at the database setting again:
set lines 300
column METRICS_NAME format a30
column WARNING_OPERATOR format a30
column WARNING_VALUE format a30
column CRITICAL_OPERATOR format a30
column CRITICAL_VALUE format a30
METRICS_NAME WARNING_OPERATOR WARNING_VALUE CRITICAL_OPERATOR CRITICAL_VALUE
———————————– —————————— —————————— —————————— ——————————
Average Users Waiting Counts GT 10 NONE
Average Users Waiting Counts GT 10 NONE
Average Users Waiting Counts GT 10 NONE
Average Users Waiting Counts GT 10 NONE
Average Users Waiting Counts GT 10 NONE
Average Users Waiting Counts GT 10 NONE
Average Users Waiting Counts GT 30 NONE
Average Users Waiting Counts GT 30 NONE
Blocked User Session Count GT 0 NONE
Current Open Cursors Count GT 1200 NONE
Database Time Spent Waiting (%) GT 99 NONE
Database Time Spent Waiting (%) GT 99 NONE
Database Time Spent Waiting (%) GT 99 NONE
Database Time Spent Waiting (%) GT 99 NONE
Database Time Spent Waiting (%) GT 99 NONE
Database Time Spent Waiting (%) GT 99 NONE
Database Time Spent Waiting (%) GT 99 NONE
Database Time Spent Waiting (%) GT 99 NONE
Database Time Spent Waiting (%) GT 99 NONE
Database Time Spent Waiting (%) GT 99 NONE
Database Time Spent Waiting (%) GT 99 NONE
Logons Per Sec GE 100 NONE
Session Limit % GT 90 GT 97
Tablespace Bytes Space Usage DO NOT CHECK 0 DO_NOT_CHECK 0
Tablespace Space Usage GE 85 GE 97
25 rows selected.
We successfully modified these metrics to a very high value; at this point you can decide to stay at 99% or you can remove that threshold in order to completely disable them.
Now let’s confirm those settings in the database:
set lines 300
column METRICS_NAME format a30
column WARNING_OPERATOR format a30
column WARNING_VALUE format a30
column CRITICAL_OPERATOR format a30
column CRITICAL_VALUE format a30
METRICS_NAME WARNING_OPERATOR WARNING_VALUE CRITICAL_OPERATOR CRITICAL_VALUE
———————————– —————————— —————————— —————————— ——————————
Average Users Waiting Counts GT 10 NONE
Average Users Waiting Counts GT 10 NONE
Average Users Waiting Counts GT 10 NONE
Average Users Waiting Counts GT 10 NONE
Average Users Waiting Counts GT 10 NONE
Average Users Waiting Counts GT 10 NONE
Average Users Waiting Counts GT 30 NONE
Average Users Waiting Counts GT 30 NONE
Blocked User Session Count GT 0 NONE
Current Open Cursors Count GT 1200 NONE
Logons Per Sec GE 100 NONE
Session Limit % GT 90 GT 97
Tablespace Bytes Space Usage DO NOT CHECK 0 DO_NOT_CHECK 0
Tablespace Space Usage GE 85 GE 97
14 rows selected.
The metrics are not there anymore and hopefully the alerts neither. This behavior is also noted for “Average Users Waiting Counts” metric, if you are receiving considerable white noise for this metric you can disable as well following the same procedure. A good practice is to create a Monitoring template to help you modify these thresholds for multiple targets at once.
Stay tuned for my next post about reducing OEM 12c noise.
Thanks,
Alfredo