PIPER-Rx - Too Many Managers

PIPER-Rx
Oracle E-Business Suite Resource Centre

Is the number of concurrent manager processes you have causing performance issues?

Is the number of concurrent manager processes you have causing performance issues?

All too often I see sites with far too many concurrent manager processes and the site wonders why they have intermittent performance issues. Remember, having more concurrent manager processes does not mean more throughput.

So, how many concurrent manager processes should you have?

Add up the number of standard and custom concurrent manager processes you have. If that value exceeds 2 times the number of CPUs (multi core adjusted) on the box you most likely have too many manager processes. If you are experiencing intermittent performance issues, particularly around high processing times like financial month ends, too many manager processes would most likely be one of your reasons.

The following Simple SQL will list your concurrent managers:

  SELECT concurrent_queue_name,
         max_processes,
         running_processes,
         nvl(sleep_seconds,0) sleep_seconds,
         cache_size,
         decode(enabled_flag, 
             'Y', 'Enabled', 'N', 'Disabled', 'Unknown' ) status
    FROM applsys.fnd_concurrent_queues
   ORDER BY decode(enabled_flag, 'Y', 1, 'N', 0, enabled_flag ) DESC,
            max_processes DESC,
            decode(concurrent_queue_name,
                   'FNDICM',   'AA',
                   'FNDCRM',   'AB',
                   'STANDARD', 'AC',
            concurrent_queue_name);
  
Example Output

A real world example of too many managers:

A site I reviewed had 54 standard manager processes on a 4 CPU box - 54 / 4 = 13.5

In this example there can be up to 54 concurrent requests running through the standard managers at any time which will just plain flood the CPUs. Generally running two (2) concurrent requests per CPU is sufficient to leave enough overhead for normal processing activities; any more than that and the risk increases of CPU flooding. And you would hope this site does run too many FSG’s... as we all know the damage they can do...

Believe it or not I have even seen 108 standard managers on an 8 CPU box..... Hmmm.....

Flooding the CPUs with concurrent requests leaves very little available CPU for normal user requests. As a result users tend to experience poor performance with their forms etc... It’s basic queuing theory... To make matters worse, these intermittent performance issues tend to occur around peak processing times when the concurrent manager load is at its peak; exactly the time users are clearly very busy and don’t want to be experiencing performance issues.

If you have a “problem” with an excessive number of standard and custom concurrent managers processes, what you need to do is lower the number of processes. This is easier said than done, as once they exist the business is very reluctant to let them go. But it is worth persisting as it will make a difference!

For more information refer to the paper I wrote: Concurrent Managers 101

Believe it or not, and much to my surprise, this is the most down loaded paper on my web site. Even though I originally wrote this paper in 2004 and have been updating it ever since, it all still holds true today.... not much changes in OEBS, and for good reason; stability in accounting and business systems is what businesses want. In fact Concurrent Managers has continued to be such a problem area for so many sites I have now also devoted 3 full PAMtutorials to the subject of Concurrent Managers so I would encourage anyone interested in better managing their Concurrent Managers to check these out as well!

The following is a TOAD report written in the TOAD Reports Manager format. In this report we list both the pending and running request counts for the manager. This can be for useful for providing evidence when the managers are hammered as this report will show the size of the back log.

Due to the complexity of this report, it is in a format that calls a stored procedure. Both the procedure and the report TRD can be down loaded using the following links:

Report - Stored Procedure
Report TRD format

For more information on calling stored procedures from a TOAD report refer to:

Lesson 2 - How to Create TOAD Report Manager reports that call PLSQL packages

Lesson 2 - A step by step guide to calling a stored procedure from a TOAD® Report (pdf)
Lesson 2 stored procedure and example report - TRD format (zip)

Last update: Mar 2012