CASE STUDY #1

CPT Performance Tuning and Capacity Planning Solution

A National Telecommunications Corporation

Early in 1995 this Telco’s billing application was experiencing a number of performance-related issues jeopardising the Company’s growing digital GSM market. At the time, the performance symptoms included batch window delays of over twelve hours and the need for daily manual interventions to ensure the completion of production processes. With business volumes set to increase three-fold, the immediate future looked very bleak for the application.

Following initial tuning assessments and short-term CPU reduction initiatives, CPT Global implemented a regime of long-term development life cycle process improvements to ensure the continued success of the application. During the last few years this Telco has achieved many capacity and performance milestones:

• Capacity walls have been averted to defer CPU upgrades saving millions of dollars; 

• Critical batch processes have been tuned to allow revenue collection processes to complete within the important batch service level agreement windows; 

• Online transactions have been tuned and redesigned to handle transaction throughput volumes required to support the growing business; 

• A Stress and Volume Test environment has been established; 

• A Capacity Planning process has been introduced that has in turn has been used as a model for other applications; 

• Agreed Service Level metrics have been established to manage online and batch performance; 

• Automated performance reporting has been implemented to proactively manage system performance; and 

• A design review process has been instituted to ensure a performance focus during the important design stages of a new application release.

Using CPT’s proven approach to performance tuning, a number of initiatives were identified, quantified and implemented. These included:

• DB2 buffer pool tuning to reduce I/O and reduce batch elapsed times; 

• DB2 SQL tuning reducing CPU path lengths to reduce CPU consumption and elapsed times; 

• Physical dataset placement tuning to reduce storage volume contention that reduced I/O response times; 

• Reducing response and elapsed times reduced DB2 Contention and the need to perform massive amounts of “re-tries” to failed transactions; 

• Tuning batch job JCL to reduce physical I/Os to reduce elapsed times; and 

• Tuning of overnight batch schedules to further contain batch processing to the overnight window and avoid costly contention with the online processes.

By late 1995, under CPT’s technical direction the project was able to demonstrate improved performance and capacity utilisation across the billing application portfolio. In addition to the capacity and performance management success story, CPT has performed a number of other tactical support roles as well as strategic consultancy roles including:

• Managing technical and production support; 

• Taking responsibility for the DBA, Production Support, Performance and Tuning, IEF Model Management and Mid-Range (Unix) support teams, comprising of 30 technical support specialists; 

• Undertaking mid-range support. The digital data network mediator required on-going support, maintenance and ultimately redesign to support the escalating business volumes in a dynamically changing area of Australian telecommunications. CPT initially supplied a specialist who then led the team over several months to oversee several significant application enhancements to the UNIX environment; and 

• Providing testing services, strategies and methods. Recognising the need to improve the quality and efficiency of the testing phases of the systems delivery life cycle, the billing management team engaged CPT to perform a baseline assessment of the testing strategies, tools and testing techniques of the billing application. Following acceptance of the short-term and long-term recommendations, CPT was retained to manage the testing transition program. Over a period of twelve months, a number of key improvements were observed, including:

   

  • A reduction in end to end testing cycle time for each release due to increased productivity; 

  • Reliable and credible management reporting of testing status and issues; and 

  • Fewer defects in production due to improved testing, identifying defects in earlier phases, improved morale of testing teams.