The purpose of this document is to define a high-level testing strategy. The document will discuss the what, where, when, and who aspects of testing, by responding to the following questions:

• What objects will be tested?
• What type of testing will be performed on the objects?
• Where will testing take place?
• When will testing occur?
• Who is involved in testing?

The purpose of testing is to validate that a system and its developed solutions satisfy business requirements as documented in approved Business Process Design, Functional Specification, and Technical Specifications documents.  During testing, issues are identified that can or will prevent the proper functioning of the system.  Once identified, the issues can be prioritized, resolved, and re-tested to business satisfaction. 

In developing any testing strategy, it is important to explicitly define what components need testing.  For the example Project, all software objects configured and/or delivered by EXAMPLE Project Members will require testing.  This involves testing of the following software objects:

• Configuration & Individual Transactions
• Development Objects (including legacy data and external system integration)
• Reports, Interfaces, Conversions, Enhancements, Forms, and Workflows
• Security Authorization & Roles

To ensure the development and delivery of these objects are built according to design and functioning correctly, the objects are tested in the following phases.   The phases support the complete development cycle of the project and as a result a benefit not only to timeliness but also to cost efficiency.   Independent case studies have shown that testing started in the earlier stage of development cycle is more cost effective than testing performed in later stages of the development cycle.  Early testing allows for early detection of defects, which allows for cheaper cost upfront to resolve than allowing the defects to pass to a more integrated environment that may possibly delay or prevent successful testing of other developments.

Testing Phases

• Feasibility Testing
• Unit Testing
• Integration/Quality Assurance Testing
• User Acceptance Testing

There are two additional testing phases to make certain the new system is ready for deployment and can handle the load of production processing.  It is critical these testing phases are executed with near all of SOFTWARE objects to be developed and delivered; Otherwise the results will not be reflective of the true state of the new system. 

• System Performance
• System Readiness 

This section defines the purpose for each level of testing in relation to the development life cycle and system environment

Feasibility Testing for example project

Purpose:    Provide wide-open exploration of system functionality used to gain firmness around a particular object or process.  This phase of testing is not mandatory and ise executed as needed by project members.

Environment:        Sandbox

Pre-Requisites:        None.

Data Requirements:    No data will be imported.  All data must be manually created, as needed, to support testing in this phase.  

End Results:     Complete understanding of system functionality regarding an object or process.

Unit Testing

Purpose:    Verify proper functioning of software objects developed and/or delivered by example Project Members.  

Environment:        Development

Pre-Requisites:    Approved Business Process Design Document, Approved Functional Specification Document, Approved Configuration or Technical Specification Document    

Data Requirements:    No data will be imported.  All data must be manually created, as needed, to support testing in this phase.

End Results:    Verified functioning of individual transactional configuration, security authorization/roles, and objects developed and/or delivered by example Project Members.  

Updated Business Process Design Document, Functional Specification Document, Configuration or Technical Specification Document (if necessary).

Integration/Quality Assurance Testing

Purpose:    Ensure that software functionality, as configured in the development environment, is properly working by verifying end-to-end processes.  This includes the testing of key transactions, authorisations/roles, and objects.  It is important to note that the configurable functionality of the system should be stabilized as much as possible before proceeding to test data conversions, interfaces, forms, and workflow objects; Doing so will avoid needless retesting of the later objects.  

Environment:            Quality Assurance

Pre-Requisites:    Completed unit testing, approved Business Process Design Document, approved Functional Specification Document, approved Configuration or Technical Specification Document, approved Test Scenarios

Data Requirements:    Clean and converted master and operational data from legacy systems.  As well, manual data will be entered as defined in the test scenario/steps and pre-conditions.

End Results:    Verified accurate and efficient functioning of configured functionality and end-to-end processes in the new system.  This includes all software objects as described earlier.  

Completed test scenario/steps.  

Finalized Business Process Design Document, Functional Specification Document, Configuration or Technical Specification Document, Test Scenarios (if necessary).

User Acceptance Testing

Purpose:    Final accuracy and completeness check of all end-to-end process and objects developed and delivery by the EXAMPLE Project before entry into the Production environment. Once the first developed object has been transported to the Production environment, UAT will also support regression testing of critical and significant changes to be made in the Production environment.         
Environment:            Pre-Production

Pre-Requisites:    Completed and Signed-Off Unit and Integration/Quality Assurance testing.  As well, completed transporting of all objects (including configuration).

Data Requirements:    Full load of clean and converted master and operational data from legacy systems.  As well, manual data will be entered as defined in the test scenario/steps and pre-conditions.

End Results:    Final functionality of system to be delivered.  All processes and objects should look and operate as they would in the Production environment.  

System Performance Testing

Purpose:    Ensure the Production environment will meet or exceed requirements around availability and response time.  This phase of testing should identify areas that may need to be adjusted or tuned to provide appropriate response times and availability.  These tests may include some or all of the following: Batch Cycle Test, Volume/Stress Test, and Month/Quarter/Year End Processing

Environment:            Pre-Production

Pre-Requisites:    Completed and Signed-Off Integration/Quality Assurance testing.

Data Requirements:    Full load of clean and converted master and operational data from legacy systems in addition to extensive transactional data to perform volume and stress testing.   

End Results:    Production system that will meet or exceed availability and response time requirements.

System Readiness Testing

Purpose:    Ensure that the technical components of the Production environment are functioning properly.  These tests may include some or all of the following: Failure Test, Cutover Testing, Disaster Recovery Test, Backup and Restore Test, System Administration Test, and Going Live Check.  

Environment:            Production

Pre-Requisites:    Completed and Signed-Off Integration/Quality Assurance testing.

Data Requirements:    Full load of clean and converted master and operational data from legacy systems. 

End Results:            Fully functioning Production environment.

This sections aims to define criteria for progressing to the next phase or type of testing.
Given the four defect and/or issue priorities of critical, high, medium, and low, a testing phase is complete when 90% of the system tests have been completed/signed-off and the 10% remaining are accepted by Business Process Owners.  For User Acceptance testing, the percentages are modified; 95% of systems tests must be completed/signed-off and Business Process Owners have accepted the 5% remaining as non-critical.

Unlike the testing of software objects, System Performance and System Readiness testing rely on a stable system (Production or Pre-Production).  These tests are final preparation for the new system and will dictate whether the Production environment is ready for actual production processing.  In terms of completion criteria, both phases are all or nothing.  This means, in both phases, tests either pass or fail as a whole based on pre-determined criteria from Business Users, Legacy Owners, and Process Owners.    

With the implementation of any new system to replace legacy systems, there is a minimal risk in developing any new object or process.  However, there are some areas that pose many risks.  These areas have been identified and will be closely monitored throughout the development cycle, starting with the drafting of the Business Process and Functional Specification documents and rigorously tested for accuracy, completeness, and efficiency.  

E.g

• Closing periods (Monthly/Quarterly/Year End)
• Data Conversion (Master and Transactional)
• Any process or interface involving integration
• Any process or interface involving SOA functionality
• Cleansing and transformation of legacy data

Other projects underway may affect the design of the example solution.  Relationships have to be established with the other project managers to make certain that common and connected processes are in sync among the new systems.  These processes will be closely monitored and tested throughout the life cycle of the example Project.

For the different phases of testing and specialty testing, an array of resources is needed.  This section of the document defines the role and responsibilities needed to enable complete and adequate testing.  

Business Roles

Business Analyst
Business Process Owner
Business Project Manager
Business User

Technical Roles

Software Architect
Software Developer
Software  Functional Analyst
Software  Functional Lead
Software Technical Analyst
Software Technical Lead
Software  System Administrator
System Infrastructure Lead
Technical Project Manager

Quality Assurance Roles

Testing Manager
Testing Coordinator
Test Functional Lead

Si - Responsible for Sign-Off         Su - Responsible for Supporting Testers     T - Responsible for executing Test scenarios

Microsoft Excel

Excel spreadsheets are often used to capture and document unit test scenarios and steps with corresponding information such as testing conditions, assigned tester, schedule/execution date, and expected/actual test results. It is important that this spreadsheet is maintained properly as it serves as the basis for Integration/Quality Assurance and User Acceptance Testing scenarios and steps.
Usage: Unit Testing

TestDirector

External software by Mercury that is used to document and execute test scenarios put forth by business project members with help from technical analysts. Test Director will keep track of test statuses per scenario. The tool will also serve as the defect repository as well as a testing monitoring tool. TestDirector can send out emails regarding open/failed tests and defects. Status reports are directly generated from TestDirector.
Usage: Integration/Quality Assurance Testing, User Acceptance Testing

LoadRunner

External software tool by Mercury used to predict end-to-end system behavior and performance.
Usage: System Performance Testing

Computer Aided Test Tools

Various tools in the SOFTWARE environment to upload mass amounts of data and to automate the execution of test scenarios.
Usage: User Acceptance Testing, System Performance Testing, System Readiness Testing

Sign-Off Document

As each test scenario passes it will be signed off (in writing).
Usage: Integration/Quality Assurance Testing, User Acceptance Testing, System Performance, Testing, System Readiness Testing

Issue Log & Risk Management

All business issues and risk analysis findings resulting from testing are logged.

A test scenario is a group of detail test steps executed in chronological order to return expected results. It may cross process boundaries in an attempt to create a truly integrated test. More likely than not, to obtain the expected results, each scenario must have test conditions and certain master and transaction data. In the end, test scenarios are a sequence group of steps used to verify requirements from the business and technical specification documents are satisfied.

Test Scenario Creation

Starting with the Unit Testing phase, scenarios must be created and loaded into the related tool to test individual functionality. Often times, unit test scenarios, which are created by technical analysts and developers, are the starting blocks for the next stage of testing. Business analysts and leads use them to finalize their business scenarios that lead to test scenarios.

Formalize Test Procedure

A formal process, that is consistent and test-friendly, must be created to allow testers to execute scheduled scenarios and update accordingly. The Test Coordinator(s) within the Quality Assurance Team on a project normally creates this process.

Defect Procedure

A formal process, that is consistent, must be created to allow for the assignment and notification of defects. The Test Coordinator within the Quality Assurance Team on a project normally creates this process.

Testing Review Procedure

A formal process, that is consistent, must be created to allow Business Process Owners, Project Managers, and Technical Leads to monitor testing process, prioritize defects, and resolve issues. The Test Coordinator(s) within the Quality Assurance Team on a project works with the associated participants to develop this process.