What is Risk based testing?

Risk based testing is basically a testing done for the project based on risks.Risk based testing uses risk to prioritize and emphasize the appropriate tests during test execution. In simple terms – Risk is the probability of occurrence of an undesirable outcome. This outcome is also associated with an impact. Since there might not be sufficient time to test all functionality, Risk based testing involves testing the functionality which has the highest impact and probability of failure.

Risk-based testing is the idea that we can organize our testing efforts in a way that reduces the residual level of product risk when the system is deployed.

• Risk-based testing starts early in the project, identifying risks to system quality and using that knowledge of risk to guide testing planning, specification, preparation and execution.
• Risk-based testing involves both mitigation – testing to provide opportunities to reduce the likelihood of defects, especially high-impact defects – and contingency – testing to identify work-around to make the defects that do get past us less painful.
• Risk-based testing also involves measuring how well we are doing at finding and removing defects in critical areas.
• Risk-based testing can also involve using risk analysis to identify proactive opportunities to remove or prevent defects through non-testing activities and to help us select which test activities to perform.

The goal of risk-based testing cannot practically be – a risk-free project. What we can get from risk-based testing is to carry out the testing with best practices in risk management to achieve a project outcome that balances risks with quality, features, budget and schedule.

How to perform risk based testing?

1. Make a prioritized list of risks.
2. Perform testing that explores each risk.
3. As risks evaporate and new ones emerge, adjust your test effort to stay focused on the current crop.

What is configuration management in software testing?

Configuration management is a topic that often confuses new practitioners. So, let me describe it briefly:

• Configuration management determines clearly about the items that make up the software or system. These items include source code, test scripts, third-party software, hardware, data and both development and test documentation.
• Configuration management is also about making sure that these items are managed carefully, thoroughly and attentively during the entire project and product life cycle.
• Configuration management has a number of important implications for testing. Like configuration management allows the testers to manage their testware and test results using the same configuration management mechanisms.
• Configuration management also supports the build process, which is important for delivery of a test release into the test environment. Simply sending Zip archives by e-mail will not be sufficient, because there are too many opportunities for such archives to become polluted with undesirable contents or to harbor left-over previous versions of items. Especially in later phases of testing, it is critical to have a solid, reliable way of delivering test items that work and are the proper version.
• Last but not least, configuration management allows us to keep the record of what is being tested to the underlying files and components that make it up. This is very important. Let us take an example, when we report defects, we need to report them against something, something which is version controlled. If it is not clear what we found the defect in, the programmers will have a very tough time of finding the defect in order to fix it. For the kind of test reports discussed earlier to have any meaning, we must be able to trace the test results back to what exactly we tested.

Ideally, when testers receive an organized, version-controlled test release from a change-managed source code repository, it is along with a test item trans-mittal report or release notes. [IEEE 829] provides a useful guideline for what goes into such a report. Release notes are not always so formal and do not always contain all the information shown.

What is the purpose and importance of test plans in software testing?

Test plan is the project plan for the testing work to be done. It is not a test design specification, a collection of testcases or a set of test procedures; in fact, most of our test plans do not address that level of detail. Many people have different definitions for test plans.

Purpose  of writing Test Plan:-

1)First, by writing a test plan it guides our thinking.  Writing a test plan forces us to confront the challenges that await us and focus our thinking on important topics.

  2)Second, the test planning process and the plan itself serve as the means of communication with other members of the project team, testers, peers, managers and other stakeholders. This communication allows the test plan to influence the project team and the project team to influence the test plan, especially in the areas of organization-wide testing policies and motivations; test scope, objectives and critical areas to test; project and product risks, resource considerations and constraints; and the testability of the item under test.As the project evolves and situations change, we adapt our plans. By updating the plan at major milestone helps us to keep testing aligned with project needs. As we run the tests, we make final adjustments to our plans based on the results. You might not have the time – or the energy – to update your test plans every time a change is made in the project, as some projects can be quite dynamic.

At times it is better to write multiple test plans in some situations. For example, when we manage both integration and system test levels, those two test execution periods occur at different points in time and have different objectives. For some systems projects, a hardware test plan and a software test plan will address different techniques and tools as well as different audiences. However, there are chances that these test plans can get overlapped, hence, a master test plan should be made that addresses the common elements of both the test plans can reduce the amount of redundant documentation.

What are the roles and responsibilities of a Test Leader?

• Test leaders tend to be involved in the planning, monitoring, and control of the testing activities and tasks.
• At the outset of the project, test leaders, in collaboration with the other stakeholders, devise the test objectives, organizational test policies, test strategies and test plans.
• They estimate the testing to be done and negotiate with management to acquire the necessary resources.
• They recognize when test automation is appropriate and, if it is, they plan the effort, select the tools, and ensure training of the team.  They may consult with other groups – e.g., programmers – to help them with their testing.
• They lead, guide and monitor the analysis, design, implementation and execution of the test cases, test procedures and test suites.
• They ensure proper configuration management of the testware produced and traceability of the tests to the test basis.
• As test execution comes near, they make sure the test environment is put into place before test execution and managed during test execution.
• They schedule the tests for execution and then they monitor, measure, control and report on the test progress, the product quality status and the test results, adapting the test plan and compensating as needed to adjust to evolving conditions.
• During test execution and as the project winds down, they write summary reports on test status.
• Sometimes test leaders wear different titles, such as test manager or test coordinator. Alternatively, the test leader role may wind up assigned to a project manager, a development manager or a quality assurance manager. Whoever is playing the role, expect them to plan, monitor and control the testing work.

Along with the test leaders testers should also be included from the beginning of the projects, although most of the time the project doesn’t need a full complement of testers until the test execution period. So, now we will see testers responsibilities.

What are the roles and responsibilities of a Tester?

• In the planning and preparation phases of the testing, testers should review and contribute to test plans, as well as analyzing, reviewing and assessing requirements and design specifications. They may be involved in or even be the primary people identifying test conditions and creating test designs, test cases, test procedure specifications and test data, and may automate or help to automate the tests.
• They often set up the test environments or assist system administration and network management staff in doing so.
• As test execution begins, the number of testers often increases, starting with the work required to implement tests in the test environment.
• Testers execute and log the tests, evaluate the results and document problems found.
• They monitor the testing and the test environment, often using tools for this task, and often gather performance metrics.
• Throughout the testing life cycle, they review each other’s work, including test specifications, defect reports and test results.

What things to keep in mind while planning tests?

 A good test plan is always kept short and focused. At a high level, you need to consider the purpose served by the testing work. Hence, it is really very important to keep the following things in mind while planning tests:

• What is in scope and what is out of scope for this testing effort?
• What are the test objectives?
• What are the important project and product risks?
• What constraints affect testing (e.g., budget limitations, hard deadlines, etc.)?
• What is most critical for this product and project?
• Which aspects of the product are more (or less) testable?
• What should be the overall test execution schedule and how should we decide the order in which to run specific tests? (Product and planning risks, discussed later in this chapter, will influence the answers to these questions.)
• How to split the testing work into various levels (e.g., component, integration, system and acceptance).
• If that decision has already been made, you need to decide how to best fit your testing work in the level you are responsible for with the testing work done in those other test levels.
• During the analysis and design of tests, you’ll want to reduce gaps and overlap between levels and, during test execution, you’ll want to coordinate between the levels. Such details dealing with inter-level coordination are often addressed in the master test plan.
• In addition to integrating and coordinating between test levels, you should also plan to integrate and coordinate all the testing work to be done with the rest of the project. For example, what items must be acquired for the testing?
• When will the programmers complete work on the system under test?
• What operations support is required for the test environment?
• What kind of information must be delivered to the maintenance team at the end of testing?
• How many resources are required to carry out the work.

Now, think about what would be true about the project when the project was ready to start executing tests. What would be true about the project when the project was ready to declare test execution done? At what point can you safely start a particular test level or phase, test suite or test target? When can you finish it? The factors to consider in such decisions are often called ‘entry criteria’ and ‘exit criteria.’ For such criteria, typical factors are:

• Acquisition and supply: the availability of staff, tools, systems and other materials required.
• Test items: the state that the items to be tested must be in to start and to finish testing.
• Defects: the number known to be present, the arrival rate, the number predicted to remain, and the number resolved.
• Tests: the number run, passed, failed, blocked, skipped, and so forth.
• Coverage: the portions of the test basis, the software code or both that have been tested and which have not.
• Quality: the status of the important quality characteristics for the system.
• Money: the cost of finding the next defect in the current level of testing compared to the cost of finding it in the next level of testing (or in production).
• Risk: the undesirable outcomes that could result from shipping too early (such as latent defects or untested areas) – or too late (such as loss of market share).

When writing exit criteria, we try to remember that a successful project is a balance of quality, budget, schedule and feature considerations. This is even more important when applying exit criteria at the end of the project.

What are the factors affecting test effort in software testing?

When you create test plans and estimate the testing effort and schedule, you must keep these factors in mind otherwise your plans and estimates will mislead you at the beginning of the project and betray you at the middle or end.

The test strategies or approaches you pick will have a major influence on the testing effort. In this section, let’s look at factors related to the product, the process and the results of testing.

In Product factors the presence of sufficient project documentation is important so that the testers can figure out what the system is, how it is supposed to work and what correct behavior looks like. This will help us do our job more efficiently.

The factors which affect the test effort are:

• While good project documentation is a positive factor, it’s also true that having to produce detailed documentation, such as meticulously specified test cases, results in delays. During test execution, having to maintain such detailed documentation requires lots of effort, as does working with fragile test data that must be maintained or restored frequently during testing.

• Increasing the size of the product leads to increases in the size of the project and the project team. Increases in the project and project team increases the difficulty of predicting and managing them. This leads to the disproportionate rate of collapse of large projects.

• The life cycle itself is an influential process factor, as the V-model tends to be more fragile in the face of late change while incremental models tend to have high regression testing costs.

• Process maturity, including test process maturity, is another factor, especially the implication that mature processes involve carefully managing change in the middle and end of the project, which reduces test execution cost.

• Time pressure is another factor to be considered. Pressure should not be an excuse to take unwarranted risks. However, it is a reason to make careful, considered decisions and to plan and re-plan intelligently throughout the process.

• People execute the process, and people factors are as important or more important than any other. Important people factors include the skills of the individuals and the team as a whole, and the alignment of those skills with the project’s needs. It is true that there are many troubling things about a project but an excellent team can often make good things happen on the project and in testing.

• Since a project team is a team, solid relationships, reliable execution of agreed-upon commitments and responsibilities and a determination to work together towards a common goal are important. This is especially important for testing, where so much of what we test, use, and produce either comes from, relies upon or goes to people outside the testing group. Because of the importance of trusting relationships and the lengthy learning curves involved in software and system engineering, the stability of the project team is an important people factor, too.

• The test results themselves are important in the total amount of test effort during test execution. The delivery of good-quality software at the start of test execution and quick, solid defect fixes during test execution prevents delays in the test execution process. A defect, once identified, should not have to go through multiple cycles of fix/retest/re-open, at least not if the initial estimate is going to be held to.

What are the estimation techniques in software testing?

There are two techniques for estimation covered by the ISTQB Foundation Syllabus.

1. One involves people with expertise on the tasks to be done and
2. Other involves consulting the people who will do the work .

The first one involves analyzing metrics from past projects and from industry data.

Let’s look at both of them one by one.

1. People with expertise on the tasks to be done:

In this process we ask the individual contributors and experts involves working with experienced staff members to develop a work-breakdown structure for the project. With that done, you work together to understand, for each task, the effort, duration, dependencies, and resource requirements. The idea is to draw on the collective wisdom of the team to create your test estimate. Using a tool such as Microsoft Project or a whiteboard and sticky-notes, you and the team can then predict the testing end-date and major milestones. This technique is often called ‘bottom up’ estimation because you start at the lowest level of the hierarchical breakdown in the work-breakdown structure – the task – and let the duration, effort, dependencies and resources for each task add up across all the tasks.

Analyzing metrics can be as simple or sophisticated as you make it. The simplest approach is to ask, ‘How many testers do we typically have per developer on a project?’ Or another more reliable approach involves classifying the project in terms of size (small, medium or large) and complexity (simple, moderate or complex) and then observing on average how long projects of a particular size and complexity combination have taken in the past. Sophisticated approaches involve building mathematical models in a spreadsheet that look at historical or industry averages for certain key parameters – number of tests run by tester per day, number of defects found by tester per day, etc. – and then plugging in those parameters to predict duration and effort for key tasks or activities on your project. The tester-to-developer ratio is an example of a top-down estimation technique, in that the entire estimate is derived at the project level, while the parametric technique is bottom-up, at least when it is used to estimate individual tasks or activities.

We prefer to start by drawing on the team’s wisdom to create the work breakdown structure and a detailed bottom-up estimate. We then apply models and rules of thumb to check and adjust the estimate bottom-up and top-down using past history. This approach tends to create an estimate that is both more accurate and more defensible than either technique by itself.

2. Consulting the people who will do the work:

Even the best estimate must be negotiated with management. Negotiating sessions exhibit amazing variety, depending on the people involved. However, there are some classic negotiating positions. It’s not unusual for the test leader or manager to try to sell the management team on the value added by the testing or to alert management to the potential problems that would result from not testing enough. It’s not unusual for management to look for smart ways to accelerate the schedule or to press for equivalent coverage in less time or with fewer resources. In between these positions, you and your colleagues can reach compromise, if the parties are willing. Our experience has been that successful negotiations about estimates are those where the focus is less on winning and losing and more about figuring out how best to balance competing pressures in the realms of quality, schedule, budget and features.