What are the Myths and Facts about Software Testing ?

1. MYTH: Quality Control = Testing.
   • FACT: Testing is just one component of Software Quality Control. Quality Control includes other activities such as Reviews.
2. MYTH: The objective of Testing is to ensure a 100% defect- free product.
   • FACT: The objective of testing is to uncover as many defects as possible while ensuring that the software meets the requirements. Identifying and getting rid of all defects is impossible.
3. MYTH: Testing is easy.
   • FACT: Testing can be difficult and challenging (sometimes, even more so than coding).
4. MYTH: Anyone can test.
   • FACT: Testing is a rigorous discipline and requires many kinds of skills.
5. MYTH: There is no creativity in testing.
   • FACT: Creativity can be applied when formulating test approaches, when designing tests, and even when executing tests.
6. MYTH: Automated testing eliminates the need for manual testing.
   • FACT: 100% test automation cannot be achieved. Manual Testing, to some level, is always necessary.
7. MYTH: When a defect slips, it is the fault of the Testers.
   • FACT: Quality is the responsibility of all members/ stakeholders, including developers, of a project.
8. MYTH: Software Testing does not offer opportunities for career growth.
   • FACT: Gone are the days when users had to accept whatever product was dished to them; no matter what the quality. With the abundance of competing software and increasingly demanding users, the need for software testers to ensure high quality will continue to grow. 

What is Release and Iteration Planning in Agile methodology?

There are 3 levels of planning in Agile. They are Release Planning, Iteration Planning and Daily Planning. These planning meetings help the Scrum Master, Product Owner and the rest of the team in understanding how the product will be delivered, the complexity involved and their day to day responsibility in the delivery of the product, among other things.

Now we will go into detail of Release Planning, Iteration Planning and Daily Planning.

Release Planning in Agile Development

Release Planning is usually performed during the Sprint zero, where there is no product increment delivered. The whole Sprint is dedicated for planning the next release. It is a way of looking ahead on defining what the release goal is, what features that need to delivered during the release, defining the release backlog, breaking features and epics into user stories, writing acceptance criteria for all stories, and estimating the user stories. Release planning also helps team members in defining the test strategy and test approach planning for all iterations. Release plans may change based on new stories added or deleted. The status of the release is tracked prospectively every sprint to understand what it takes to meet the release

At the start of the release planning, the Product Owner sets the release goal and release time frame. The Product Owner also collaborates with the team , based on the user stories the team performs high architecture evaluation and high level effort estimation (see Section 3.2).

Testers are involved in release planning and perform the following activities:

• Write User stories and acceptance criteria
• Seek clarification on those user stories where there is insufficient information.
• Determine the high level test strategy for the whole release
• Point out any testing risks they might occur
• Do a high level test planning
• Define the number of test levels to be performed.

Iteration Planning in Agile Development

Iteration 1 planning is done after release planning. The team pulls the stories into the sprint backlog from the product backlog and tasks them into independent tasks of fewer 8 hours each. The also performs risk assessment of stories and decides on a light weight plan on how to address the risks as the sprint moves forward.  The team asks questions to the Product Owner on clarifications that might make them to understand the stories in more realistic way. The number of stories that the team pulls in the sprint may depend on team’s capacity or velocity (if known). The capacity planning is done in hours and the velocity planning is done in story points. The team makes a commitment to the sprint backlog and changes the sprint backlog as it emerges.

The testers are also involved in the iteration and can contribute to the same in the following ways:

• Breaking user stories into testing tasks
• Determining test coverage of every story
• Creating acceptance tests for user stories
• Estimate the testing tasks like creating test strategy, test plans and test cases specific to user stories
• Understand and plan for automating user stories and support various levels of testing

Release plans are not static documents and they are liable to change as the external and internal factors change as the iteration executions take place. The release plans may also change due to various internal factors like delivery team capabilities, velocity of the team and technical competencies. The examples of external factors are change in target segments, threat from substitutes, change triggered by superior competitor product. The release dates may be adjusted or scope may be cut down to adjust for the changes.

As per Scrum, no changes are allowed during the sprint by the Product Owner. The Product Owner cannot add new stories to the sprint backlog during the sprint and distract the team from the executing the sprint goal. The Scrum master must help the team to see that the product owner does not disturb the team, by inserting new work into the team.

The team, especially testers much understands the big picture in order to release a fine quality product. Any ambiguity regarding the test planning must be carefully considered after due consultations with the rest of the team.

Release and iteration planning broadly helps the team to prepare test planning checklist related to the following items:

• The team members responsible for testing
• The scope of testing and testing goals to be carried out
• The test environment that needs to be setup
• The test data to be collected that helps the testing team
• The sequence of test activities, dependencies and interfaces related to testing
• Any project related quality tests that surface during testing 

Daily Planning Meeting in Agile Development

Daily Planning meeting in agile are also called as ‘Stand up meetings’ where the development team and testing team meets daily to discuss What progress in the assigned task they made yesterday, What tasks they have planned to do today and what tasks they will do tomorrow. In addition to these details each member has to tell that how long they are going to take to complete the assigned task within the sprint. If any remaining or pending task of any member of the team is not completed in that particular sprint then they notify such details in the daily meetings. Accordingly the pending task(s) from the previous sprint are taken care in the next sprints. Likewise if there are any changes in the requirement from the customers or any build issues or any blocking issues then such details are also discussed in the meetings.

What is Continuous Integration in Agile methodology?

Continuous Integration (CI) is a practice in Software Engineering, where all the developers local working code base will be merged to share with a common repository several times during the product development. It was first adopted as a part of Extreme Programming (XP). The main purpose of CI is prevent developers stepping over each other code and eliminate integration issues. CI works in tandem with other best practices like Configuration management, compilation, software build, deployment, and testing which are bundled into a single automated and repeatable process. Due to rapid integration of code, it is more likely that defects surface faster than it could compare to normal manual integration.

CI has the following built in automation to check the validity of the code that was checked in:

• Static code analysis: Reporting the results of static code execution
• Compile: Generating the executable files by linking the code and compiling after
• Unit test: Writing unit tests, executing them, checking code coverage and reporting the results
• Deploy: Build the code and install it into a test/production environment.
• Integration test: Providing results by executing the integration tests.
• Report (dashboard): Indicating the status of key parameters by posting Red, Green, and Yellow to a publicly visible location.

The Cycle of CI has been shown below:-

Cycle of Continuous Integration

CI helps testers to perform automation effectively to uncover defects at a faster rate and improve the regression results. The automation will also give the percentage of test coverage area. The automation report also covers the number of user stories and functionality mapped to the product increment. Automation also reduces the manual testing effort of the testers and makes their life easy. Testers can also manually test the areas of failure and collaborate with the developers to fix the defects.

There are several tools that are being used in organizations as a part of continuous integration to build the automation scripts. Few examples of the tools are JUnit, Selenium, SONAR etc. CI automation will reduce the time to deliver, since it replaces the traditional manual testing. Build tools are linked to automation server and build server and deploy to testing, staging and production environments. Organizations started using these build tools to replace the traditional quality control.

CI provides the following benefits to its users

• Enables a quick feedback mechanism on the build results
• Helps collaboration between various product teams within the same organization
• Decreases the risk of regression since the code is often churned by other developers.
• Maintains version control within for various product releases and patch releases.
• Reduces the technical debt within the code.
• Allows earlier detection and prevention of defects
• Reduces manual testing effort
• Provides a facility of falling back to previous versions in case of any problem to the current build

CI also has the following risks and challenges:

• CI tools maintenance and their administration have associated costs to it.
• CI guidelines need to be well established before starting it.
• Test automation is a rare skill in the market and existing testers may have to be trained on that.
• Full fledge test coverage is required to see the benefits to automation.
• Teams sometimes depend too much on the unit testing and ignore automation and acceptance testing.

So in summary, CI is advantageous to a project team, if the required tools are put in place for automating the build process.

What is Retrospective meeting or Retrospection in Agile project management?

In Agile project management, a retrospective meeting is held at the end of every iteration to discuss what really went well and what didn't go well and any action items to carry over the next Sprint. Retrospective meetings are a useful to identify the ways of continuous improvement of an Agile team. Few topics for discussion could be about metrics and velocity of the team, tracking tools, defect density, team dynamics, self-organization, any potential impediments that surfaced during the iteration etc.

Retrospectives may also result in test coverage improvements, giving more importance to unit testing, test strategy and test plans. Root causes of various testing related issues can be identified and action items are suggested. Thus it also helps for continuous testing process improvement. Testers bring their unique testing perspective to the retrospective.

Retrospectives ideally may be held at the same place and same location every iteration. It is highly recommended that Scrum Master, Product Owner and the team only attend the retrospective. Managers and senior management may be barred for entering a retro meeting; the team may not speak up due to inherent fear of management. There is an eco-system of mutual trust and openness be built in a retrospective.

Sprint Retrospective meetings can be facilitated by asking each person in the team to answer a variation on the above questions. Instead of asking what went well, what didn’t go well some teams prefer -

• What should we start doing
• What should we stop doing
• What should we continue to do

Teams are asked to be specific in their answers so that effective actions can be taken. The retrospective meeting is usually conducted immediately after the Sprint review meeting. It is recommended that the entire team participate in this exercise so that any issues or concerns that the teams face during the previous Sprint are addressed during the teaming and avoided in upcoming Sprints.

What is Planning Poker? Effort estimation in Agile methodology

In the Sprint zero as a part of release planning, the Agile team has come up with effort estimation for all the stories in the release. The Planning Poker is a popular method of effort estimation which ensures that the entire team is involved in the estimation exercise.

The Planning Poker is a consensus based technique and is used to size the stories (in terms of story point) or effort estimate (in terms of days).

• It is a non-liner scale of estimation technique.
• Fibonacci series is used while playing the planning poker with higher numbers rounded off (0, 0.5, 1, 2, 3, 5, 8, 13, 20, 40, 60, 100).
• For example : You have been asked to provide your estimate for a Story, your estimates have to be rounded off to one of the number of the Fibonacci series.
• If you feel the task will take 10 days to complete, the estimate you give will have to be either 8 or 13.

Each team member takes a deck of cards and estimates the effort independently for each story.

• The stories are wall mapped and arranged relatively with respect to small story which can be coded and tested in a day – taken as a reference.
• All the stories are arranged relatively with respect to small story from left to right.
• The parameters on which the stories are estimated are complexity, ideal time taken (in days) and uncertainty.
• It is always a good idea to break the bigger stories into small ones to make them fit into a Sprint.

The following are the steps followed by the team while using the Planning Poker technique:

1. Product Owner explains the story and its acceptance criteria in detail
2. Team discusses the work involved and asks questions to the Product Owner
3. Everyone estimates individually without getting influenced by others.
4. Each team member selects a poker card that reflects their estimate of the effort required for that story
5. Everyone reveals their estimates simultaneously.
6. Team members who voted with the lowest and highest estimates explain their reasoning behind their estimation and any misunderstanding of scope is neutralized.
7. After discussion, the team members reselect the cards to reflect their estimates based current discussion.
8. Take the agreement of final number as the team estimate.
   
9. Repeat the process for all the stories in the Sprint backlog.

This estimation technique makes the estimation and planning exercise much more fun. It also ensures that everyone in the team is on the same page in terms of understanding the requirements for a particular story and the complexity involved.

What is Behavior-Driven development?

• The suite of unit tests provides constant feedback that each component is still working.
• The unit tests act as documentation that cannot go out-of-date, unlike separate documentation, which can and frequently does.
• When the test passes and the production code is refactored to remove duplication, it is clear that the code is finished, and the developer can move on to a new test.
• Test-driven development forces critical analysis and design because the developer cannot create the production code without truly understanding what the desired result should be and how to test it.
• The software tends to be better designed, that is, loosely coupled and easily maintainable, because the developer is free to make design decisions and refactor at any time with confidence that the software is still working. This confidence is gained by running the tests. The need for a design pattern may emerge, and the code can be changed at that time.
• The test suite acts as a regression safety net on bugs: If a bug is found, the developer should create a test to reveal the bug and then modify the production code so that the bug goes away and all other tests still pass. On each successive test run, all previous bug fixes are verified.
• It also reduces debugging time.

What is Acceptance Test-Driven Development?

Acceptance tests are from the user’s point of view – the external view of the system. They examine externally visible effects, such as specifying the correct output of a system given a particular input. In general, they are implementation independent, although automation of them may not be. Acceptance tests are a part of an overall testing strategy. 

Acceptance tests are created when the requirements are analyzed and prior to coding. Failing tests provide quick feedback that the requirements are not being met. The tests are specified in business domain terms. The terms then form a ubiquitous language that is shared between the customers, developers, and testers. Tests and requirements are interrelated.

A requirement that lacks a test may not be implemented properly. A test that does not refer to a requirement is an unneeded test. An acceptance test that is developed after implementation begins represents a new requirement. Acceptance criteria are a description of what would be checked by a test.

Given a requirement such as “As a user, I want to check out a book from the library”, an acceptance criterion might be “Verify the book is marked as checked out.” An acceptance test for this requirement gives the details so that the test can be run with the same effect each time.