Test-driven development

Test-driven development (TDD) is a software development technique that relies on the repetition of a very short development cycle: First the developer writes a failing automated test case that defines a desired improvement or new function, then produces code to pass that test and finally refactors the new code to acceptable standards. Kent Beck, who is credited with having developed or ‘rediscovered’ the technique, stated in 2003 that TDD encourages simple designs and inspires confidence.

Test-driven development is related to the test-first programming concepts of extreme programming, begun in 1999, but more recently has created more general interest in its own right.Programmers also apply the concept to improving and debugging legacy code developed with older techniques.

Test-driven development cycle

1. Add a test

2. Run all tests and see if the new one fails

3. Write some code

4. Run the automated tests and see them succeed

5. Refactor code

Repeat:

Starting with another new test, the cycle is then repeated to push forward the functionality. The size of the steps should always be small, with as few as 1 to 10 edits between each test run. If new code does not rapidly satisfy a new test, or other tests fail unexpectedly, the programmer should undo or revert in preference to excessive debugging. Continuous Integration helps by providing revertible checkpoints. When using external libraries it is important not to make increments that are so small as to be effectively merely testing the library itself, unless there is some reason to believe that the library is buggy or is not sufficiently feature-complete to serve all the needs of the main program being written.

Development style

There are various aspects to using test-driven development, for example the principles of “keep it simple, stupid” (KISS) and “You ain’t gonna need it” (YAGNI). By focusing on writing only the code necessary to pass tests, designs can be cleaner and clearer than is often achieved by other methods. In Test-Driven Development by Example Kent Beck also suggests the principle “Fake it, till you make it”.

To achieve some advanced design concept (such as a design pattern), tests are written that will generate that design. The code may remain simpler than the target pattern, but still pass all required tests. This can be unsettling at first but it allows the developer to focus only on what is important.

Write the tests first. The tests should be written before the functionality that is being tested. This has been claimed to have two benefits. It helps ensure that the application is written for testability, as the developers must consider how to test the application from the outset, rather than worrying about it later. It also ensures that tests for every feature will be written. When writing feature-first code, there is a tendency by developers and the development organisations to push the developer onto the next feature, neglecting testing entirely.

First fail the test cases. The idea is to ensure that the test really works and can catch an error. Once this is shown, the underlying functionality can be implemented. This has been coined the “test-driven development mantra”, known as red/green/refactor where red means failand green is pass.

Test-driven development constantly repeats the steps of adding test cases that fail, passing them, and refactoring. Receiving the expected test results at each stage reinforces the programmer’s mental model of the code, boosts confidence and increases productivity.

Advanced practices of test-driven development can lead to Acceptance Test-driven development (ATDD) where the criteria specified by the customer are automated into acceptance tests, which then drive the traditional unit test-driven development (UTDD) process.This process ensures the customer has an automated mechanism to decide whether the software meets their requirements. With ATDD, the development team now has a specific target to satisfy, the acceptance tests, which keeps them continuously focused on what the customer really wants from that user story.

WET Web Tester

WET Web Tester is a web testing tool that drives an IE Browser directly and so the automated testing done is equivalent to how a user would drive the web pages. The tool allows a user to perform all the operations required for testing web applications – like automatically clicking a link, entering text in a text field, clicking a button etc. One may also perform various checks as a part of the testing process by using Checkpoints. The latest version of WET is 0.9.8.

WET sits on top of Watir, an automated test tool which uses the Ruby scripting language. WET retains all the features that Watir has and adds many usability related functionality like Data table support, Object depot (aka object repository) and inbuilt Html reporting

History

WET started off as an Extension Toolkit to Watir, a framework for Web testing, released as an opensource product and called it as WET (Watir Extension Toolkit).

Technical details

Working Principle

A UI Test automation is conventionally done by either using a record and playback technique or by scripting completely.

• In the record-playback technique, the tool automatically captures the events generated by the tester’s actions and converts them to test scripts. These scripts can then be played back subsequently. While the record and playback technique allows testers to quickly create tests, experienced testers tend to detest using this approach due to the reliability and maintainability problem posed by these.
• The scripting technique relies on an experienced tester writing test scripts from scratch. Experienced test automation engineers follow a complete software engineer process and treat the test automation effort as another development effort. While this technique offers the advantages of a well maintainable and reliable code, it suffers from certain drawbacks like requiring a greater experience pool, a larger budget, etc.

The third technique is a middle solution that offers some of the conveniences of recorders while retaining the complete scripting power. WET has adopted this technique, called as the Proxied UI technique for test automation. Using this methodology, a tester instructs an IE browser (using some sort of a wizard) to perform various steps like clicking a link, setting text in a textfield, selecting a list box, etc. The tool then converts these instructions to test scripts. Unlike the recorders, where scripts are generated by the tool automatically based on the actions that the user performed on the actual browser, in this case scripts are created only when the tester asks the tool to do so. This by itself increases the accuracy of the generated scripts. Using these scripts as the baseline, a tester can increase the test coverage by writing scripts to suit his application under test.

Be it a total scripting, record and playback or the Proxied UI solution, each comes with its own benefits and liabilities. One advantage of the Proxied UI technique is that it gives control to the tester to decide how much of script generation is to be used vs. how much of hand written scripts. This is a decision that needs to be taken based on the application under test and the test coverage required. In a good project, there should be a proper mix between these techniques so that the benefits of each technique can be availed.

Features

The following are the important features of WET:

• WET UI – Preliminary Script development can be done using the WET UI which is easy to use. Using the WET UI, a tester can create Test Definitions, Object Repositories and the first draft of the test scripts.
• Scripting using Ruby – WET uses Ruby, an object oriented scripting language, which in turns gives WET a powerful scripting ability
• Object depot – The Object depot (aka Object Repository) allows a tester to map all the application’s objects into centralized repositories. This helps in a higher maintainability of scripts.
• Object identification using multiple parameters – Many web pages are designed in such a way that the same page has elements with similar attributes – For example, there may be text fields with the label ‘name’ – one may be for the User’s name while the other may be for the Developer’s name. WET allows a tester to identify even these kind of objects by letting to search for objects using multiple parameters.
• ‘Test definitions’ to define tests – The structure and flow of a WET test is controlled by a test definitions file. This flow closely mimics the regular manual testing process.
• Slick HTML results – After the tests are completed, the results are printed out in neat HTML Format
• Integrated Data table support – Testers can write data-driven tests by using the Integrated data table support. Data tables can either be as Excel files or XML format.
• Popup handling – WET handles Win32 popups quite reliably.

Limitations

The following are some shortcomings in WET:

• Supports only the IE browser. Cannot be used for compatibility testing.
• The Simulated browser view of WET doesn’t work accurately for a complex pages. For these sort of pages, one has to view objects in a tree view only.
• No integration with SCM / bug tracking tools
• Does not have support for keyword driven tests

Compliance Testing

What is compliance testing?  It’s basically an audit of a system carried out against a known criterion.  A compliance test may come in many different forms dependant on the request received but basically can be broken down into several different types:

1. Operating Systems and Applications: A verification that an operating system and/or applications are configured appropriately to the companies needs and lockdown requirements, thus providing adequate and robust controls  to ensure that the Confidentiality, Integrity and Availability of the system will not be affected in its normal day to day operation.
2. Systems in development: verification that the intended system under development meets the configuration and lockdown standards requested by the customer.
3. Management of IT and Enterprise Architecture: A verification that the in-place IT management infrastructure encompassing all aspects of system support has been put in place.  This is to ensure effective change control, audit, business continuity and security procedures etc. have been formulated, documented and put in place.
4. Interconnection Policy: A verification that adequate security and business continuity controls governing the connection to other systems, be they Telecommunications, Intranets, Extranets and Internet etc. have been put in place, have been fully documented and correspond to the stated customer requirements.

A normal compliance check will encompass some if not all of the types listed above.  I will mostly discuss the lockdown policies that can be applied, (or have been applied), to the underlying operating system and applications.  There are a plethora of these about, some provided by vendors, some from other respective parties.  The ones listed below are just a sample of what is available online, there are many more.  They could be consulted as a reference and used as guidance by customers when you have performed a Compliance Check but also possibly after a Vulnerability Assessment or Penetration Test so that they can apply extra security measures to their enterprise to enhance its security.  With the ones quotes, I am just detailing generic Security settings and recommendations that you could apply and audit against, their are also country specific human relations and statutory regulations that you also must adhere to which these guides do not cover.

What are six software defect origins?

These are the origins

1. Requirements
2. Design
3. Source code
4. User manuals/training material
5. “Bad fixes” or mistakes made during repairs
6. Flawed test cases used by the application