Volume 3, Issue 1

Welcome to the first 2009 edition of the Testing Times. This year brings with it a lot of challenges in the world economy that are reflected in everyone's local environment. In this issue we've focused on how to gain efficiencies in your operation or "how to get more with less". Let us know what you think at info@ascert.com.

Is it Spending or Investing?
Let's face it, many companies put buying software for testing in the "spending" category. In some cases this may be a correct assumption, but if testing software can deliver quantifiable results in dollars and time savings, it really belongs in the "investment" category.
More Details


Barclays Breaks New Ground in Automated Testing
Barclays has deployed an end-to-end automated testing solution based on VersaTest Automator from Ascert and BRIDGE:test from Level Four as a key part of a refresh of its network of nearly 4,000 ATMs. Using VersaTest Automator for message and host testing and BRIDGE:test for virtual ATM testing has produced the most advanced end-to-end automated testing solution of its kind in the world. With the new ATM test environment, Barclays has quadrupled its test coverage. Under the manual process, the bank took at least 23 man days to execute 800 test scripts at the ATM alone, typically over a three month period. With the new testing environment, Barclays has now expanded its coverage to include over 4,000 test scripts, which it can execute in just 48 hours. A huge time and money savings!
More Details

Is it Spending or Investing?

Let's face it, many companies put buying software for testing in the "spending" category. In some cases this may be a correct assumption, but if testing software can deliver quantifiable results in dollars and time savings, it really belongs in the "investment" category.

Most companies start their testing program with internally created software and scripting, downloading "free-ware" from a testing web site, or investing in some externally created software package that addresses a specific need. Often times this results in a "hodge-podge" of programs, systems and processes, with a very poor return on investment. Free-ware is only free to use - there are still costs to get it working in a corporate environment and resolving any issues.

So when is it time to take that next step and invest in an integrated testing solution? Or, when is it time to look at upgrading your current external software solution? There is no stock answer for that, but probably the best response is that the time is always right to be aware of the costs of your own testing processes and the benefits and gains to be realized from state of the art current integrated testing solutions. One might argue that in these challenging economic times it is irresponsible not to look at the ROI of other options, since any costs saved by improving the efficiency of your testing process can go straight to your company's bottom line.

Once you're ready to take the leap, the budgeting process is key, but unlocking those funds in today's financial environment is challenging at best. Some Project Managers view testing as a necessary evil; something that has to be done at the completion of the project. These are usually the people who look at testing as a hassle. It takes too long...it doesn't help build the product or project...it costs too much. These beliefs/attitudes usually lead to an effort to spend as little as possible on testing, which leaves the goal for quality products and systems an anemic one at best. It also produces few metrics that would enable the real cost of the testing process and overall quality to be assessed.

What is this real cost of testing, or not taking testing seriously? Real costs of testing include things like training, investment in testing software, development of test cases and data compilation. They also include the time to test processes, find bugs and fix them. These costs are fairly easy to measure. The cost of not taking quality and testing seriously is a little harder to measure. It usually costs more to fix bugs after a customer finds the problem, than it does if bugs are found prior to product release. The consequences and relating intangible costs of finding the problem later can result in damage to the company image or brand, lawsuits, loss of future business, and angry customers.

So how does a successful Quality Manager get through the budgetary process? By understanding that achieving quality products and services requires investing in testing technologies. They view their goal as identifying and resolving defects, measuring and quantifying continuous improvement of their products, and streamlining their testing operations. They spend their money wisely and use simple industry ROI formulas to qualify testing expenditures. It's probably not a surprise that these managers view testing as contributing a necessary element to the product/project. They also understand that to obtain budgetary funds they need to demonstrate the return on investment as early as possible and in measurable ways, by starting out on small projects that successfully demonstrate proof of concept. These small proofs of concept are the key to unlocking the larger budgets that will return the more significant contribution to the company's bottom line - and the gratitude of the CFO!

In summary, prioritizing testing as an important and valued part of your organization will help minimize the cost of quality; the reason being the investment required to find the problems early in your own testing environment will usually be significantly less than the cost to fix them once a customer encounters them. In this issue of the Testing Times, you can read about how one company, Barclays, used testing software from multiple companies to achieve significantly streamlined testing operations and big savings.

Barclays Breaks New Ground in Automated Testing

Barclays has deployed an end-to-end automated testing solution based on VersaTest Automator from Ascert and BRIDGE:test from Level Four as a key part of a refresh of its network of nearly 4,000 ATMs. Using VersaTest Automator for message and host testing and BRIDGE:test for virtual ATM testing has produced the most advanced end-to-end automated testing solution of its kind in the world.

The project is a result of Barclays moving away from proprietary ATM software to an open standards environment, implementing Windows and packaged software such as BASE24 and ProCash/NDC. This is the largest tier one Bank ATM refresh in the UK for over a decade so requires an advanced ATM testing solution.

The Barclays ATM refresh project includes simultaneous upgrades of operating system, application, networking, encryption and monitoring components. The project is also unique in the extent of its collaboration between the bank and its testing technology partners, Ascert and Level Four. Together, they have developed an advanced strategy for the rapid testing of the entire end-to-end transaction process and alerting and monitoring environment, taking automated ATM testing to a completely new level.

Barclays sought to address the complex testing requirements resulting from the shift to product software with mismatched release cycles. There are thousands of test scenarios at the ATM alone and manual testing of these complex scenarios is much slower as it can involve significant hardware manipulation. A single test could take over an hour to complete manually. The new testing environment addresses this challenge and fulfils a crucial need to fit with the product life cycle.

Jeff Eastham, Technology Partner at Barclays, commented, "We've now reduced the cost and complexity of implementing change across our refreshed ATM network and can quickly respond to the market ahead of the competition." With the new ATM test environment, Barclays has quadrupled its test coverage. Under the manual process, the bank took at least 23 man days to execute 800 test scripts at the ATM alone, typically over a three month period. As well as taking too long to be viable, this process did not address ATM operating system issues which could result in ATM downtime. Barclays has now expanded its coverage to include over 4,000 test scripts, which it can execute in just 48 hours using simulated external systems and virtual ATMs, enabling it to significantly reduce the time to market for new products.

The automated solution enables Barclays to cope with the monthly hot fixes for the Windows O/S by facilitating rapid regression tests of the ATM software stack and all connected interfaces and host systems. This can now be done monthly and quarterly, as well as each time a new software update is installed. Jim Tomaney, Systems Integrator at Barclays, commented "Under the conventional model, even if this coverage could have been achieved, it would have been a one-off cost for a one-off benefit. However, with this strategic approach using VersaTest Automator and BRIDGE:test, it is a reduced one-off cost for continuous, accumulating benefits." So far, the time savings from applying this new model have been worth over £1m compared to the projected manual costs of testing. Subsequent annual savings, in the maintenance cycle, is predicted at around £1m per annum.

Mike Wainwright, Business Development Director in Ascert's London office said, "The approach taken by Barclays, looking to automate as much testing as they can and implementing the best of breed tools for the job, validates a number of approaches that we as a company have been promoting for some years. It proves the benefits that can be gained through such an approach in terms of both reduced costs and increased test coverage."

Do you want to know how testing automation can save your company time and money? For more information, contact us at info@ascert.com or call one of our local offices.

Questions & Answers

Q: When attempting to download a file from a link supplied by Ascert support staff I get the following error. "No matching accessible download found - try logging in".

A: The first port of call here is to make sure that a) you have successfully and completely registered yourself on the Ascert web-site b) you are indeed logged into the Ascert web-site c) that you allow cookies from http://www.ascert.com/. Your identity and logon state is maintained in an Ascert cookie.

Note that successfully completing registration on our website involves the following steps: 1) You complete the registration form; 2) Our website sends you an automated email to verify your email address, to which you must respond; 3) Unless you have provided an 'Organization Key' to identify yourself as one of our customers, Ascert Support will examine your registration request and decide whether or not to grant access.

If you still can't download the file, then please contact the Ascert support team.


Q: Is there a way to specify TCP/IP message length without altering the message definition?

A: As of VersaTest version 5.1 there is a new option in the TCP/IP communications construct called PREPENDED. It can be used to declare that a message has a message length present at the very start of the message without having to define a message length field within the message structure. Here's an example:

    TCP_IP my_client
        POLARITY CLIENT
        REMOTE_ADDRESS
        ENV_PARAM_GET("ClientAddress","","127.0.0.1")
        REMOTE_PORT ENV_PARAM_GET("ClientPort","","30083")
        MESSAGE LENGTH PREPENDED FIELD IS INT16
        AFTER_CONNECT PERFORM session_mark_up
        AFTER_DISCONNECT
        PERFORM session_mark_down
        ;

    SESSIONS
        (
         SESS00000 (TCP_IP my_client)
        )

When this option is used, VersaTest automatically adds the message length field to outgoing messages and removes it from incoming messages.

So where a driver without PREPENDED would have had a message structure of:

    STRUCT iso_msg_def;
        BEGIN
            FIELD IS INT16 msg_len;
            FIELD IS ASCII FOR 4 msg_type;
             ! other fields...
        END;

With PREPENDED it would have a message structure of...        
    STRUCT iso_msg_def;
        BEGIN
            FIELD IS ASCII FOR 4 msg_type;
            ! other fields...
        END;

It also means that there is no longer a need to have custom code on the INBOUND and OUTBOUND constructs to handle the TCP/IP length param.


Q: It seems that while using Internet Explorer to communicate with VersaTest Automator, its memory usage increases over time. Is there a way to reduce this?

A: To reduce the amount of memory allocated by Internet Explorer, simply minimize the Internet Explorer window. Minimizing IE triggers .NET garbage collection to start freeing memory no longer required by the browser.

Photos

Copyright © 2009 Ascert, LLC. All rights reserved. Duplication or republication of any portion of the Testing Times is permitted only with prior written approval from Ascert, LLC.