Can You Handle the Risk? The Importance of Software Testing

As a Citisoft consultant, I frequently travel to my clients and rely heavily on the airline’s phone applications and websites to arrange my travel. My preferred major airline has impressed me over the years with their intuitive, comprehensive, robust and innovative app and website. Each new code release made travel easier to arrange, change, monitor and keep me, the traveler, well informed. 

However, recently I was required to change a flight and after confidently logging into the app I was disappointed to find that the screen for the flight change option would not work. Puzzled, I attempted to accomplish this task via their website from my desktop. Once at my desk I logged in, selected my flight options and proceeded with the change. I continued onward only to receive an error at the end once again, indicating I had to call customer service to have my ticket reissued. After a call to customer service my flight was reticketed. But when I went back to the website to confirm the status of my new ticket, I saw that all my reserved flights had disappeared, and the re-ticketed reservation was still in an inactive status. Dumbfounded, I called the airline customer service for the second time and was transferred to their Online Help Desk. I explained my situation and was reassured by the representative that, “thousands of other customers were having the same issue and they were working on getting it resolved in the next few days.” Good grief, this was my reassurance? 

What would happen if my clients, highly regarded asset management firms, had similar issues with their software?

These interactions with the airline’s app, website, and customer service made me wonder:

• Did they not understand the reputational risk of deploying shoddy code to their widely used phone application and website?
• Had they conducted integrated, regression and performance testing of their code release before deploying that code to their extremely large client base? Did they not have a rollback plan?
• Did they not have a communication plan in place to reassure their puzzled, alarmed and frustrated client base?
• Did they realize the number of clients that had now lost confidence in their firm’s ability to handle their travel?

Next time, when an asset management client wants to minimize the testing of new software before rolling it out to their clients, I will relay my experience with this airline and emphasize the following key points:

Test plans should be a standard practice

Test plans are comprehensive documents that define the scope, objectives, timeline, resources and environments of the testing effort and hold everyone in the organization accountable to the same plan of action and to the documented exit criteria.

Extensive and comprehensive test scripts are essential

Test scripts can be automated or manual steps to take in the application that should yield defined results to be marked successful. Test scripts are important as they capture the functionality introduced and tested over time as well as the new functionality being introduced. They also serve as the foundation for all unit, integration, regression and UAT testing that follows.

Unit, integration, regression, performance and user acceptance testing should be conducted before rollout to production

Unit testing validates the new functionality works in isolation. Integration testing ensures the new code plays well with the old code in the application and continues to interact well with other linked applications. Regression testing ensures you didn’t break anything within that application with the new code and performance testing ensures the application continues to be responsive with expected standard and peak usage. Lastly, user acceptance testing ensures you have met the requirements defined by the business.

Communication is key

Ensure your client base understands the new functionality and its release time frame and that they know how to address and escalate any potential issues. This will help to ensure confidence in the application and minimal aggravation on the part of your user if they do happen to encounter issues.

Asset managers and financial institutions have a fiduciary responsibility to ensure the validity of their software applications. Software errors have cost companies billions and could even lead to devastating events such as people losing their homes as a Wells Fargo computer glitch revealed in 2018. The millions of dollars Wells Fargo may pay to remediate damages caused by the software error is just the tip of the iceberg of harm done to the institution’s reputation.

It’s been two days since my call to the airline’s customer service and all my reservations are still missing and my flight still shows an inactive status. As my travel challenges continue, consider the importance of software testing at your firm to avoid future pitfalls and ensure your firm’s continued success.