27th May, 2019: Commercial Bank of Africa (CBA) has selected Tenjin—Yethi’s Robotic Test Automation platform to test its core banking solution by Temenos (T24 – Version R18). CBA was in pursuit of a low-maintenance, seamless, effective, and easy-to-use Test Automation Solution. Yethi’s Tenjin met their requirements after a detailed evaluation among the leading players in the market. This is Yethi’s first T24 project in the African Continent, which will demonstrate the versatility and robustness of Tenjin given the complex nature of the ongoing T24 upgrade.
Commercial Bank of Africa, headquartered in Nairobi, Kenya, has been rated by Global Credit Ratings (GCR) with a national scale long-term rating of A+ and short-term rating of A1. CBA is a pioneer in leveraging technology innovations and aims to transform the African banking sector with a unique range of banking products and solutions. M-Shwari, a mobile banking application launched in partnership between Safaricom and CBA, reached 10 million customers within 18 months of its launch.
Moses Abwoga – Head of Enterprise Projects Management said, “We have always been associated with using innovative technology to augment the manner of interaction that our customers have with our bank. Effectively testing our banking solution to prevent any future errors is a major step towards fulfilling our motive of creating engaging customer experience journeys. We are extremely happy to select TENJIN to assist us in our testing activities during T24 upgrade. We look forward to reap the benefits associated with Tenjin and reduce our overall testing efforts.”
Speaking on the engagement, Srirang Srikantha, COO, Yethi added, “We are extremely happy to be working with the Commercial Bank of Africa. Tenjin’s core features such as naviflow learner, result validator, test management, and task scheduler will help in developing a robust solution enabling CBA to elevate its customer experience.”
About Tenjin
Tenjin is an advanced test automation tool which meets the changing needs of testing in banking enterprises. Its core features such as naviflow learner and robotic units that spontaneously determine and learn the application interface are the reason why some of the most complicated systems in the world use it to improve the efficiency of operations. As a plug-and-play and banking aware solution, Tenjin works within the bounds of regulatory requirements and is instinctively conscious of different transactions, making it a blueprint for successful testing of banking applications. The high-speed solution includes changes automatically without additional programming, relying on robotic learning for quick integration with high levels of consistency. Tenjin Test Automation works with major core banking and digital platforms and is adopted by banks across 15 countries.
Commercial Bank of Africa, headquartered in Nairobi, Kenya, is one of the largest privately owned banks in Kenya. A member of the Commercial Bank of Africa Group, CBA is licensed by the Central Bank of Kenya. Apart from providing a wide range of financial services, CBA’s core mission is to enhance the wealth and financial goals of their customers.
Quality assurance and software testing are the key areas of the software development phase. The current scenario is that the software development lifecycle (SDLC) demands integrating testing simultaneously with development. To assess the quality of test codes with ease, teams find it easy to execute testing soon after the development stage across the CI/CD pipeline for better quality checks. It is referred to as shift left testing.
Shift left, initially conducted at the end of the software development lifecycle, is now executed simultaneously in each stage of development (testing comes to the left of the development stage in the workflow process). This technique is becoming exceedingly popular and driving the growth of DevOps and other modern development techniques.
Testing is introduced in the early stage of the product development lifecycle, which allows the team to detect potential bugs, improve brand affinity by building products that customers desire and save time and money for their organizations. After the developers added the end-to-end browser and API testing to CI/CD pipelines, the scope of testing and quality assurance practices have extended.
What is Browser testing?
Browser testing is a type of non-functional testing that validates the performance of a code on different browsers. In addition to this, browser testing also checks how test codes interact with essential data sources. By introducing browser testing to the early stage of the development process, the developers have added another layer of quality testing to check the code compatibility across different browsers. The QA practices are reformed with the addition of browser testing along with other shift-left testing techniques such as unit and security testing in CD/CI pipelines.
The key areas that are analyzed in browser testing
Browser testing validates the basic functionality to ensure that most of these work on different browser combinations. The functionalities such as dialogue boxes, menus, the input fields in the forms, website handling first-party cookies, and seamless touch inputs for mobile and tablets are validated by browser testing.
The website designer chooses a uniform design such as applications fonts, images, and layouts. The team executes browser testing to ensure that the design elements appear the same as the design shared before the execution of the step.
The team executes browser testing also to check if it meets the accessibility criteria. The account must be compliant with Web Content Accessibility Guidelines (WCAG) and accessible to all users with ease.
Browser testing also validates the website responsiveness. The design team ensures that the website design seamlessly adapts and fits the different screen sizes and orientations.
What is API testing?
API testing is a software testing type where we test functionality, security, performance, reliability, accessibility, usability, and API attributes of an Application Programming Interface (API). API is a group of codes or collective codes replicating an interface between two computers. With the help of API, organizations can deploy external systems with their existing programs and extend the verification and computing services beyond the purview of the programs.
Let us start with an understanding of what the Application Programming Interface is. An application is integrated with many data, and how users can access these data through communication with the application interface. Now consider that an organization wants to deploy an external system with their existing one to extend services through other channels; API makes this process effortless.
This connected program and framework need an adequate assessment and verification. Organizations conduct testing at the API level to focus on assessing the business processes, application security, and data responses. As API testing is ideal for DevOps and the continuous testing process, the development and QA teams use and automate it.
API testing is important
API testing is an integral part of the Agile software development process, which needs to incorporate feedback immediately for a smooth process flow. The User Interface or UI testing is inadequate for validating API service functionality and fails to cover necessary aspects of back-end testing. It may lead to the rewriting of innumerable codes, often causing a delay in the product release and a rise in the cost of production. If not detected earlier, it may also result in bugs going unnoticed within the server or unit levels.
API and Unit tests are easy to maintain and effective, and organizations prefer them over other tests. Tests like the GUI test are hard to keep up with the pace of frequent changes in an Agile environment and require continuous reworking.
The developers can begin with API testing early in the development cycle. It allows the developers to resolve the bugs before they become a critical issue. API testing allows the development team to provide improved services and products to their customers. Incorporating API testing into the Agile methodologies can help engineering and development teams to improve the entire development lifecycle and provide outstanding quality products and services to their customers.
The key areas that are analyzed in API testing
The key areas that are analyzed in API are the responses of,
Reply time
Quality of data
Confirmation of authorization
HTTP status code and
Error codes
Benefits of API testing
API testing ensures that the connection among the shared platforms is accessible, reliable, and safe.
API test automation is faster and cost-efficient as it requires less code than automated GUI testing.
API testing helps developers identify errors in the early development stage and fix the issues before the product release. Developers can access the app to discover the grey areas early without a user interface, allowing them to save time and money.
API tests use JSON or XML, and it contains HTTP requests and responses, making them independent of language criteria.
API tests analyze applications of harmful codes and breakages. It protects applications by removing all possible vulnerabilities.
API tests can be easily integrated with GUI tests before performing the latter. API tests allows to integrate new users to facilitate the testing process.
API tests help in analyzing the non-functional tests like security and performance testing as well.
How are both “shift left testing”?
Before we investigate how the browser and API testing is shift-left testing, let us understand what shift-left testing is? Shift-left testing is also defined as “Early and Often” because it analyses the functional aspects of the product at the early stage of the Software Development Lifecycle.
It is easy for the development and testing team in the agile framework to detect the issues early rather than putting them up for the later stage of the software development lifecycle. DevOps executes comprehensive end-to-end automation to enhance product accuracy and efficiency. But, there are other essential functional and non-functional aspects of software testing that need consideration but often go unnoticed.
API testing and browser testing are the perfect examples of shift-left testing, which needs serious consideration to validate the overall quality of the software. By including browser testing and API testing, the production team combines all functional and non-functional aspects into CD/CI pipelines and Continuous testing in a cost-effective and time-effective manner. It helps in reducing operational expenses and maintaining high-quality standards with greater ease.
API and browser testing is introduced in the agile workflow early during the software development lifecycle and simultaneously runs along with integration, user acceptance, performance, security testing and more. Testing of unit and element can be done separately and integrated with the product during assembling. The practice does not just enhance the product quality but also helps in reducing time, money, and effort.
Developers verify end-to-end workflows by combing HTTP requests and API calls through API tests. It helps the team validate all the layers of their systems from worldwide locations. Often there can be changes in the UI features that can affect the application performance on the website. The users validate complex navigation with the help of browser testing to ensure that no changes should hamper the application performance and look. It captures user view early at the development stage.
Tests like API and browser are “shifting left” to earlier stages of the development process to maintain the rapid development pace. Companies are moving towards leveraging CI/CD practices to ensure that the codes are ready to be deployed. By taking the API and browser codes through the CD/CI pipelines, developers can analyze the product compatibility with different browsers. It can also help in detecting bugs earlier in the development cycle.
Running browser and API tests in your CI pipelines can help the teams to compare the tests with data from different stacks like the front end, back end, application, and infrastructure. They will be able to detect and troubleshoot failed tests during the development stage that boosts the QA process and development lifecycle.
Shifting browser and API testing to the left are considered the best option. By doing this development team can monitor any fundamental changes immediately before the final code goes live. It will also help in improving an organization’s business and operational activities.
Conclusion
We follow an agile methodology and framework in Yethi. We have a well-structured and planned environment for continuous testing. Our codeless test automation platform, Tenjin, is a plug-and-play banking-aware solution that is easy to integrate and learn. Tenjin’s robotic capabilities enable to learn and adapt to the application and its updates. Regardless of the complexity and number of updates, Tenjin facilitates continuous testing, minimizing the manual effort and speeding up the test execution, thereby helping you launch your product early in the market.
Our testing solution allows you to test your software at the beginning of the development stage. Testing is executed simultaneously with the development process so that you can test as you build. Tenjin covers comprehensive end-to-end testing, ensuring you have the quality codes and the product.
Software Testing Automation has gained significant acceptance in enterprises in recent years. Many organizations struggle to identify the value drivers in a test-automation purchase and might take decisions based on partial data.
This article is an attempt to help people understand why automation would make sense, how to understand the various platforms out there, how to balance the economics and present an ROI to ensure proper communication of the expectations.
Software is consistent – Any software will consistently perform actions as per the instructions. While the instruction set might be suspect, the software would be consistent irrespective of the day of the week, the temperature outside or any of the factors that impact humans.
Software is exact – Logical structures have no room for common sense. So, the onus on the developer to understand intent and implications is high. Given that majority of enterprise software, today consists of a myriad set of options, the chances that a particular combination of options (an option set) might work contrary to intentions is high.
The software will evolve – Any software that you use is amorphous and would continue to evolve as long as there are customers and/or developers that wish to tinker. Change is typically due to innovation, competition, regulation, or adaptation.
Software is fragile – New features and options must be delicately introduced, considering the existing options and their usage. It is very much possible to unintentionally “break” an “option-set” while bringing in some new capabilities.
Software Testing – Considering these attributes, it is imperative that software is tested – when it is designed, built, deployed, or modified. The discipline of software testing has been evolving in parallel to software development, albeit in the shadows. Considering the repetitive nature of software testing, a lot of research and effort has been spent on reducing the “drudgery” of the repetitive tasks by adopting automation in the practice of testing. Automated testing is a large part of the public debate and we have explored why exploratory and automated testing needs to coexist in our article here.
In this article, we would attempt to share how test automation supports the mission of consistently deploying reliable software and how one could explore the ROI of these efforts. This is important as Test Automation is primarily a tool to improve efficiency and not a tool to improve the effectiveness of your tests.
What Constitutes Test Automation
The practice of software testing essentially is a stimulus-response study. You provide stimulus to the software (“Application under-test” or AUT) and validate if its response meets a predetermined criterion.
The profession of software testing is to
identify various stimulus-response conditions (test-cases) that could be encountered during the use of the software
plan and execute a sequence of such test-cases
capture and report software behavior,
Test Automation is an aid to a software tester, where the execution and reporting of a certain subset of cases are delegated to the automation software.
Automation allows teams to focus on “net-new”
As software gets enriched, the testing burden shifts from testing “new” feature to testing the “existing” features. In the graph, we have shown 5 sprints where each bar represents newsprint. The “new” functionality in each sprint is shown in “blue” and the “existing” features from previous sprints are in “grey”. You would notice that while the volume of new features is relatively steady, the volume of “existing” features has grown with every release.
While leaders are happy to spend time/effort on testing the blue portions, the repetitive testing of the grey portions is a shoe that pinches. Re-testing of functionality that has been available for a few releases represents a unique set of challenges across all stakeholders. From a tester’s perspective, it is repetitive and therefore boring. It is very likely to only get a cursory look-over and not an in-depth review. From a sponsor’s perspective, it represents an ever-increasing volume of work that often has a low-return element. Therefore, we risk under-testing and letting defects slip into “deemed-stable” portions of our software. Test-automation represents a middle path, where the code can be tested for a larger sample of usage conditions at a lower cost-time construct where only the exceptions are picked-up for further evaluation.
What benefits can I expect? What benefits should I not expect?
The investment in the right automation tool once set up offers potential benefits like:
Coverage: You could achieve a higher test coverage / larger sampling of the software functionality, faster and at a lower execution effort, than what you would get with pure manual efforts.
Speed: Automated tests can run faster and round the clock. These could reduce your overall time-to-test significantly, giving more time to development teams to fix any potential issues.
Consistency: Test automation tools offer maximum consistency. The same test case runs multiple times without even a minor deviation.
Effort: Test Automation can be set up to run with little or no manual intervention.
The Caveats
Setup: Setting up tests on any automation tool is a laborious task. It requires planning and foresight that only experience would bring. Knowledge of how data changes across time, what variables would need to be dynamically updated and how to validate expected outcomes etc. require experience and time.
Maintenance: Once you have set up a large library of cases to be automatically executed, it should be smooth sailing; or so we think!! The challenge of test automation (and therefore where money is generally made or lost) is how easy it would be to maintain this comprehensive library when your underlying application changes. When the “Application-Under-Test” changes, you would have to identify all aspects of your automation that consequently need a change.
Test Automation – Understanding Costs & Benefits
Companies are increasingly investing in test automation considering the increased frequency of changes and anticipated cost-benefit. The investment in the right automation tool is worth it, as it offers potential benefits like:
Coverage: Test automation offers high test coverage with nearly 100% results, which otherwise cannot be achieved by manual testing.
Consistency: Test automation tools offer maximum consistency. The same test case was run multiple times without deviation.
Speed: Automation is multiple times greater than the manual testing speed.
Effort: The manual effort in executing a test automation tool is almost negligible.
Test-automation solutions are varied and bring different categories of over-bearing costs
The cost of the test automation can be categorized as:
Acquisition cost
Maintenance cost
Acquisition cost: There are some free test automation tools available in the market. The free versions mostly perform basic functions and are not recommended for thorough end-to-end or regression testing.
To buy an efficient test automation tool, you will require to make some investment. A licensed and open-source automation tool may start with as low as $4000, depending on the brand and functionality it offers. The total acquisition cost is often inclusive of the tool cost and construction cost. Before investing in the tool, it is recommended to do thorough research and understand the right tool that will complement your requirements perfectly.
Maintenance cost: The maintenance cost of automated tests drives expenses up. Maintenance may involve a range of factors:
Process
User interface
Data
Environment
However, maintenance of automated tests may cost you a bit, statistics reveal the fact that maintenance can be a lot cheaper affair than creating new test scripts. Furthermore, frequent maintenance will keep the system efficient and seamless. Maintenance is a good option while considering the cost model of the project, as manual testing or implementing new automation tools can be expensive and time-consuming.
Test Automation ROI
Now that we have given you an understanding of the costs and benefits of test automation, we go further deep into exploring the ROI of these efforts. Companies are constantly investing in test automation, but investment in test automation without proper calculation of ROI would be a half-baked effort, which is unacceptable for the companies.
At the beginning of this article, we have stated that test Automation is primarily a tool to improve efficiency, and the calculation of ROI determines the efficiency of your test automation process. Test automation is a one-time investment. Configuring the test automation at the initial stage of project inception can maximize the benefits in the long run. By calculating ROI, companies can decide to invest in test automation.
Calculating ROI does not necessarily mean that we would focus on the overall cost, which is required to evaluate the quality of the product. ROI calculation also includes the parameters like late releases, defects overlooked, and lack of quality.
One of the biggest challenges that the companies face is investing a large sum of money for conducting the end-to-end testing of software, which often they are unable to recoup. Once the companies decide what test sets or subsets they must automate, they will generate a high return on investment.
Due to poor decision-making, companies may spend a lot to adopt automation at the later stage of development, and if the project fails, they may incur a high loss. ROI-based automation testing resolves this issue. The Quality Assurance team decides which section must be automated based on the clients’ environment, system, and machine. They can do a cost comparison analysis of manual test runs and automation test runs and determine the types of scripts that can be automated.
Following are the significant metrics, which can be considered while calculating automated testing ROI.
Measure product quality – Ensuring the overall quality of the product by identifying defects and mean time to detection of defects.
Test execution speed – Comparing the test run duration percentage between automated and manual testing.
Thorough cost and resource analysis – Analyzing cost and resources are important as the test automation process involves software and hardware costs, prioritizing defects based on their severity, operational costs, the requirement of new resources, training costs, and more.
With the metrics mentioned above, we can calculate Automated Testing ROI. The cost that companies bear on resources, tools used, setting up the testing framework and more fall under companies’ investments. The net gain that the companies receive over their investments would be the ROI for the automation testing.
Following are the parameters required for performing ROI calculation.
Automating new tests – There are always a certain amount of costs involved in automating new tests. Before initiating the automation of new tests, companies must consider the predictable costs of developing, executing, and maintaining the tests. Based on the test categorization, you would segregate the test to automate or run manually.
Automate regression testing – Adding new features or changes in software would introduce new bugs in the existing modules. It makes regression testing one of the most expensive phases of testing. It is a necessary step to calculate the cost of automating regression testing.
Managing Multiple Environments – As companies develop products, they must be convinced that their product works equally in different environments. Hence, companies set up multiple platforms, devices, and browsers to ensure the success of their products. For organizations, it is necessary to calculate the ROI while setting up test environments.
Minimizing Defect Leakage – There are instances where bugs were detected at the production phase, as they were not resolved at the development stage. It often leads to defect leakage in the production phase, making it an expensive issue for organizations. Calculating ROI is necessary for companies to control such overhaul expenditures in certain instances.
Reusable test cases – Using reusable test cases saves time and effort by avoiding task duplication. When you already have created test cases, you can reuse them in multiple scenarios. Hence this will impact while calculating ROI.
Protecting product knowledge – Companies may still gain control over their systems and software, but they need to take a calculative risk on their resources. If an engineer leaves, the company must look for a replacement and provide training for the job. It may impact the company’s long-term automation testing ROI.
Ways to improve Automated Testing ROI
Instead of automating all the tasks at hand, it is best to automate repetitive tasks. Automating the applications, which are not steady, may not show an accurate result. Moreover, it may increase your team’s effort and affect while calculating ROI.
Automation testing works best for regression testing. Hence, when new tests and integrated with the older ones and applications are tested across various platforms, automation testing can be implemented to save time, effort, and overall costs. Further, if you need to complete the test run in a shorter time, you can run parallel testing like a smoke test.
Changes are introduced frequently in software. Hence, the developers and testers are relying on exploratory testing which is widely used in agile models. The focus of exploratory testing is discovering, investigating, and learning through the process. The testers here own the freedom and responsibility to think and note down ideas before the test execution. As exploratory testing is strategic, it helps in improving automated testing ROI.
The testing process has evolved, and today shift-left testing is a popular step in the software development lifecycle. Shift-left testing allows you to detect defects in the early stage, which helps in reducing effort, time, and money. You could also use test management tools to identify redundant test cases at the early stage of the development process. Finally, keep thorough documentation of the process so that the newly hired resources can pick up where your previous resources have left.
Conclusion
The company adopts automation testing to improve its test efficiency and test coverage. Of all the possible methods, which could be applicable for calculating the automated testing ROI, the company must adopt the strategies, which work best for them and suffice their requirements.
When it is about improving the quality of your product, the right methodology would be to test your applications simultaneously while developing them. If defects are detected at the postproduction stage, the company will suffer monetary loss, which may be unacceptable for any organization. Hence, a little strategy, the right tool, and differentiation between manual and automation testing can successfully help you implement test automation ROI.
Organizations do not immediately execute software testing after receiving the project details. From acquiring project details to test execution there is a critical step of requirement validation that deals with related documentation and helps in smoothening the testing process. It is an essential step in the software development and testing processes, which cannot be neglected. While testing helps understand the efficiency of the code and other functionalities and identifies any errors or discrepancies that affect the quality of the software, requirement validation and documentation help you prepare for software test execution. The checking of this documentation comes under the purview of static testing.
There are two critical methods of Testing; they are Static Testing and Dynamic Testing.
Static Testing: It is a testing method that allows a user to examine the software/program and all the related documents without executing the code.
Dynamic Testing: On the other hand, dynamic testing checks the application when the code is executed.
Both these methods are essential and frequently used together to ensure that the applications are functional. However, this article highlights the static testing approach, which is crucial for the software development lifecycle but often is taken for granted. It is an assessment process to check the code and requirement documents to find errors early in the development phase.
Why is Static Testing used?
Static testing is performed to check the code and design documents and requirements for coding errors. The aim is to find the flaws in the early stage of development. Static Testing makes it easy to find sources of potential errors.
The users and developers static test the code, design documents, and requirements before executing the code. Checking the functional requirements is also possible. The process explicitly reviews the written material that gives a broader view of the software being tested.
Following are some documents that are checked during the Static Software Testing Process.
Requirement Specifications
Design documents
User documents
Web page content
Source code, Test cases, test data, and test scripts
User documents
Specification and matrix documents
What are the errors that can be detected during Static Software Testing?
Types of defects that can be easier to find during static testing are as follows –
Deviations from standards
Non-maintainable code
Design defects
Missing requirements
Inconsistent interface specifications
Why is Static Testing used?
Static Testing is specifically used to identify any flaws in functionalities or possible coding errors in the initial stages before the code is executed.
Following are some benefits of using Static Testing –
Detection and correction of potential coding errors
Cost Efficiency – A reduced cost that goes into rework to fix errors
Time Efficiency – A reduced time that goes into rework
Feedback received at this stage helps improve the functioning of the project
Once the developer moves to Dynamic Testing, the number of errors is limited. It makes the code maintainable
The process also helps the developers identify the quality issues in the software
There are automated tools available that make this process of reviewing the code and other documents faster and easier
Static Testing also boots communication among inter-functional teams
What are Static Testing Techniques?
Static Testing is carried out in two steps –
Static Review
Static Analysis
Static review is done to find and rectify ambiguities and errors in supporting documents like requirement specification, software design documents, and test cases.
These documents can be reviewed in various ways, such as –
Walkthrough
Peer review
Inspection
In the second step of Static Analysis, the code written by developers is analyzed. This step helps us identify and rectify structural defects that may become errors or bugs when the code is executed.
Static Analysis helps the developer to find the following types of errors –
1. Wrong Syntax
2. Unused variable or variables with undefined values
3. Dead code
4. Infinite loops
Static Analysis is of 3 types –
1. Data Flow – Related to stream processing
2. Control Flow – Determining how the statements and instructions are executed
3. Cyclomatic Complexity – Determining the complexity of the program that is related to the number of independent paths in the control flow graph of the program
There are various other techniques used while performing Static Testing; following are some of the common ones –
Use Care Requirement Validation: This technique ensures that the end-user functionalities are defined properly
Functional Requirement Validation: This technique identifies all requirements for the project
Review of Architecture: In this technique, all business-level processes are analyzed
Field Dictionary Validation: This technique helps us analyze all User Interface related fields
Static Software Testing Process may be conducted in the following ways –
Manually
With the use of Automated Testing Tools
How is Static Software Testing Reviewed?
Review is the most crucial step in the Static Software Testing process. These reviews are conducted to identify and rectify any potential errors in the supporting documents. The reviews can be walkthroughs, informal reviews, technical reviews, or inspections.
Walkthrough – The author of the specific document explains the document to the team and peers. The author also answers the questions and queries from the team.
Technical Review – Technical Specifications are reviewed by the peers to ensure all the functionalities are reflected in the software and the potential errors are identified and rectified.
Inspection – A dedicated moderator conducts strict reviews to ensure that the Static Testing process is completed efficiently to make the application as robust as possible.
Informal Reviews – Informal reviewers do not follow any specific process. Co-workers review the documents and provide internal comments.
Conclusion
Used to evaluate the code and requirement documents, static testing is essentially used to assess the written code. Organizations incorporate static testing methodologies either manually or by automation to detect the code error early in the test lifecycle, which plays a critical role in improving the quality and reducing cost and effort.
At Yethi, we offer a thorough requirement analysis, planning/scenario design, and reviews. We ensure maximum quality at different test stages. Before executing the tests, we review business processes, products, applications, and integrations to ensure optimum test coverage. We neatly arrange the steps like reviews and analysis to execute a well-structured testing process.
In recent years, many digital platforms were developed to address user requirements, and innovation is still underway. Since we use the digital platform in every sphere of our life, technology and innovation will keep emerging. Just while we continue developing more digital platforms, we need to be confident about their performance and user acceptance. Testing becomes extremely important as it ensures that the software performance is top-notch, and end-users can reap the benefits of these applications. As digital platforms are evolving, so is testing. From manual testing to automation, the testing phase has moved further to Cloud testing.
What is Cloud testing?
Testing of applications using cloud computing services is known as cloud testing. Cloud testing focuses on faster availability while ensuring scalability and flexibility, and helps in saving time and cost, which is an ultimate requirement for software testing. The purpose of cloud testing is to check and validate the functional and non-functional requirements of software using cloud computing. Cloud testing allows testing of software over an internet-based platform and remotely provides many computing services like hardware, software, and other computer-related services.
Following are the main models of Cloud Computing,
Models of Cloud Computing
Types of Testing in the Cloud
The applications can undergo following types of testing in the cloud.
1. Functional testing
Functional software testing examines all the features and functional requirement of software. It also checks how software interacts with hardware.
System testing — Through this testing system’s compliance is evaluated for its functional and system requirements.
Acceptance testing — Acceptance testing ensures that an application can meet customer requirements.
Integration testing — This testing checks if the application is compatible with different platforms and works even when it is moved from one cloud infrastructure to another.
2. Non-functional testing
Non-functional testing in the cloud allows you to validate the non-functional features of software such as performance, usability, and reliability.
Security testing — Security testing of application ensures that data is stored and transmitted safely.
Stress testing — Executing Stress testing help you in evaluating your application performance under excessive stress.
Performance testing — Performance testing ensures that the application performs correctly with various numbers of users, while cloud solutions should be scalable on-demand.
Load testing — Load testing helps in measuring software response time when the load is increased in system.
Latency testing — Latency test allows you to measure the application response delay after deploying it in the cloud.
3. Ability testing
To verify whether users really receive application services on-demand, ability testing is necessary.
Compatibility testing — As the name suggests, the compatibility testing helps in evaluating the application’s compatibility with various environments and platforms.
Disaster recovery testing — Disaster recovery testing helps in evaluating disaster recovery time and ensure that the application is available to users with minimum data loss.
Multi-tenancy testing — Multi-tenancy testing verifies application’s ability to ensure the level of security and access control for multiple users in the cloud.
Benefits of Cloud Testing
Cloud-based testing is widely accepted over traditional manual testing. Here are the following reasons:
Scalability — Business requirements keep changing frequently. Based on these requirements, cloud computing resources can be increased or decreased, ensuring the scalability of the technology.
Time-efficient — Cloud testing allows you to run your applications on various hardware. Hence, testers have enough time to fix the bugs making the technology more time-efficient.
Cost-efficient — Cloud testing allows you to pay only for those resources, testing environment, hardware, and software that you might use.
Easy test environment configuration — Cloud-based tools allow you to set up a test environment on multiple devices, saving you time and money. It can also detect the errors during environment configuration across all devices.
Easily customizable — Cloud testing provides easily customizable options by using cloud-based tools and services. Testers can easily emulate an end-user-centric environment saving you time and money.
Ensure comprehensive testing — Cloud testing creates an all-device platform and configuration, eliminating the need for multiple devices and facilitating comprehensive testing.
Faster testing — Cloud-based testing tools enable software test automation reducing, time to the market. It has eliminated the need for having separate testing infrastructure while enhancing team collaboration and improving testing.
Continuous availability — Software testing is available to the testers anytime and anywhere to speed up their software development lifecycle with its continuous availability.
Improve team collaboration— Companies can improve their team collaboration by implementing DevOps in the cloud. The team collectively can try different configurations and data and automate testing processes using tools.
Four main categories of Cloud testing
SaaS testing in Cloud – Based on the application requirements, functional and non-functional testing is executed.
Testing using entire cloud – Based on the cloud features, where the cloud is considered as an entity, testing is executed by Cloud and SaaS vendors, as well as end-users.
Testing within a cloud – Cloud vendors can check internal features to carry out testing.
Testing across cloud – Utilizing different types of cloud environment, the testing is executed.
Cloud Testing’s Core Components
The testing is executed of these core components to ensure the following aspects.
Cloud Core Component
Conclusion
The expert team at Yethi understands the importance of testing across multiple devices and networks. As swift changes in digital environment redefine consumer and retail banking, we are providing exhaustive test-coverage throughout all aspects of digital transformation and application testing like Functional, UI/UX, Performance, Compatibility, Security, Geo-based and network testing, by enabling testing across multiple devices, OSes, and browsers,
With cloud testing, we ensure timely testing of digital and mobile assets by incorporating real-time testing of both native and hybrid mobile applications across multiple platforms, thereby eliminating delays and redundancies and saving time and money.
Early in 2018 four major Banks in the US had clients up in arms when, for several hours, banking services were not accessible via their mobile banking apps. The interrupted services were attributed to a major spike in traffic due to it being payday. Notwithstanding, reputational damage had been done to the banks and, adding to their woes, a social media tirade on platforms like Twitter, where irate clients vented their frustrations and further lambasted the banks, ensued. The experience of these banks again highlighted the importance of software system testing in banking and financial services. However, software system testing is only as good as its test data management strategy, which should be adopted and implemented to:
Manage test and development processes to meet testing and application development requirements
Secure data and streamline cloning processes, delivering clones needed to meet upgrade and patch cycles as well as maintain data security
Identify appropriate replicable accounts and transactions from production to meet test criteria
Mitigate the threat of identity theft concerns among consumers and regulators
Improve turn-around-times during system upgrades through improved planning of data refreshes and overall data utilization.
The importance of test data management in testing
At its core, data management is the science of creating and maintaining the data sets generated by software system testing, driven by cause-effect relationships, producing predictable outcomes and responses. These data sets can be generated entirely by the tester, by simulating each stage of the customer or transaction journey, producing what is called synthetic data. Alternatively, data generated by actual transactions and previously recorded is called migrated data, providing authentic sets. Each of these forms of data generation has its value and challenges.
Synthetic data needs to be interoperable, able to account for diverse systems in varying environments without diluting the complexity of inter-relationships. Besides, the data needs to be refreshed and valid for multiple rounds of testing and if in an automated environment, should not need to be updated by manual intervention. Migrated data, in turn, is drawn from authentic transactions has limited reusability across different test-cycles, hamstrung further by data confidentiality obligations.
Mitigating the risks inherent in Test Data Management
Implementing a Test Data Management strategy requires that several risks and their potential impacts be considered. The complexity of the data, structured or unstructured, whether the databases are new or from legacy systems and how the data is stored. If kept in multiple environments, it may have the added challenge of access sensitivities and potential confidentiality breaches. Assessing the time available for data discovery, its generation and management is another factor that is paramount in ensuring representative samples are obtained. Besides, test data management requires different types of data for different types of testing, from performance through to user acceptance testing, so data needs to be organized correctly to fit within the required time and budgetary constraints. Another potential risk is the measure to which the organization operates in a distributed multi-supplier or outsourced environment, with multiple users accessing the data in multiple locations. Operating in such an environment also highlights the importance of data security and protection. As a standard, new security protocols will be required as well as staff training, highlighting the importance of protecting live production data as well as guarding test environments.
Empowered Software Testing through Test Data Management
Given the dynamic nature of the banking environment and the need to ensure optimized and efficient process continuity, any process that does not facilitate this expediency will have a negative impact on the entire system. Accordingly, if the process of choosing, interpreting and scrutinizing test results is time-consuming, arduous, and requiring specific knowledge of underlying applications, it will have a direct negative impact on the entire system. The time it takes for reporting and information sharing in an organization can also be a significant factor in the efficiency of the entire organization.
To meet these dual needs of time and efficiency, banks should implement the following test data management techniques, while taking care to mask sensitive information:
Database cloning through the copying of production data
Data sub-setting by substituting production data when appropriate, &
Synthetic data generation, through the production of synthetic data based on a clear understanding of the underlying data model, which requires no de-identification
Yethi, a leader in software solutions and quality assurance in global banking and financial services has developed substantial expertise in creating end-to-end enterprise-wide test data management services which include data quality assessment, data masking, data subset, data archiving, data cleansing and data optimization. Yethi also assures that through rigorous control the thorough management of test data is ensured, beginning with quality assured consistent data. Data security and privacy are also safeguarded, while storage requirements and software costs are significantly reduced.
Software testing in the banking industry is an important exercise performed to accommodate and reflect operational changes. Given the surge in use of banking applications and technology, the industry is dependent on testing. While manual testing is still required to test the application before it goes live, automated testing helps reduce time to market and improves the quality of the application in the production stage.
While automated testing tools make testing easier, they come with their own set of challenges. Test automation projects require unique scripting skills that testers sometimes lack. The most demanding phase of an automation project is the initial stage when tools are set up and test scripts are written.
Even when an automated testing solution is implemented, chances of failure often frustrate banks. This is mainly because of the flawed implementation of the solution or investments in automation failing to deliver on their promise.
Let us look at other reasons why a bank’s software test automation project fails:
1. Automation testing requires special programming skills
Testing an automation software solution requires unique programming skills and most testers come with domain and application knowledge. Consequently, they struggle with testing automated software. The solution is to use a script-less automated testing software for better efficiency.
2. Timely reflection of changes
Another reason for automation projects to fail is that the changes, be it by the regulatory authority or product updates take too much time to reflect on the bank’s system as well as the third-party systems.
3. Underestimating the complexity of the automated testing tool
Most automation tools are tested to handle simple cases with 5 or 10 screen workflows. When it comes to a banking solution, there could be over 5,000 screens with over 50,000 fields. This could be challenging for even the best automation solution. It could further lead to wrong ROI calculations and pose some serious technical challenges while dealing with different technologies of the bank’s underlying applications.
4. Inflexibility in testing
Factors such as dual authentication can be frustrating in the testing activity. Not solving this issue can be one of the primary reasons why a bank’s software testing automation project fails.
5. Use of various technologies
Most banks have diverse applications in use, on various technologies. Accommodating changes on all the user interfaces and APIs requires handling multiple UI types –web, desktop, and mainframes. Failure to do so could mean failure of the automation project.
6. Choosing a wrong protocol to automate
While planning an automated testing activity, one must plan the testing activity and the test process efficiently to see which is the process that needs manual testing and which ones could be automated. Any wrong move in planning can result in failure of the automation testing project.
7. Not being able to reuse the testing solution
As a testing solution has a higher initial cost, banks must plan in such a way that the same testing solution can be used for multiple functions and must be easy to navigate.
8. Having no validation
A testing solution must make room for validation—validating the software at the end of the development process to see if the software meets the expectations and requirements. In the banking domain, the testing solution must be able to test loan schedules, interest charged, charges, accounting entries, exchange rates, messages, and client communications.
Failure to do so could cause the automation project to fail. It all boils down to this—plan proactively and do not underestimate the tool, the deadline, and the resources to ensure the success of the testing project.
A script-less easy-to-use automation solution can help banks meet their testing needs. The Tenjin Automation Suite makes uninterrupted banking operations a possibility.
I have been a fan of online casino games for many years, and I know that software testing is an important part of the process. This is where Yethi Consulting comes to the rescue - they are experts in software testing and help online casino real money provide the best service to their customers. The company's services include software testing and quality control. Yethi Consulting's team of experienced professionals often assists online casino operators with numerous tasks, including unit testing and manual testing. The company's clients highly appreciated its services for thoroughness and efficiency.
Testing software is important before its launch. Testing an application ensures that it meets the objectives to which it was designed. With the frequency of changes to software increasing, the percentage of tests for “new features” progressively diminishes. Teams are increasingly spending time to ensure existing features continue to work. Therefore, the practice of automating tests to reduce the “regression” burden has steadily increased.
What is Manual Testing?
A QA Analyst executes manual testing to detect bugs or defects found in software at the development stage. Software is built with several essential features. A skilled and experienced tester checks all these features, execute test cases and generate reports without the help of any software automation testing tools. This traditional method helps in detecting bugs in software systems. To accomplish the software testing process, it is conducted by the experienced testers.
Benefits of Manual Testing
In manual testing, the testers execute test cases manually without any support from tools or scripts. Manual testing allows you to detect the errors.
Manual testing allows you to explore a little more possibility. Ad hoc testing allows trying different avenues of detecting errors.
Manual testing can eliminate the errors, which automated testing fails to detect.
Manual testing is useful when automated testing is not technically possible or is not cost-effective.
Manual Testing can help you to detect the issues better, identify the whole problem and fix them.
What is Automation Testing?
Automation allows the testing team to offload some of the test execution efforts to software to execute against a set of pre-determined inputs and validate against expected outputs. Automated testing is a pre-scripted test that runs automatically to help the tester determine whether the behaviour of the application is as expected. Automated testing lowers the run time of repetitive tasks or regression test execution. Automation is an investment that would reduce your efforts over a period of time. Initially, additional effort and costs are required for procuring the software and creating the test scripts, thereafter it may run automatically with lack or no human effort.
Benefits of Automation Testing
Automation allows the testing team to focus more on “new features” and reduces the repetitive, detailed oriented regression testing.
By completing tasks faster, any potential errors can be identified faster to enable faster closure of defects.
It reduces business risks by increasing coverage of the tests that you can perform within a given period-of-time.
Ensures increased quality of test results by decreasing the necessity of fixing glitches after release and reduces project costs
It increases high test coverage leading to testing more features. Automated test cases can be used multiple times through different approaches
Automation allows you to detect software defects at an early stage so that the glitches can be fixed in a cost-effective way
Automation allows you to launch the product in the market faster while protecting the potential risks of data manipulation
Despite its benefits, many organizations do not fully implement test automation, either because of the initial cost or the efforts required to maintain the level of automation. This is a case where the intangible benefits are equally important to the tangible benefits.
Both manual and automated testing have advantages and disadvantages, and both forms of testing are equally important to get the best results of testing. It is not a case of one or the other, but a question of using the right tool for the right task.
A few key differentiation are:
Manual Testing is done manually by an experienced QA analyst.
Automation Testing is done using scripts, codes, and automation tools by a tester.
In the manual testing process, there are possibilities of human errors.
The automation process is code and script-based, hence it is reliable.
Manual Testing is time-consuming.
Automation Testing is very fast.
Manual Testing is possible without programming knowledge.
Automation Testing needs programming knowledge.
Manual Testing allows for random testing.
There is no scope for random testing in the automation process.
What manual testing is best suitable for?
Ad-hoc Testing: It is an unplanned method of testing. The important factor is the tester’s understanding and insights.
Exploratory Testing: This test is not possible without human skills. A tester must have the knowledge, experience, analytical/logical skills, creativity, and intuition to execute the testing process.
Usability Testing: Human observation is an important factor to measure user-friendliness, efficiency, or the convenience of the software or product for the end-users.
How to think about the complexity of the testing automation?
Application Under Enhancement
Cost of automation is relatively low, ROI is moderate. Reduce cost of automation build.
Cost of Automation / Investment is high. ROI is also high. Reduce cost of maintenance.
Application Under Development
Cost of automation build is high – repetitive builds. Focus tool on build / change management.
The differences between these two forms of testing are enlisted below to help you achieve quality output well within your budget and timeline.
Difference Between Manual Testing and Automation Testing
About Yethi:
Yethi has taken the enterprise software testing to the next level with Tenjin’s superior automated test suite. It is powered by an intuitive and path-breaking algorithm that automatically “learns” the software’s UI and “re-learns” any changes without manual assistance. We are led by a team of functional experts, who can conduct testing without relying alone on technical resources. We thrive to help our clients achieve their business objectives within the stipulated time-frame, without compromising on quality. With multiple test case instances, we have proved our technical capabilities.
User Acceptance Testing (UAT) Service ensures that a software continues to perform and provide you with expected outcome under all business scenarios. UAT is an essential part of the software development process, regardless of whether the system is developed in-house and purchased from a licensed firm.
The methodology of User Acceptance Testing is complicated because it is not always easy for a developer to understand what customers seek. The developers may misconstrue and edit specifications, which may not be useful for the customers.
User Acceptance Testing involves multiple stages and some companies may not have time, bandwidth, and experienced staff to conduct the tests. Under this scenario, it is extremely profitable to outsource testing to an experienced team of professionals who can perform all the stages of UAT without any errors.
Outsourcing the manual and automation testing work to professionals allow you to focus on your business priorities with an assurance that all your testing needs are taken care off by the professionals. It does not let you drive away your focus from the fundamental requirements of your business.
Business Challenges
The software testing cycle includes a few challenges. The common business challenges are as mentioned below:
The reluctance of users to perform the test
Lack of participation
Allocating resources for UAT testing
Improper test design and planning
Untrained and inexperienced UAT testers
The advantage of outsourcing User Acceptance Testing
A team of experienced testing professionals with thorough knowhow of the business domain executes your project
Develops a thorough UAT strategy and improves the quality of your project outcome
Projects are successfully executed within strict deadlines
Not just developing, but a specialised outsourcing firm provides guidance and support for seamless implementation of User Acceptance Test plans
You can get access to a large repository of test cases and real-life scenarios based on your business requirements
Availability of well-defined models like the risk-based testing approach that aligns perfectly with the ever-evolving business needs and schedules
With a large pool of specialised consultants being available, there is no additional responsibility for hiring and managing internal consultants
Outsourcing user acceptance testing services reduce risk, ensuring customer satisfaction. UAT services comes is wrapped with the following features: –
Enhance the productivity of your team without hiring new resources
Get your UAT professionals, testers, and consultants to develop relevant test cases based on user/business requirements
Set up relevant defect management system as per customer preferences
Make informed decisions based on accurate reporting and proper communication protocol
Benefits of outsourcing UAT
Cost Management: Outsourcing user acceptance testing will help you to manage your budget constraint by reducing the costs. You can be sure of high-quality services at affordable prices.
Risk Management: Outsourcing UAT services can help you to minimize risks associated with operations, testing automation, test management and resource planning as only the domain experts can make a secured test plan to control the risk.
Time Management: In addition to cost and risk management, an experienced UAT outsourcing firm can help you with time management, which is an important criterion for your project inception. The domain expert understands your time constraint and takes care of your project from start to end.
Business Expansion: When you are outsourcing the user acceptance testing process, you will gain a competitive edge across the market. The experienced outsourcing firm can optimise the use of your resources and provide you with additional support to improve your delivery schedules and service quality.
Skilled Resources: You can get access to the skilled resources who are experts in their domain practice. This allows you to free your in-house resources and delegate them on your business priorities, based on their core competencies.
Why Choose Yethi?
With years of experience, we have developed solutions of varied scales and complexities for our BFSI clients so that they can conduct comprehensive testing and auditing of their services. Our team has helped the clients with error-free implementations whether they needed to upgrade or test the applications for new modules. We offer extensive solutions within the stipulated timelines by leveraging domain expertise and script-less in-house automation platform. We offer varied testing types to unlock their business potential through new avenues and enhance application quality while managing the budget and time constraints. Our experienced team has worked with multiple technologies to seamlessly implement a user acceptance test plan based on our client’s requirements. With a team of experienced UAT consultants, advanced tools, and testers, we ensure a quality to match your organization’s reputation.
Yethi’s achievement of being 6 years in the operation:
Yethi’s Achievement
300 highly experienced consultants
Tenjin – 5th Generation Test Automation Accelerator
