Software testing is a crucial part of building an IT product. Manual Testing is still important for effective software delivery. However, for foolproof results, you cannot survive without complementing it with automation testing. Hybrid Test Automation is one of biggest trends in the software industry these days.

Perhaps, you may also be vying to implement automation testing, but to no avail. The reason is pressure of quick and continuous applications releases, busy schedules or non-availability of internal skilled resources. So, manual testing has been no cakewalk for you, especially, while you’ve been running projects using Agile Methodology where you must be testing stories of particular sprint at least 3-4 times before releasing the sprint to the client.

The fact is fixing bugs detected early is a lot easier and cheaper than finding them later in the development process. This means creating automated tests for checking each added feature as early as possible. However, knowing what and when to automate is very important and there are some best practices for automated testing.

Today, I’m going to show you a tool using Automation Framework that Can Eliminate Manual Testing Woes. With AI-Powered Test Automation Dashboard, this Framework is based on the hybrid model that applies the best features of Keyword-Driven Framework, and Data-Driven Testing Framework.

With this Framework, you can create simple functional tests in the earlier stages of development, test the application, piece by piece, and improve your automated testing success rate without having to learn a scripting language. This can extremely improve the test process and reduce cost. The tester can automate any web and mobile applications by writing test scripts in Excel using the simple English language with little training.

The major difference between the Keyword Driven Framework and others is the use of “Keywords”. Based on the usage of Keywords, you can broadly classify the framework in 2 different types:

Type 1: Keywords are mapped to the lowest level operation on an object.

Type 2: Keywords are mapped to a function or action which contains a collection of statements.

In this article, I will cover the first type in detail. We will see how you can create a keyword driven framework where most of the keywords are mapped to the operations at the object level. The important point you should note here is that in this type of framework you have to “first identify keywords and then map them with your functions”.

In this framework, all of test suites, test cases, test data are generated from external files Excel. Mark that the Excel file formats are solely user defined.

I will explain the structure of each Excel file based on the below images:

Test Case is divided into four different parts. First is called as Test Step, second is Object of Test Step, third is Action on Test Object and fourth is Data for Test Object.

Normally, test cases are written in excel file along with test data and then automation framework executes the test steps by reading the excel row one by one.

To run the test cases against multiple sets of data, you can store the test data in separate excel sheets. When you store the data separately, the data from the Excel sheets will be fetched according to column names in the data sheets. The number will be added at the end of test case name depend on the number of test data.

With web-based applications, Selenium supports various locator types such as id, css, xpath, name…

For mobile, Appium is a great choice for test automation framework as it can be used for all these different app/web types.

Appium is like Selenium but for mobile apps and games. Appium is also suitable for mobile web testing when real devices and real browsers are used in testing. In a nutshell, Appium is a mobile test automation framework (with a tool) that works for all: native, hybrid and mobile web apps for iOS and Android.

In order to find elements in a mobile environment, Appium implements a number of locator strategies that are specific to, or adaptations for, the particulars of a mobile device. There are available for both Android and iOS: name, xpath, id, cssSelector…

Appium provides you with a neat tool that allows you to find the elements you’re looking for. With Appium Desktop Inspector, you can find any element and its locators.

For mobile games that you cannot catch the element by locators such as Unity games, you can integrate this framework with Sikuli API library in order to implement image recognition. Sikuli will detect and perform actions on detected images.

The Object values represent the image names that you want to perform the actions instead of the element’s name in this case.

The image will be recognized and returned in coordinates by Sikuli library. As you can see in the image above, the settings button is detected.

And now let’s dig in for you understand the operating mechanism of this framework.

You would need to write some code that will open excel files and handle information from each sheet of them. To read or write an Excel, Apache provides a very famous library POI. This library is capable enough to read and write both XLS and XLSX file format of Excel.

Keywords has been used to create the test cases. Actions library will read the keywords and when they are encountered and execute the functions associated with those keywords.

According to the keywords written in Excel files, the framework will perform the actions on UI.

Selenium is used for Browser automation whereas Appium is used for Mobile app automation. So, it depends on your testing scenario, whether you are targeting mobile app or a web browser. If your target is testing a mobile app then you should go with Appium actions and if your target is web browser then use Selenium actions.

To reduce the time and efforts of testing, I implemented parallel execution concept to run the test suites on multiple browsers or devices simultaneously by using ExecutorService and SwingWorker.

While parallel testing has saved us huge amounts of time and effort, it still has hardware limitations (you always need more devices/processing power)

You totally can run a test parallel on browsers and devices but in case responsive issue does not affect your test.

Your tests can run in parallel in locally (local computer) or across remote machines by using Selenium Grid.

Finally, the key part of a good testing infrastructure is test result reporting. Reports are arguably the most valuable tool available to project teams and stakeholders. Reporting plays a very important role in Automation so we can create different type of reports like testng Report, Xslt report, customized reports with listeners but today we will cover unique reporting which allow us to generate log and add screenshot and many other feature like Graph, DashBoard and so on.

Report Portal is a real-time reporting system for automated tests with custom dashboards, widgets, metrics, customizable defect type, history and smart analysis based on Machine Learning.

I rounded up the top eight advantages you can expect from report portal:

– You can easily integrate Reportportal with your test frameworks, as well as collect and browse all project tests in one place.

– Receive analytics in real-time: Tests are reflected in ReportPortal in a real time, so you can begin an investigation job of the failure reasons immediately after the test is completed.

– With Report Portal you have access to executions of all auto tests on your project and find info about any test item easily.

– Aggregate all info about test in one place: test history, logs, attachments.

– Classify test failures and quickly observe statistics around the launch as a whole.

– Reduce time cost and analyze the failure reasons by Auto-Analyzer based on Machine Learning.

– Integrate with Bug tracking systems Jira or Rally.

– Track release readiness: Use widget for creating the report about a product status, effectiveness of automation and a readiness to the Release.

To integrate with Report Portal, I built a function that simulates ITestNGService to implement operations for interaction Report Portal.

And now it’s showtime! Here is my automation testing tool based on Hybrid framework. This tool was written in Java and used Selenium, Appium, Report Portal.

The first and the foremost step is to create excel files of test suites and test cases that you want to run test. In this demo, I created 2 test suites. The first suite executes the searching action on Google page. The other one executes the login action on Facebook with invalid information.

The test cases of 2 test suites:

And step of each test case:

With test data:

After creating excel files, open the automation testing tool and choose the project that you want to run test.

In the main form, the excel data will be loaded into data list accordingly.

Choose browsers or devices for each test suite:

With the Facebook test suite, run 2 chrome and 1 Firefox browsers in parallel. In this test suite, we set run mode “Yes” for only test case Login so the test case “Register” will not be displayed.

With the Google test suite, run with 1 chrome and 1 Firefox browsers in parallel. Because this test suite has 2 test data “Automation Testing” and “Manual Testing” (Data Sheets image) so it has 2 test case with number at the end of name “Google – 1” and “Google – 2”.

Let’s run it!

After clicking on Start button, 5 web browsers will be run simultaneously (3 for Facebook test suite, 2 for Google test suite)

When all of them finish, you should be able to see the results in your ReportPortal

Facebook test suite result:

Google test suite result:

Logs of an error with Excel info, trace and screenshot:

And the metrics dashboard after few tests run:

That’s it! You’ve run an automation test successfully with steps in the form of simple and convenient spreadsheets. Everyone of the team can build sophisticated tests without writing a single line of code. This tool can ultimately help you in bringing down the bid price for testing for the customer.

Finally, thanks for reading, I hope that this article provided some useful information about one of automation testing frameworks and the benefits of automation testing.

Quan Nguyen

back to insights

Share with us your business goals and product ideas, we will support you to develop, scale and succeed together.