UI Automation: Graph based Approach

Abhish Agarwal

Abhish Agarwal on January 14, 2016

A Graphical User Interface (GUI) is a fundamental component in the software we engineer today. It provides an interactive and easy connect to the underlying program making the overall software more usable, e2e workflows more passable and flexible. However, at the quality layer, GUIs pose a number of challenges to software testers. Bringing in the right test coverage, balancing test scenarios between automated and manual test techniques, choosing the right tools are all important questions to address early on in the test cycle, to ensure users are able to reap the true benefits of what a GUI has to offer.

To mitigate these challenges, especially around test case identification and coverage, I want to share a new graph based approach in workflow decomposition. In this approach, each UI component is regarded as a graph node and movement from one GUI component to the other is accounted for to build a connected graph covering the entire workflow. An algorithmic, Breadth First Search (BFS) is used to generate spanning trees and each spanning tree represents one test case in the GUI flow.

Once the optimized set of test cases are identified and scripted, they can be bucketed into the regression and sanity suites, as in any other test effort.

Let’s look at approach in greater detail with an example of a simple login form and see how test case generation and execution would look like -

Test Case Generation

A robust test case generation process covering all required workflows in an optimized manner, is the first important step in this approach. A sample graph to accomplish this is as seen below:

login graph2.jpg

The graph has three UI components and two spurious nodes, start and end. To generate test cases, we need to find the “n” number of ways a user can traverse from “start” to “end” node. This will result in a number of spanning trees for the above graph. To generate these spanning trees either a BFS or Depth First Search (DFS) can be used, but as mentioned above we will use BFS.

The resultant test cases could look like the ones below:

Test Case: 1

Priority: P0

Iteration: 2

Flow: Username->Password->Submit

Data: {Username => XYZ, Password => 123}

Verification: {“Next page”=>”Page Title”}

Test Case: 2

Priority: P0

Iteration: 1

Flow: Password->Submit

Data: {Password => 123}

Verification: {“Error Message”=>”Some Message”}

Test Case: 3

Priority: P0

Iteration: 1

Flow: Username->Submit

Data: {Username => XYZ}

Verification: {“Error Message” =>”Some Message”}

Terminologies:

Priority: As in any test effort, this definition of priority will help bucket test cases into varied test suites to determine the order, frequency and need for test execution. By default, the value is P0 which is the highest priority.

Iteration: By default, iteration is 1. But for a given flow, user can give multiple data values which will lead to multiple iterations for the same case. For example, Test case 1: we can test the flow for various values of username and password. Each iteration is one test case – one data value combination, leading to one instance of the test case.

Flow: This defines the path to be followed during execution.

Data: This refers to the input values to be used for execution.

Verification: This is the core step of ensuring outcomes map with expected results – for example, is the redirection happening correctly, are the right error messages thrown at the right times, do the right pop ups show up etc.

The three test cases listed above, show how automated test cases can be generated from the input UI graph. Manual intervention is still required to determine the right data sets to be used. Data sets often determine how solid the entire test effort is. . These inputs of tests and data sets together will feed into the test execution process.

Test Execution

Along with the node details, graph definition also has methods for each UI node. During execution, the parser will call these methods and process the corresponding UI components.

For example in test case 3, the parser will move from the start node to username - it will call the “FillText()” method along with the data values defined in the input file for username. It will then move from the username to submit where the parser will call the “PressButton()” method.

Once we reach the end node all the required verifications are complete.

Defining UI methods in the graph facilitates any editing that may be needed in future, making the graph maintenance manageable. Since the number of UI components are limited, the number of methods processing these UI components will also be limited. So when UI changes are made and more nodes are added into the graph, methods can be re-used without touching their code base.

Conclusion

Test automation is an expensive effort. As much as there is value in taking it up especially in the Agile cycles we operate in, a lot of diligent upfront planning is important in identifying the right test cases to be automated. The above graph node model is one such step in identifying the right test cases from a complex set of workflows which will make the test automation effort more relevant in bringing in repeatable and consistent quality in the product under test.

Specifically, how does the above model address the challenges we discussed earlier on?

  • Anytime UI changes occur, we need not change our parser code. Only graph changes are required to incorporate the updates. For example: If “Forgot password” is a new UI component, we just need to add one more node and update its XPATH location in the graph definition.
  • Code changes will be required only when we need to write newer methods to handle new UI components which were not used so far. But since UI components are limited, eventually a lot of new code may not be needed.
  • This approach enables us to automate test cases quickly and comprehensively. Instead of figuring out the varied possible permutations, the tester is able to spend time on defining data sets and prioritizing the identified workflows. All of this together enhances the effectiveness of the UI test effort.