Our latest solutions supporting testing and quality engineering of high-end mission critical and business critical applications

Event-based Software Testing

  • Automated Testing of Platform Games

    Game development industry has recently emerged as one of major software development industry which is due to the availability of increasingly powerful processing devices, including hand-held, and mobile devices. The number of games produced in the last few years has increased exponentially. The common practice followed by the game testers is manual game testing in which the testers play a large number of potential scenarios to test the functionality of the games. The scenarios are selected at random or keeping in mind some particular goals depending on the testing target, for example, killing an enemy, completing a level. Manual testing of such games is a labor-intensive and time-consuming task. The test scenarios have to be manually executed on every game change, which becomes a monotonous and tedious task for frequently changing games.

    Semi-automated techniques also exist for game testing. The test cases still need to be recorded and evaluated manually. The tester has to design test cases, select test sequences, generate test data and generate the test oracle manually. This becomes a major challenge as the games are frequently modified due to serious market competition. All test cases have to be re-recorded and evaluated manually, which is neither efficient nor scalable. Consequently, there is a need for an approach for testing games that allows automated test case generation, execution, and evaluation.

    The project we are working on is an automated functional testing approach for platform games. The model-based approach allows for automated test case generation and automated test execution of platform games. The models are developed in UML using a proposed model-based testing profile for platform games. The models include a domain model and state machines representing the game play behavior of an avatar. The state machine is used to generate test cases using the N+ testing strategy. The generated test cases result in short execution sequences, which can then be combined to generate interesting game testing scenarios.

    Contact Person: Muhammad Zohaib Iqbal

  • 2- Product-line Model-driven Engineering, Generation and Testing of Mobile Application Variants

    Mobile application development has emerged as one of the most focused areas in software industry due to exponential growth of mobile users and applications. The current industrial practices of mobile software engineering have not kept pace with the rapid development and are inadequate to address the needs of mobile application development industry. Model-driven engineering practices are ignored, which results in low reusability and lack of portability along with other challenges. Mobile applications have an inherent complexity, the variability of the mobile platforms, their versions and hardware devices. New platforms and devices are being introduced rapidly, thus aggravating the problem.

    Mobile applications have to support multiple platforms, as an application written for one platform (e.g., Android) cannot run on another platform (e.g., Windows Phone). Each of the platforms in turn suffers from fragmentation. This results in multiple versions of an application that need to be simultaneously tested and maintained. Moreover, the limitation of the mobile device components (i.e., memory, power, processing speed, graphical resolutions, and screen size), and its context (or environment) plays a challenging role for the mobile application testing. This is a huge burden on the development and testing team, both in terms of cost and effort. Software product-line engineering addresses variability management and has been successfully applied to develop families of related software products. Model-driven software engineering has been successfully applied in other domains to address the issues related to portability and reusability.

    We use product-line engineering in collaboration with model-driven engineering to generate feature-based mobile application variants for multiple platforms, and developed a tool named MOPPET. Specifically, we deal with three types of variations in mobile applications: variation due to operation systems and their versions, variations due to software and hardware capabilities of mobile devices, and variations based on the functionalities offered by the mobile application. We define a modeling methodology (using UML) for mobile application modeling and feature model for mobile application variability.

    To support the performance testing of the generated feature-based mobile application variants for multiple platforms, we developed a PELLET tool that tests of the performance measuring parameters for mobile applications, i.e., the consumption of time, memory, and battery. We define a performance modeling profile for specifying the mobile domain specific performance measuring concepts and automated the generation of mobile platform-specific test cases from the UML diagrams.

    Contact Person: Muhammad Usman

Search-based Software Testing

  • Test Data Generation by Solving OCL Constraints

    Object Constraint Language (OCL) is an international modeling standard for writing constraints on Unified Modeling Language (UML) diagrams. These constraints can be written at various levels, e.g., they can represent class and state invariants, guards in state machines, constraints in sequence diagrams, and pre and post conditions of operations.

    Depending on the goals, we might need to solve the OCL constraints on the models for various purposes. For example, if the models are developed for automated model-based testing, solving the OCL constraints on the test models is essential for test data generation. Similarly, we need to solve the constraints on meta-models, if we intend to automatically create models based on these models. Another use can, for example, be to identify the inconsistencies between constraints.

    For industrial systems, typically solving the constraints written in a language as expressive as OCL is a very complex task. Most of the tools available apply random strategy for this purpose, which do not scale to the requirements of solving complex constraints. Therefore, to achieve the goals mentioned above, it is a key requirement to have an automated, scalable, and robust constraint solver that is useful in various contexts.

    As part of the work to be carried out under this research project, we have developed such an automated OCL constraint solver. This tool is developed in Java using Eclipse IDE. It uses an API named as Eclipse OCL which provides facility to parse and evaluate OCL expressions. It takes UML class diagram and constraints written in OCL (Object Constraint Language) as input. It loads UML model and parses OCL constraint to extract relevant information. After that it uses search techniques to solve those constraints. For this purpose various search algorithms such as AVM, SSGA, (1+1) EA and RS are available. Finally when search algorithms successfully find the solution that satisfies given OCL constraints, first this solution is evaluated using Eclipse OCL evaluator and then UML object diagram is generated that contains test data.

    Moreover to perform thorough testing of critical systems, this tool also provides mechanism to generate multiple solutions according to various coverage criteria. Coverage criteria available for this purpose are: Clause Coverage, Partition Coverage, Branch Coverage and predicate (MC/DC) coverage.

    Contact Person: Zohaib Iqbal

  • Model Transformation Testing Environment(MOTTER)

    Model transformations (MT) are a fundamental part Model Driven Engineering (MDE). Model transformation involves transforming model from one representation to other. MT takes an input a source model, that conforms to a source meta-model and generate a target model that conforms to a target meta-model. Like any other software program, correction of model transformation program is of significant importance. Existing traditional software testing techniques cannot be adopted for testing transformations because of a number of specific challenges. The foremost is the complexity involves in the generation of test which in the case of model transformation is the instance of input meta-models.

    Typically, a meta-model comprises of a large set of elements. These elements have attributes along with relationships which are further restricted by constraints defined in OCL. Automated generation of instances from a meta-model is itself a difficult process, which is further complicated when the constraint specified over the meta-model are solved. The complexity of instance generation process is depending upon the language elements used in the meta-model and the solving of constraints so that model instances become valid. Manual generation of valid meta-model instances manually is not possible. However, automated generation of instances require solving of OCL constraints. The generation of meta-element instances are to achieve the coverage of either the meta-model or the structural coverage of the transformation under test.

    The Model Transformation Testing Environment (Motter) is a tool-set that implements searchbased approach to provide automated structural testing of model transformations and metamodel. Motter generate set of test cases (instances of source meta-model as test model) that not only covers various execution paths of the transformation under test but also generates valid meta-model instances. Therefore, the use of Motter tool is two-folded, one in the generation of valid meta-model instances to maximize the meta-model coverage and other in the generation of test models, that automated the process of test generation by generating set of test that successfully achieves various structural coverage criteria including, statement coverage, branch coverage etc.

    The overall architecture of the Motter Tool-set is shown in Figure 1. The tool implemented search-based strategies for maximize structural coverage of transformation and meta-models. To guide the search fitness function have been developed which is based on approach level and branch distance. Additionally, Motter tool also solve the constraint specified at meta-model.

    Current version of the Motter tool support structural coverage of the transformation written in Atlas Transformation Language and MofScript.

    Contact Person: Atif Jilani

  • Automated Test Data Generation for Data-Intensive Systems

    Data-intensive systems such as battlefield management systems, air control traffic system, and transaction processing systems make use of different forms of data and their essential operations are highly dependent on the nature of data present in such systems. As data is a crucial element for such systems, it is important to ensure the correctness and generalizability of data.

    Experts from the industry have proposed and used different rules for ensuring the correctness and integrity of data. These rules have been applied in the form of various constraints, such as domain constraints on relational schemas, entity integrity constraints, referential integrity constraints, and key constraints. These constraints can be specified using different languages such as OCL. These can also be specified in the form of stored procedures or SQL queries for different data-intensive systems that belong to a similar domain.

    We are developing an automated tool to test the constraints applied to data-intensive systems. The tool generates test data by using search based techniques such as genetic algorithm and alternate variable methods.

    Contact Person: Maheen Arshad

Avionics and Ground Control System

  • Cockpit Display System

    Software systems play a vital role in mission and safety critical domains. The avionics sector is one of the leading industries where cockpit interface technology is continuously coming up with advanced solutions, seeking to flight operational efficiency. The advanced cockpits are composed of several integrated displays that are used to show information from multiple sources. These integrated displays are known as "Multi-function Displays"(MFD) that are used in Cockpit Display Systems (CDS). The MFDs comprise of small LCD screens that are used to display a large amount of flight management and guidance system to the aircraft crew.

    Since aircraft are used worldwide for different purposes such as traveling, cargos, and air defense, hence safety of these systems cannot be compromised. Therefore, the correct behavior of the CDS is quiet crucial for the aircraft crew to perform various safety critical flight operations. Safety critical standards demand that the behavior of the components of the CDS must be reliable and accurate enough in all scenarios. Thus in order to increase the dependability of CDS, thorough and effective testing is required.

    We are developing an automated tool to test the A661 compliant CDS of various aircraft. The focus is on functional testing of the user applications and that the information from the user applications is being displayed correctly on the CDS. The information displayed on different CDS widgets during the aircraft flight is recorded using a camera. The tool uses image processing and artificial intelligence to identify the faults in the display of various CDS widgets.

    Contact Person: Hassan Sartaj

  • Ground Control System

    The use of Unmanned Aerial Vehicles (UAV)s in long endurance and long-range missions depend heavily on the existence of a Ground Control Station (GCS) that must meets quality standards. The UAV flights might be performed beyond the line-of-sight, making aircraft control, video feedback, and real-time telemetry vital for their success. The objective of this project is to design and build a ground control station that can be deployed in the field as part of the UAV.

    A ground station is a software application running on a ground-based computer, which communicates with UAV via wireless telemetry. It displays real-time data on the UAVs performance and position, serving as a “virtual cockpit. The GCS can also be used to control a UAV in flight, uploading new mission commands and setting parameters. It is also used to monitor the live video streams from a UAV’s cameras.

    We have designed a GCS compatible with any aircraft. Once developed, it will be capable of controlling multiple UAVs remotely from a single point simultaneously. The GCS consists of multiple screens which perform multiple tasks. The GCS allows to constrict a mission on a live map at anywhere around the world. There is a live simulation of the aircraft on the map which represents its actual location. In case of emergency there is failsafe control which allows the safe return of the aircraft. These facilities of flight situation awareness include controlling and monitoring of multiple UAVs, live video streaming and tracking of telemetry data. It will also be able to facilitate monitoring and controlling of payload in real-time.

    Contact Person: Farhan Aslam

Web Applications Testing

  • Automated Model Based Test Case Repair Tool for Evolving Web Applications

    Automated testing of web applications is classically performed using capture-replay tools. Such tools allow the testers to record the various test scenarios, re-executing them later with same or different data to test a web application. These scenarios also contain the expected results which are usually embedded as assertions. These tools record the test scenarios as a sequence of user actions (such as mouse clicks, keyboard entries, and navigation commands) performed on the Graphical User Interface (GUI) of the web application. Most of the capture-replay tools generate a test script corresponding to the recorded scenarios. For such kind of testing, a significant amount of effort and cost is invested in developing the various test scripts.

    Web applications are evolving rapidly with the evolving requirements and changing technology. The test scripts written for automated capture-replay tools are strictly tied to the interfaces making them sensitive to any change in the interface of the web application. Therefore, even simple changes, such as slight modifications in a web page layout may break the previously recorded test scripts. For interfaces, such simple changes can require around 75% of the test suites to be fixed.

    In order to address this problem, we are working on a model-based approach, based on a UML profile, to automatically repair the test breakages of capture-replay tools. This model-based approach provides a generic test script repair mechanism that is independent of the underlying Capture-Replay tools. Existing approaches (Selenium, TestComplete) only tend to repair a specific type of broken test scripts whereas our automated solution repairs almost every kind of automated test script generated through any capture-replay tool.

    Contact Person: Javeria Imtiaz

  • MuWeb: A Mutation Testing Tool

    As web applications are incessantly evolving along with time testing such applications pose new challenges to testing community. Faulty web applications can not only cause user inconvenience, it may also result in an organization’s business loss.

    Mutation testing is considered to be an effective technique for improving the quality of test cases for a web application. There are many tools that have been produced for mutation testing of web application. All of them consider some specific type of mutation in a web application to evaluate the quality of test cases that can kill these mutants.

    We are developing a tool “MuWeb” with different mutation operators that will create mutants for web applications. Mutation testing helps in evaluating the quality of test suite and it has been applied successfully in a number of applications such as automated test generation, program repair, and real fault simulations etc. Our tool will help testers in producing an effective test suite to detect faults in non-trivial web applications.

    Contact Person: Abeera Naveed