AI-based generation of test case definitions from natural language requirements by Martin Heininger & August Weiss

The automatic derivation of test cases from textual requirements is a much-discussed topic in Requirements Engineering. Previous studies on this topic show sobering results and tend to exclude such automation.

In our presentation, a new approach will be introduced, which was developed and investigated as part of a master's thesis. This was supervised by Vector Informatik GmbH. A machine learning model was trained to extract test-relevant attributes from natural language requirement texts. Such attributes can be parameters, their values, and possible initial conditions. The requirements used are real safety-critical requirements exported from an aviation project. These requirements must comply with standards such as DO-178C and are therefore semi-formally specified. This characteristic enabled the creation of a dataset on which the machine learning model was trained. In the further course, the attributes collected from the requirements can be processed along with the remaining texts to achieve the ultimate goal of automated test case generation. Until then, the interim results can be used to support requirements and testing engineers. The presentation will discuss the results and describe the challenges.

Finally, we will show what capabilities well-known AI models, such as ChatGPT, already possess, but also in which areas current models still have weaknesses. We will discuss how AI could simplify the bridge between requirements and testing and what opportunities exist in this exciting field.