Publications

2025

1. FSE

   Automated Extraction and Analysis of Developer’s Rationale in Open Source Software

   Mouna Dhaouadi, Bentley Oakes, and Michalis Famelis

   Proceedings of the ACM on Software Engineering, 2025

   Abstract DOI Bib PDF

   Contributors to open source software must deeply understand a project’s history to make coherent decisions which do not conflict with past reasoning. However, inspecting all related changes to a proposed contribution requires intensive manual effort, and previous research has not yet produced an automated mechanism to expose and analyze these conflicts. In this article, we propose such an automated approach for rationale analyses, based on an instantiation of Kantara, an existing high-level rationale extraction and management architecture. Our implementation leverages pre-trained models and Large Language Models, and includes structure-based mechanisms to detect reasoning conflicts and problems which could cause design erosion in a project over time. We show the feasibility of our extraction and analysis approach using the OOM-Killer module of the Linux Kernel project, and investigate the approach’s generalization to five other highly active open source projects. The results confirm that our automated approach can support rationale analyses with reasonable performance, by finding interesting relationships and to detect potential conflicts and reasoning problems. We also show the effectiveness of the automated extraction of decision and rationale sentences and the prospects for generalizing this to other open source projects. This automated approach could therefore be used by open source software developers to proactively address hidden issues and to ensure that new changes do not conflict with past decisions.

   @article{Dhaouadi2025FSE,
     author = {Dhaouadi, Mouna and Oakes, Bentley and Famelis, Michalis},
     doi = {10.1145/3729383},
     journal = {Proceedings of the ACM on Software Engineering},
     number = {FSE},
     pages = {2548--2570},
     title = {Automated Extraction and Analysis of {Developer's} Rationale in Open Source Software},
     volume = {2},
     year = {2025}
   }

2. MSR

   CoMRAT: Commit Message Rationale Analysis Tool

   Mouna Dhaouadi, Bentley Oakes, and Michalis Famelis

   In Proceedings of the 22nd International Conference on Mining Software Repositories, 2025

   Abstract DOI Bib PDF

   In collaborative open-source development, the rationale for code changes is often captured in commit messages, making them a rich source of valuable information. However, research on rationale in commit messages remains limited. In this paper, we present CoMRAT, a tool for analyzing decision and rationale sentences rationale in commit messages. CoMRAT enables a) researchers to produce metrics and analyses on rationale information in any Github module, and b) developers to check the amount of rationale in their commit messages. A preliminary evaluation suggests the tool’s usefulness and usability in both these research and development contexts.

   @inproceedings{Dhaouadi2025MSR,
     author = {Dhaouadi, Mouna and Oakes, Bentley and Famelis, Michalis},
     booktitle = {Proceedings of the 22nd International Conference on Mining Software Repositories},
     doi = {10.1109/MSR66628.2025.00120},
     pages = {831--835},
     title = {{CoMRAT}: Commit Message Rationale Analysis Tool},
     year = {2025}
   }

3. SAM

   Model-Based Systems Engineering Perspectives: A Survey of Practitioner Experiences and Challenges

   Maged Elaasar, Abdelwahab Hamou-Lhadj, Bentley Oakes, and Mohammad Hamdaqa

   In 2025 ACM/IEEE 28th International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C), 2025

   Abstract DOI Bib PDF

   Model-based systems engineering (MBSE) is an established field, aiming to bring traceability and complexity management to the systems engineering process. However, multiple conceptual, technical, and organizational challenges continue to impede the effective deployment of MBSE in practice. This paper reports the results of a survey completed by 76 MBSE researchers and/or practitioners, on their organization’s use of MBSE. Our analysis indicates that many organizations have yet to fully leverage MBSE. Several have not completely transitioned to MBSE in their systems engineering processes, or do not adhere to any specific method, indicating a lack of a comprehensive and organization-wide MBSE approach. We find that challenges such as change management, cross-team collaboration, and tool customization persist. We report on these challenges and provide recommendations as potential solutions.

   @inproceedings{Elaasar2025,
     author = {Elaasar, Maged and Hamou-Lhadj, Abdelwahab and Oakes, Bentley and Hamdaqa, Mohammad},
     booktitle = {2025 ACM/IEEE 28th International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)},
     doi = {10.1109/MODELS-C68889.2025.00055},
     pages = {367--376},
     title = {Model-Based Systems Engineering Perspectives: A Survey of Practitioner Experiences and Challenges},
     year = {2025}
   }

4. EDTconf

   DTInsight: A Tool for Explicit, Interactive, and Continuous Digital Twin Reporting

   Kérian Fiter, Louis Malassigné-Onfroy, and Bentley Oakes

   In 2025 ACM/IEEE 28th International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C), 2025

   Abstract DOI Bib PDF Poster

   With Digital Twin (DT) construction and evolution occurring over time, stakeholders require tools to understand the current characteristics and conceptual architecture of the system at any time. DTInsight is a systematic and automated tool and methodology for producing continuous reporting for DTs. DTInsight offers three key features: (a) an interactive conceptual architecture visualization of DTs; (b) generation of summaries of DT characteristics based on ontological data; and (c) integration of these outputs into a reporting page within a continuous integration and continuous deployment (CI/CD) pipeline. The tool enables stakeholders to generate detailed, up-to-date reports when provided with a modeled description of the Digital Twin that conforms to the proposed DT Description Framework.

   @inproceedings{Fiter2025,
     author = {Fiter, K{\'e}rian and Malassign{\'e}-Onfroy, Louis and Oakes, Bentley},
     booktitle = {2025 ACM/IEEE 28th International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)},
     doi = {10.1109/MODELS-C68889.2025.00030},
     pages = {139--143},
     title = {{DTInsight}: A Tool for Explicit, Interactive, and Continuous Digital Twin Reporting},
     year = {2025}
   }

5. EDTconf

   Engineering a Digital Twin for the Monitoring and Control of Beer Fermentation Sampling

   Pierre-Emmanuel Goffi, Raphaël Tremblay, and Bentley Oakes

   In 2025 ACM/IEEE 28th International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C), 2025

   Abstract DOI arXiv Bib PDF

   We present an experience report describing the implementation of a safety-critical digital twin for beer fermentation monitoring. The system reduces manual sampling time by 91% through continual automated sampling. Key contributions include a three-phase engineering methodology, multi-layered safety protocols, hardware-software integration across Arduino and Unity platforms, and real-time synchronization approaches. The work demonstrates how the constellation reporting framework enables interdisciplinary collaboration and emphasizes the importance of safety-first design and simulation-driven development for bidirectional control in pressurized systems.

   @inproceedings{Goffi2025,
     author = {Goffi, Pierre-Emmanuel and Tremblay, Rapha{\"e}l and Oakes, Bentley},
     booktitle = {2025 ACM/IEEE 28th International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)},
     doi = {10.1109/MODELS-C68889.2025.00033},
     pages = {167--173},
     title = {Engineering a Digital Twin for the Monitoring and Control of Beer Fermentation Sampling},
     year = {2025}
   }

6. MODELS-C

   DTChecker: A Real-Time Signal Monitoring and Property Specification Tool for Digital Twins

   Abdelhamid Rouatbi, Eugene Syriani, and Bentley Oakes

   In 2025 ACM/IEEE 28th International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C), 2025

   Abstract DOI Bib PDF

   Specifying and monitoring temporal requirements in Digital Twin (DT) systems is challenging, as writing formal specifications in temporal logic is often complex and inaccessible to domain experts. This typically necessitates close collaboration with software engineers, introducing communication overhead and slowing development. We present DTChecker, a reusable selfcontained monitoring tool for DT systems built on RabbitMQbased architectures. The tool enables domain experts to write temporal specifications in a browser-based editor with language server support. These specifications are automatically translated into Signal Temporal Logic (STL) formulas and evaluated in real-time on data streams from sensors or services. Robustness scores are streamed to a front-end dashboard to visualize how well the system satisfies the specified requirements over time. This enables domain experts to write and verify temporal properties easily, thereby improving the real-time monitoring of the DT. We demonstrate the tool through integration with an open-source incubator DT case study. Video demonstration: https://youtu.be/elyhSOiGuc4?si=V2TTsXgRcYaAU2X

   @inproceedings{Rouatbi2025,
     author = {Rouatbi, Abdelhamid and Syriani, Eugene and Oakes, Bentley},
     booktitle = {2025 ACM/IEEE 28th International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)},
     doi = {10.1109/MODELS-C68889.2025.00020},
     pages = {84--88},
     title = {{DTChecker}: A Real-Time Signal Monitoring and Property Specification Tool for Digital Twins},
     year = {2025}
   }

2024

1. MODELS

   Toward Intelligent Generation of Tailored Graphical Concrete Syntax

   Meriem Ben Chaaben, Oussama Ben Sghaier, Mouna Dhaouadi, Nafisa Elrasheed, Ikram Darif, Imen Jaoua, Bentley Oakes, Eugene Syriani, and Mohammad Hamdaqa

   In Proceedings of the ACM/IEEE 27th International Conference on Model Driven Engineering Languages and Systems, 2024

   Abstract DOI Bib PDF

   In model-driven engineering, the concrete syntax of a domain-specific modeling language (DSML) is fundamental as it constitutes the primary point of interaction between the user and the DSML. Nevertheless, the conventional one-size-fits-all approach to concrete syntax often undermines the effectiveness of DSMLs, as it fails to accommodate the diverse constraints and specific requirements inherent to diverse users and usage contexts. Such shortcomings can lead to a significant decline in the performance, usability, and efficiency of DSMLs. This vision paper proposes a conceptual framework to generate concrete syntax intelligently. Our framework considers multiple concerns of users and aims to align the concrete syntax with the context of the DSML usage. Additionally, we detail a baseline process to employ our framework in practice, leveraging large language models to expedite the generation of tailored concrete syntax. We illustrate the potential of our vision with two concrete examples and discuss the shortcomings and research challenges of current intelligent generation techniques.

   @inproceedings{ben2024toward,
     author = {Ben Chaaben, Meriem and Ben Sghaier, Oussama and Dhaouadi, Mouna and Elrasheed, Nafisa and Darif, Ikram and Jaoua, Imen and Oakes, Bentley and Syriani, Eugene and Hamdaqa, Mohammad},
     booktitle = {Proceedings of the ACM/IEEE 27th International Conference on Model Driven Engineering Languages and Systems},
     doi = {10.1145/3640310.3674085},
     pages = {160--171},
     title = {Toward Intelligent Generation of Tailored Graphical Concrete Syntax},
     year = {2024}
   }

2. ICPC

   Rationale Dataset and Analysis for the Commit Messages of the Linux Kernel Out-of-Memory Killer

   Mouna Dhaouadi, Bentley Oakes, and Michalis Famelis

   In Proceedings of the 32nd IEEE/ACM International Conference on Program Comprehension, 2024

   Abstract DOI Bib PDF

   Code commit messages can contain useful information on why a developer has made a change. However, the presence and structure of rationale in real-world code commit messages is not well studied. Here, we detail the creation of a labelled dataset to analyze the code commit messages of the Linux Kernel Out-Of-Memory Killer component.

   @inproceedings{Dhaouadi2024,
     address = {New York, NY, USA},
     author = {Dhaouadi, Mouna and Oakes, Bentley and Famelis, Michalis},
     booktitle = {Proceedings of the 32nd IEEE/ACM International Conference on Program Comprehension},
     doi = {10.1145/3643916.3644413},
     isbn = {9798400705861},
     pages = {415--425},
     publisher = {Association for Computing Machinery},
     series = {ICPC '24},
     title = {Rationale Dataset and Analysis for the Commit Messages of the {Linux} Kernel Out-of-Memory Killer},
     year = {2024}
   }

3. SIMULATION

   Towards a Systematic Reporting Framework for Digital Twins: A Cooperative Robotics Case Study

   Santiago Gil, Bentley Oakes, Claudio Gomes, Mirgita Frasheri, and Peter G. Larsen

   SIMULATION, 2024

   Abstract DOI Bib PDF Video

   Digital Twins (DTs) can be constructed for many different applications, leading to substantial differences between different case studies. To be able to learn from the challenges and lessons learned by other DT practitioners, it is important that experience reports be consistent to facilitate comparisons. In this paper, we merge three reference description frameworks for DTs, one generated from a systematic mapping study, one generated from an analysis of experience reports, and one from a systematic literature review, to come up with a unified characterization of DT applications. This analysis has identified six non-overlapping and three cross-cutting characteristics in the reference frameworks. This paper showcases the unified characterization with 21 characteristics to report on a DT case study called the Flex-cell, a manufacturing cell with two robotic arms used for cooperative assembly. The generalizability of this unified characterization is validated using a multi-case approach with another case study in robotics and another in the food industry. We call on the DT community to integrate these systematic reporting principles in their future DT experience reports such that other practitioners can learn from each other more effectively.

   @article{Gil2024towardssystematicreporting,
     author = {Gil, Santiago and Oakes, Bentley and Gomes, Claudio and Frasheri, Mirgita and Larsen, Peter G.},
     doi = {10.1177/00375497241261406},
     journal = {SIMULATION},
     number = {3},
     pages = {313--339},
     publisher = {SAGE Publications},
     title = {Towards a Systematic Reporting Framework for Digital Twins: A Cooperative Robotics Case Study},
     volume = {101},
     year = {2024}
   }

4. Book

   Foundational Concepts for Digital Twins for Cyber-Physical Systems

   Cláudio Gomes, John Fitzgerald, Bentley Oakes, Ken Pierce, Peter Høgh Mikkelsen, Santiago Gil Arboleda, Till Böttjer, and Michael Sandberg

   In The Engineering of Digital Twins, 2024

   DOI Bib

   @incollection{Gomes2023,
     author = {Gomes, Cl{\'a}udio and Fitzgerald, John and Oakes, Bentley and Pierce, Ken and Mikkelsen, Peter H{\o}gh and Gil Arboleda, Santiago and B{\"o}ttjer, Till and Sandberg, Michael},
     booktitle = {The Engineering of Digital Twins},
     doi = {10.1007/978-3-031-66719-0},
     editor = {Fitzgerald, John and Gomes, Cl{\'a}udio and Larsen, Peter Gorm},
     publisher = {Springer},
     title = {Foundational Concepts for Digital Twins for Cyber-Physical Systems},
     year = {2024}
   }

5. TOSEM

   Building Domain-Specific Machine Learning Workflows: A Conceptual Framework for the State of the Practice

   Bentley Oakes, Michalis Famelis, and Houari Sahraoui

   ACM Transactions on Software Engineering and Methodology, Apr 2024

   Abstract DOI Bib PDF

   Domain experts are increasingly employing machine learning to solve their domain-specific problems. This article presents to software engineering researchers the six key challenges that a domain expert faces in addressing their problem with a computational workflow, and the underlying executable implementation. These challenges arise out of our conceptual framework which presents the "route" of transformations that a domain expert may choose to take while developing their solution. To ground our conceptual framework in the state of the practice, this article discusses a selection of available textual and graphical workflow systems and their support for the transformations described in our framework. Example studies from the literature in various domains are also examined to highlight the tools used by the domain experts as well as a classification of the domain specificity and machine learning usage of their problem, workflow, and implementation. The state of the practice informs our discussion of the six key challenges, where we identify which challenges and transformations are not sufficiently addressed by available tools. We also suggest possible research directions for software engineering researchers to increase the automation of these tools and disseminate best-practice techniques between software engineering and various scientific domains.

   @article{Oakes2024BuildingDomainSpecific,
     articleno = {91},
     author = {Oakes, Bentley and Famelis, Michalis and Sahraoui, Houari},
     doi = {10.1145/3638243},
     issn = {1049-331X},
     journal = {ACM Transactions on Software Engineering and Methodology},
     month = apr,
     number = {4},
     numpages = {50},
     publisher = {Association for Computing Machinery},
     title = {Building Domain-Specific Machine Learning Workflows: A Conceptual Framework for the State of the Practice},
     volume = {33},
     year = {2024}
   }

6. EDTconf

   Towards Ontological Service-Driven Engineering of Digital Twins

   Bentley Oakes, Claudio Gomes, Eduard Kamburjan, Giuseppe Abbiati, Elif Ecem Bas, and Sebastian Engelsgaard

   In Proceedings of the ACM/IEEE 27th International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C), 2024

   Best Paper Abstract DOI Bib PDF Video Slides

   EDTconf 2024 Best Short Paper

   The systematic engineering of Digital Twins (DTs) requires the establishment of clear methodologies supported by intelligent tooling. We propose an approach to guide the user in the creation and deployment of services for DTs utilizing ontologies and workflows. In our approach, the user selects a desired DT service from an array of options. This selection is then used to suggest a) enablers and models to place in the DT, and b) development and deployment workflows for the DT service. The aim is to provide DT engineering guidance to assist non-software engineering experts to develop DT services more rapidly with less effort. We describe our initial work on applying this approach to a derived version of an industrial wind turbine generator case study, utilizing openCAESAR for ontology definition and enacting the workflows with Jupyter notebooks.

   @inproceedings{Oakes2024ServiceDriven,
     author = {Oakes, Bentley and Gomes, Claudio and Kamburjan, Eduard and Abbiati, Giuseppe and Bas, Elif Ecem and Engelsgaard, Sebastian},
     booktitle = {Proceedings of the ACM/IEEE 27th International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)},
     doi = {10.1145/3652620.3688261},
     pages = {464--469},
     title = {Towards Ontological Service-Driven Engineering of Digital Twins},
     year = {2024}
   }

7. SoSyM

   Improving Repair of Semantic ATL Errors Using a Social Diversity Metric

   Zahra VaraminyBahnemiry, Jessie Galasso, Bentley Oakes, and Houari Sahraoui

   Software and Systems Modeling, Apr 2024

   Abstract DOI Bib PDF

   Model transformations play an essential role in the model-driven engineering paradigm. However, writing a correct transformation requires the user to understand both what the transformation should do and how to enact that change in the transformation. This easily leads to syntactic and semantic errors in transformations which are time-consuming to locate and fix. In this article, we extend our evolutionary algorithm (EA) approach to automatically repair transformations containing multiple semantic errors. To prevent the fitness plateaus and the single fitness peak limitations from our previous work, we include the notion of social diversity as an objective for our EA to promote repair patches tackling errors that are less covered by the other patches of the population.

   @article{VaraminyBahnemiry2024,
     author = {VaraminyBahnemiry, Zahra and Galasso, Jessie and Oakes, Bentley and Sahraoui, Houari},
     doi = {10.1007/s10270-024-01170-4},
     issn = {1619-1374},
     journal = {Software and Systems Modeling},
     month = apr,
     number = {6},
     pages = {1547--1568},
     title = {Improving Repair of Semantic {ATL} Errors Using a Social Diversity Metric},
     volume = {23},
     year = {2024}
   }

2023

1. MODELS-C

   Towards Understanding and Analyzing Rationale in Commit Messages using a Knowledge Graph Approach

   Mouna Dhaouadi, Bentley Oakes, and Michalis Famelis

   In 2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C), 2023

   Abstract DOI Bib PDF

   Extracting rationale information from commit messages allows developers to better understand a system and its past development. Here we present our ongoing work on the Kantara end-to-end rationale reconstruction pipeline to a) structure rationale information in an ontologically-based knowledge graph, b) extract and classify this information from commits, and c) produce analysis reports and visualizations for developers. We also present our work on creating a labelled dataset for our running example of the Out-of-Memory component of the Linux kernel. This dataset is used as ground truth for our evaluation of NLP classification techniques which show promising results, especially the multi-classification technique XGBoost.

   @inproceedings{Dhaouadi2023towards,
     author = {Dhaouadi, Mouna and Oakes, Bentley and Famelis, Michalis},
     booktitle = {2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)},
     doi = {10.1109/MODELS-C59198.2023.00101},
     title = {Towards Understanding and Analyzing Rationale in Commit Messages using a Knowledge Graph Approach},
     year = {2023}
   }

2. SAM

   openCAESAR: Balancing Agility and Rigor in Model-Based Systems Engineering

   Maged Elaasar, Nicolas Rouquette, David Wagner, Bentley Oakes, Abdelwahab Hamou-Lhadj, and Mohammad Hamdaqa

   In 2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C), 2023

   Abstract DOI Bib PDF

   Model-Based System Engineering (MBSE) employs models and formal languages to support development of complex (systems-of-) systems. NASA Jet Propulsion Laboratory (JPL) sees MBSE as a key approach to managing the complexity of system development. However, balancing agility and rigor in MBSE has been reported as a challenging task not yet addressed by modeling tools and frameworks. This is because existing MBSE approaches may enable agility but compromise rigor, or enhance rigor but impede agility. We discuss the challenges of balancing agility and rigor in MBSE across seven systems engineering architectural functions defined by the JPL Integrated Model-Centric Engineering (IMCE) initiative. We demonstrate how openCAESAR, an open-source MBSE methodology and framework created at JPL, can strike a balance between agility and rigor through a case study of the Kepler16b project and discussion of lessons learned from past projects.

   @inproceedings{Elaasar2023opencaesar,
     author = {Elaasar, Maged and Rouquette, Nicolas and Wagner, David and Oakes, Bentley and Hamou-Lhadj, Abdelwahab and Hamdaqa, Mohammad},
     booktitle = {2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)},
     doi = {10.1109/MODELS-C59198.2023.00051},
     title = {{openCAESAR}: Balancing Agility and Rigor in Model-Based Systems Engineering},
     year = {2023}
   }

3. MODELSWARD

   A Digital Twin Description Framework and its Mapping to Asset Administration Shell

   Bentley Oakes, Ali Parsai, Bart Meyers, Istvan David, Simon Van Mierlo, Serge Demeyer, Joachim Denil, Paul De Meulenaere, and Hans Vangheluwe

   In Model-Driven Engineering and Software Development, Communications in Computer and Information Science, Aug 2023

   Abstract DOI arXiv Bib PDF

   The pace of reporting on Digital Twin (DT) projects continues to accelerate both in industry and academia. However, these experience reports often leave out essential characteristics of the DT, such as the scope of the system-under-study, the insights and actions enabled, and the time-scale of processing. A lack of these details could therefore hamper both understanding of these DTs and development of DT tools and techniques. Our previous work developed a DT description framework with fourteen characteristics as a checklist for experience report authors to better describe the capabilities of their DT projects. This report provides an extended example of reporting to highlight the utility of this description framework, focusing on the DT of an industrial drilling machine. Furthermore, we provide a mapping from our description framework to the Asset Administration Shell (AAS) which is an emerging standard for Industry 4.0 system integration. This mapping aids practitioners in understanding how our description framework relates to AAS, potentially aiding in description or implementation activities.

   @incollection{Oakes2023DigitalTwinDescription,
     author = {Oakes, Bentley and Parsai, Ali and Meyers, Bart and David, Istvan and Van Mierlo, Simon and Demeyer, Serge and Denil, Joachim and De Meulenaere, Paul and Vangheluwe, Hans},
     booktitle = {Model-Driven Engineering and Software Development, Communications in Computer and Information Science},
     doi = {10.1007/978-3-031-38821-7_1},
     month = aug,
     organization = {Springer},
     pages = {1--24},
     title = {A Digital Twin Description Framework and its Mapping to {Asset Administration Shell}},
     volume = {1708},
     year = {2023}
   }

4. ANNSIM

   Examining Model Qualities and Their Impact on Digital Twins

   Bentley Oakes, Claudio Gomes, Joachim Denil, Julien Deantoni, Joao Cambeiro, John Fitzgerald, and Peter Gorm Larsen

   In 2023 Annual Modeling and Simulation Conference (ANNSIM), 2023

   Abstract Bib PDF Slides

   Digital Twins (DTs) are built using modelling and simulation techniques in complex domains such as cyber- physical systems. However, further formal investigation is required for how a DT and the services it provides relate to the qualities of the models used by a service. Specifically, this article examines when a DT service can be said to have the qualities of relevant, verifiable, substitutable, and faithful based on the results of checking properties in comparison to the actual system. Using an incubator system as our running example, we show how a DT service relies on multiple models, present the consequences when these qualities are violated, and discuss strategies for adapting models to ensure these qualities.

   @inproceedings{Oakes2023examining,
     author = {Oakes, Bentley and Gomes, Claudio and Denil, Joachim and Deantoni, Julien and Cambeiro, Joao and Fitzgerald, John and Larsen, Peter Gorm},
     booktitle = {2023 Annual Modeling and Simulation Conference (ANNSIM)},
     organization = {IEEE},
     pages = {220--232},
     publisher = {IEEE},
     title = {Examining Model Qualities and Their Impact on Digital Twins},
     year = {2023}
   }

5. SoSyM

   Fault Localization in DSLTrans Model Transformations by Combining Symbolic Execution and Spectrum-Based Analysis

   Bentley Oakes, Javier Troya, Jessie Galasso, and Manuel Wimmer

   Software and Systems Modeling, Sep 2023

   Award Abstract DOI Bib PDF

   SoSyM/MODELS 2024 Journal-First Award

   The verification of model transformations is important for realizing robust model-driven engineering technologies and quality-assured automation. Many approaches for checking properties of model transformations have been proposed. Most of them have focused on the effective and efficient detection of property violations by contract checking. However, there remains the fault localization step between identifying a failing contract for a transformation based on verification feedback and precisely identifying the faulty rules.

   @article{Oakes2023FaultlocalizationDSLTrans,
     author = {Oakes, Bentley and Troya, Javier and Galasso, Jessie and Wimmer, Manuel},
     doi = {10.1007/s10270-023-01123-3},
     issn = {1619-1374},
     journal = {Software and Systems Modeling},
     month = sep,
     number = {3},
     pages = {737--763},
     title = {Fault Localization in {DSLTrans} Model Transformations by Combining Symbolic Execution and Spectrum-Based Analysis},
     volume = {23},
     year = {2023}
   }

2022

1. ASE

   End-to-End Rationale Reconstruction

   Mouna Dhaouadi, Bentley Oakes, and Michalis Famelis

   In 37th IEEE/ACM International Conference on Automated Software Engineering, 2022

   Abstract DOI Bib PDF

   The logic behind design decisions, called design rationale, is very valuable. In the past, researchers have tried to automatically extract and exploit this information, but prior techniques are only applicable to specific contexts and there is insufficient progress on an end-to-end rationale information extraction pipeline. Here we outline a path towards such a pipeline that leverages several Machine Learning (ML) and Natural Language Processing (NLP) techniques. Our proposed context-independent approach, called Kantara, produces a knowledge graph representation of decisions and of their rationales, which considers their historical evolution and traceability. We also propose validation mechanisms to ensure the correctness of the extracted information and the coherence of the development process. We conducted a preliminary evaluation of our proposed approach on a small example sourced from the Linux Kernel, which shows promising results.

   @inproceedings{Dhaouadi2022,
     author = {Dhaouadi, Mouna and Oakes, Bentley and Famelis, Michalis},
     booktitle = {37th IEEE/ACM International Conference on Automated Software Engineering},
     doi = {10.1145/3551349.3559547},
     pages = {1--5},
     title = {End-to-End Rationale Reconstruction},
     year = {2022}
   }

2. Book

   An Architecture and Reference Implementation for WSN-Based IoT Systems

   Burak Karaduman, Bentley Oakes, Raheleh Eslampanah, Joachim Denil, Hans Vangheluwe, and Moharram Challenger

   In Emerging Trends in IoT and Integration with Data Science, Cloud Computing, and Big Data Analytics, 2022

   Abstract DOI Bib PDF

   The Internet of Things and its technologies have evolved quickly in recent years. It became an umbrella term for various technologies, embedded devices, smart objects, and web services. Although it has gained maturity, there is still no clear or common definition of references for creating WSN-based IoT systems. In the awareness that creating an omniscient and ideal architecture that can suit all design requirements is not feasible, modular and scalable architecture that supports adding or subtracting components to fit a lot of requirements of various use cases should be provided as a starting point. This chapter discusses such an architecture and reference implementation. The architecture should cover multiple layers, including the cloud, the gateway, and the edges of the target system, which allows monitoring the environment, managing the data, programming the edge nodes and networking model to establish communication between horizontal and vertical embedded devices. In order to exemplify the proposed architecture and reference implementation, a smart irrigation case study is used.

   @incollection{Karaduman2022,
     author = {Karaduman, Burak and Oakes, Bentley and Eslampanah, Raheleh and Denil, Joachim and Vangheluwe, Hans and Challenger, Moharram},
     booktitle = {Emerging Trends in IoT and Integration with Data Science, Cloud Computing, and Big Data Analytics},
     doi = {10.4018/978-1-7998-4186-9.ch005},
     pages = {80--103},
     publisher = {IGI Global},
     title = {An Architecture and Reference Implementation for {WSN-Based IoT} Systems},
     year = {2022}
   }

2021

1. MODELSWARD

   Improving Digital Twin Experience Reports

   Bentley Oakes, Ali Parsai, Simon Van Mierlo, Serge Demeyer, Joachim Denil, Paul De Meulenaere, and Hans Vangheluwe

   In Proceedings of the 9th International Conference on Model-Driven Engineering and Software Development – Volume 1: MODELSWARD, 2021

   Abstract DOI Bib PDF Video Slides

   : Digital twins (DTs) are prevalent throughout industrial domains as evidenced by the rapid pace of experience reports in the literature. However, there remains disagreement about the precise definition of a DT and the essential characteristics in the DT paradigm, such as the scope of the system-under-study and the time-scale of its communication with the DT. These experience reports could therefore be hampering further classification and research insights by not reporting all of these relevant details about the DT solutions. We address these concerns by providing a conceptual structure for DTs as a common understanding and checklist for researchers and practitioners to precisely describe the characteristics and capabilities of their DT solutions. We express five experience reports using our structure to demonstrate its applicability and role as a guideline to improve the reporting of characteristics and increase the clarity of future experience reports.

   @inproceedings{Oakes2021,
     author = {Oakes, Bentley and Parsai, Ali and Van Mierlo, Simon and Demeyer, Serge and Denil, Joachim and De Meulenaere, Paul and Vangheluwe, Hans},
     booktitle = {Proceedings of the 9th International Conference on Model-Driven Engineering and Software Development -- Volume 1: MODELSWARD},
     doi = {10.5220/0010236101790190},
     isbn = {978-989-758-487-9},
     organization = {INSTICC},
     pages = {179--190},
     publisher = {SciTePress},
     title = {Improving Digital Twin Experience Reports},
     year = {2021}
   }

2. SAFECOMP

   Machine Learning-Based Fault Injection for Hazard Analysis and Risk Assessment

   Bentley Oakes, Mehrdad Moradi, Simon Van Mierlo, Hans Vangheluwe, and Joachim Denil

   In Computer Safety, Reliability, and Security: SAFECOMP 2021, 2021

   Abstract DOI Bib PDF

   Current automotive standards such as ISO 26262 require Hazard Analysis and Risk Assessment (HARA) on possible hazards and consequences of safety-critical components. This work attempts to ease this labour-intensive process by using machine learning-based fault injec- tion to discover representative hazardous situations. Using a Simulation- Aided Hazard Analysis and Risk Assessment (SAHARA) methodology, a visualisation and suggested hazard classification is then presented for the safety engineer. We demonstrate this SAHARA methodology using machine learning-based fault injection on a safety-critical use case of an adaptive cruise control system, to show that our approach can discover, visualise, and classify hazardous situations in a (semi-)automated man- ner in around twenty minutes.

   @inproceedings{Oakes2021a,
     author = {Oakes, Bentley and Moradi, Mehrdad and Van Mierlo, Simon and Vangheluwe, Hans and Denil, Joachim},
     booktitle = {Computer Safety, Reliability, and Security: SAFECOMP 2021},
     doi = {10.1007/978-3-030-83903-1_12},
     organization = {Springer},
     pages = {178--192},
     title = {Machine Learning-Based Fault Injection for Hazard Analysis and Risk Assessment},
     year = {2021}
   }

3. OntoDT

   Structuring and Accessing Knowledge for Historical and Streaming Digital Twins

   Bentley Oakes, Bart Meyers, Dennis Janssens, and Hans Vangheluwe

   In First Workshop on Ontology-Driven Conceptual Modeling of Digital Twins, 2021

   Abstract Bib PDF

   Organisations are intensely developing Digital Twins (DTs) to correctly and efficiently answer questions about the history and be- haviour of physical systems. However, it is not clear how to construct these DTs starting from the data, information, knowledge, and wisdom available in the organisation. In this work, we present our approach to DT construction which involves a layered knowledge graph (KG) architecture communicating with the organisation’s data repositories. We explain the components and timelines for using the KG to build both historical and streaming DTs, and what kinds of questions can be answered for our drivetrain use case.

   @inproceedings{Oakes2021b,
     author = {Oakes, Bentley and Meyers, Bart and Janssens, Dennis and Vangheluwe, Hans},
     booktitle = {First Workshop on Ontology-Driven Conceptual Modeling of Digital Twins},
     pages = {1--13},
     title = {Structuring and Accessing Knowledge for Historical and Streaming Digital Twins},
     year = {2021}
   }

2020

1. MODELS-C

   Towards Adaptive Abstraction for Continuous Time Models with Dynamic Structure

   Romain Franceschini, Bentley Oakes, Simon Van Mierlo, Moharram Challenger, and Hans Vangheluwe

   In Proceedings of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings, 2020

   Abstract DOI Bib

   Humans often switch between multiple levels of abstraction when reasoning about salient properties of complex systems. These changes in perspective may be leveraged at runtime to improve both performance and explainability, while still producing identical answers to questions about the properties of interest. This technique, which switches between multiple abstractions based on changing conditions in the modelled system, is also known as adaptive abstraction.

   @inproceedings{Franceschini2020,
     address = {New York, NY, USA},
     articleno = {93},
     author = {Franceschini, Romain and Oakes, Bentley and Van Mierlo, Simon and Challenger, Moharram and Vangheluwe, Hans},
     booktitle = {Proceedings of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings},
     doi = {10.1145/3417990.3421443},
     isbn = {978-1-4503-8135-2},
     publisher = {Association for Computing Machinery},
     series = {MODELS '20},
     title = {Towards Adaptive Abstraction for Continuous Time Models with Dynamic Structure},
     year = {2020}
   }

2. DSN-W

   Exploring Fault Parameter Space Using Reinforcement Learning-Based Fault Injection

   Mehrdad Moradi, Bentley Oakes, Mustafa Saraoglu, Andrey Morozov, Klaus Janschek, and Joachim Denil

   In 2020 50th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W), 2020

   Abstract DOI Bib PDF

   Assessing the safety of complex Cyber-Physical Systems (CPS) is a challenge in any industry. Fault Injection (FI) is a proven technique for safety analysis and is recommended by the automotive safety standard ISO 26262. Traditional FI methods require a considerable amount of effort and cost as FI is applied late in the development cycle and is driven by manual effort or random algorithms. In this paper, we propose a Reinforcement Learning (RL) approach to explore the fault space and find critical faults. During the learning process, the RL agent injects and parameterizes faults in the system to cause catastrophic behavior. The fault space is explored based on a reward function that evaluates previous simulation results such that the RL technique tries to predict improved fault timing and values. In this paper, we apply our technique on an Adaptive Cruise Controller with sensor fusion and compare the proposed method with Monte Carlo-based fault injection. The proposed technique is more efficient in terms of fault coverage and time to find the first critical fault.

   @inproceedings{Moradi2020,
     author = {Moradi, Mehrdad and Oakes, Bentley and Saraoglu, Mustafa and Morozov, Andrey and Janschek, Klaus and Denil, Joachim},
     booktitle = {2020 50th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W)},
     doi = {10.1109/DSN-W50199.2020.00028},
     pages = {102--109},
     title = {Exploring Fault Parameter Space Using Reinforcement Learning-Based Fault Injection},
     year = {2020}
   }

3. SAFECOMP

   Machine Learning-Assisted Fault Injection

   Mehrdad Moradi, Bentley Oakes, and Joachim Denil

   In Computer Safety, Reliability, and Security (SAFECOMP 2020 Workshops), 2020

   Position paper

   Abstract Bib PDF

   Fault Injection (FI) is a method for system validation and verification in which the tester evaluates the system behavior resulting from the introduction of faults into the system under test. This paper proposes a model-based approach to improve the efficiency of the FI process by utilizing Machine Learning (ML) and formalized domain knowledge. This ML algorithm uses a probabilistic automaton to reduce the manual effort required in the testing procedure as the algorithm can automatically make decisions and predictions about catastrophic fault parameters. This assists the tester in dealing with complicated and broad-scale systems by enabling higher fault coverage with fewer simulations.

   @inproceedings{Moradi2020a,
     author = {Moradi, Mehrdad and Oakes, Bentley and Denil, Joachim},
     booktitle = {Computer Safety, Reliability, and Security (SAFECOMP 2020 Workshops)},
     note = {Position paper},
     title = {Machine Learning-Assisted Fault Injection},
     year = {2020}
   }

4. SIMULTECH

   Hint-Based Configuration of Co-Simulations with Algebraic Loops

   Bentley Oakes, Cláudio Gomes, Franz Rudolf Holzinger, Martin Benedikt, Joachim Denil, and Hans Vangheluwe

   In 9th International Conference, SIMULTECH 2019, Revised Selected Papers, 2020

   Abstract DOI Bib PDF

   . Co-simulation is a powerful technique for performing full-system sim- ulation. Multiple black-box models and their simulators are combined together to provide the behaviour for a full system. However, the black-box nature of co- simulation and potentially infinite configuration space means that configuration of co-simulations is a challenging problem for today’s practitioners. Our previous work on co-simulation configuration operated on the notion of hints, which allow system engineers to encode their knowledge and insights about the system. These hints, combined with state-of-the-art best practices, can then be used to semi-automatically configure the co-simulation. We summarize our previous hint-based configuration work here, and explore the challenging problem of scheduling co-simulations which contain algebraic loops. Solving or “breaking” these loops is required for scheduling, yet this breaking process can induce errors in the co-simulation. This work formalizes this schedul- ing problem, presents our insights gained about the problem, and details an op- timal search algorithm as well as greedy scheduling algorithms. These heuristic algorithms are evaluated on (synthetic) co-simulation scenarios to determine their relative speedup and optimality.

   @incollection{Oakes2020,
     author = {Oakes, Bentley and Gomes, Cl{\'a}udio and Holzinger, Franz Rudolf and Benedikt, Martin and Denil, Joachim and Vangheluwe, Hans},
     booktitle = {9th International Conference, SIMULTECH 2019, Revised Selected Papers},
     doi = {10.1007/978-3-030-55867-3_1},
     pages = {1--28},
     publisher = {Springer},
     title = {Hint-Based Configuration of Co-Simulations with Algebraic Loops},
     volume = {1260},
     year = {2020}
   }

5. MODELS-C

   Validity Frame Concept as Effort-Cutting Technique within the Verification and Validation of Complex Cyber-Physical Systems

   Bert Van Acker, Bentley Oakes, Mehrdad Moradi, Paul Demeulenaere, and Joachim Denil

   In Proceedings of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings, 2020

   Abstract DOI Bib PDF

   The increasing performance demands and certification needs of complex cyber-physical systems (CPS) raise the complexity of the engineering process, not only within the development phase, but also in the Verification and Validation (V&V) phase. A proven technique to handle the complexity of CPSs is Model-Based Design (MBD). Nevertheless, the verification and validation of complex CPSs is still an exhaustive process and the usability of the models to front-load V&V activities heavily depends on the knowledge of the models and the correctness of the conducted virtual experiments. In this paper, we explore how the effort (and cost) of the V&V phase of the engineering process of complex CPSs can be reduced by enhancing the knowledge about the system components, and explicitly capturing it within their corresponding validity frame. This effort reduction originates from exploiting the captured system knowledge to generate efficient V&V processes and by automating activities at different model life stages, such as the setup and execution of boundary-value or fault-injection tests. This will be discussed in the context of a complex CPS: a safety-critical adaptive cruise control system.

   @inproceedings{VanAcker2020,
     address = {New York, NY, USA},
     articleno = {80},
     author = {Van Acker, Bert and Oakes, Bentley and Moradi, Mehrdad and Demeulenaere, Paul and Denil, Joachim},
     booktitle = {Proceedings of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings},
     doi = {10.1145/3417990.3419226},
     isbn = {978-1-4503-8135-2},
     publisher = {Association for Computing Machinery},
     series = {MODELS '20},
     title = {Validity Frame Concept as Effort-Cutting Technique within the Verification and Validation of Complex Cyber-Physical Systems},
     year = {2020}
   }

6. ICSMM

   Exploring Validity Frames in Practice

   Simon Van Mierlo, Bentley Oakes, Bert Van Acker, Raheleh Eslampanah, Joachim Denil, and Hans Vangheluwe

   In Proceedings of the First International Conference, ICSMM 2020, 2020

   Abstract DOI Bib PDF

   . Model-Based Systems Engineering (MBSE) provides work- flows, methods, techniques and tools for optimal simulation-based design and realization of complex Software-Intensive, Cyber-Physical Systems. One of the key benefits of this approach is that the behavior of the re- alized system can be reasoned about and predicted in-silico, before any prototype has been developed. Design models are increasingly used after the system has been realized as well. For example, a (design) digital twin can be used for runtime monitoring to detect and diagnose discrepancies between the simulated and realized system. Inconsistencies may arise, however, because models were used at design time that are not valid within the operating context of the realized system. It is often left to the domain expert to ensure that the models used are valid with respect to their realized counterpart. Due to system complexity and automated Design-Space Exploration (DSE), it is increasingly difficult for a human to reason about model validity. We propose validity frames as an explicit model of the contexts in which a model is a valid representation of a sys- tem to rule out invalid designs at design time. We explain the essential and conceptual, yet practical, structure of validity frames and a process for building them using an electrical resistor in the optimal design of a high-pass filter as a running example. We indicate how validity frames can be used in a DSE process, as well as for runtime monitoring.

   @inproceedings{VanMierlo2020,
     author = {Van Mierlo, Simon and Oakes, Bentley and Van Acker, Bert and Eslampanah, Raheleh and Denil, Joachim and Vangheluwe, Hans},
     booktitle = {Proceedings of the First International Conference, ICSMM 2020},
     doi = {10.1007/978-3-030-58167-1_10},
     pages = {131--148},
     publisher = {Springer, Cham},
     title = {Exploring Validity Frames in Practice},
     year = {2020}
   }

2019

1. MODELS-C

   Validating Industrial Requirements with a Contract-Based Approach

   Matthias Bernaerts, Bentley Oakes, Ken Vanherpen, Bjorn Aelvoet, Hans Vangheluwe, and Joachim Denil

   In 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C), Sep 2019

   Abstract DOI Bib PDF Slides

   This paper presents our contract-based design technique for formalizing requirements during the design phase of a complicated and safety-critical automotive component. In our approach, contracts are created using property specification patterns to eliminate ambiguous unstructured natural language requirements, which could lead to misinterpretations or mismatched interfaces in the integration phases of the design process. These patterns are then automatically transformed into Signal Temporal Logic (STL) formulas. The STL formulas are verified on a modeled system of the component, utilizing the Matlab® toolbox Breach. This approach validates the industrial requirements described in the contracts, and can help achieve the requirement-based testing demanded by automotive safety standard ISO 26262.

   @inproceedings{Bernaerts2019,
     author = {Bernaerts, Matthias and Oakes, Bentley and Vanherpen, Ken and Aelvoet, Bjorn and Vangheluwe, Hans and Denil, Joachim},
     booktitle = {2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)},
     doi = {10.1109/MODELS-C.2019.00010},
     month = sep,
     pages = {18--27},
     title = {Validating Industrial Requirements with a Contract-Based Approach},
     year = {2019}
   }

2. SIMULTECH

   HintCO – Hint-Based Configuration of Co-Simulations

   Cláudio Gomes, Bentley Oakes, Mehrdad Moradi, Alejandro Torres Gámiz, Juan Carlos Mendo, Stefan Dutré, Joachim Denil, and Hans Vangheluwe

   In Proceedings of the 9th International Conference on Simulation and Modeling Methodologies, Technologies and Applications – Volume 1: SIMULTECH, 2019

   Best Paper Abstract DOI Bib PDF Slides

   Best Student Paper Award at SIMULTECH 2019

   Simulation-based analyses of Cyber-Physical Systems are fundamental in industrial design and testing approaches. The utility of analyses relies on the correct configuration of the simulation tools, which can be highly complicated. System engineers can normally judge the results, and either evaluate multiple simulation algorithms, or change the models. However, this is not possible in a co-simulation approach. Co-simulation is a technique to perform full-system simulation, by combining multiple black-box simulators, each responsible for a part of the system. In this paper, we demonstrate the difficulty of correctly configuring a co-simulation scenario using an industrial case study. We propose an approach to tackle this challenge by allowing multiple engineers, specialized in different domains, to encode some of their experience in the form of hints. These hints, together with state-of-the-art best practices, are then used to semi-automatically guide the configuration process of the co-simulation. We report the application of this approach to a use case proposed by our industrial partners, and discuss some of the lessons learned.

   @inproceedings{Gomes2019,
     author = {Gomes, Cl{\'a}udio and Oakes, Bentley and Moradi, Mehrdad and G{\'a}miz, Alejandro Torres and Mendo, Juan Carlos and Dutr{\'e}, Stefan and Denil, Joachim and Vangheluwe, Hans},
     booktitle = {Proceedings of the 9th International Conference on Simulation and Modeling Methodologies, Technologies and Applications -- Volume 1: SIMULTECH},
     doi = {10.5220/0007830000570068},
     isbn = {978-989-758-381-0},
     organization = {INSTICC},
     pages = {57--68},
     publisher = {SciTePress},
     title = {{HintCO} -- {Hint-Based} Configuration of Co-Simulations},
     year = {2019}
   }

3. MODELS-C

   A Model-Driven Engineering Framework to Support the Functional Safety Process

   Bart Meyers, Klaas Gadeyne, Bentley Oakes, Matthias Bernaerts, Hans Vangheluwe, and Joachim Denil

   In 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C), Sep 2019

   Abstract DOI Bib PDF

   The design of safety-related systems traditionally has long and costly development cycles due to the highly manual safety engineering process, which is guided by industry standards. In this paper, we present a modelling framework that supports DevOps principles of continuous testing and fast development iterations for the design of safety-critical systems. We show how modelling can help introducing DevOps in the context of functional safety analysis, and we also report how DevOps was used during the development of the framework.

   @inproceedings{Meyers2019,
     author = {Meyers, Bart and Gadeyne, Klaas and Oakes, Bentley and Bernaerts, Matthias and Vangheluwe, Hans and Denil, Joachim},
     booktitle = {2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)},
     doi = {10.1109/MODELS-C.2019.00094},
     month = sep,
     pages = {619--623},
     title = {A Model-Driven Engineering Framework to Support the Functional Safety Process},
     year = {2019}
   }

4. SummerSim

   Optimizing Fault Injection in FMI Co-Simulation through Sensitivity Partitioning

   Mehrdad Moradi, Cláudio Gomes, Bentley Oakes, and Joachim Denil

   In Proceedings of the 2019 Summer Simulation Conference, 2019

   Abstract Bib PDF Slides

   As society and industry relies extensively on Cyber-Physical Systems (CPS), any malfunctions can have unforeseen catastrophic failures. Fault Injection (FI) techniques perturb a model of a CPS with the intention of causing a failure and measuring the robustness of the CPS. Naturally, the success of a FI simulation depends on three factors: (i) the realism of the faults injected; (ii) how quickly the faults cause catastrophic failure; and (iii) the fidelity of the model used. This paper proposes to improve the success rate of FI studies by addressing each one of these factors. An algorithm is presented that leverages traditional sensitivity analysis in hybrid systems to reduce an uncount- able fault search space to a optimal finite set (factors i,ii), and we use co-simulation as the model integration technique (factor iii). We evaluate our contribution on the power window system developed by MathWorks®.

   @inproceedings{Moradi2019,
     address = {San Diego, CA, USA},
     articleno = {32},
     author = {Moradi, Mehrdad and Gomes, Cl{\'a}udio and Oakes, Bentley and Denil, Joachim},
     booktitle = {Proceedings of the 2019 Summer Simulation Conference},
     pages = {1--12},
     publisher = {Society for Computer Simulation International},
     series = {SummerSim '19},
     title = {Optimizing Fault Injection in {FMI} Co-Simulation through Sensitivity Partitioning},
     year = {2019}
   }

5. MODELS-C

   The Computational Notebook Paradigm for Multi-Paradigm Modeling

   Bentley Oakes, Romain Franceschini, Simon Van Mierlo, and Hans Vangheluwe

   In 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C), Sep 2019

   Abstract DOI Bib PDF Slides

   Computational notebooks are gaining widespread acceptance as a paradigm for storage, dissemination, and re-production of experimental results. In this paper, we define the computational notebook paradigm (CNP) consisting of entities and processes and discuss how the reproducibility of the experimental process and results is enhanced by each element. This paper also details the interactions of CNP and multi-paradigm modeling (MPM), with an aim of understanding how to support MPM within the CNP, and improve the reproducibility aspects of both the CNP and MPM.

   @inproceedings{Oakes2019,
     author = {Oakes, Bentley and Franceschini, Romain and Van Mierlo, Simon and Vangheluwe, Hans},
     booktitle = {2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)},
     doi = {10.1109/MODELS-C.2019.00072},
     month = sep,
     pages = {449--454},
     title = {The Computational Notebook Paradigm for Multi-Paradigm Modeling},
     year = {2019}
   }

2018

1. Ph.D.

   A Symbolic Execution-Based Approach to Model Transformation Verification Using Structural Contracts

   Bentley Oakes

   McGill University, 2018

   Abstract Bib PDF Slides

   As the complexity of software systems increases, the engineering effort for developing those systems must deal with that complexity. One paradigm for software development is model-driven engineering, where the models of the system become the first-order artefacts. These models may be used for simulation or analysis of the system, or be transformed into executable code or documentation. The intention is to represent each facet in the system in the most appropriate formalism at the most appropriate level of abstraction. Model transformations provide a structured and understandable way of manipulating these models, and are often rooted in a mathematical approach which enables precise specification and analysis. However, it may be difficult for a user to reason about what elements will be matched and written by a particular transformation. Our research is focused on the verification of model transformations for a particular model transformation language. In particular, we are interested in the proving of pre-condition/ post-condition contracts, which relate the elements in the input models to the transformation with the elements present in the corresponding output elements. i

   @phdthesis{Oakes2018,
     author = {Oakes, Bentley},
     school = {McGill University},
     title = {A Symbolic Execution-Based Approach to Model Transformation Verification Using Structural Contracts},
     year = {2018}
   }

2. SoSyM

   Full Contract Verification for ATL Using Symbolic Execution

   Bentley Oakes, Javier Troya, Levi Lúcio, and Manuel Wimmer

   Software and Systems Modeling, 2018

   Abstract DOI Bib PDF

   The Atlas Transformation Language (ATL) is currently one of the most-used model transformation languages and has become a de-facto standard in model- driven engineering for implementing model transforma- tions. At the same time, it is understood by the com- munity that enhancing methods for exhaustively verify- ing such transformations allows for a more widespread adoption of model-driven engineering in industry. A va- riety of proposals for the verification of ATL transfor- mations have arisen in the past few years. However, the majority of these techniques are either based on non- exhaustive testing or on proof methods that require hu- man assistance and/or are not complete. In this paper, we describe our method for statically verifying the declarative subset of ATL model trans- formations. This verification is performed by translat- ing the transformation (including features like filters, OCL expressions, and lazy rules) into our model trans- formation language DSLTrans. As we handle only the declarative portion of ATL, and DSLTrans is Turing- incomplete, this reduction in expressivity allows us to use a symbolic-execution approach to generate repre- sentations of all possible input models to the transform- Bentley James Oakes School of Computer Science, McGill University, Canada E-mail: bentley.oakes@mail.mcgill.ca Javier Troya Department of Computing Languages and Systems, Univer- sidad de Sevilla, Spain, E-mail: jtroya@us.es Levi L ́ucio fortiss GmbH, M ̈unchen, Germany, E-mail: lucio@fortiss.org Manuel Wimmer Business Informatics Group, TU Wien, Austria, E-mail: wimmer@big.tuwien.ac.at ation. We then verify pre-/post-condition contracts on these representations, which in turn verifies the trans- formation itself. The technique we present in this paper is exhaus- tive for the subset of declarative ATL model transfor- mations. This means that if the prover indicates a con- tract holds on a transformation, then the contract’s pre- /post-condition pair will be true for any input model for that transformation. We demonstrate and explore the applicability of our technique by studying several rela- tively large and complex ATL model transformations, including a model transformation developed in collabo- ration with our industrial partner. As well, we present our ‘slicing’ technique. This technique selects only those rules in the DSLTrans transformation needed for con- tract proof, thereby reducing proving time.

   @article{Oakes2018a,
     author = {Oakes, Bentley and Troya, Javier and L{\'u}cio, Levi and Wimmer, Manuel},
     doi = {10.1007/s10270-016-0548-7},
     journal = {Software and Systems Modeling},
     number = {3},
     pages = {815--849},
     title = {Full Contract Verification for {ATL} Using Symbolic Execution},
     volume = {17},
     year = {2018}
   }

3. MDEbug

   Debugging of Model Transformations and Contracts in SyVOLT

   Bentley Oakes, Clark Verbrugge, Levi Lúcio, and Hans Vangheluwe

   In Proceedings of the MDEbug Workshop at Model Driven Engineering Languages and Systems (MODELS), 2018

   Abstract Bib PDF Slides

   The SyVOLT tool verifies DSLTrans transformations by generating a state-space for the transformation’s execution, and then proving structural contracts on that state-space. As with any verification activity, it is non-trivial to ensure that these contracts are error-free and correspond to the user’s intention. SyVOLT detects and localizes errors in the input artefacts for the verification activity to provide the user with assistance in de- bugging the transformation and/or the contracts. This experience report details the techniques built into the analysis, monitoring, and reporting stages of the tool. These techniques include detection of invalid rules and contracts, a form of reachability analysis during state-space generation, and assisting the user in understanding why a contract fails to be satisfied. CCS CONCEPTS • Computing methodologies →Model verification and vali- dation; • Software and its engineering →Domain specific lan- guages; Automated static analysis; ACM Reference Format: Bentley James Oakes, Clark Verbrugge, Levi Lucio, and Hans Vangheluwe.

   @inproceedings{Oakes2018b,
     author = {Oakes, Bentley and Verbrugge, Clark and L{\'u}cio, Levi and Vangheluwe, Hans},
     booktitle = {Proceedings of the MDEbug Workshop at Model Driven Engineering Languages and Systems (MODELS)},
     pages = {532--537},
     title = {Debugging of Model Transformations and Contracts in {SyVOLT}},
     year = {2018}
   }

2015

1. MODELS

   SyVOLT: Full Model Transformation Verification Using Contracts

   Levi Lúcio, Bentley Oakes, Cláudio Gomes, Gehan Selim, Juergen Dingel, James Cordy, and Hans Vangheluwe

   In Model Driven Engineering Languages and Systems (MODELS), 2015

   Abstract Bib PDF Video Poster

   —We introduce SyVOLT, a plugin for the Eclipse development environment for the verification of structural pre- /post-condition contracts on model transformations. The plugin allows the user to build transformations in our transformation language DSLTrans using a visual editor. The pre-/post-condition contracts to be proved on the transformation can also be built in a similar interface. Our contract proving process is exhaustive, meaning that if a contract is said to hold, then the contract will hold for all input models of a transformation. If the contract does not hold, then the counter-examples (i.e., input models) where the contract fails will be presented. Demo: https://www.youtube.com/watch?v=8PrR5RhPptY I. INTRODUCTION Model transformations are at the center of model-driven development, making pragmatic and usable tools for their verification indispensable. In this paper we introduce SyVOLT (Symbolic Verifier of mOdeL Transformations) [5], an Eclipse plugin that allows verifying pre-/post-condition structural con- tracts on model transformations. SyVOLT’s operation relies on a theoretical framework that has been developed for the DSLTrans model transformation language. In this framework, pre-/post-condition contracts can be shown to either hold for all input/output pairs resulting from executing a given DSLTrans model transformation, or not to hold for at least one of those input/output pairs [16]. Extensive work exists on the verification of different aspects of model transformations [7]. In [26] the authors describe a method where ‘Tracts’ can be specified for model trans- formations. Tracts resemble SyVOLT’s contracts and define a set of constraints on the source and target metamodels, a set of source-target constraints, and a tract test suite, i.e., a collection of source models satisfying the source constraints. The accompanying TractsTool can then automatically test the transformation by verifying if all source/target model pairs satisfy the constraints in the tract test suite. Several other approaches support the testing of model transformations based on different kinds of contracts such as model fragments [18], graph patterns [13], [9], Triple Gr

   @inproceedings{Lucio2015,
     author = {L{\'u}cio, Levi and Oakes, Bentley and Gomes, Cl{\'a}udio and Selim, Gehan and Dingel, Juergen and Cordy, James and Vangheluwe, Hans},
     booktitle = {Model Driven Engineering Languages and Systems (MODELS)},
     pages = {24--27},
     title = {{SyVOLT}: {Full} Model Transformation Verification Using Contracts},
     year = {2015}
   }

2. MODELS

   Fully Verifying Transformation Contracts for Declarative ATL

   Bentley Oakes, Javier Troya, Levi Lúcio, and Manuel Wimmer

   In Model Driven Engineering Languages and Systems (MODELS), 2015

   Abstract DOI Bib PDF Slides

   —The Atlas Transformation Language (ATL) is today a de-facto standard in model-driven development. It is under- stood by the community that methods for exhaustively verifying such transformations provide an important pillar for achieving a stronger adoption of model-driven development in industry. In this paper we propose a method for verifying ATL model transformations by translating them into DSLTrans, a transformation language with limited expressiveness. Pre-/post- condition contracts are then verified on the resulting DSLTrans specification using a symbolic-execution property prover. The technique we present in this paper is exhaustive for the declarative ATL subset, meaning we are certain that if a contract holds, it will hold when any input model is passed to the ATL transformation being checked. We explore the scalability of our technique using a set of examples, including a model transformation developed in collaboration with our industrial partner. I. INTRODUCTION Graph-based model transformations have become in the last few years the main means for manipulating models in model-driven development. Their simplicity, their allowance for mathematical treatment, and the fact that they can natively manipulate domain-specific concepts expressed in metamod- els, all make graph-based model transformations an excel- lent compromise between strong theoretical foundations and applicability to real-world problems. In particular, the Atlas Transformation Language (ATL) [3] has come to prominence in the model-driven development community. This success is due to ATL’s flexibility, support of the main meta-modelling standards, usability that relies on good tool integration with the Eclipse world, and a supportive development community. Because of the importance of ATL in both the academic and the industrial arenas, specification verification is of prime importance: firstly because the correctness of software built using model-driven development techniques typically relies on the correctness of many operations executed using model transformations; and secondly because tools that allow build- ing verified software are in strong demand, especially in industry where quali

   @inproceedings{Oakes2015,
     author = {Oakes, Bentley and Troya, Javier and L{\'u}cio, Levi and Wimmer, Manuel},
     booktitle = {Model Driven Engineering Languages and Systems (MODELS)},
     doi = {10.1109/models.2015.7338256},
     pages = {256--265},
     title = {Fully Verifying Transformation Contracts for Declarative {ATL}},
     year = {2015}
   }

3. AMT

   Finding and Fixing Bugs in Model Transformations with Formal Verification: An Experience Report

   Gehan Selim, James Cordy, Juergen Dingel, Levi Lúcio, and Bentley Oakes

   In Proceedings of Analysis of Model Transformations Workshop at Model Driven Engineering Languages and Systems, 2015

   Abstract Bib PDF

   We report on the use of a formal verification tool for a graph-based transformation language in the context of a case study. The tool iden- tified two bugs in the transformation that had eluded all previous test- ing efforts. The paper describes what we learned about the analysis of model transformations and how we intend to use these insights to improve the verification tool.

   @inproceedings{Selim2015,
     author = {Selim, Gehan and Cordy, James and Dingel, Juergen and L{\'u}cio, Levi and Oakes, Bentley},
     booktitle = {Proceedings of Analysis of Model Transformations Workshop at Model Driven Engineering Languages and Systems},
     pages = {26--35},
     title = {Finding and Fixing Bugs in Model Transformations with Formal Verification: {An} Experience Report},
     year = {2015}
   }

2014

1. TECH

   A Technique for Symbolically Verifying Properties of Graph-Based Model Transformations

   Levi Lúcio, Bentley Oakes, and Hans Vangheluwe

   2014

   Abstract Bib PDF

   As model transformations are a required part of model-driven development, it is crucial to provide techniques that address their formal verification. One approach that has proven very successful in program verification is symbolic ex- ecution. The symbolic abstraction in these techniques allows formal properties to be exhaustively proved for all executions of a given program. In our approach we apply the same ab- straction principle to verify model transformations. Our algo- rithm builds a finite set of path conditions which represents all concrete transformation executions through a formal ab- straction relation. We are then able to prove properties over all transformation executions in a model-independent way. This is done by examining if any created path condition vi- olates a given property, which will produce a counterexam- ple if the property does not hold for the transformation. We demonstrate that this property proving approach is both valid and complete. Implementation results are also presented here which suggest that our approach is feasible and can scale to real-world transformations. Key words Model Transformations, Symbolic Verification, Translation

   @techreport{Lucio2014,
     author = {L{\'u}cio, Levi and Oakes, Bentley and Vangheluwe, Hans},
     institution = {McGill University},
     number = {SOCS-TR-2014.1},
     title = {A Technique for Symbolically Verifying Properties of Graph-Based Model Transformations},
     year = {2014}
   }

2. TECH

   Optimizing Simulink Models

   Bentley Oakes

   2014

   Abstract Bib PDF Poster

   The Simulink R ⃝modelling tool is used to diagram and study cyber-physical systems. One advantage of modelling the systems in this way is that embeddable code can generated from the models directly. However, this process means that inefficiencies in the model may be propagated to the code. Code generation optimizations are available, but may lead to an unacceptable loss of traceability in determining which parts of the model were modified or removed during code generation. In our work, we focus on defining model-to-model optimizations. This means that the optimized model can be loaded back into Simulink for further development or analysis, improving traceability and allowing model specialization for different platforms. An analysis framework has been created, based on dataflow analysis from the compiler optimization domain. This allows fast and accurate definition of new optimizations. As well, an initial optimization classification was developed to aid in the discovery of new optimizations. At the present time, we have implemented three optimizations in our framework: constant folding, dead-block removal, and hierarchy flattening. These optimizations are intended to simplify the model and potentially increase model performance when simulated. Our framework allows us to communicate directly with a Simulink instance to import and export models, allowing us to test these optimizations on a number of sample Simulink models. In order to test the performance benefits of our optimizations, our experiments generated simulation code for the model before and after optimization. Examining the run-time of these simulations indicated that the constant folding optimization decreased the run-time on all applicable models when code generation optimizations were not used. This decrease in run-time is well above the fraction of a second that the analysis and transformation took for this optimization. Thus, a net gain in performance was obtained. Other optimizations did not show performance results, but did produce a simplified model, which is also desirable for a modeller.

   @techreport{Oakes2014,
     author = {Oakes, Bentley},
     institution = {McGill University},
     number = {CS-TR-2014.5},
     title = {Optimizing {Simulink} Models},
     year = {2014}
   }

3. ICGT

   Specification and Verification of Graph-Based Model Transformation Properties

   Gehan Selim, Levi Lúcio, James Cordy, Juergen Dingel, and Bentley Oakes

   In Proceedings of International Conference on Graph Transformation, 2014

   Abstract DOI Bib PDF

   . We extend a previously proposed symbolic model transfor- mation property prover for the DSLTrans transformation language. The original prover generated the set of path conditions (i.e., symbolic trans- formation executions), and verified atomic contracts (constraints on input- output model relations) on these path conditions. The prover evaluated atomic contracts to yield either true or false for the transformation when run on any input model. In this paper we extend the prover such that it can verify atomic contracts and more complex properties composed of atomic contracts. Besides demonstrating our prover on a simple transfor- mation, we use it to verify different kinds of properties of an industrial transformation. Experiments on this transformation using our prover show a speed-up in verification run-time by two orders of magnitude over another verification tool that we evaluated in previous research.

   @inproceedings{Selim2014,
     author = {Selim, Gehan and L{\'u}cio, Levi and Cordy, James and Dingel, Juergen and Oakes, Bentley},
     booktitle = {Proceedings of International Conference on Graph Transformation},
     doi = {10.1007/978-3-319-09108-2_8},
     organization = {Springer},
     pages = {113--129},
     title = {Specification and Verification of Graph-Based Model Transformation Properties},
     year = {2014}
   }

2013

1. M.Sc.

   Practical and Theoretical Issues of Evolving Behaviour Trees for a Turn-Based Game

   Bentley Oakes

   McGill University, Aug 2013

   Abstract Bib PDF Slides

   The concept of evolving components of an artificial intelligence (AI) has seen increased interest in recent years as the power and complexity of AI has grown. In entertainment software, this AI can impact the player’s experiences and enjoyment through elements such as the level of difficulty of the player’s competition. There- fore AI development is an important research topic, especially as development is considered difficult by the video game industry. This work applies the evolutionary computing paradigm to a turn-based domain by evolving team strategies. These strategies are represented as behaviour trees, a formalism found in the video game industry and well-suited to the evolutionary algorithm due to their flexibility and tree structure. During the evolutionary process, strategies are evaluated in Battle for Wesnoth, an open-source game with a stochastic nature. A fitness function is defined to assign strategies a numerical strength value, along with a second perfor- mance metric that is robust to the variance found in the domain. The evolutionary algorithm then recombines strategies with high strength values, using evolutionary operators from the literature such as crossover and mutation. Later experiments focus on evolutionary algorithm parameters, including comparing a variety of fitness functions to provide insights into their use. Starting from a number of initial states, numerous strategies are evolved using this algorithm. These initial states are a null strategy, randomly-generated strategies, and a number of hand-built strategies. The evolved strategies are then evaluated in- game, and will be shown to be measurably stronger than the initial strategies. i

   @mastersthesis{Oakes2013,
     author = {Oakes, Bentley},
     month = aug,
     school = {McGill University},
     title = {Practical and Theoretical Issues of Evolving Behaviour Trees for a Turn-Based Game},
     year = {2013}
   }

2012

1. TECH

   Navigating Social Spaces

   Bentley Oakes and Clark Verbrugge

   Sep 2012

   Abstract Bib PDF

   Behaviour of robots within a human-populated space can be disruptive, as robot motion does not necessarily conform to social norms. Typical movement models are oblivious to social expectations, and so easily violate personal space and other social rules, magnifying the unnatu- ral behaviour of robot agents and causing discomfort to human occupants. This paper presents a navigation algorithm that incorporates human proxemics into a modified Rapidly-exploring Random Tree (RRT) algorithm. Our Socially-Realistic RRT algorithm (SRRRT) includes both a cost function based on a realistic model of human interaction distances, as well as a human motion model in order to produce movement patterns that better integrate with human so- cial behaviour. We experiment with our algorithm in simulation, comparing it with both a naive RRT and an A* implementation in both static and dynamic movement contexts. SRRRT demonstrates quantifiably better paths in terms of social cost, while maintaining a simple and easily extensible implementation design. Inclusion of such a design in robot motion enables more socially transparent behaviour, improving the ability of humans and robots in real or virtual contexts to coexist.

   @techreport{Oakes2012,
     author = {Oakes, Bentley and Verbrugge, Clark},
     institution = {McGill University},
     month = sep,
     number = {GR@M Technical Report No.2012-2},
     title = {Navigating Social Spaces},
     year = {2012}
   }

2. TECH

   Embedding Causal Block Diagrams within Behaviour Trees

   Bentley Oakes

   Apr 2012

   Abstract Bib PDF Slides

   Causal block diagrams are a formalism to model systems using math- ematical values. However, their text output can be difficult to visualize. This paper aims to describe a system where causal block diagrams can be visualized using a Java simulation. This is achieved by embedding causal blocks within a behaviour tree formalism. With this new hybrid formalism, a number of advantages are realized such as increased flexibility and express- ibility. Two simulations using these hybrid trees are presented, as well as a discussion of the new formalism’s suitability to these problems. These sim- ulation experiments are the circle test and a personal space simulation, and the resulting hybrid trees are also examined.

   @techreport{Oakes2012a,
     author = {Oakes, Bentley},
     institution = {McGill University},
     month = apr,
     number = {COMP 522 - Modelling and Simulation Course Project},
     title = {Embedding Causal Block Diagrams within Behaviour Trees},
     year = {2012}
   }