Measuring Progress and Defining Productivity Metrics in Model-based Engineering

Have you ever walked into a model review session or looked into a model and wondered about its level of maturity toward the deliverables?

Over the past decade, the adoption of a model-driven approach to engineering has dramatically changed the delivery and coordination methods in capital projects. Many firms that are already engaged in model-driven engineering processes have a positive perception of the value they receive for the time, money, and effort they expend on their modeling programs. Although the industry is adjusting to a data-rich, model-driven approach to engineering, there are no established and standardized processes for measuring progress and productivity of a model-driven engineering process; particularly when the deliverable type, schedule reporting, and their impacts have changed. The key underlying challenge is how the progress and level of effort in a model-driven engineering process can be measured and reported.

Any process or metric should give project stakeholders accurate and timely information about both modeling progression and productivity. It should also measure factors that provide insight, manage, and control opportunities, and it must account for the unavailability of vendor data, design changes, rework and design development, model review comments, operability reviews, hazard and operability studies, among other factors. These processes and metrics should be easy to use, flexible, and extensible for implementation across various project types and lifecycle phases of today’s capital projects.

CII Research Team 332 (RT-332), Measuring Progress and Defining Productivity Metrics in Model-based Engineering, set out to create a solution for these challenges. This research aims to address the following essential question:

With this essential question as its central catalyst, the RT-332 research offers a new guideline for defining productivity metrics and measuring progress in a model-driven approach to engineering. Part of this guideline is a series of standardized definitions created to measure the maturity of various modeling disciplines. These definitions can be used to analyze the maturity of the design components and to assess the quality of the information used in the modeling process. The Model Maturity Index (MMI) Definitions – categorized into a set of discrete MMI levels ranging from 100 to 600 – provide owners and engineering firms with a clear set of modeling requirements that must be fulfilled during each engineering phase.

To collect data and test the research findings, RT-332 built a Model Maturity Risk Index (Model MRI) Toolkit in Microsoft Excel. To successfully implement the research findings, a company would coordinate with its model software vendor to incorporate the content of the research team’s Model MRI Toolkit into its software. This would allow the user to quickly and easily determine the MMI levels for each model discipline and Work Breakdown Structure (WBS) location. It would also offer a detailed analysis of the risk associated with the modeling work remaining to achieve a certain MMI level within the same discipline and across other related modeling disciplines. The Toolkit could be used externally with clients as part of any model review session to communicate modeling progress, or internally within the engineering team to assess actual modeling progress against client expectations or the project schedule.

A ModelXP Addendum is also needed to bring transparency to modeling by outlining the modeling process and assuring that all project stakeholders are aware of their responsibilities for achieving certain maturity levels for each modeling discipline, per model review session, or by another project milestone. By connecting the modeling expectations to the MMI levels, a project team can compare actual progress against expected progress. The expected progress per project-specific project milestone should be outlined in a ModelXP Addendum.

The MMI Definitions provided with this report will work together with the Model MRI Toolkit and ModelXP Addendum that users will create in their modeling software. When aligned with each other, these three tools establish procedures and define metrics by which project stakeholders can reliably measure progress and productivity in a model-driven engineering process. While the resources outlined in this research are adoptable and adaptable for different types of projects and applications across the process industrial sector, they can also be extended for use by other construction industries, such as commercial or industrial buildings. The framework created as part of this research has great potential to automate progress measurement in the modeling environment. This potential for automation has been demonstrated via a real-world project example using industry modeling solutions, and is also being explored as part of ongoing research.