Robust detection of hidden material damages

Overview

Machine Learning (ML) techniques are widely used in Structural Health Monitoring (SHM) and Non-destructive Testing (NDT), but the learning process, the learned models, and the prediction consistency are poorly understood.

This work investigates and compares a wide range of ML models and algorithms for the detection of hidden damages in materials monitored using low-cost strain sensors.

The investigation is performed using a multi-domain simulator imposing a tight coupling of physical and sensor network simulation in the real-time scale. The device under test is approximated by using a mass-spring network and a multi-body physics solver.

Introduction

Sensorial Materials

A sensorial material poses tight coupling of structure, sensors, data processing, communication, and energy supply, integrated in a host material ⇒ Acquisition of the state of the structure material

Sensorial materials extend structural materials with the following functions:

1. Load Monitoring
2. Structural Health Monitoring
3. Non-destructive Testing

Machine Learning can be utilised to detect damages and to acquire the state of the material.

Test and Prediction

Damage diagnostic and prediction is the outcome of testing:

Non-destructive Testing (NDT)

Semi-automated or manual detection of damages and relevant changes of materials and structures

Structural Health Monitoring (SHM)

Automated recording of the state of a technical structure or device at run-time (including damages, but not limited to)

• NDT, SHM, and the prediction of damages is still a challenge even in conventional monolithic materials

• New materials and hybrid materials, e.g., fibre-metal laminates, are subject to hidden damages without externally visible change of the material

• Well established measuring techniques are ultra-sonic monitoring and computer tomography

  • Both techniques suffer from their high instrumental effort and difficulties in diagnostic robustness

Test and Prediction

External monitoring of internal damages of such materials and structures with simple and low-cost external sensors, e.g., strain-gauge sensors, under run-time conditions is of high interest.

• Different parameters and constraints have influence on the test result, accuracy, and its probability of trust:

  • Noise (external, internal)
  • Sensor failure
  • Sensor node failure in a sensor network
  • Change of material and structure properties at run-time
  • Gap between base-line and run-time conditions
  • Inaccurate models

Machine Learning

Models and Algorithms

Models and algorithms have to be distinguished. Models are functions, graphs, trees, and tables. Algorithms perform training, testing, and classification (i.e., prediction).

The following learning algorithms and models were used for damage prediction:

• Classical decision tree learner (C45)
• Advanced decision tree learner with interval arithmetic and nearest-neighbour approximation (ICE)
• Random forest tree learner (RF)
• Single layer perceptrons (one layered artificial neural network, SLP)
• Multi layer perceptrons (deep learning with hidden layers, MLP)
• Multi-label Support Vector Machines (SVM)

Architectures

Machine learning aims to find a model function M that maps an input vector x on an output vector y:

Machine learning is divided in three phases:

1. Learning of a model function M with training sensor data (labelled with target variable values commonly by an expert in case of supervised learning)
2. Testing of the computed model function with test data to estimate the model accuracy (using training data and additional labelled data sets)
3. Application of M to new sensor data

Architectures

There are two main classes of sensor data and learning strategies that can be used:

1. One spatially distributed data set D(t) sampled at a specific time t (or averaged in a time interval) → Global Learning with one instance

2. A set of time-resolved sensor data d(p) at a specific spatial position p → Local Learning with multiple instances and global fusion

Architectures

Learning and Noise

• Noise (including sensor failures) has a high impact on the model function M and its prediction accuracy

• Traditional learners like decision trees do not address noisy sensor data

• To cope with noisy sensor data, a new decision tree algorithm ICE is introduced, derived from classical ID3/C45 decision tree learners

• Instead using sensor variables directly, each sensor variable x_i is transformed to an interval variable with a noise margin ε, i.e., x_i → [x_i-ε_i,x_i+ε_i]

• This noise margin and interval arithmetic used by the decision tree learner improves the model quality and prediction accuracy significantly

Multi-domain Simulation

In this work, a multi-domain simulation tool is used to compare and evaluate different ML algorithms and models.

Domains

To enable the physical simulation of mechanical structures and the response of sensor networks on dynamic changes of the structure two relevant domains and models have to be coupled tightly:

Physical Model

Multi-body physics (MBP) using the CANNON physics engine to solve dynamic equations of mass-spring systems modelling a mechanical structure

Computational Model

Multi-agent systems and sensor networks using the JAM agent platform to implement centralised and decentralised sensor processing and damage prediction

Physical Model

Multi-body Physics

• Traditionally the mechanic behaviour of structures is computed by using Finite Element Methods (FEM)

• FEM poses high computation times

• To enable fast and real-time simulation of arbitrary shaped structures a simplified Multi-body physics (MBP) approach and Multi-body simulation (MBS) are used in this work

• A MBP model consists of a set of bodies (rigid or elastic) and a set of connections between the bodies

• Forces between bodies and friction is considered in MBS

Mass-Spring Systems

• Elastic materials are modelled by a set of rigid masses M connected by a set of springs Sp, creating a mass-spring graph network St=<M,Sp>

• Each mass node is connected with up to 12 neighbour nodes

Physical Model

Sensor Model

Sensor

• There is a set of pairs of orthogonally orientated strain-gauge sensors S applied virtually to the surface of the DUT

Sensor Node

• The sensors are attached to a set of sensor nodes N performing sensor signal pre-processing, data processing, and communication

Sensor Network

• The sensor nodes are connected in a sensor network SN providing 2D grid connectivity for communication
  • The sensor network is the organisational structure of the damage prediction system

Synthetic Sensor Data

• Sensor data is computed directly from MBS (via displacement of mass nodes) → Synthetic sensor data used for ML!

Computational Model

• Data processing is performed by agents (in simulation as well as in a real technical system)

• An agent is composed of a set of activities, e.g., sensor acquisition, sensor processing, data distribution, and implementing local learning

• Each sensor node is deployed with a node and learner agent

• There is a global world instance with an agent performing simulation control and implementing global learning

• Agents are processed by the JAM agent processing platform

SEJAM Framework

SEJAM Framework

SEJAM Framework

Live Demo

Experiments and Evaluation

Experimental Setup

• Experiments were performed with a simple plate (Device under Test, DUT) consisting of a homogeneous elastic material

  • The plate was modelled with 14 × 8 × 3 mass nodes connected with springs (elastic model)
  • The plate was fixed at two sides

• The sensor network virtually applied to the surface of the DUT consists of 4 × 3 sensor nodes with each node connected to one strain-gauge sensor pair

• Artificial defects inside the material can be added to the plate by removing mass nodes (nine different positions)

• Additional loads can be applied to the surface of the DUT to add disturbance

• Monte Carlo method is used to add random noise to sensor data and material properties

• Training data is derived from artificial sensor data

Experimental Setup

Experiments

Two different learning domain strategies were investigated:

1. Global learning with one learner instance (see Fig. 1 a) using a sensor snapshot (after the DUT dynamics is below a threshold)

2. Local learning with 12 node-based learner instances and global fusion (see Fig. 1 b) using time records of the sensor signals (starting with the begin of DUT dynamics)

Results

Global Learning

• With 12 sensor nodes and each sensor node connected with a pair of strain-gauge sensors, there are 24 input variables.

• The neural networks consist of 24 input neurons (12 sensor pair feature variables) and 10 output neurons (nine damage locations and the undamaged case).

• The multi-label SVM consists of 10 parallel binary classifier SVMs (one for each damage feature).

Decision Tree learner create the smallest models (by means of data size).

The ICE learner is the fastest learner with high prediction accuracy.

ANN (SLP, MLP) and SVM are the slowest learner

Hidden layer (MLP) have no advantage over single layer networks (SLP)

Results

Comparison of different learned models (200 data sets of all sensors; global learning) showing learning time and model data size.

Results

Results

Local Learning

• In contrast to the previously evaluated global learning approach using sensor snapshots of all spatially distributed sensors at a specific time, the local learning approach uses time-resolved records of the sensor signals with a given capture window (32 samples).

• Furthermore, each sensor node implements a single learner instance learning local models that are applied to local sensor data only. The input data vector x consists of 64 variables.

• There are only two target labels for each learning instance that have to be classified:

  • DAMAGE: Meaning a damage was detected within a region around the sensor node (radius of 1.5 sensor distance lengths)
  • NODAMAGE: Meaning no damage within this region was detected. In the evaluation, the number of “No damage” cases for each node is much higher than the number of “Damage near by” cases (naturally).

Results

Decentralised local learning can compete with global single-instance learning

Although the single instance prediction accuracy can be low, the global majority fusion results in an always high prediction accuracy of a damage

Disturbances like sensor failure or additional loads changing the damage diagnostic base-line have low effect on prediction accuracy ⇒ High robustness of the damage detection system

Results

Summary

• A multi-domain simulation framework realizing tight coupling of physical simulation of mechanical structures and computational simulation of sensor networks was used to investigate different ML approaches and algorithms used for the prediction of hidden damages.

• It could be shown that simple decision tree models are generally suitable for damage prediction.

• A new advanced decision tree learner ICE was introduced. This learner takes noise of sensor data into consideration leading to improved models and prediction accuracy.

• Additionally, this new learner outperforms classical decision tree learners and neural networks regarding learning time (in milliseconds) and model data size.

• The comparison of single and multi-layer neural networks (deep learning) poses no significant advantage over multi-layer networks with respect to prediction accuracy.

• All learners can be applied in the sensor spatial and time-domain, and in centralised single-instance and multiple-instance decentralised learning architectures.

• Although SVMs require the highest learning time, they outperform all other algorithms in multiple-instance decentralised learning architectures using time records of sensor signals.

References

1. S. Bosse, D. Lehmhus, Robust detection of hidden material damages using low-cost external sensors and Machine Learning, Proceedings of the ECSA 2019, MDPI
2. S. Bosse, D. Lehmhus, Material-integrated cluster computing in self-adaptive robotic materials using mobile multi-agent systems, Cluster Computing, doi 10.1007/s10586-018-02894-x, Volume 22, Number 3, pp. 1017-1037, 2019 ISSN 1386-7857
3. S. Bosse, A. Lechleiter, A hybrid approach for Structural Monitoring with self-organizing multi-agent systems and inverse numerical methods in material-embedded sensor networks, Mechatronics, (2016), DOI:10.1016/j.mechatronics.2015.08.005,