greek english
intermek-logo

ThermaBot

Robotic Inspection of Marble Slabs
using Active Thermography

Operational Program "Competitiveness, Entrepreneurship and Innovation (EPAnEK)", NSRF 2014 – 2020 Managing Authority of EPAnEK and Managing Authority of ETAK National Energy Action "Research - Create - Innovate B' Cycle, priority sector 8-TPE, region 8.4 Robotics" project code Τ2ΕΔΚ-00238

The Project.

Scope and Objectives of the Project

The scope of the project is industrial research for the automation of the resin process that all marble slabs produced undergo, to obtain a glossy surface and mechanical strength. The automation will be implemented using a robotic arm guided by machine vision software that will automatically detect surface cracks and discontinuities. The vision system will take images of the plate from a thermal camera as the plate will be artificially heated (active thermography).

Resinification is a standard process in the marble processing industry. The marble blocks as they come from the quarry, are cut into slabs, washed and then subjected to resin. During this process, epoxy resin is applied to the surface of the marble slab and penetrates pores and cracks, on the one hand in order to give a smooth and polished surface, and on the other hand, in order to enhance the mechanical strength of the slab, especially where there are cracks and discontinuities.

These points are the most susceptible to breakage, due to notch stresses. At these points, after the first layer and the treatment of the plate in a vacuum chamber, a second layer is applied, as the resin has already penetrated into the cracks and gaps. In other words, only these sensitive areas need a second spread.

To date, the resinification process is carried out, almost exclusively, manually by skilled workers in both stages. This presents problems both in the final polishing quality and safety issues for workers from inhalation of chemicals. Robotic arms are available from foreign companies programmed to spray the first layer of resin across the plate.

However, there is no effective system available on the global market to detect cracks and sensitive areas that need a second coating. The aim of the project is the automatic detection of exactly these sensitive areas (cracks & gaps), so that both stages of the coating are fully carried out by a robotic arm.

After a scientific literature search and review of technological proposals for detecting cracks and gaps in marble slabs (e.g. by ultrasound, or simple artificial vision), the research team decided to use active thermography for this purpose. On the one hand, a fairly extensive bibliography is available for this technology and it is a method that easily gives an overall picture of the cracks of the entire plate, compared to e.g. ultrasound that gives point results and requires 2D scanning of the plate.

thermabot-photo1 thermabot-photo2 thermabot-photo3 thermabot-photo4

Τimetable.

Work Packages and Deliverables

WP1. Support

Actions: Support actions include a series of actions aimed at the smooth preparation and implementation of the research project, taking into account the most up-to- date data and data. These actions ensure coordination, management, and better visibility of the project.

  • 1.1 Technology Review

    In this context, emphasis is placed on updating technological knowledge through an exhaustive overview of available products, technological solutions, as well as current scientific research relevant to the purpose of the project.

  • 1.2 Market Trends

    The research focuses on recording current market trends for marble products. Opportunities and growth forecasts based on applications and geographies are examined.

  • 1.3 Development of Research Methodology

    The approach of the research process is defined, identifying the basic parameters, objectives and expected results, so that there is a clear direction and effectiveness in research activity.

  • 1.4 Project Management

    During the implementation of the project, the continuous, effective and coordinated management of all activities will be ensured, ensuring the cooperation and harmonized operation of the different departments and bodies involved.

  • 1.5 Publicity

    Outreach and dissemination actions of the project results are crucial for the recognition and wider understanding of the importance and achievements of the project to the wider public.

thermabot-photo6

WP2. Laboratory Research on Thermography

Thermography is a critical process in detecting surface cracks and discontinuities, mainly in materials such as marble. This research aims to improve and automate this process.

  • 2.1 Process Recording

    The aim is to record the resin process of different types of marble, with particular emphasis on Greek, such as the marbles of Drama, Kavala and Thassos. With the use of modern equipment, such as cameras, information will emerge to automate the process.

  • 2.2 Laboratory mapping of discontinuities

    The marble slabs will be studied with various techniques, destructive or not, in order to identify discontinuous areas that require resin interventions.

  • 2.3 Laboratory application of thermography

    Analysis of heating and environmental conditions is crucial for effective crack detection through thermography.

  • 2.4 Thermography software development

    Based on the conclusions, software will be developed that will improve the analysis of thermographic images, in order to automate the process.

  • 2.5 Resin head design

    It aims to optimize the resin spraying process by integrating it into a robotic system.

  • 2.6 Experimental Layout Study

    The design research of the laboratory will focus on the creation of an integrated system that will include all the necessary tools and technologies to automate the thermography process.

In general, laboratory research aims to improve methods of detection and repair of marble surfaces through the automation of the thermography process.

thermabot-photo7

WP3. Artificial Vision System for Detecting Discontinuities in Marble Slabs

Within the framework of the proposed system, an innovative solution will be developed for the automated detection and mapping of cracks and discontinuities in marble slabs.

  • 3.1 Development of theoretical model

    The investigation of the problem will focus on two main directions:

    • Active thermography images: Through the thermal camera, images of the marble slabs will be extracted, in order to analyze their thermal footprint with the help of artificial vision and machine learning.
    • Images in the visible spectrum: These images will provide a detailed depiction of the cracks, allowing an accurate estimation of their dimensions.
  • 3.2 Algorithm Development

    At this stage, two algorithms will be developed:

    • For thermography images: This algorithm will focus on the processing of thermographic images, by applying edge detection, image threshold and segmentation methods.
    • For images in the visible spectrum: This algorithm will include methods of description of texture and texture analysis to accurately identify cracks.
  • 3.3 Software Implementation

    The algorithm will be implemented in Python, taking advantage of the capabilities of OpenCV, TensorFlow, etc. libraries for image processing and training of artificial intelligence models.

  • 3.4 Evaluation and Optimization

    After development, the system will be tested for accuracy, sensitivity, and specificity in crack detection, as well as processing speed. The parameters will be optimized in order to increase efficiency.

The proposed system is expected to offer significant benefits to the marble processing industry, as it will allow fast and accurate identification of discontinuities, thus contributing to increasing product quality and reducing control costs.

Industrial Application.


Development of other subsystems, their integration into an industry standard and the execution of experiments-tests. Subsystem improvements and re- testing. Validation of technologies and verification of functionality. Application of the use of the system under development, for demonstration purposes. Reliability check for industrial use. The EU includes the following activities:

  • Guidance software development: The software will receive 2D spatial coordinate data from the above artificial vision software and create trajectories of motion for the robotic arm in its programming language.

  • Resinator head development: Construction of a resin spraying system that will be placed at the end of the arm for robotic resination. It will have an electronically adjustable spray beam, controlled by the above guidance software.

  • Experimental Setup: Construction of a laboratory device that will include (a) a roller mixer with electric motor and inverter for movement of marble slabs 65X40 cm, (b) support frame and transverse movement with servo motor system, the thermal camera, (c) curtains, air heating system of the marble slab and appropriate lighting for taking thermographic images.

  • Arm Programming: Development of the hardware and software for connecting optical vision with the robotic arm for guidance, interoperability and synchronization and with the programmable drive belt.

  • Systems Ιntegration: It includes the integration of the above systems and the integration at laboratory level, of the automation of mechanical thermographic vision and the robotic control of an industrial arm with the electronic resinator. The overall unified layout will be installed in the laboratory areas of the TEI of Thessaloniki ready for final demonstration of its functions and capabilities.

  • Laboratory Tests: Design and conduct a series of experiments and tests of automatic resination. Detailed recording and classification of results. Reliability testing for industrial use and upgrading TRL (Technology Readiness Level, according to NASA & Horizon 2020) to level 6 (System/subsystem model or prototype demonstration in a relevant environment). Development of training tools.

  • Results - Conclusions: Elaboration and presentation of the results obtained from the project. Processing and scientific – technological documentation and analysis of industrial research tests. Recording, presentation and analysis of functions that need further optimization.

  • Technical Implementation Study: Study of the analysis of the required resources, processes and technical requirements, in order for the results of industrial research to be applied on a production scale and to produce a new product, commercially available in the near future.

thermabot-photo5

Results - Articles.

Towards Robotic Marble Resin Application: Crack Detection on Marble Using Deep Learning

Eleni Vrochidou, George K. Sidiropoulos, Athanasios G. Ouzounis, Anastasia Lampoglou, Ioannis Tsimperidis, George A. Papakostas, Ilias T. Sarafis, Vassilis Kalpakis, and Andreas Stamkos

Cracks can occur on different surfaces such as buildings, roads, aircrafts, etc. The manual inspection of cracks is time-consuming and prone to human error. Machine vision has been used for decades...

READ ARTICLE

RGB and Thermal Image Analysis for Marble Crack Detection with Deep Learning

Eleni Vrochidou, George K. Sidiropoulos, Athanasios G. Ouzounis, Ioannis Tsimperidis, Ilias T. Sarafis, Vassilis Kalpakis, Andreas Stamkos and George A. Papakostas1

Surface damage identification implies visual inspection, which is tra- ditionally performed manually with the bare eye. The latter is a time-consuming and error-prone process. Effectively automated...

READ ARTICLE

Fusion of Thermal and RGB Images for Automated Deep Learning Based Marble Crack Detection

Eleni Vrochidou, George K. Sidiropoulos, Ioannis Tsimperidis, Athanasios G. Ouzounis, Ilias T. Sarafis, Vassilis Kalpakis, Andreas Stamkos and George A. Papakostas

Research is constantly turning towards the development of image-based inspection tools that leverage deep learning models to automate surface crack detection in materials. Yet, most efforts involve color...

READ ARTICLE

Contact.

thermabot@intermek.gr +30 251 039 2288 +30 251 039 2488
  • Amisiana, Municipality of Paggaio, 64100

espa