Research Article - (2024) Volume 12, Issue 1
Received: Jun 07, 2024, Manuscript No. IJCSMA-24-138455; Editor assigned: Jun 09, 2024, Pre QC No. IJCSMA-24-138455 (PQ); Reviewed: Jun 12, 2024, QC No. IJCSMA-24-138455 (Q); Revised: Jun 14, 2024, Manuscript No. IJCSMA-24-138455 (R); Published: Jun 22, 2024
Measuring sound pressure is a complex and tedious task in industrial environments. Classical methods can be costly, inaccurate, and potentially dangerous for workers. Integrating robotics and computer vision into acoustic measurement processes offers an innovative solution to these problems. This paper uses a Doosan Robotic Vision System to investigate an innovative automatic acoustic pressure measurement solution. The Doosan robot, equipped with a scanning camera and microphone, captures acoustic data and produces a map of the machine's acoustic radiation using computer vision algorithms. The measurement technique is accurate, efficient, and safe for workers. It can also be used for continuous monitoring of the industrial acoustic environment. The developed measurement method offers a promising way to automate acoustic measurement, which can be performed faster and with better accuracy to help industries improve their product sound quality and reduce noise levels. The method was compared to analytical predictions. A 3D acoustic mapping was conducted to visualize the spatial distribution of sound sources within the environment. The experimental results show a good correlation with the theory.
Sound pressure measurement; Robotics; Acoustic data; Scanning camera; Mapping; Noise reduction
Over the years, several research and measurement techniques have been proposed to measure and localize noise sources. A summary of the methods can be found in Refs [1].
Jung and Ih introduced a comprehensive microphone array approach to locate sound sources through acoustic intensimetry accurately [2]. Their study concentrated on creating Three-Dimensional (3D) intensimetry techniques for precise localization of compact sound sources. The analysis emphasized addressing spatial bias errors arising from the irregularity of directivity, leading to the investigation of potential compensation methods.
Botteldooren et al. conducted industrial sound source localization using microphone arrays under difficult meteorological conditions [3]. Their investigation highlighted the need for long-term monitoring of industrial noise. The study identified difficult meteorological conditions, characterized by highly variable sound attenuation, as significant challenges.
Cherif and Atalla developed a hybrid method to measure the radiation efficiency of complex structures across a wide frequency range [4]. This technique combines direct measurements in an anechoic room with an indirect approach utilizing Statistical Energy Analysis (SEA) in a reverberant room. Validation was performed on two types of constructions: metallic (flat and stiffened aluminum panels) and sandwich composite (honeycomb core). Experimental results were compared with analytical predictions based on Leppington's asymptotic formulas. Their studies demonstrated that the technique accurately identifies radiation efficiency and distinguishes between intrinsic and radiation damping.
Simmons C. conducted a comprehensive study focused on the measurement of sound pressure levels at low frequencies within the context of room acoustics [5]. This research involved a meticulous comparison of various methods documented in a literature survey. The primary objective was to evaluate the performance of these methods, particularly in scenarios dominated by low-frequency sounds or when measured in octave or third-octave bands.
Other studies have delved into various aspects of sound pressure measurement, including measurement of sound pressure level in voice and speech (Jan G and Svante). The research contributes to improving the accuracy and reproducibility of sound pressure level measurements in the domain of voice and speech studies, while (Yunus, M. et al.) propose an advanced method for using noise measurement as an effective tool for analysis before or after noise control efforts for industries using mechanical machinery [6,7].
These studies provide valuable insights into different facets of sound pressure measurement. However, they often rely upon arrays of multiple microphones, which have come with significant limitations, such as the necessity of intricate calibration procedures, raised costs, and extended time required for data collection, occasionally hampering real-time noise source identification. To overcome the limitations of traditional methods, there has been a growing interest in using automation and robotics for acoustic pressure measurements. Several recent studies have explored using robotic systems in various settings.
For instance, Wang et al. developed an automated system that can measure the sound power level of HVAC systems. Meanwhile, Yang et al. developed a robotic platform for sound power and intensity measurements in open spaces. Nonetheless, these methods still require the intervention of an operator to define the accurate calibration for best mapping, which can lead to a semi-automatic measurement process [8, 9].
M. Blazek et al. present a case study of sound mapping using a scanning microphone system in an urban environment. The authors used a scanning microphone system mounted on a vehicle to measure sound levels along a predefined route through the city [10].
C. Mansouri et al. present an experimental investigation of sound power mapping of a vibrating machine using a scanning microphone and a robot [11]. The researchers used a Hemisphere method to measure the sound pressure at different points on the surface of a hemispherical array surrounding the machine. The scanning microphone and the robot were used to automate the measurement process and obtain a fully automatic scan of the machine. The results showed that the proposed method could accurately map the sound power of the vibrating machine, which could be helpful in industries to identify and mitigate noise sources.
In the mentioned papers, the author’s present varied experimental methods to measure sound power levels using robotic systems in different settings. These methodologies rely on pre-programmed instructions guiding the robots to execute repetitive tasks. Notably, none of these methods have yet integrated AI for autonomous measurements.
This paper aims to investigate and improve the acoustic pressure measurement in vibrating machines using a Doosan robot. The objective is to automate the measurement process, enabling a fully automated scan using a Doosan Robotic Vision System and a single microphone, allowing precise source localization.
This section briefly overviews sound pressure and sound pressure level measurement techniques, focusing on the fundamental principles of sound energy and its distribution. It also explores the impact of Artificial Intelligence (AI) on acoustic measurement.
Overview of Approaches
Sound or acoustic pressure is the local deviation from the ambient atmospheric pressure caused by a sound wave. In air, sound pressure can be measured using a microphone. The SI unit for sound pressure p is the Pascal (Pa). Sound Pressure Level (SPL) or sound level is a logarithmic assessment of the effective sound pressure in relation to a reference value. Expressed in decibels (dB) above a standard reference level, it utilizes 20 μPa as the standard reference for sound pressure in air or other gases. This value is commonly acknowledged as the threshold of human hearing, particularly at 1 kHz. The subsequent equation outlines the process of calculating the Sound Pressure Level Lp in decibels [dB] from the sound pressure (p) in Pascal [Pa]
Where pref is the reference sound pressure and prms is the Root Mean Square (RMS) sound pressure being measured. Most sound level measurements will be made relative to this level, meaning 1 Pascal will equal an SPL of 94 dB.
Sound intensity is defined as sound power per unit of an area. It depends on the distance from the sound source and the acoustic environment in which the sound source is [12].
Sound power is the characteristic of a sound source. It is independent of distance and is a practical way of comparing various sound sources. Sound power can be measured in different ways by sound pressure or by sound intensity:
Where W is the sound power [W] and [W0]is the reference sound power [W].
The fixed-array microphone method is less stringent regarding the test environment and measurement setup requirements. This method supports all noise sources (continuous, intermittent, fluctuating, isolated bursts) and does not limit source size, provided the measurement conditions are met. Measurements can be taken indoors or outdoors, with one or more sound reflection planes present. Additional procedures are suggested for applying corrections if the environment is less than ideal.
The standard outlines various arrangements of microphones around a sound source. The measurement radius, r, must satisfy several conditions for accurate assessment. Firstly, it should be at least 2d0 or 3ℎ0, whichever is greater, where d0 represents the characteristic dimension of the tested noise source, and ℎ0 is the distance between the acoustic center of the source and the ground. Secondly, r should be greater than or equal to λ/4, where λ denotes the wavelength of the sound at the lowest frequency of interest. Finally, the radius must be at least 1 meter. These conditions ensure precise measurements of sound characteristics [13].
Integrating automated acoustic pressure measurement using robots like Doosan cobots involves a combination of hardware and software approaches. The hardware component includes a scanning camera and microphone, which capture acoustic environmental signals. These signals are then processed and analyzed using algorithms that extract meaningful information about the acoustic pressure of the sound source. Software algorithms for processing and analyzing captured acoustic signals can employ different techniques such as Fast Fourier Transform (FFT) and Short-Time Fourier Transform (STFT) for frequency domain analysis, Spectral Subtraction for noise reduction, and Gaussian Mixture Models (GMM) for clustering and classification in audio processing [14].
AI and Acoustic Measurements
As technology advances, AI has become an integral component in enhancing the efficiency and accuracy of sound-related analyses. The exploration will focus on how AI algorithms, particularly in conjunction with robotics and computer vision, contribute to automating and optimizing the measurement processes.
AI plays a crucial role not only in detecting specific objects, but also in calculating their position relative to the microphone. It enables the robot to position itself strategically, guided by the information obtained from the camera's detection of objects. This information is essential for the following steps. The camera first detects the sound source object, calculating and obtaining its precise position relative to the microphone. After successful detection, the measurement process is started offline using the Robot DK software. Once the process has been validated, the robot seamlessly switches to the automatic scanning phase to take acoustic measurements, using the microphone mounted on its robotic arm.
This integration solves the complex problems associated with traditional methods and opens up new possibilities for real-time monitoring and a complete understanding of the acoustic environment in industrial settings.
This section describes the measurement process of the sound pressure using a Doosan robot, a scanning camera, and microphone. The analysis is performed in a third-octave frequency band starting from 10 Hz to 3.2 kHz. Comparisons between experimental and theoretical results.
The measurement process involves several steps: first, acquiring and processing images, then detecting, and localizing the sound source within the OnRobot's Eye tool system. Afterward, the robot arm, equipped with a microphone, is moved, followed by the initiation of the automatic scan.
Doosan Robotics' collaborative robots, also known as cobots, are highly regarded for their precision, versatility, and ease of use, making them ideal solutions for various industrial applications. In the context of this study, Figure 1 provides a visual representation of the built-in processor within the Doosan collaborative robot [15].
Figure 1: A Built-in Processor in the Doosan Collaborative Robot [15].
This illustration offers insights into the technological capabilities of the robot used, emphasizing the crucial role played by its integrated processor in ensuring optimal performance.
On Robot's Eye tool is an intelligent camera system specifically designed for robotic applications. It employs advanced computer vision algorithms to detect objects accurately in real-time, making it an optimal tool for locating objects and calculating the distance between the source and the microphone.
The Eye tool includes a built-in processor that runs On Robot's proprietary software, allowing it to perform real-time object detection, classification, and tracking. Additionally, with the measuring microphone connected to the Doosan robot and the Eye tool, it is possible to measure the distance between the detected object and the robot. Then, based on the object's location, orientation, and distance measurement, the robot's parameters can be adjusted automatically to perform more precise scans.
With these technological advancements, the Doosan robot offers a promising way to automate sound measurement.
Devices Calibration
Microphone Configuration and Calibration: Initiating precise data collection with BK Connect software involves a crucial starting point: configuring the microphone. This step is pivotal for accuracy. Key parameters such as sampling frequency and microphone sensitivity need validation and other adjustments to optimize microphone performance. The TYPE 4189-A-021 microphone is employed for measurements, featuring a dynamic range of 16.5 dB – 134 dB and a frequency range of 20 Hz – 20000 Hz. With an inherent noise level of 16.5 dBA, its sensitivity is 50 mV/Pa. The microphone operates within a temperature range of -20℃–80℃. Subsequently, meticulous planning is vital to define measurement positions around the vibrating source, where acoustic pressure data will be captured [16]. In this context, the process of microphone calibration takes on paramount importance. Ensuring the calibration is accurate is key to getting precise acoustic data. During calibration, critical parameters like sampling frequency and microphone sensitivity are rigorously validated and adjusted as necessary.
Camera Calibration: Calibration is an essential step in the setup process of the On Robot Eye vision system. Its purpose is to ensure that the camera is accurately aligned with the robot arm and that the image captured by the camera precisely reflects the real-world environment. The calibration process, depicted in requires placing a calibration object in front of the camera and capturing images from various angles using the On Robot Eye software Figure 2.
Figure 2: External Camera Calibration.
The software then employs these images to determine the position and orientation of the camera relative to the robot arm. Once the calibration is complete, the On Robot Eye can detect and locate objects within the robot's workspace with unparalleled accuracy, enabling it to execute complex manipulation tasks with exceptional precision and efficiency.
Communication between Devices
Effective communication between devices is a crucial aspect of this project, as it enables the coordination and synchronization of the various stages of data acquisition, processing, and analysis. The Transmission Control Protocol (TCP) plays a vital role in ensuring the reliability of end-to-end connections, significantly simplifying their establishment.
The compatibility of both systems with TCP/IP protocols facilitates seamless communication between the computer and the robot. This standardized communication protocol ensures efficient data exchange. The connection is established through a direct Ethernet cable link, forming a point-to-point connection. To ensure the proper functioning of this direct connection, it was imperative to configure the IP addresses to align them within the same network domain. An overview of the entire system, illustrating the fundamental functional scheme of the different interconnected devices used, is shown in Figure 3.
Figure 3: Communication between Devices within the System.
Object Detection and Positioning Strategy
Related work: Object recognition and detection has been the subject of intense research for several decades. Initially, these techniques were based on descriptors such as HOG and LBP, combined with classifiers. However, recent advances in recognition algorithms have led to significant adoption in recent years. Among these advances, Convolutional Neural Networks (CNNs) have established themselves as the main reference for image recognition and visual classification [17]. CNNs are distinguished by their multi-layer architecture, where each layer learns specific characteristics. They have been hailed for their outstanding performance and efficiency in generic object detection and classification tasks. Recent research has focused on improving deep learning methods to create more robust models. In addition, current approaches exploit unsupervised convolutional coding to reapply filtering techniques at each step, further improving detection accuracy [18]. A new detection approach called You Only Look Once (YOLO) was introduced in research by Redmon et al. It has contributed significantly to the advancement of real-time applications due to its fast execution [19]. You Only Look Once (YOLO) proposes using an end-to-end neural network that predicts bounding boxes and
class probabilities all at once. This approach differs from the one taken by previous object detection algorithms, which reused classifiers for detection. While algorithms like Faster RCNN detect possible regions of interest using the region proposal network and then perform recognition on these regions separately, YOLO makes all its predictions using a single fully connected layer.
Sound Source Detection and Positioning Strategy: The robotic system's functionality for object detection and positioning through the camera involves a series of well-defined steps. The high-resolution camera embedded in the system is designed to meticulously capture detailed images of the object or scene being examined. These images serve as the foundation for the subsequent stages of the process. For our detection needs, we opted to utilize the YOLOv5 version of the You Only Look Once (YOLO) algorithm, which stands out as one of the most potent solutions presently accessible. To accomplish this, we employed a YOLOv5 model built upon an open-source algorithm, which we further trained and tailored by fine-tuning specific parameters to align with the requirements of speaker detection within the images. The algorithm relies on Convolutional Neural Networks (CNNs) to predict both the class and the location of a sound source in an image. To train the model, a database consisting of images representing different types of microphones was used to develop the learning model. The dataset was divided into an 80:20 ratio, with 80% allocated for training and the remaining 20% reserved for validation. This process involves identifying objects in the images and marking them for the model training process. Subsequently, the model utilizes this annotated data to detect the specified objects in the images. Figure 4 illustrates the detection of speakers as sound sources.
Figure 4: Detection of the Sound Source.
Following this phase, the system enters the decision-making stage. This pivotal step involves real-time adjustments and strategic positioning strategies based on the acquired visual information. The robotic system autonomously assesses the object's characteristics, evaluates the optimal positioning for accurate measurements, considering the distance between the source and the microphone and dynamically adjusts its position as necessary ensuring compliance with the measurement conditions of the hemisphere method.
Based on this information, the robot can be programmed to start the automatic scan. This integration of AI enhances the overall efficiency and accuracy of the source detection and positioning strategy, as exemplified in Figure 5.
Figure 5: Source Detection and Positioning Strategy.
This adaptive process ensures that the robot is precisely positioned for optimal sound pressure measurements guaranteeing optimal mapping outcomes tailored to specific frequency bands.
Python for Automating Sound Measurement and Mapping
Automatic Scanning: Once the machine learning algorithm classifies the objects and determines the distance between the source and the measuring microphone, the next step involves programming the robot to measure the sound pressure at different points within the space. This can be achieved using Python programming language, which can be used to control the robot's movement and the microphone's positioning. The robot can be instructed to move and scan in different locations within the space to measure sound pressure at various points. At each location, the microphone can take sound pressure measurements, which can be used to generate an acoustic map using the Hemisphere method. Figure 6 visually depicts the solution for the automatic measurement using the cobot. The algorithm manages data collection and orchestrates the robot's optimal path through the space, a critical aspect in ensuring thorough and efficient coverage of the investigation area.
Figure 6: Doosan Robot Equipped with a Microphone that follows a Well-Defined Trajectory.
Collaboration with RobotDk proves pivotal in pursuing precise acoustic pressure measurement and mapping. RobotDk, a leading platform for robotic control and offline trajectory programming, significantly enhances the efficiency and precision of the robot's movements. Utilizing RobotDk's specialized tools enables meticulous offline planning of the robot's
trajectory, ensuring comprehensive coverage of all specified locations with precision.
It facilitates the integration of trajectory planning into the algorithm, resulting in a more reliable operation. The synergy between the algorithm's data collection and analysis capabilities, powered by Python, and RobotDk's trajectory programming expertise promises to deliver a comprehensive and highly effective solution for sound mapping.
Moreover, Figure 7 visually illustrates the offline programming of the spiral path, showcasing the coordinated movement of the robot during the sound measurement process. This graphical representation adds an essential visual dimension to enhance the understanding of the approach.
Figure 7: Off-line Programming of the Doosan Robot.
The LAN-XI Open API module is a powerful tool that allows easy interaction with modules like the TYPE 3160 and offers extensive control options. The LAN-XI web interface provides a user-friendly graphical interface, making it simple for users to configure settings, monitor measurements, and access advanced features [20]. The developed code relies on the LAN-XI Open API to automate data collection. Results are automatically saved for subsequent analysis.
The code operation is designed to coordinate a sequential process of measurements, intelligently synchronized with the robot's movements as it transitions between different positions. Upon triggering the script, the LAN-XI module automatically initiates the first measurement. Upon completion of the initial measurement, the code attentively monitors the
robot's status, patiently waiting for it to reach the next pre-defined position. Once the robot is appropriately positioned, the script autonomously triggers the second measurement, continuing this iterative process until the completion of the predefined measurement sequence. The synchronization between robot movements and measurement initiation ensures a structured and consistent data collection process. It also optimizes the overall process efficiency by minimizing unnecessary waiting times between successive measurements.
Acoustic Map Generating
The initial step of generating an acoustic map using our developed method involves calculating the Sound Pressure Level (SPL) at each measurement point. This calculation relies on the measured sound pressure and the distance between the measuring microphone and the noise source. Once the SPL is calculated ateach point, the data is plotted on a coordinate system to create the acoustic map.
This study employed a Bk Connect acquisition station to measure acoustic pressure, adding an extra layer of accuracy and reliability to the measurement process. The Bk Connect station is specifically designed to deliver highly accurate and precise acoustic pressure measurements, capable of capturing data at high sample rates, allowing for real-time SPL measurements.
The integration of AI algorithms extends to guide the robot's decisions in real time. This holistic approach guarantees a seamless and efficient workflow, where the robot not only detects objects but also actively adapts its positioning strategies for enhanced accuracy. Through the synergy of AI and the robot, the measurement points become highly precise with fixed positions, offering the advantage of reproducibility for future measurements, in contrast to traditional methods. The outcome is a cohesive system that excels in dynamic decision-making and real-time adjustments, culminating in optimal mapping.
For the development of our learning model, the model is trained with Google Colab on a GTX 1080 GPU and 64 GB RAM. Training time is considerably reduced thanks to the high performance of the GPU. The model performs relatively well with a precision of around 0.833, recall is around 0.9, and the F1 score is around 0.86.
After recording the sound, the next step is to process it for acoustic analysis. The process focuses on figuring out the shape of the recorded audio device and capturing the acoustic response automatically. This shape is determined by identifying its outer boundaries using instructions from an interface in the MATLAB program.
The acoustic data acquired constitute a complete set of sound pressure readings recorded at various points in the designated area. Each measurement point provides valuable information on the acoustic characteristics of the environment.
Figure 8 shows four curves that describe changes in sound at different places. The first curve is for the point closest to the sound source, and the others are for places farther away. These curves reveal how sound pressure varies over time and space, showing possible significant changes and patterns.
Figure 8: Acoustic Analysis: Frequency-Dependent Sound Signal Variations.
The process begins by importing the acoustic data into MATLAB for simulation, which can then be utilized to extract relevant features, as shown in Figure 9. The creation of the measurement contour often involves the use of signal processing tools and spectral analysis techniques.
Figure 9: Robot Trajectory with Measurement Points.
Once the measurement contours have been established, acoustic data are added to enrich the sound mapping. This stage involves integrating information gathered by the microphone-equipped robot in specific areas. Linear interpolation was used to estimate sound levels between discrete measurement points, creating a continuous surface representing acoustic variations.
Figure 10 shows the 3D sound map created using linear interpolation on measurement data. This visual provides important information about the distribution of sound levels in the studied environment. The color variations clearly illustrate the acoustic dynamics of the area. Warmer areas indicate higher sound levels, while cooler areas, represented by lighter colors, signify lower sound levels. This visual analysis enables the identification of areas with more intense noise, helping to understand which regions experience stronger sound disturbances.
Figure 10: Acoustic Mapping of Measurement Data.
A comparison table between the developed approach and an alternative method is presented. The developed approach utilizes a single microphone mounted on a robotic arm with a scanning camera, while the alternative method employs a hemisphere approach with multiple microphones positioned around the noise source. Table 1 outlines the key distinctions between these methods, encompassing aspects such as microphone positioning, cost, and ease of use, measurement accuracy, measurement environments, measurement time, data processing, and acoustic radiation visualization.
Table 1. A comparison of the two methods.
Measurement Method with a Single Microphone Mounted on the Robot Arm |
Hemisphere Measurement Method with Multiple Microphones |
|
---|---|---|
Microphone Positioning | Single microphone mounted on the robot arm | Multiple microphones placed on a sphere around the noise source |
Cost | Relatively low cost due to the use of a single microphone |
Higher cost due to the use of multiple microphones. |
Measurement Accuracy | Measurements can be accurate if the microphone is correctly positioned |
Considered the reference method for loudness measurements |
Measurement Environments | Useful for measurements in open environments | Can be used for measurements in complex environments |
Measurement Duration | Can be shorter because only one measurement can be taken at a time |
Can be longer because several measurements must be taken at different points. |
Data Processing | Data can be processed quickly because there are fewer measurement points |
Can take longer because there are several measurement points to process |
The measurement method with a single microphone mounted on the robot arm offers several advantages over the hemisphere measurement method with multiple microphones. From a financial point of view, the developed method provides relatively low cost thanks to using a single microphone. It should be noted, however, that this method can offer superior accuracy. This is particularly true when the microphone is correctly positioned, highlighting the quality of the results obtained with this simplified approach.
Finally, the combination of microphone and camera facilitates integration with other technologies, making it more suitable for complex industrial environments, unlike the previous method, which could encounter integration difficulties due to complex communication protocols. Finally, the developed method stands out for its simplicity, low cost, versatility, and ease of integration with other technologies, making it a promising alternative to the previous method. Still, it is considered the reference method for acoustic measurements.
This paper investigated the measurement of sound pressure in industrial environments, addressing the challenges associated with traditional methods. An innovative and automated solution was explored in response to the complexity, cost, and potential risks to workers posed by classical approaches. The study leveraged the Doosan Robotic Vision System, featuring a scanning camera and microphone, to capture acoustic data. Guided by computer vision algorithms, this system not only facilitated accurate and efficient measurement but also generated a detailed map showcasing the spatial distribution of sound sources within the industrial setting.
The methodology underwent comprehensive evaluation, including comparisons with analytical predictions. Experimental results, complemented by a 3D acoustic mapping, revealed a strong correlation between the measured data and theoretical expectations.
The developed method demonstrated a significant advancement in safety and precision compared to conventional techniques. The automated measurement, enabled by the Doosan robot, ensures worker safety and allows for continuous monitoring of the industrial acoustic environment.
Combining robotics and computer vision, this innovative approach provides a safer and more precise alternative for sound pressure measurement. It holds promise for industries aiming to enhance product sound quality and reduce noise levels. With its automated measurement capabilities, the Doosan Robotic Vision System integration stands out as a recommended method for swiftly acquiring accurate data and contributing to improving industrial acoustic environments.
Based on the Doosan robot and equipped with advanced sensors, the automated measurement system has a wide range of applications in various industrial contexts. It can be used in an industrial production line in the final phase after product manufacture, such as daily-use machines or other equipment. The model enables precise noise control, helping to ensure the sound quality of products before they go to market.
This study's synergy of robotics, computer vision, and IA streamlines the measurement process. It opens up new avenues for continuous monitoring and real-time analysis of acoustic environments.
The authors thank the “Université de Québec à Rimouski (UQAR)” for supporting this study by providing researchfacilities and resources.