Welding Research Lab, Center for Manufacturing, University of Kentucky

 

 

 

 

 

Figure 3-1, System principle diagram

System Components

A typical machine vision system consists of several among the following components [46]:

• One or more digital or analog camera (black-and-white or color) with suitable optics for acquiring images;
• Lighting;
• Camera interface for digitizing images (widely known as a "frame grabber");
• A processor (often a PC or embedded processor, such as a DSP);
• Computer software to process images and to detect relevant features;
• A synchronizing sensor for part detection (often an optical or magnetic sensor) to trigger image acquisition and processing;
• Input/Output hardware (e.g. digital I/O) or communication links (e.g. network connection or RS-232) to report results;
• Some form of actuators used to sort or reject defective parts.

In our proposed machine vision system, most of the components listed above are included. During the welding process, the reflected images captured by the sensing system are sent to a PC to be processed using developed image processing and reconstruction algorithms. After weld pool surface reconstruction, its three-dimensional parameters are extracted and analyzed, which can be further used as a basis to adjust welding current and speed. As can be seen, in fact this is a typical feedback control system, and our research is emphasized on the welding pool surface sensing part and the control part is not included currently. Here some important components of the system are discussed below:

• Structured light Laser

Structured light is the projection of a light pattern (parallel lines, grid, or more complex shape) at a known angle onto an object. This technique can be very useful for imaging and acquiring dimensional information in machine vision applications. For example, when a sheet-of-light intersects with an object, a bright line of light can be seen on the surface of the object. By viewing this line of light from an angle, the observed distortions in the line can be translated into height variations.

In our experimental system, StockerYale's Lasiris? SNF uniform intensity laser projector is chosen. It is especially useful for structured light applications, including machine vision, inspection, and alignment. By using different diffractive lens, different projection patterns can be produced, such as parallel lines, dot matrix, crosshair and single circle etc. In most cases, scanning the object with the light can construct 3D information about the shape of the object. This is the basic principle behind depth perception for 3D machine vision. In our application, there are some special characteristics. The observed object is a dynamic weld pool rather than a solid object, and the reflection of structured light is observed instead of direct viewing the structured light on the object. To measure the three-dimensional weld pool surface more accurately by using reflection law, the dot-matrix projection pattern is preferred in the designed system.

• Imaging plane

The objective of the imaging plane used in the designed system is to intercept the reflected laser light and avoid strong welding arc directly entering camera to disturb observation. The most important is that it provides a plane to show reflected image of projected laser pattern, which can be recorded by a camera.

In Figure 3-1, it can be seen that the laser light reflected from weld pool surface is projected to the back side of the imaging plane and a high-speed camera is used to capture the laser light on its front side. In order to ensure the visibility of the reflected laser pattern from the front side, a piece of 4”-by-4” square glass attached with a grid paper is designed as the imaging plane in the system. The paper on the imaging plane is marked with coordinate axes (system), which can be used to localize the position of the reflected laser light for the precise measurement. Actually there are more reasons of using glass as an imaging plane in addition to its transparency. One is the glass can prevent the paper from burning by the high-temperature of welding arc. The other reason is to reflect away some strong arc from imaging plane.

• High-speed camera

Camera is the core component in the machine vision system. Though most machine vision systems rely on black-and-white cameras, the use of color cameras is becoming more common. It is also increasingly common for machine vision systems to include digital camera equipment for direct connection rather than a camera and separate frame grabber, thus reducing signal degradation. Another kind of camera, called “smart” camera, with built-in embedded processors are capturing an increasing share of the machine vision market. The use of an embedded processor eliminates the need for a frame grabber card and external computer, thus reducing cost and complexity of the system while providing dedicated processing power to each camera.

In the proposed system, an Olympus i-SPEED high-speed monochrome camera, known as a kind of smart camera, with a band-pass filter (lens) is placed about 1~2 meters away from the imaging plane to capture the reflected laser light on it. Since the camera is focused on the imaging plane, the whole 4”-by-4” plane can be clearly recorded by the camera. Since the 20 nm band-pass filter is centered at 685 nm (the wavelength of the laser), the camera can view the reflected laser pattern clearly despite of the existence of strong arc, whose spectrum covers widely. The speed of camera ranges from 60 to 33,000 frames per second. Thus the minimal changes of reflected images with a short period can still be captured by it, which is very important to analyze the status change of the weld pool surface and to realize online quality control.

• GTAW welding machine

Gas tungsten arc welding (GTAW) commonly known as tungsten inert gas (TIG) welding is an arc welding process that uses a nonconsumable tungstenelectrode to produce the weld, shown in Figure 3-2. The weld area is protected from atmospheric contamination by a shielding gas and a filler metal is normally used. GTAW is often used to produce high-quality weld because of its capability in precision control of the fusion process. Here GTAW is used as the test welding process for the designed observation system, and there is no filler metal used. In our system, PULSETM current-constant welding machine is used and the welding current can be adjusted by changing the input control voltage.

Figure 3-2, General GTAW weld area (no filler rod used in our system)

• PC (and software)

In a typical machine vision system, the camera's image is captured by the frame grabber, which is a digitizing device that converts the output of the camera to digital format and places the image in computer memory so that it may be processed by the machine vision software. The software will typically take several steps to process an image. Often the image is first manipulated to reduce noise or to convert many shades of gray to a simple combination of black and white (binarization). Following the initial simplification, the software will count, measure, and/or identify objects, dimensions, defects or other features in the image. As a final step, the software passes or fails the part according to programmed criteria. In our system, MATLAB programming language with Imaging Processing Toolbox is used to process the captured images and developed corresponding reconstruction algorithm is applied to rebuild the weld pool surface.

Observation Result

Different structured light patterns (multiple-line and dot-matrix) are tested in the designed machine vision sensing system. For multiple-line pattern experiment, a commercial available 5-line pattern (model SNF-505L(0.23)-685-20-5) is used. For dot-matrix projection pattern, a 19-by-19 dot-matrix pattern (model SNF-519X(0.77)-685-20) is selected since it can cover the whole weld pool surface and the lack of its center point can be easily recognized in the reflected image for locating the dots. The procedures of the experiment and the observation results are presented and discussed. As can be seen, the high-quality reflected images captured by the camera verified the success of this new sensing system.

Multiple-line pattern

(a)(b)

Figure 3-3, Multiple-line pattern and its application in the system

A multiple-line structured light pattern (five-line) is used in the sensing system, which is shown in Figure 3-3. In the figure, some more lines with less intensity appear beside the five bright lines, which are the diffraction fringes produced by the laser diode. The proposed machine vision system has been shown in Figure 3-1. A 20 mW illumination laser at a wavelength of 685 nm with variable focus is used to generate the multiple lines. The fan angle of the laser is 5 degree and the interline angle is 0.23 degree. In one of the experiments, 5-line laser pattern is projected on the weld pool surface under the torch electrode at 27 degree in the OYZ plane. The distance from the laser to the weld pool is approximately 90 mm. An imaging plane is parallel to the OXZ plane at a known distance of 53.6 mm from the axis of the electrode (i.e., axis Z). To minimize the influence of the arc, the camera observes the imaging plane with a band-pass filter of band width 20 nm centered at a wavelength of 685 nm. The welding direction is the negative direction of Y axis, which is opposite direction shown in Figure 3-1. The detailed experiment instructions are attached in Appendix II.

• Observation results

Although the illumination laser is continuous and has low power (20 mW) in comparison with that of the arc, clear images as shown in Figure 3-4 are obtained in the presence of bright arcs . This is because the system takes advantage of the difference between propagation in the illumination laser and arc plasma. In fact, the arc light intensity decreases with the square of the distance. However due to the coherent and unidirectional property of the laser light waves, the laser’s travel direction remains unchanged and its intensity or power loss over the distance traveled by the laser light is insignificant in comparison with that of the arc. In addition, the specular surface of the weld pool reflects nearly all the intensity of the projected illumination laser lines. Hence, if the imaging plane is placed reasonably far from the torch, the intensity of the laser light falling on it will be much stronger than that of the arc light.

Figure 3-4 shows the reflected images using different work piece with dissimilar reflective characteristic. The clear image in Figure 3-4 (a) was acquired when a piece of mild steel was used, which has less reflection property. As can be seen, only the distorted laser lines reflected from specular weld pool are shown on it. This kind of image is simple and intuitive, so it is suitable for the weld pool observation in our research.

(a) Result using mild steel plate (b) Result using stainless steel plate

Figure 3-4, Reflected images of multiple-line pattern

In addition, the result using a stainless steel plate is presented in Figure 3-4 (b).Except for the deformed lines reflected from the weld pool surface, there are more straight reflected lines in the middle of the image, which are reflected from outside of weld pool due to the high reflective characteristic of the work piece. Compared with Figure 3-4 (a), this overlapped image is more complicated and less clear. But this image provides more information about the weld pool. From the image, we can not only get the information about the 3D shape of the weld pool, but also acquire its 2D shape from the laser lines since they are straightly reflected from outside of the weld pool. This finding is interesting and deserves further study.

Dot-matrix pattern

As can be seen above, clear reflected images can be achieved by using multiple-line pattern of structured light, and the images contain 3D information of the weld pool surface. But these images with reflected lines are difficult to be used to reconstruct the weld pool surface based on reflection law. According to the knowledge of optics, the reflection of light ray is easy to be tracked. Thus dot rays are needed for the proposed system to be projected onto the weld pool.

(a)(b)

Figure 3-5, (a) Dot-matrix pattern and (b) its application in the experiment

In this step, many methods were tested in order to produce appropriate dot matrix laser source, such as using printed transparency with vertical lines to blocking multiple-line pattern and putting diffractive grating in front of dot laser. However some problems were encountered by using these approaches. At last, considering the needed dimension of dot matrix and the safe distance between laser diode and tungsten electrode, a SNF laser with a 19-by-19 dot matrix pattern lens is used for our application, whose projection area can cover the whole weld pool area with reasonable close distance between dots (about 0.5-1 mm with 50 mm projection distance). Its inter-beam angle is 0.77 degree. In Figure 3-5 (a), it can be seen that the center point (at 10th row and 10th column) in the dot matrix is made absent (or can be thought much less bright), which can be used to locate the position of multiple dots as a reference (dot). In the experiment, the dot matrix is projected to the work piece at a certain angle covering the whole possible area of the weld pool surface, just as shown in Figure 3-5 (b).

• Observation results

In an experiment using dot-matrix projection pattern, 2 mm thick mild steel sheet is used as the work piece. The welding current is kept at 70 Amp, and the welding speed is about 2 mm/s. The 19-by-19 laser dot matrix is projected on the weld pool under the torch electrode at 30 degree in the OYZ plane. The distance from the laser to the weld pool is approximately 40 mm. An imaging plane is parallel to the OXZ plane at a known distance of 49 mm from the axis of the electrode (i.e., axis Z). The distance between torch tip and the work piece is about 3mm. In this experiment, the welding direction is also the negative direction of Y axis, which is opposite to the direction shown in Figure 3-1.

 

Figure 3-6, Acquired reflected images by using dot-matrix pattern

In Figure 3-6, four images acquired by the camera during the stable period of the experiment are shown. Although the contrast of the images is a little low, the reflected dots can still be seen, which were distorted by the specular weld pool surface. In these images, some reflected dots on the upper row are blocked and the positions of the reflected laser dots change a little, which shows the variation behavior of the weld pool surface under the same welding condition.

Another experiment using 19-by-19 dot matrix is conducted by changing the welding direction to positive Y axis direction. During the experiment, a sheet of 2 mm thick mild steel is used as the work piece and the welding current is kept on 75 A with the constant welding speed of 3 mm per second. The distance between the torch and imaging plane is 50 mm and the projection angle of laser diode is about 31 degree. Figure 3-7 shows four captured images. As can be seen, under nearly the same condition more rows and more dots are reflected onto the imaging plane compared with the results shown in Figure 3-6. These images are more useful for the accurate reconstruction of weld pool surface.

Figure 3-7, Acquired reflected images by change welding direction

• Center reference point location

In Figure 3-6 and Figure 3-7, the corresponding position of the center reference point can also be easily, and it lies in the 2nd (5th) row from bottom up and between the 5th (5th) and 6th (6th) dots respectively. In Figure 3-8, the corresponding positions of the reference point are shown. With the information of reference point, it can be known which row and column the other dots around the reference dot in the image is reflected from which point in the projected dot matrix. Thus the corresponding relationships between projected dots and the reflected dots on the imaging plane are established, which can be used for reconstructing the weld pool surface.

From above observation results, it can be seen that reflected images of the weld pool surface with the reflection pattern can be acquired using the proposed approach with a low power illumination laser. The clearness of the images assures that they can be processed to accurately extract the laser lines/dots shaped by the specular weld pool surface and thus be used to accurately compute the weld pool surface. The novelty of the proposed approach which takes advantage of the specular surface and the difference between propagation in an illumination laser and arc plasma is responsible for achieving such quality of clear images.

Figure 3-8, Corresponding positions of reference point in the images

However there are also some interferential factors existed in the experiments, which affect the quality of acquired reflected images. The unstable shielding gas and unbalanced electromagnetic field vibrate the arc and make weld pool surface unstable. During practical welding, impurities or oxides existing on the pool surface also block light reflection and make the reflected images unstable. In order to ensure reflected image quality, these bad factors should be minimized, even eliminated.