Median Filters are Important
Median filters are a way to reduce unwanted signals, better known as noise. Effective filtering reduces as much of an unwanted signal, leaving as much of a desired signal as possible. Non-destructive testing (NDT) inspectors cannot presume to know all the signals they are looking for in advance. It’s therefore necessary to be very aware of what desired signals—target defects—a median filter could remove. You must always use median filters with the utmost care and understanding their functions thoroughly. It’s advisable to review filtered and unfiltered data to make sure nothing is missing.
What to Filter?
Median filters are very good at filtering two types of noise: high-frequency noise and low-frequency noise.
High-Frequency Noise
It can be electrical noise entering the data stream from external sources. It can also be spiking from electrical or electronic issues inside the probe. Finally, this type of noise can be from the instrument and natural signal changes due to variations or deposits in the material under test. One very common source of high-frequency noise is over-amplification, such as gain, beyond the optimal settings of a given inspection technique.
Low-Frequency Noise
It can be caused by liftoff variations encountered by a probe in a tube or on a surface. It can also come from variations in deposit conditions on a surface under test. Finally, it can be from changes in material, composition, geometry, or thickness or the part under test.
Median Filters, What They are and What They do
Median filters perform digital signal processing and come in two types: low pass and high pass.
Low-Pass Filter
This type of filter allows low-frequency signals to pass (filters high-frequency noise).
High-Pass Filter
This type of filter allows high-frequency signals to pass (filters low-frequency noise).
This can be a bit confusing as these names describe what is unfiltered rather than what is. Another way to distinguish these two filters is to remember that a low-pass median filter is most often used with a few data points and a high-pass median filter is most often used with many data points.
Median Filter
The word median is a mathematical term used to describe the middle of a sorted data set. For example, in Table 1, there are 11 samples in the data set and the middle value is at sample number 6 with a value of 20. Figure 1 shows a line graph of the data.
Table 1: Example data set with 11 data points
Low-Pass Median Filter
Returns the median value of a sorted set of values centered on the current data point. The number of data points in the set is the width of the filter. Although this sounds complicated, it isn’t. To illustrate how this filter works, look at Table 2 and Figure 2.
As you can see in Figure 2, one data point in the data set has a higher value than the rest. Assume that this higher value is a noise signal and that a low-pass median filter is used to reduce its influence.
Table 2: Low-pass median filter example data set with 33 data points
Low-Pass Median Filter Working Principles
1. Filter width – the low-pass median filter in this example will use five data points (more about choosing the width of the filter later).
2. Centered on a current data point – center the filter on point 17 so the data runs between points 15 and 19.
3. Sorted values — take data points 15–19 and sort their values from lowest to highest. The original data point numbers are no longer tracked.
This is repeated on all the data in the set. See Table 3 and Figure 3. You can see that the filtered data follows the original closely, but deviates slightly on a point-by-point basis. The filtered data is shorter by two data points at each end.
Low-pass median filters are most effective at removing spike-like noise from data streams. It has a minimal impact on data integrity, especially if the the filter is as narrow as possible. Spike signals are filtered regardless of their amplitude, where an averaging filter retains some traces of a large amplitude spike signal, especially when the spike voltage is very large.
Table 3: 5-data-point-wide low-pass median filter of Table 2 data
Low-Pass Median Filter Example on Actual Data
Here is an example of a low-pass median filter reducing the influence of electronic noise. The upper C-scan and Lissajous are from the unfiltered data and the lower ones had a low-pass filter applied.
Figure 4: High-resolution Magnifi® C-scan of calibration tube data showing unfiltered and filtered outputs
Take special care when using low-pass median filters. There’s always a danger of attenuating signals (amplitude reduction), but a certain degree of signal attenuation is normal. You can see different signal attenuations for several low-pass median filter widths in Figure 5. The signal is from a 0.1 mm (0.004 in) wide EDM notch. The noise adjacent to the notch signal is approximately 0.20 Vpp. The unfiltered signal was measured at 1.93 Vpp (Figure 5a). Assuming the noise adds to the signal, it’s reasonable to assume that the notch is as small as 1.73 Vpp.
Figure 5: High-resolution Magnifi C-scans of a 0.1 mm (0.004 in) wide EDM-notch signal with a) no filter, b) a 3-point low-pass median filter, and c) a 17-point low-pass median filter
Applying a 3-point low-pass median filter reduces the amplitude of the signal to 1.83 Vpp (Figure 5b). Similarly, the attenuated amplitudes with filters up to 17 data points wide (Figure 5c) appear in Figure 6.
A low-pass median filter 3 or 5 data points wide is acceptable for this application. With such a filter, the signal amplitude is not attenuated under the arbitrary limit of the original signal amplitude minus the approximate amplitude of the noise being filtered. But every situation is different. A wider low-pass median filter will remove more noise, but can also attenuate more signal. The degree of attenuation is a function of the inspection speed, the length of defects, and the filter’s width. The maximum width of a filter should always be ascertained using calibration samples before it’s used on actual inspection data. If you’re unsure, always err on the side of using a narrower low-pass median filter.
Figure 6: Signal attenuation according to low-pass median filter width
Low-pass median filters are an effective means of mitigating short-duration noise events such as spiking, electronic noise, and surface conditions. Low-pass median filters should therefore be as narrow as possible. Be sure to assess the degree of attenuation of a filter on known defects before deploying it. You should, finally, examine your data in its filtered and unfiltered states to avoid missing anything.
Read part 2 of this series to learn about high-pass median filters.