Figure 1.Many DEM data sets generated by the various Gov't agencies have errors associated with them. These errors can take many forms and vary from quadrangle to quadrangle.
Here I will describe a simple method to remove the most common of errors - striping. Striping errors are composed of semi-uniform horizontal striping similar to the example below.
The objective of this project was to generate a map of contour lines for an eight quad area in Casper, Wyoming. The contours are to be 40 foot increments.
There are a number of problems associated with this task. First, the accuracy of the DEM data may be greater than the contour interval. Minimum error associated with DEM's is approx. 7 meters with some as high as 15 meters. Therefore, generating contour intervals of 40' increments means that any contour can be anywhere from 23 to 50 feet off the mark. This is a limitation to the dataset.
In addition to the accuracy of the DEM, the scan line errors would create a contour map showing the scan line errors. Therefore the goal is to generate a DEM without scan line errors. A simple way to remove these scan lines is to apply a smoothing convolution filter to remove or subdue the striping problem. However, using a smoothing filter will alter the original dataset by averaging the elevation values.
This is the process I went through to subdue the striping and determine how much change was applied to the dataset. First all eight DEM's were imported from standard USGS ASCII data files using the ARC/INFO 'DEMLATTICE' command:
The LATTICEMERGE command was used to combine all DEM's into one file covering the study area.
The resulting file looked like this:
It was at this point that the striping was noticed (Figure 1).
The ARC/INFO DEM GRID file was converted to ERDAS Imagine format files by using the data import/export utility in Imagine. The resulting image (.img) file was an unsigned 16-bit format which is common to DEM files. Another standard format for 7.5 minute DEM files is that all elevation data are measured in meters. To generate contour lines in feet, I needed to convert the file to units of feet.
I wrote an Imagine SML model using the Spatial Modeler. The model looked like this:
The input file was the original DEM in meters. The function circle is a simple arithmetic formula which multiplied the input file by 3.281. The output file was named and set to a floating point single precision format.
I looked at the statistics of both files and listed the histograms.
Here are the statistics for the data set in meters:
And the statistics and histogram for the data set in feet:
As you can see there is no change in the histogram, but there is a change in the units. The average elevation in meters is 1762m and in feet it is 5783ft.
And now to the good part:
I used the Image Interpreter, Spatial Enhancement function of Imagine to run a Convolution filter over the dataset. I had many filter size choices and my problem now was to choose the appropriate smoothing filter size. I tested the 3x3 smoothing filter and found that it did not change the data set significantly. The striping was still there (similar to Figure 1.)
My other choices were a 5x5 filter, a 7x7 filter or a filter size of my design. I opted to test the available 5x5 and 7x7 smoothing filters before I made my own.
I first ran the 5x5 smoothing filter over the DEM and looked at the result. Figure 8 shows a small portion of the data set matching the area in Figure 1. Compare Figure 1 with Figure 8 and see the difference
There is a substantial difference between the two files. This made me nervous so I decided to test the files to determine what the difference was between the two. I made another Imagine model to subtract the files from one another (Figure 9).
In this model, the original and filtered files are fed into the function box to make a difference image. The function is a simple subtraction of the filtered from the original. The output file was a signed 16-bit file since I knew the output was going to contain negative numbers.
The resulting file and statistics is shown in Figures 10, 11, and 12.
Figure 10.
Figure 11.
Figure 12.
Notice that there is very little contrast in Figure 10, and that the statistics show a very wide variation in difference values. This, I realized is predominantly a result of edge pixels in the file as a result of the convolution filtering and how it behaves around the edges of files
I used an Area of Interest (AOI) to subset the file and basically cut the edge pixels from the dataset so I could get a more realistic estimate of the variation of elevation values between the two files.
The result of this exercise is shown in Figures 13, 14, and 15.
Figure 13.
Figure 14.
Figure 15.
Now observe the variation in the subseted file without the edge pixels biasing the result. Using a 5x5 smoothing filter the mean value of the difference image was 0.019 and the standard deviation was 4.277. This means that 95% of my data set was within 8.5 feet (two SD's) of the original data set. This was not too bad. Further, it reduced the striping of the DEM significantly.
I tried the same proceedure on the 7x7 smoothing filter with the following results:
Figure 16.
Figure 17.
Figure 18.
The 7x7 filter came out with a mean difference of 0.025 and a standard deviation of 7.113. Meaning that 95% of the data was within 14.226 feet from the original position.
After I determined that the amount of change in elevation was not significant. I converted the Imagine (.img) files back to ARC/INFO GRID files and generated contours using the 'LATTICECONTOUR' command.
The result of the 5x5 smoothing filter with a 40 foot contour interval is shown below:
Notice the heavy white line around the edges of the data set. Since I was using only thin lines in ARCEDIT, this heavy line is a combination of hundreds of thin lines adjacent to each other. These represent contours going from the true elevation at the edge of the dataset to 0 elevation (0 is the value around the edges.)
I want to remove these lines since they do not represent reality and they add significantly to the size of the dataset. With the edge contours the contour coverage is about 60 megabytes in size. Without the edge contours the coverage is about 17.4 megabytes in size.
To remove these, I created a clip coverage by hand digitizing the border just inside the heavy white line. I tried to get as close to the heavy white line as possible since I knew that I was trimming data from my dataset and did not want to loose very much of the true information.
I used the CLIP command in ARC to subset the "true" contours from the edge.
This is what the file looks like now.
Zooming in to the area that is shown in Figure 1 to examine the amount of striping in the 5x5, 40' contour coverage showed some residual stripes, but for the most part the striping has been removed.
I also generated contours for the 7x7 file to compare. Here are the results with a zoom in view of the same area.
And for comparison here is the 40' contour coverage for the original DEM file:
Figure 2.
Figure 3.
Figure 4.
Figure5.
Figure 6.
Figure 7.
Figure 8.
Figure 9.
Figure 19.
Figure 20.
Figure 21.
Figure 22.
Figure 26.