Processing Procedures

• Identification of Existing DEM's


• Datum and Elevation Conversions


• Projection Transformation and Resampling


• Artifact Corrections


• Merge of Multiple Data Sources


• Edge Matching

Specifications

• Technical Specifications
• Operational Specifications


• Business Specifications

Processing Procedures:

Development of the NED required the merging of over 50,000 different DEM data files. A processing system was designed to assemble a seamless dataset from multiple data sources, resolutions, and production methods. Procedures were developed to maintain the database with periodic updates and to insure the integration of higher resolution elevation data as they become available. A raster data model referenced to a geographic grid was used for NED. The data model is logically seamless but uses an internal tile structure initially selected as a 1- by 1-degree area. The NED dataset currently achieves complete national coverage by integrating the "best" available data in the USGS Sales Database database. Even with the "best" available, there could be a wide range of source dates and some artifacts in the source DEM. The NED assemble process identifies all existing DEM's available from data producers. The system filters production artifacts, and performs any necessary datum conversions and coordinate transformations. The NED data is only as good as the orginal source DEM. Individual DEM files are appended together into the larger tile structure specified for the database. Edge matching and metadata generation are applied lastly in assembling each NED tile. The specific procedures adopted and the issues addressed in building the NED are discussed in the following steps:

• Identification of Existing DEM's


• Datum and Elevation Conversions


• Projection Transformation and Resampling


• Artifact Corrections


• Merge of Multiple Data Sources


• Edge Matching

Identification of Existing DEM's

The first step in the process of assembling NED is the identification and acquisition of all DEM files available for the geographic area being assembled. This step was designed to be automated and to work for a variable geographic extent. The process was implemented by developing a query tool that scanned selected DEM availability indices and returned the name and location of the files found within the specified region.

The query tool that was developed required the compilation of DEM availability indices for all existing data sources (7.5-minute,30-minute,1-degree,SAST). The availability indices were derived from a database search of the NMD Sales Database (SDB). The latest search provided a list of the 50,000 DEM files currently in the SDB. The list is used to extract and store the header record from each DEM. The resulting list of DEM header records is subsequently processed to create DEM availability indices, GIS polygon files, and associated attribute tables. The information extracted from the DEM header record database can be easily redefined and currently includes the full DEM file and path name, state code, production method, horizontal datum, vertical datum, elevation units, SW corner coordinates, DEM level, production codes, and mapping center codes.

Understanding the varying data characteristics of existing DEM's was necessary to determine the data conversions and transformations required to assemble the NED database. National digital elevation data production began over 20 years ago. The data produced have varied in production method and data quality. Varying specifications have lead to the production of DEM's with different horizontal datums, vertical datums, and elevation units. Many of these inconsistencies were eliminated during NED processing.

Datum and Elevation Conversions

Many source DEM's required datum and elevation conversions to the specifications of the NED database. The NED development process incorporated National Geodetic Survey and U.S. Army Topographic Engineering Center(TEC) software to enable the necessary horizontal and vertical datum conversions. The predominant horizontal datum conversion was from North American Datum of 1927 (NAD 27) to North American Datum of 1983 (NAD 83). The predominant vertical datum conversion was from National Geodetic Vertical Datum of 1929 (NGVD 29) to North American Vertical Datum of 1988 (NAVD 88). Other required horizontal datum conversions were World Geodetic System 1972 Datum (WGS 72) to World Geodetic System 1984 (WGS 84), and conversion of Old Hawaiian Datum (OHD) and Puerto Rico Datum of 1940 (PRD) to NAD 83. Other vertical datum conversions included conversion of local mean sea level (LMSL) to NAVD 88. Alaska has a horizontal datum of NAD27 and a vertical datum of NAVD29.

DEM files were also found with elevations stored in either meters or feet. Dataset development required conversion of all elevations to a common unit. Conversion of elevations to decimal meters was adopted in order to preserve the higher precision of data already in feet. These datum and elevation conversions were performed as each DEM file was transformed into a geographic projection.

Projection Transformation and Resampling

Projection transformation was required to transform the 7.5-minute DEM data referenced horizontally in the UTM coordinate system to the geographic (latitude/longitude) coordinate system adopted for the NED dataset. The transformation is applied to the input elevation data, creating a 1-arc second dataset. Resampling of the 2-arc second and 3-arc second DEM's was necessary at this point to create coregistered data from all data sources at the 1-arc second resolution. A bilinear interpolation algorithm was used to perform the resampling.

Artifact Corrections

NED development included procedures to correct and/or minimize the artifacts found in source DEM files. DEM's produced by manual profiling techniques were evaluated for data quality and filtered if necessary. A "mean profile filtering" algorithm was used to reduce the banding artifact observed in profile DEM's. The algorithm isolates the elevation differences that cause the banding, and then subtracts them from the DEM.

The first step is to smooth each row in the DEM independently using a large Gaussian-weighted convolution kernel, up to 101 pixels wide. After this operation, each column consists of a weighted mean of many columns, and constitutes a "mean profile" in the y dimension. The banding artifacts are caused by a systematic operator bias in operating the profiler, and alternate sign from band to band. Since they lie along the x-axis, they are not significantly attenuated by the smoothing operation, and are visibly present in the y-axis mean profiles. By gently smoothing each mean profile with a small convolution window, about the size of a band (7 pixels works well), an array may be constructed of the residuals obtained by subtracting each mean profile from its lightly smoothed version. The resulting profile effectively models the banding. By subtracting these residuals from the original DEM, a largely band-free DEM is obtained. The magnitude of the residuals is under two meters, with an RMS of one meter typical. The resulting data is saved and replaces the input DEM file. This same method works well in removing patch boundary artifacts from DEM produced by the GPM II, though it must be run in two directions.

Merge of Multiple Data Sources

The customary production of DEM's based on quadrangle boundaries neccesitates the paneling of multiple DEM files to compile data for any large study area. In building NED, the various source DEM files were appended (or paneled) together. A 1- x 1-degree tile with 0.125 degrees of overedge required as many as 100 7.5-minute DEM files, 36 30-minute DEM files, or 9 1-degree DEM files.

Individual DEM files were read in the standard USGS ASCII format, converted to binary, rotated so that north was to the top, and placed in the proper geographic position within a temporary array in memory.

Small slivers of missing data were occasionally found within the seams of the paneled 7.5-minute image data. A bilinear interpolation algorithm was used to correct these discrepancies during the assemble process.

The "best available" elevation data was assembled by merging multiple overlapping data sources. The "best available" data can be determined by either a logical operation or a priority in the overlay sequence. Logical operations can include cell based or moving window based operations of the average, minimum, or maximum elevations found in the source materials. A priority specification uses a preferred sequence for merging the elevation values from the input source materials. A priority sequence based on the highest spatial resolution of the source material is currently used to define the "best available" data. The 7.5-minute quadrangles were given highest priority for the output. If 7.5-minute data didn't exist the 2-arc or 3-arc second data was then used to populate the output.

Edge Matching

Seams or discontinuities in a composite elevation file will occur due to differences in the quality and accuracy of the input DEM data files. These seams must be corrected or minimized to provide the user with data useful for a range of applications. Seams generally arise when spikes or an offset occurs in elevations values between two DEM data files. An edge matching operation is used to detect spikes or offsets along a seam. If a spike is found the bad elevation is replaced by an bilinear interpolated value. If an offset is found the edge-matching algorithm forces the DEM to fit along the edge, respecting slope continuity, and distributes the residuals of any mismatch outwards from the seam in an inverse-distance-squared fashion. The magnitude of the residuals will range from a few meters up to 60 meters.

Specifications

NED characteristics are described by Technical Specifications (issues related to the specifications of the data), Operational Specifications (issues related to the implementation), and Business Specifications (guiding principles to encourage use of the framework data).

Technical Specifications:

• Data encoded using a raster data model.
• Database is comprised of the "best available" elevation data.
• Data is in one seamless geodetic framework.
• Data is logically seamless. Edges between contributed data and adjoining tiles are adjusted.
• Uses a geographic coordinate reference system based on the NAD83 horizontal datum and NAVD88 vertical datum.
• Source data are reformatted to a 1-arc second resolution which is approximately equivalent to the 30-meter resolution of the majority of existing USGS Sales Database data.
• Supports multiple resolutions of output data defined by the user.
• Vertical units are in decimal meters.

Operational Specifications:

• Supports data contributed from multiple sources (primarily 7.5-minute DEM's and 30-minute DEM's)
• Provides for variable accuracy documented by spatially referenced metadata
• Obtains complete national coverage using the existing USGS Sales Database and other high resolution data sources
• Supports access to NED data by information networks and digital media.
• Users will be able to find the NED data through the National Geospatial Data Clearinghouse
• Metadata detailing the characteristics and quality of the NED data will be provided. Quality information includes positional accuracy, completeness, logical consistency, and lineage.

Business Specifications:

• Avoids restrictive practices that would inhibit use of the NED. Uses principles from the U.S. Office of Management and Budget Circular A-130 to guide the implementation
• Provides information about limitations of data, including suggested or optimal uses of data, disclaimers, and liability clauses
• Is available in public, non-proprietary formats
• Conforms to approved standards. This allows users to know the characteristics of the data. At a minimum, conformance to relevant FGDC standards should be required and subject to verification