Chapter 11 - Data Products
11.1 Level 0R Product
Unlike earlier Landsat programs, the Landsat 7 system was not originally designed to produce high level (i.e. Level 1) products for users. The baselined program philosophy was to provide raw data only which would leave the value added domain for commercial companies. A prevailing "wait and see" position by commercial vendors prompted NASA to add a systematic correction capability to ensure product availability. The primary product for users and vendors seeking higher level processing, however, is 0R data - an essentially raw data form that is marginally useful prior to radiometric and geometric correction. This is readily apparent when viewing a simulated 0R image . A Landsat 7 0R product, however, does contain all the ancillary data required to perform these corrections including a calibration parameter file (CPF) generated by the Landsat 7 IAS.
LPS spatially reformats earth imagery and calibration data into Level 0R data. This involves shifting pixels by integer amounts to account for the alternating forward-reverse scanning pattern of the ETM+ sensor, the odd-even detector arrangement within each band, and the detector offsets inherent to the focal plane array engineering design. All LPS 0R corrections are reversible; the pixel shift parameters used are documented in the IAS CPF.
11.1.1 Product Size
Three options, depicted in Figure 11.2, exist when defining the size or spatial extent of a Landsat level 0R product ordered from the EDC-DAAC.
The distance between WRS center points along a path is 161.1 kilometers. A path distance of 85 kilometers before and after a WRS center point defines the standard scene length of 170 km. The standard WRS scene overlaps neighboring scenes along a path by 5% and has a width or cross track distance of 185 km.
Landsat 7 browse is framed according to WRS scenes. An ordered scene will cover the same geographic extent observed in the browse with the following caveat. Standard WRS scenes have 375 scans. Partial scenes (less than 375 scans) may exist at the beginning or end of a subinterval due to the fact that imaging events do not always start and end on scene boundaries. Browse and scene metadata for these occurrences accurately reflect their partial scene nature and geographic extent although partials are currently not offered due to complexities associated with level 1 processing.
11.1.2 Product Components
A complete scene-sized 0R product ordered from the EDC-DAAC consists of 19 data sets derived from the wideband telemetry an IAS-generated calibration parameter file, a product specific metadata file, a geolocation index generated by EOSDIS Core System (ECS), and an HDF directory. Therefore, if you order a complete (i.e. all bands) scene-based 0R product it will have 23 distinct files. A brief description of each follows.
· 1 - 9. Earth Image Data - The unique bands of ETM+ image data comprise nine of the data sets. The data is laid out in a scan line sequential format in descending detector order (i.e. detector 16 followed by detector 15 and so on for the 30 meter bands). Band 6 is captured twice - once in low and the other in high gain mode. Under nominal satellite configuration the low gain form of band 6 will be present in format 1. All image samples or pixels are 8 bits in size.
· 10. Internal calibrator (IC) data - format 1 - IC data for format 1 consists of scan line ordered internal lamp and shutter data for bands 1-5 and blackbody radiance and shutter data for low gain band 6. The data is collected once per scan and structured in a band sequential format in descending detector order (e.g. detector 16 followed by detector 15 and so on for the 30 meter bands).
· 11. Internal calibrator (IC) data - format 2 - IC data for format 2 consists of scan ordered internal lamp and shutter data for bands 7 and 8 and blackbody radiance and shutter data for high gain band 6. The data is collected once per scan and structured in a band sequential format in descending detector order (e.g. detector 16 followed by detector 15 and so on for the 30 meter bands).
· 12. MSCD - format 1. A logical record of MSCD exists for each data scan present in the 0R product ordered. Each logical record consists of 3 MSCD data values - the first half scan error, the second half scan error, and the scan line direction. This information, which actually applies to the previous scan, is used to compute deviations from nominal scan mirror profiles as measured on the ground and reported in the calibration parameter file. Also included in the MSCD file are scan based values such as time code, gain status and processing errors encountered by LPS. The MSCD is trimmed to fit the product ordered although one additional record is added to the file during the subsetting process due to the fact that scan error and direction information corresponds to the prior scan.
· 13. MSCD - format 2. A duplicate set of MSCD is generated when format 2 is processed and is kept with the product in the event format 1 MSCD is lost or corrupted.
· 14. PCD - format 1 The PCD for format 1 consists of attitude and ephemeris profiles as well high frequency jitter measurements. PCD for the entire subinterval is included with the 0R product regardless of the size of the data set ordered.
· 15. PCD - format 2 A duplicate set of PCD is generated when format 2 is processed and is kept with the product in theevent format 1 is lost or corrupted.
· 16. Scan line offsets - format 1. During LPS processing image data is shifted in an extended buffer to account for predetermined detector and band shifts, scan line length, and possible bumper wear. The scan line offsets represent the actual starting and ending pixel positions for valid (non-zero fill) earth image data on a data line by data line basis for bands 1 through 6 low gain. The left starting pixel offsets also apply to the IC data.
· 17. Scan line offsets - format 2. During LPS processing image data is shifted in an extended buffer to account for predetermined detector and band shifts, scan line length, and possible bumper wear. The scan line offsets represent the actual starting and ending pixel positions for valid (non-zero fill) earth image data on a data line by data line basis for bands 6 high gain through 8. The left starting pixel offsets also apply to the IC data.
· 18. Metadata - format 1. During LPS format 1 processing metadata is generated that characterizes the subinterval's spatial extent, content, and data quality for bands 1 through 6 low gain. This file, in its entirety and original form, accompanies the 0R product.
· 19. Metadata - format 2. Format 2 metadata is similar but not identical to format 1 metadata. The subinterval-related metadata contents are identical; the scene-related metadata is specific to bands 6 - high gain, 7, and 8. Also, the format 2 metadata does not include cloud cover assessment data or references to browse data products. This file, in its entirety and original form, accompanies the 0R product.
· 20. Metadata - ECS. A third metadata file generated by ECS during order processing. This file contains product specific information such as corner coordinates and number of scans.
· 21. Geolocation Index. The geolocation index is also produced by ECS. This table contains scene corner coordinates and their product-specific scan line numbers for bands at all three resolutions. Its purpose is provide for efficient subsetting of a 0R product.
· 22. Calibration parameters. The IAS regularly updates the CPF to reflect changing radiometric and geometric parameters required for level 1 processing. These are stamped with applicability dates and sent to the EDC-DAAC for storage and bundling with outbound 0R products.
· 23. HDF Directory. A file containing all the pointers, file size information, and data objects required to open and process the 0R product using the HDF library and interface routines.
A user may order a subset of the available bands which will affect the actual file count in a 0R product. In all cases, however, every product includes two PCD files, two MSCD files, three metadata files, the CPF, and the HDF directory. Only the internal calibrator, scan line offset, and earth image file counts are affected by a product possessing less than the full complement of bands.
11.1.3 Product Format
The product delivered to Landsat 7 data users is packaged in HDF - an open standard selected by NASA for Earth Observing System (EOS) data products. HDF is a self-describing format that allows an application to interpret the structure and contents of a file without outside information. HDF allows Landsat 0R products to be shared across different computer platforms without modification and is supported by a public domain software library consisting of access tools and various utilities.
Product users are directed to the Landsat 7 0R Distribution Product Data Format Control Book, Volume 5 (PDF) for details regarding the HDF design used for the 0R product. Included are references to NCSA-authored documentation. New users should begin with Getting Started with HDF while the HDF User's Guide and HDF Reference Manual are excellent resources for the HDF programmer.
11.2 Level 1R Product
The Level 1R product is a radiometrically corrected 0R product. Radiometric correction is performed using either the CRAM gains in the CPF or gains computed on the fly from the IC data. The choice is available to a user when the product is order. The biases used are alway calculated from the IC data. Image artifacts such as banding, striping, and scan correlated shift are removed prior to radiometric correction. Radiometric corrections are not reversible. The 1R product geometry is identical to the input Level OR data.
During 1R product rendering image pixels are converted to units of absolute radiance using 32 bit floating point calculations. Pixel values are then multiplied by 100 and converted to 16 bit integers prior to media output. Two digits of decimal precision are thus preserved. One merely divides each pixel value by 100 to convert the 1R image data back to radiance units. The 16 bit 1R product is twice the data volume of an alike 0R product.
11.2.1 Product Size
Two options exist for users when defining the size or spatial extent of a Landsat level 1R product ordered from the EDC-DAAC.
· Standard Worldwide Reference System (WRS) Scene. The standard WRS scene, as defined above for the 0R product, is an orderable 1R product. Partial scenes that may exist at the beginning and end of subintervals may be also be ordered.
· Partial Subinterval A partial subinterval can also be ordered in 1R form, although this capability is not scheduled for release until early 2000. Unlike the 0R product the 1R is limited to a maximum of 3 WRS scenes in size. The variably sized 1R product can float or be positioned at any scan line starting point within a subinterval. Alternatively, the product can be defined by up to three contiguous WRS locations.
11.2.2 Product Components
A complete scene-sized 1R product ordered from the EDC-DAAC consists of 17 data sets derived from the wideband telemetry, an IAS-generated calibration parameter file, a product specific metadata file, a geolocation index generated by EOSDIS Core System (ECS), and an HDF directory. Therefore, if you order a complete (i.e. all bands) scene-based 0R product it will have 21 distinct files. There are two fewer data files than an alike 0R product due to the fact that the multiple PCD and MSCD files are merged into single consensus files. Please reference the 0R file product for individual file descriptions.
A user may order a subset of the available bands which will affect the actual file count in a 1R product. In all cases, however, every product includes one consensus PCD file, one consensus MSCD files, three metadata files, the CPF, and the HDF directory. Only the internal calibrator, scan line offset, and earth image file counts are affected by a product possessing less than the full complement of bands.
11.2.3 Product Format
The 1R product is delivered to users only in the HDF format. The HDF 0R and 1R formats are nearly identical. Exceptions include the united PCD and MSCD files and an enhanced product specific metadata file that reflects 1R correction characteristics. Please refer to the Landsat 7 0R Distribution Product Data Format Control Book, Volume 5 (PDF) for details regarding HDF specifics. Additional information unique to the 1R product can be found in the ESDIS Level 1 Product Generation system Output Files DFCB .
11.3 Level 1G Product
The 1G product available to users from the EDC-DAAC is a radiometrically and systematically corrected 0R image. The term systematic refers to the nature of geometric corrections applied. The correction algorithms model the spacecraft and sensor using data generated by onboard computers during imaging events. Primary inputs are the PCD, which includes the attitude and ephemeris profiles, and the MSCD. Refined parameters from the CPF are also used to improve the overall geometric fidelity of the 1G product.
During processing the 0R image data undergoes two-dimensional resampling according to user-specified parameters including output map projection, rotation angle, pixel size, and resampling kernal. The seven map projections supported are:
· Universal Transverse Mercator
· Lambert Conformal Conic
· Transverse Mercator
· Polar Stereographic
· Hotine Oblique Mercator A
· Hotine Oblique Mercator A
· Space Oblique Mercator
Associated with each projection is a unique set of projection parameters that must be specified when ordering a level 1 product. These parameters flow from the USGS General Cartographic Transformation Package and are specified similarly.
The WGS84 ellipsoid is employed as the Earth model for coordinate transformation. The end result is a geometrically rectified product free from distortions related to the sensor (e.g. jitter, view angle effects), satellite (e.g. attitude deviations from nominal), and Earth (e.g. rotation, curvature). A subset of a full scene in Figure 11.3 illustrates the distortion-free characteristics of a 1G product.
The systematic 1G correction process does not employ ground control or relief models to attain absolute geodetic accuracy. Residual error in the systematic 1G product will be appoximately 250 meters (1 sigma) in flat areas at sea level. Precision correction employs ground control points to reduce geodetic error of the output product to approximately 30 meters. This accuracy is attained in areas where relief is moderate. Terrain correction processing employs both ground control points and digital elevation models to reduce geodetic error of the output product to less than 30 meters in areas where terrain relief is
substational. Users requiring higher level products would be best served ordering level 0R data for in-house processing or contacting a value added service organization with such capabilities.Conversion to Radiance
During 1G product rendering image pixels are converted to units of absolute radiance using 32 bit floating point calculations. Pixel values are then scaled to byte values prior to media output. The following equation is used to convert DN's in a 1G product
to back to radiance units:
Radiance = gain * DN + bias
which is also expressed as:
Radiance = (LMAX - LMIN)/255 * DN +LMIN
The LMINs and LMAXes are the spectral radiances for each band at digital numbers 0 and 255, respectively. One set exists for each gain state. These values will change slowly over time as the ETM+ detectors lose responsivity. Table 11.2 lists the LMINs and LMAXes as of September 14, 1999. Always check the most recent CPF for any updates to these values.
Radiance to Reflectance
For relatively clear Landsat scenes, a reduction in between-scene variability can be achieved through a normalization for solar irradiance by converting spectral radiance, as calculated above, to planetary reflectance or albedo. This combined surface and atmospheric reflectance of the Earth is computed with the following formula:
= Unitless effect planetary reflectance
= Spectral radiance at the sensor's aperture
= Earth-Sun distance in astronomical units
from nautical handbook
= Mean solar exoatmospheric irradiances
from Table 11.3
= Solar zenith angle in degrees
ETM+ Band 6 imagery can also be converted from spectral radiance (as described above) to a more physically useful variable. This is the effective at-satellite temperatures of the viewed Earth-atmosphere system under an assumption of unity emmissivity and using pre-launch calibration constants listed in Table 11.4. The conversion formula is:
= Effective at-satellite temperature in
= Calibration constant 2 from Table 11.4
= Calibration constant 1 from Table 11.4
= Spectral radiance in watts/(meter
squared * ster * µm)
11.3.1 Product Size
The same two 1R options exist for users when defining the size or spatial extent of a Landsat level 1G product ordered from the EDC-DAAC.
· Standard Worldwide Reference System (WRS) Scene. The standard WRS scene, as defined above for the 0R product, is an orderable 1G product. Partial scenes that may exist at the beginning and end of subintervals may be also be ordered.
· Partial Subinterval. A partial subinterval can also be ordered in 1G form. Unlike the 0R product the 1G is limited to a maximum of 3 WRS scenes in size. The variably sized 1G product can float or be positioned at any scan line starting point within a subinterval. Alternatively, the product can be defined by up to three contiguous WRS locations.
11.3.2 Product Components
The 1G product ordered from the EDC-DAAC consists of the corrected image files and descriptive metadata. All other ancillary files delivered with the 0R and 1R products are not included. A user may order a subset of the available bands which affects the actual file count in a 1G product.
11.3.3 Product Format
The 1G product can be packaged into one of following user-specified output formats:
· HDF. The HDF packaging format used for the 0R and 1R products is also used for structuring the 1G. The design employs external elements for the band files and metadata. These are standalone files that are referenced via tags and pointers residing in an HDF directory. External elements provide users with two processing options - exploit the NCSA HDF libraries for data access or process the data files directly using homegrown code.
The number of files comprising an HDF-formatted 1G product will vary according to the number of bands ordered. A product with a full band complement has 11 files - the HDF directory, a metadata file, and a separate file for each band. The HDF directory and metadata files are always present regardless of bands ordered. Please refer to the Landsat 7 0R Distribution Product Data Format Control Book, Volume 5 (PDF) for details regarding band file specifics. The 1R metadata file description can be found in the ESDIS Level 1 Product Generation System Output Files DFCB.
The HDF format can be specified for any type of 1G product ordered from the EDC-DAAC.
· Fast. The Fast Format was originally developed by EOSAT as a means for quickly accessing Landsat 4 and 5 image data. Its structure is straightforwardly simple. Each band is self contained in its own file (i.e external element style). A header file containing three records accompanies the image data. The three records in order of appearance are labeled administrative, radiometric, and geometric respectively. Sensor specific information is placed in the administrative record, gains and biases can be found in the radiometric record while projection information and image coordinates are stored in the geometric record. A single header file along with the image files constitute the Fast product.
A derivative of the Fast Format (Fast-L7) used by EOSAT for Landsat (FAST-B) and Indian Remote Sensing products (Fast-C) was created for Landsat 7. Several differences are worth noting. File names are now included in the administrative record which allows for direct file access. A separate header file now accompanies the panchromatic, thermal and VNIR/SWIR band groups for Landsat 7. For Fast-B and Fast-C all bands were resampled to a common grid cell size thus permitting a single header file. In all likelihood each of the band groups for Landsat 7 will be resampled to a common resolution (i.e. 15, 30, & 60 meters) thus requiring a distinct header file for each.
All critical fields required for product ingest were left unchanged in the Fast L-7 Format. As a consequence Heritage Fast readers residing on user systems can be used for the Landsat 7 Fast formatted product. A full layout of the Fast L-7 Format can be found in the the ESDIS Level 1 Product Generation system Output Files DFCB.
The Fast-L7 format supports all variations of the 1G product.
· GeoTIFF .Geographic tagged image file format (GeoTIFF) is based on Adobe's TIFF - a self-describing format developed to exchange raster images such as clipart, logotypes, and scanned images between applications and computer platforms. Today, the TIFF image file format is used to store and transfer digital satellite imagery, scanned aerial photos, elevation models, and output from digital cameras. TIFF is the only full-featured format in the public domain, capable of supporting compression, tiling, and extension to include geographic metadata.
The TIFF file consists of a number of label (tags) which describe certain properties of the file (such as gray levels, color table, byte format, compression size). After the initial tags comes the image data which may be interrupted by more descriptive tags. GeoTIFF refers to TIFF files which have geographic (or cartographic) data embedded as tags within the TIFF file. The geographic data can then be used to position the image in the correct location and geometry on the screen of a geographic information display.
Baseline TIFF image types can be bilevel, grayscale, palette color, and full color (24 bit). For simplicity's sake the grayscale model was implemented for the Landsat 7 GeoTIFF product. Under this implementation each ordered band is delivered as its own 8 bit greyscale GeoTIFF image. A standard WRS scene possessing the full band complement would thus be comprised of nine separate GeoTIFF images or files. No other files accompany the product. For detailed information regarding the Landsat 7 GeoTIFF implementation please refer to the ESDIS Level 1 Product Generation system Output Files DFCB. For GeoTIFF details, please refer to the GeoTIFF Format Specification.
At the present time GeoTIFF format cannot be used for the Space Oblique Mercator and Oblique Mercator projections. Products projected into these reference systems must be formatted using HDF or Fast-L7.