Chapter 10 - Calibration Parameter File

 10.1 File Description

The IAS is responsible for the sustained radiometric and geometric calibration of the Landsat 7 satellite and ETM+ and passing this knowledge to the user community. This is achieved by assessing new imagery on a daily basis, performing both radiometric and geometric calibration when needed, and developing new processing parameters for creating level 1 products. Processing parameters are stored in the Calibration Parameter File (CPF) which is stamped with applicability dates and sent to the EDC-DAAC for storage and eventual bundling with outbound Level 0R products. The CPF also is sent to international ground stations via the Landsat 7 Mission Operations Center.

10.1.1 Calibration Parameter File Updates

IAS updates and distributes the calibration parameter file at least every 90 days. Updates will likely be more frequent during early orbit checkout and will also occur between the regular 90-day cycles whenever necessary. Irregular updates, however, will not affect the regular 90 day schedule. The timed release of a new calibration parameter file must be maintained because of the UT1 time corrections and pole wander predictions included in the file. These parameters span a 180 day interval time centered on the effective start date of the new IAS CPF.

Time Stamps.

The calibration parameter file is time stamped by IAS with an effective date range. The first two parameters in the file, Effective_Date_Begin and Effective_Date_End, designate the range and are of the form YYYY-MM-DD. The Effective_End_Date for the most recent parameter file is its Effective_Date_Begin plus 90 days. After this date the file is without applicable UT1 time predictions. The parameter file that accompanies an order has an effective date range that includes the acquisition date of the image ordered.

Through the course of the mission, a serial collection of CPFs is generated and sent to the EDC-DAAC for coupling to 0R products. A distinct probablity exists that a CPF will be replaced due to improved calibration parameters for a given periord or perhaps due to file error. The need for unique file sequence numbers becomes necessary as file contents change. The following file naming procedure is used by IAS to name the CPF:

                                      L7CPFyyyymmdd_yyyymmdd.nn

                                    where:

                                            L7 = Constant for Landsat 7

                                            CPF = 3-letter CPF

                                            designator

                                            yyyy = 4-digit effectivity

                                            starting year

                                            mm = 2-letter effectivity

                                            starting month

                                            dd = 2-letter effectivity

                                            starting day

                                            _ = Effectivity

                                            starting/ending date

                                            separator

                                            yyyy = 4-digit effectivity

                                            ending year

                                            mm = 2-letter effectivity

                                            ending month

                                            dd = 2-letter effectivity

                                            ending day

                                            nn = Sequence number for

                                            this file

As an example, suppose four calibration files were created by the IAS on 90-day intervals and sent to the EDC-DAAC during the first year of the mission. Further suppose that the first file was updated twice and the second and third files were updated once.

The assigned file names would be as follows:

                                    File 1

                                         L7CPF19980601_199808210.00

                                         L7CPF19980601_199808210.01

                                         L7CPF19980601_199808210.02

                                    File 2

                                         L7CPF19980830_19981127.01

                                         L7CPF19980830_19981127.02

                                    File 3

                                         L7CPF19981128_19990225.01

                                         L7CPF19981128_19990225.02

                                    File 4

                                         L7CPF19990226_19990526.01

It is worth nothing the 00 sequence number assigned to the origianl CPF. This reserve sequence number uniquely identifies the pre-launch CPF. Sequence numbers for subsequent time periods all begin with 01. New versions or updates are incremented by one.

This example assumes the effectivity dates do not change. The effectivity date range for a file can change, however, if a specific problem (e.g. detector outage) is discovered somewhere within the nominal 90-day effectivity range. Assuming this scenario, two

CPFs with new names and effectivity date ranges are spawned for the time period under consideration. The effective_date_end

for a new pre-problem CPF would change to the day before the problem occurred. The effective_date_begin remains unchanged. A post-problem CPF with a new file name would be created with an _effective_dage_begin corresponding to the imaging date the problem occurred. The effective_date_end assigned would be the original effective_date_end for the time period under consideration. New versions of all other CPFs affected by the erroneous parameter also would be created.

Using this example, suppose a dead detector is discovered to have occurred on January 31, 1999. Two new CPFs are created that supersede the time period represented by file number three, version 2, and a new version of file number four is created. The new file names and sequence numbers become:

                                    File 3

                                          L7CPF19981128_19990225.01

                                          L7CPF19981128_19990225.02

                                          L7CPF19981128_19990131.03

                                          L7CPF19990201_19990225.03

                                    File 4

                                          L7CPF19990226_19990526.01

                                          L7CPF19990226_19990526.02

10.1.2 File Structure

All calibration parameters are stored as American Standard Code for Information Interchange (ASCII) text using the ODL syntax developed by JPL. ODL is a tagged keyord language developed to provide a human-readable data structure to encode data for simplified interchange. The body of the file is composed of two statement types:

  1.Attribute assignment statement used to assign values to parameters.

  2.Group statements used to aid in file organization and enhance parsing granularity of parameter sets.

To illustrate consider the first three parameters in the file: Effective_Date_Begin, Effective_Date_End, and the CPF_File_Name. These three parameters form their own group which is called FILE_ATTRIBUTES. The syntax employed for this collection of

parameters in the CPF appears as:

                                    GROUP = FILE_ATTRIBUTES

                                      Effective_Date_Begin = 1999-02-26

                                      Effective_Date_End = 1999-05-26

                                      CPF_File_Name =

                                      L7CPF19990226_19990526.01

                                    END_GROUP =

                                    FILE_ATTRIBUTES

 10.2 File Content

The CPF supplies the radiometric and geometric correction parameters required during Level 1 processing to create superior products of uniform consistency across the Landsat 7 system. They fall into one of three major categories: geometric parameters, radiometric parameters, or artifact removal parameters.

 10.2.1 Geometry Parameters

The geometric parameters are classified into 11 first tier groups. A brief description of each group and their use various Landsat 7 systems follows. The heading for each group is the actual ODL group name used in the CPF.

• Earth Constants
• Orbit Parameters
• Scanner Parameters
• Spacecraft Parameters
• Mirror Parameters
• Scan Line Corrector
• Focal Plane Parameters
• Attitude Parameters
• Time Parameters
• Transfer Function
•  UT1 Time Parameters

 10.2.2 Radiometric Calibration Parameters

The radiometric parameters are classified into 15 first tier groups. A brief description of each group and their use in various Landsat 7 systems or by user follows. The heading for each group is the actual ODL group name used in the CPF.

• Detector Status The Detector Status parameters provide a five element code that describes the current health status of each ETM+ detector. The five codes indicate detector status (live or dead), low gain signal noise, high gain signal noise, low gain dynamic range quality, and high gain dynamic range quality.

• Detector Gains Analysis of the SIS calibration transfer to the IC and output from the CRAM model used by IAS results in the Detector Gain parameter set. For each detector there is a prelaunch gain, postlaunch gain, and a current gain for each of the two gain settings. The prelaunch and postlaunch gains are based on the SIS calibration and remain remain static while the current gain is updated as a function of CRAM model improvement and detector responsivity over time. The Detector Gain parameters are used to radiometrically correct ETM+ image data prior to LPS automatic cloud cover assessment (ACCA) algorithm and optionally by LPGS for as an alternative to computing gains on the fly from the IC data.

• Bias Locations The bias location parameters refer to the IC data. They specify the starting pixel location for the bias (dark current restore), the length in pixels of the bias region, and the length of useable IC data including the pulse. A set of parameters exists for each of the three band groups - reflective, panchromatic, and thermal. They are used during radiometric correction for rapid retrieval of calibration pulse and shutter data.

• Detector Biases B6 During level 1 processing band 6 biases are generally computed from the IC for the image being processed. This is a complex task and may be subject to anomolies. This parameter group is computed both prelaunch and at regular intervals over the life of the mission. These are baselined band 6 biases and are used during level 1 processing if the image specific IC-derived biases prove unreliable.

• Scaling Parameters The Scaling Parameter set consists of the lower and upper limit of the post-calibration dynamic range for each band in each gain state. These are the LMIN and LMAX values and are expressed in units of absolute spectral radiance. These values are used by LPGS to convert 1G products to scaled 8-bit values and by users for the reverse transformation. There is an LMIN/LMAX pair per band for each of the gain modes.

• MTF Compensation All image systems, including Landsat 7, cause a blurring of the scene radiance field during image acquisition. Accurate characterization of this blurring is referred to as the modulation transfer function (MTF). Retoration processing compensates and corrects for systemic degradations to yield greater radiometric accuracy. The MTF compensation parameters are weighting functions for each band. Five weighting parameters for both pixel and line directions were selected to best fit the optimal MTFC response. These are applied to the components of the piecewise cubic convolution kernal to generate the optimal MTF reconstruction kernel.

• Sensitivity Temperatures The temperature of the detectors on the primary focal plane of the ETM+ are not controlled and tend to warm up as the instrument operates. The cold focal plane is controlled but may operate at different set points. Most detectors show some dependence of responsivity with temperature. The sensitivity temperature parameters describe the relationship between gain change and operating temperature for each detector and are used to adjust the gains derived from multi-calibration sources. Gains derived soley from IC data are not temperature adjusted.

• Reference Temperatures The sensitivity temperature coefficients just described are used to adjust gains for varying focal plane temperatures. The reference termperatures are used to normalize the gains to a stable temperature. A single reference temperature is calculated prelaunch and postlaunch for each band at both gain states.

• Lamp Radiance The lamp radiance parameters contain the actual radiance of the two IC lamps in three possible configurations (i.e. lamp 1 on - lamp 2 off, lamp 1 off - lamp 2 on, lamp 1 on - lamp 2 on). For each reflective band there are pre-launch, post-launch, and current values for the low and high gain settings. Pre-launch values are established by transferring the SIS calibration to the IC lamps via the ETM+. Post launch are determined using PASC and FASC calibration data. The lamp radiance parameters used to compute the gains used for converting raw ETM+ data to units of absolute radiance.

• Reflective IC Coeffs Radiance levels produced by the internal calibrator, or seen by the detectors vary as a function of instrument state. Severalparameters affecting instrument state are tracked and used for correcting this effect. These parameters are instrument on time, position in orbit, and temperatures of the internal calibrator components and focal plane arrays. The reflective IC coefficients are used in the model that corrects for these effects. For each detector there are 18 coefficients for both the low and high gain states.

• Lamp Reference As explained above, the radiance levels produced by the internal calibrator, or seen by the detectors vary as function of instrument state. The model that compensates for these effects requires as input 14 temperatures of the internal calibrator components and focal plane arrays. In general, these temperatures are extracted from the PCD for the image being processed. However, the IAS also performs a pre-launch calibration of the ETM+ and a post calibration using the combined radiometric model. The lamp reference parameters represent the instrument state at the time of calibration.

• B6 View Coeffs The band 6 view coefficients are used in computing the actual shutter (i.e. bias) values when processing the emissive IC data. The offset algorithm takes into account radiance of the shutter flag as well as contributions from other instrument components such as the scan mirror and scan line corrector. Each band 6 detector has a different view of the contributing components. The band 6 view coefficients capture this view and are used to adjust the contributing spectral radiances accordingly.

• B6 Temp Model Coeffs The Band 6 temperature coefficients are used to calculate the temperature of the scan mirror. The emissive IC algorithm requires scan mirror temperature for computing band 6 gains and offsets. The scan mirror's contribution to the band 6 response must be calculated and accounted for.

• Lamp Current Coeffs Included in the PCD are the currents for the two IC lamps. The currents are in a raw data format and require conversion to engineering units (i.e. milli-amps) prior to their use. The lamp coefficient parameters are used to linearly transform the raw counts to milli-amps. There are two coefficients for each lamp.

• Thermistor Coeffs Included in the PCD are a variety of ETM+ component temperatures. The temperatures are in a raw data format and require conversion to valid numbers prior to their use. The thermistor coefficients parameters are used for this purpose. Six conversion coefficients are supplied for each of the 28 different temperatures that accompany the PCD.

 10.2.3 Artifact Parameters

The artifact parameters are classified into 9 first tier groups. A brief description of each group and their use in various Landsat 7 systems follows. The heading for each group is the actual ODL group name used in the CPF.

• Memory Effect Memory effect is a noise pattern commonly known as banding. It's observed as alternating lighter and darker horizontal scan-wide stripes. The memory effect parameters were derived by the IAS and are static. They consist of a magnitude and time constant for each detector. These are used in an inverse filtering operation to remove the memory effect artifact.

• Ghost Pulse The ghost pulse is a faint secondary image of the internal calibrator lamp pulse. It appears in bands 5 and 7. The ghost pulse parameters identify the beginning and ending minor frames that bound this ghost image.

• Scan Correlated Shift Scan correlated shift is a sudden change in bias that occurs in all detectors simultaneously. The scan correlated shift parameters are derived by the IAS and are static. They consist of a bias magnitude for each detector and are used to compensate for the shift when it occurs.

• Striping Striping is defined as residual detector to detector gain and offset variations within a band of radiometrically corrected (1R) data. The 1R process is intended to remove detector to detector variations through the generation of relative gains and bias from histograms. These are included in the absolute gains and biases eventually applied. Nonetheless, the possibility of residual striping remains. The striping parameters are correction methodology flags. Two processing options are possible: linearly adjust the 1R data to match the means and standard deviations of each detector to a reference detector or to an average of all the detectors. There is one striping parameter per band for each of the gain modes.

• Histogram Histogram analysis estimates the relative gains and biases for all detectors by characterizing the response behavior of individual detectors in a band relative to the other detectors in a band. Results are used to adjust the gains and biases applied during radiometric correction. The histogram parameters control the algorithm by specifying detector noise, a normalization reference detector for each band, saturation metrics, and histogramming window size.

• Impulse Noise Impulse noise within a digital signal manifests itself in a sample as a departure from the signal trend far in excess of that expected from random noise. The impulse noise parameters specify a median filter width and an impulse noise threshold for each detector. The IAS employs these parameters for identifying and trending impulse noise in otherwise homogeneous data such as night scenes and FASC imagery.

• Coherent Noise Coherent noise is a low-level periodic noise pattern that was found in all Landsat 5 imagery and characterized by the IASfor Landsat 7. The coherent noise parameters consist of the number of noise components and a set of wave form characteristics that describe each component for each band. The wave form characteristics are the mean, sigma, minimum, and maximum for each component's frequency, phase, and magnitude.

• Detector Saturation In addition to normally observed saturation (i.e. 0, 255) two other types of detector saturation can occur. An analog to digital converter may saturate below 255 counts at the high end, or above 0 at the low end. The detector saturation parameters identify these levels for each detector. The analog electronic chain may saturate at a radiance corresponding to a level below 255 counts and above 0 counts on the low end. The detector saturation parameters also identify these levels for each detector.

• Fill Patterns LPS uses two values to fill minor frames to distinguish missing or bad band data from good data. The two fill values used are zeros for odd detectors and 255s for even detectors. The fill data is placed on a minor frame basis - if data is missing from part of a minor frame the whole minor frame is filled. The alternating 0/255 fill pattern was selected to unambiguously flag artificial fill from reflectance values that naturally could occur.

 10.2.4 ACCA Parameters

Each scene processed by LPS undergoes automatic cloud cover assessment prior to being archived. The cloud cover score become searchable metadata and are used to filter out undesirable scenes from an archive interrogation. The ACCA algorithm uses a variety of threshold and band indices for cloud identification. These may change during the mission and are therefore included in the CPF for LPS use.

·       ACCA Biases The LPS automatic cloud cover recognition (ACCA) algorithm requires radiometrically corrected image data. The ACCA Biases parameter set is used in conjunction with the Detector Gains described above for converting raw digital numbers to units of absolute radiance. There is one bias parameter per detector per band for each of the two gain modes. Although ACCA uses only bands 2 through 6, the other band biases are included for completeness. Biases are reported in units of digital counts.

·       ACCA Thresholds The LPS ACCA algorithm uses bands 2 through 6 in a combination of thresholds, ratios, and indices to separate clouds from land. Results are reported in metadata that eventually is used in client data searches. The ACCA Threshold parameters are listed in the CPF for use by LPS and possibly IGSs.

·       Solar Spectral Irradiances The LPS ACCA algorithm converts radiometrically corrected data to units of planetary reflectance prior to cloud filtering. This involves normalizing image data for solar irradiance which reduces between-scene variability. The parameter values listed in Table 10.1 are the mean solar spectral irradiances for bands 1 through 5, 7 and 8. There is one value for each band.

• Thermal Constants

    ACCA converts Band 6 from spectral radiance to a more physically useful variable, namely the effective at-satellite temperatures of the viewed Earth-atmosphere system. The transformation equation requires two calibration constants which are listed in table 10.2