search

isce-framework / isce3

InSAR Scientific Computing Environment ***NOTICE - This repository is currently a mirror of an internally hosted project and is in the process of migrating the development to this public repository. In near future we will accept external pull requests.***
Other
128 stars 35 forks source link

Unable to open object (object 'RD' doesn't exist)" #21

Closed AmazingSAR closed 8 months ago

AmazingSAR commented 8 months ago

When testing NISAR Sample Data Product with ISCE3, KeyError occurred in "~/isce3/packages/nisar/products/readers/Raw/Raw.py" line 603.

Exception has occurred: KeyError
"Unable to open object (object 'RD' doesn't exist)"
  File "/home/amazingsar/install/isce3.3/packages/nisar/products/readers/Raw/Raw.py", line 603, in getRdWdWl
    return (fid[path_txrx]["RD"][()], fid[path_txrx]["WD"][()],
  File "/home/amazingsar/install/isce3.3/packages/nisar/antenna/pattern.py", line 289, in __init__
    rd_all[pol], wd_all[pol], wl_all[pol] = raw.getRdWdWl(
  File "/home/amazingsar/install/isce3/packages/nisar/workflows/focus.py", line 1505, in focus
    antpat = AntennaPattern(raw, dem, antparser,
  File "/home/amazingsar/install/isce3/packages/nisar/workflows/focus.py", line 1609, in main
    focus(cfg, args.run_config_path)
  File "/home/amazingsar/install/isce3/packages/nisar/workflows/focus.py", line 1613, in <module>
    main(sys.argv[1:])
KeyError: "Unable to open object (object 'RD' doesn't exist)"

Here is my rslc_config.yaml.

``` runconfig: groups: input_file_group: input_file_path: - NISAR_L0_PR_RRSD_001_005_A_128S_20081012T060910_20081012T060926_P01101_F_J_001.h5 qa_input_file: ./slc.h5 dynamic_ancillary_file_group: dem_file: dem_file_description: orbit: pointing: external_calibration: internal_calibration: REE_INSTRUMENT_TABLE.h5 antenna_pattern: REE_ANTPAT_CUTS_DATA.h5 corner_reflector_file: product_path_group: product_path: /out scratch_path: . sas_output_file: ./rslc.h5 sas_config_file: ./rslc_config.yaml qa_output_dir: ./qa primary_executable: product_type: RSLC product_version: 0.1.0 processing_type: CUSTOM mission_id: processing_center: JPL partial_granule_id: None geometry: relative_orbit_number: 1 frame_number: 1 worker: gpu_enabled: false gpu_id: 0 processing: output_grid: start_time: end_time: start_range: end_range: output_prf: time_snap_interval: 1.0 ephemeris_crop_pad: 10.0 range_window: kind: Kaiser shape: 1.6 azimuth_window: kind: Kaiser shape: 0.0 radio_frequency_interference: detection_enabled: true mitigation_enabled: false mitigation_algorithm: ST-EVD cpi_length: 32 num_range_blocks: 1 max_emitters: 16 num_max_trim: 0 num_min_trim: 0 max_num_rfi_ev: 2 num_cpi_per_threshold_block: 20 threshold_hyperparameters: x: - 2.0 - 20.0 y: - 5.0 - 20.0 range_common_band_filter: attenuation: 40.0 width: 0.15 rdr2geo: tol_height: 1e-05 look_min_deg: 0.0 look_max_deg: 90.0 geo2rdr: tol_aztime: 1e-07 time_start: time_end: doppler: azimuth_boresight_deg: 0.0 interp_method: bilinear spacing: range: 2000.0 azimuth: 1.0 rangecomp: mode: full block_size: range: 0 azimuth: 1024 azcomp: block_size: range: 64 azimuth: 64 azimuth_resolution: 6.0 kernel: type: Knab halfwidth: 4 approx_oversample: 1.7 fit: Table fit_order: 2048 dry_troposphere_model: nodelay dem: reference_height: 0.0 interp_method: biquintic nominal_antenna_size: range: 12.0 azimuth: 12.0 encoding_scale_factor: 1e-08 delete_tempfiles: true debug_dump_height: false is_enabled: presum_blu: true rangecomp: true eap: true range_cor: true azcomp: true qa: workflows: validate: true qa_reports: true absolute_radiometric_calibration: true noise_estimation: true point_target_analyzer: true qa_reports: backscatter_img: linear_units: true nlooks_freqa: nlooks_freqb: longest_side_max: 2048 percentile_for_clipping: - 5.0 - 95.0 gamma: 0.5 tile_shape: - 1024 - 1024 histogram: decimation_ratio: - 10 - 10 backscatter_histogram_bin_edges_range: - -80.0 - 20.0 phs_in_radians: true tile_shape: - 1024 - -1 range_spectra: az_decimation: 8 hz_to_mhz: true tile_height: 512 absolute_radiometric_calibration: nchip: 64 upsample_factor: 32 peak_find_domain: time nfit: 5 power_method: box power_threshold: 3.0 point_target_analyzer: nchip: 64 upsample_factor: 32 peak_find_domain: time num_sidelobes: 10 predict_null: true fs_bw_ratio: 1.2 window_type: rect window_parameter: 0.0 ```

Does anyone know what happened here?

bhawkins-jpl commented 8 months ago

The error message could be better, but I'd guess that what's happening is you're processing data from ALOS that's been packed in a NISAR-like raw data format. It's trying to read SweepSAR parameters from the raw data and not finding them. To work around this issue you can disable elevation antenna pattern correction by setting eap: false in the runconfig, e.g.,

runconfig:
    groups:
        processing:
            is_enabled:
                eap: false
AmazingSAR commented 8 months ago

I disabled elevation antenna pattern correction in the runconfig. The focus workflow works fine. But the result seems wrong.

  1. Focusing raw data
  2. Data extraction
  3. Data visulization

1.Focusing process

Input:

amazingsar@geekpro-17irb:~/data1$ python3 ~/install/isce3.3/packages/nisar/workflows/focus.py rslc_config.yaml

Output:

```shell 2024-03-05 16:47:04.778, WARNING, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:88, "Skipping input validation." 2024-03-05 16:47:04.778, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1657, "Logging configuration to file ./rslc_config.yaml." 2024-03-05 16:47:04.787, INFO, RSLC, Raw, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/readers/Raw/Raw.py:41, "Reading L0B file /home/amazingsar/data1/ALPSRP037370690.h5" 2024-03-05 16:47:04.790, WARNING, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:310, "No DEM given, using ref height 0.0 (m)." 2024-03-05 16:47:04.792, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1339, "output mode = PolChannelSet({PolChannel(freq_id='A', pol='HH', band=Band(center=1269999750.0604727, width=28000000.0))})" 2024-03-05 16:47:04.877, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1347, "Processing using CUDA device 0 (NVIDIA GeForce RTX 4060 Ti)" 2024-03-05 16:47:04.878, INFO, RSLC, Raw, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/readers/Raw/Raw.py:41, "Reading L0B file ALPSRP037370690.h5" 2024-03-05 16:47:04.882, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1355, "Raw data time spans [22197.003100315524, 22213.402546775807] seconds since 2006-10-07T00:00:00.000000000." 2024-03-05 16:47:04.882, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1356, "Raw data range swath spans [743438.0, 776429.2231514688] meters." 2024-03-05 16:47:04.882, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:233, "Loading orbit from L0B file." 2024-03-05 16:47:04.883, INFO, RSLC, Raw, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/readers/Raw/Raw.py:41, "Reading L0B file ALPSRP037370690.h5" 2024-03-05 16:47:04.887, INFO, RSLC, Raw, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/readers/Raw/Raw.py:41, "Reading L0B file ALPSRP037370690.h5" 2024-03-05 16:47:04.891, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:203, "Original orbit data file spans time interval [2006-10-07T05:58:00.000000000, 2006-10-07T06:25:00.000000000]" 2024-03-05 16:47:04.891, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:205, "Cropping orbit to 3 points beyond [2006-10-07T06:09:47.003100316, 2006-10-07T06:10:23.402546776]" 2024-03-05 16:47:04.891, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:212, "Cropped orbit to interval [2006-10-07T06:06:00.000000000, 2006-10-07T06:14:00.000000000]" 2024-03-05 16:47:04.891, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:256, "Loading attitude from L0B file." 2024-03-05 16:47:04.891, INFO, RSLC, Raw, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/readers/Raw/Raw.py:41, "Reading L0B file ALPSRP037370690.h5" 2024-03-05 16:47:04.894, INFO, RSLC, Raw, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/readers/Raw/Raw.py:41, "Reading L0B file ALPSRP037370690.h5" 2024-03-05 16:47:04.898, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:203, "Original attitude data file spans time interval [2006-10-07T06:09:55.013000000, 2006-10-07T06:10:16.013000000]" 2024-03-05 16:47:04.898, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:205, "Cropping attitude to 1 points beyond [2006-10-07T06:09:47.003100316, 2006-10-07T06:10:23.402546776]" 2024-03-05 16:47:04.898, WARNING, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:210, "Failed to crop attitude. Trying to proceed anyway." 2024-03-05 16:47:04.898, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1361, "Verifying ephemeris covers time span of raw data." 2024-03-05 16:47:04.898, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:433, "Generating Doppler LUT from pointing" 2024-03-05 16:47:04.898, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:233, "Loading orbit from L0B file." 2024-03-05 16:47:04.898, INFO, RSLC, Raw, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/readers/Raw/Raw.py:41, "Reading L0B file ALPSRP037370690.h5" 2024-03-05 16:47:04.902, INFO, RSLC, Raw, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/readers/Raw/Raw.py:41, "Reading L0B file ALPSRP037370690.h5" 2024-03-05 16:47:04.906, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:203, "Original orbit data file spans time interval [2006-10-07T05:58:00.000000000, 2006-10-07T06:25:00.000000000]" 2024-03-05 16:47:04.906, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:205, "Cropping orbit to 3 points beyond [2006-10-07T06:09:47.003100316, 2006-10-07T06:10:23.402546776]" 2024-03-05 16:47:04.906, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:212, "Cropped orbit to interval [2006-10-07T06:06:00.000000000, 2006-10-07T06:14:00.000000000]" 2024-03-05 16:47:04.906, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:256, "Loading attitude from L0B file." 2024-03-05 16:47:04.906, INFO, RSLC, Raw, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/readers/Raw/Raw.py:41, "Reading L0B file ALPSRP037370690.h5" 2024-03-05 16:47:04.910, INFO, RSLC, Raw, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/readers/Raw/Raw.py:41, "Reading L0B file ALPSRP037370690.h5" 2024-03-05 16:47:04.913, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:203, "Original attitude data file spans time interval [2006-10-07T06:09:55.013000000, 2006-10-07T06:10:16.013000000]" 2024-03-05 16:47:04.913, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:205, "Cropping attitude to 1 points beyond [2006-10-07T06:09:47.003100316, 2006-10-07T06:10:23.402546776]" 2024-03-05 16:47:04.913, WARNING, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:210, "Failed to crop attitude. Trying to proceed anyway." 2024-03-05 16:47:04.913, WARNING, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:310, "No DEM given, using ref height 0.0 (m)." 2024-03-05 16:47:04.914, INFO, RSLC, Raw, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/readers/Raw/Raw.py:41, "Reading L0B file ALPSRP037370690.h5" 2024-03-05 16:47:04.916, INFO, RSLC, Raw, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/readers/Raw/Raw.py:41, "Reading L0B file ALPSRP037370690.h5" 2024-03-05 16:47:04.919, INFO, RSLC, Raw, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/readers/Raw/Raw.py:41, "Reading L0B file ALPSRP037370690.h5" 2024-03-05 16:47:04.923, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:400, "Attempting to find reasonable EL search bounds." 2024-03-05 16:47:04.924, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:407, "Preliminary EL bounds are [-6.241, 4.061] deg" 2024-03-05 16:47:04.926, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:447, "Made Doppler LUT for fc=1269999750.0604727 Hz, az=0.0 deg with mean=77.76628922315598 Hz" 2024-03-05 16:47:04.926, INFO, RSLC, Raw, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/readers/Raw/Raw.py:41, "Reading L0B file ALPSRP037370690.h5" 2024-03-05 16:47:04.930, DEBUG, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:534, "Approximate synthetic aperture duration is 2.2264258217058535 s." 2024-03-05 16:47:04.931, DEBUG, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:578, "offset0 = 0.0" 2024-03-05 16:47:04.931, DEBUG, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:584, "offset1 = 0.0" 2024-03-05 16:47:04.931, DEBUG, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:595, "Reskew time offset at start 0.136591743081226 s" 2024-03-05 16:47:04.931, DEBUG, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:596, "Reskew time offset at end 0.13322250116834766 s" 2024-03-05 16:47:04.931, DEBUG, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:597, "Reskew range offset at start -0.6430301590589806 m" 2024-03-05 16:47:04.931, DEBUG, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:598, "Reskew range offset at end -0.6016614780528471 m" 2024-03-05 16:47:04.931, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:602, "Approximate fully focusable time interval is [2006-10-07T06:09:58.252904969, 2006-10-07T06:10:12.422556366]" 2024-03-05 16:47:04.931, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:603, "Approximate fully focusable range interval is [743437.3569698409, 772381.4233069908]" 2024-03-05 16:47:04.931, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1382, "Output grid A is RadarGridParameters(sensing_start=22198.252904969457, wavelength=0.2360571, prf=1862.1969999999324, starting_range=743437.3569698409, range_pixel_spacing=4.461197291666666, lookside=LookSide.Right, length=26387, width=6488, ref_epoch=2006-10-07T00:00:00.000000000)" 2024-03-05 16:47:06.971, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1390, "Need to process raw data time span [22196.90070108831, 22213.50708700978] seconds since 2006-10-07T00:00:00.000000000 to produce requested output grid." 2024-03-05 16:47:06.971, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1402, "Creating output RSLC product /home/amazingsar/data1/rslc.h5" 2024-03-05 16:47:06.972, DEBUG, RSLC, SLC, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/writers/SLC.py:317, "Creating group /science/LSAR/RSLC" 2024-03-05 16:47:06.972, INFO, RSLC, SLC, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/writers/SLC.py:563, "Writing orbit to SLC" 2024-03-05 16:47:06.973, INFO, RSLC, SLC, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/writers/SLC.py:583, "Writing attitude to SLC" 2024-03-05 16:47:06.976, INFO, RSLC, SLC, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/writers/SLC.py:661, "Populating identification based on /home/amazingsar/data1/ALPSRP037370690.h5" 2024-03-05 16:47:06.982, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1224, "No calibration file provided. Using defaults." 2024-03-05 16:47:06.989, INFO, RSLC, SLC, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/writers/SLC.py:409, "Saving Doppler for frequency A" 2024-03-05 16:47:07.178, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1224, "No calibration file provided. Using defaults." 2024-03-05 16:47:07.185, INFO, RSLC, SLC, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/writers/SLC.py:802, "Creating geolocationGrid." 2024-03-05 16:47:07.186, DEBUG, RSLC, SLC, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/writers/SLC.py:317, "Creating group /science/LSAR/RSLC/metadata/geolocationGrid" 2024-03-05 16:47:07.186, DEBUG, RSLC, SLC, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/writers/SLC.py:313, "Creating dataset /science/LSAR/RSLC/metadata/geolocationGrid/epsg" 2024-03-05 16:47:07.186, DEBUG, RSLC, SLC, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/writers/SLC.py:313, "Creating dataset /science/LSAR/RSLC/metadata/geolocationGrid/slantRange" 2024-03-05 16:47:07.187, DEBUG, RSLC, SLC, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/writers/SLC.py:313, "Creating dataset /science/LSAR/RSLC/metadata/geolocationGrid/zeroDopplerTime" 2024-03-05 16:47:07.187, DEBUG, RSLC, SLC, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/writers/SLC.py:313, "Creating dataset /science/LSAR/RSLC/metadata/geolocationGrid/heightAboveEllipsoid" journal: cube height: 20 journal: -- cube length: 217 -- cube width: 58 journal: EPSG: 4326 2024-03-05 16:47:07.455, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1480, "Processing frequencyA HH" 2024-03-05 16:47:07.455, INFO, RSLC, SLC, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/writers/SLC.py:460, "Creating SLC image for frequency=A pol=HH" 2024-03-05 16:47:07.457, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1487, "Using raw data channel PolChannel(freq_id='A', pol='HH', band=Band(center=1269999750.0604727, width=28000000.0))" 2024-03-05 16:47:07.458, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1224, "No calibration file provided. Using defaults." 2024-03-05 16:47:07.461, INFO, RSLC, DataDecoder, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/readers/Raw/DataDecoder.py:47, "Decoding raw data with lookup table." 2024-03-05 16:47:07.461, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1495, "Raw data shape = (30540, 7044)" 2024-03-05 16:47:07.462, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1501, "Using pulses [0, 30540]" 2024-03-05 16:47:07.467, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1511, "Number of sub-swaths = 1" 2024-03-05 16:47:07.467, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1514, "Decoding raw data to memory map /home/amazingsar/data1/tmpg6q_4963_raw.c8." 2024-03-05 16:47:28.047, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:983, "Running radio frequency interference (RFI) detection" 2024-03-05 16:47:34.477, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1011, "RFI likelihood = 0.1708595387840671" 2024-03-05 16:47:34.478, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1534, "Uniform PRF, using raw data directly." 2024-03-05 16:47:34.479, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1123, "Generating chirp" 2024-03-05 16:47:34.479, INFO, RSLC, Raw, 999999, /home/amazingsar/install/isce3.3/packages/nisar/products/readers/Raw/Raw.py:103, "Chirp(-1037037037037.037, 2.7000000000000002e-05, 32000000.0)" 2024-03-05 16:47:34.479, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1126, "Chirp length = 865" 2024-03-05 16:47:34.479, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:117, "Kaiser(beta=1.6)" 2024-03-05 16:47:34.480, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1150, "Normalizing chirp to unit white noise gain." 2024-03-05 16:47:34.480, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1171, "Renormalizing chirp by signal gain terms wavelength * PRF * sqrt(TBP) = 12086.58436684436" 2024-03-05 16:47:34.480, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1177, "Scaling chirp by calibration factor = (1+0j)" 2024-03-05 16:47:34.480, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1181, "Preparing range compressor with mode=Mode.Full" 2024-03-05 16:47:36.592, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1196, "Adjusting starting range by delay calibration = 0.0 m" 2024-03-05 16:47:36.593, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1550, "Writing range compressed data to /home/amazingsar/data1/tmp106_cpeu_rc.c8" 2024-03-05 16:47:36.610, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1552, "Range compressed data shape = (30540, 7908)" 2024-03-05 16:47:36.610, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1560, "Range compressing block at pulse 0" 2024-03-05 16:47:36.670, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1576, "Compensating range loss" 2024-03-05 16:47:36.693, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1560, "Range compressing block at pulse 1024" 2024-03-05 16:47:36.754, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1576, "Compensating range loss" 2024-03-05 16:47:36.781, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1560, "Range compressing block at pulse 2048" 2024-03-05 16:47:36.826, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1576, "Compensating range loss" ......... 2024-03-05 16:55:45.838, DEBUG, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:847, "scaled max component = 2.474508098426753e-10" 2024-03-05 16:55:45.857, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1626, "Image statistics A HH = StatsRealImagFloat32(real=StatsFloat32(n_valid=171198856, mean=-2.1622777353347278e-12, min=-6.899431070905848e-08, max=5.664902147373141e-08, sample_stddev=1.8534578039020033e-10), imag=StatsFloat32(n_valid=171198856, mean=8.585247313411259e-13, min=-4.509920614736984e-08, max=9.678218049202769e-08, sample_stddev=1.8471717211365757e-10))" 2024-03-05 16:55:45.876, INFO, RSLC, focus, 999999, /home/amazingsar/install/isce3.3/packages/nisar/workflows/focus.py:1630, "All done!" ```

2.Data extraction

Input:

amazingsar@geekpro-17irb:~/data1$ gdal_translate -of ENVI HDF5:"rslc.h5"://science/LSAR/RSLC/swaths/frequencyA/HH HH.slc

Output:

Input file size is 6488, 26388
0...10...20...30...40...50...60...70...80...90...100 - done.

3.Data visulization

Run python script from SDS-on-command

```python import numpy as np import osgeo.gdal as gdal import matplotlib.pyplot as plt ds = gdal.Open("/home/amazingsar/data1/HH.slc", gdal.GA_ReadOnly) x0 = 0 y0 = 0 x_offset = 1000 y_offset = 1000 slc = ds.GetRasterBand(1).ReadAsArray(x0, y0, x_offset, y_offset) ds = None fig = plt.figure(figsize=(14, 12)) #display amplitude of the slc ax = fig.add_subplot(1,2,1) ax.imshow(np.abs(slc),cmap='gray') ax.set_title("amplitude") #display phase of the slc ax = fig.add_subplot(1,2,2) ax.imshow(np.angle(slc)) ax.set_title("phase") plt.show() slc = None ```

Output:

output

Are there any example config file for the official example data set?

bhawkins-jpl commented 8 months ago

First, SLC amplitudes tend to have a large dynamic range, so we usually view the images on a log scale (or some other nonlinear color scale). So instead you might try

power_db = 20 * np.log10(np.abs(slc))
plt.imshow(power_db, cmap="gray")

You may need to add some vmin and vmax parameters to imshow to get a satisfactory plot. I usually plot a dynamic range of 40 dB or so.

Second, depending on what version of isce3 you're running, the pixel values may be stored using 16-bit floating point. In that case you need to have the values at least roughly calibrated or you risk losing precision. I notice in your config file you have encoding_scale_factor: 1e-08 and then in the log I see the stats reports SLC values of similarly small magnitude. While you could supply a separate calibration file, for your purposes it's probably simpler to just change encoding_scale_factor: 1.0 to get more reasonable values.

AmazingSAR commented 8 months ago

@bhawkins-jpl Thanks a lot for your reply! The result seems reasonable now except artificial traces in dark areas such as lakes. The results are displayed as follows: full area full lake area horizontal The artificial traces along horizontal parallel to radar range direction. Are these artificial traces related to the absence of EAP processing? And how can I do to remove it?

bhawkins-jpl commented 8 months ago

Those artifacts are due to radio frequency interference (RFI). Removing RFI is an ongoing research topic. There's already an algorithm the code, but it's not enabled by default. You could try setting mitigation_enabled: true in your config file (example) and see if that improves the image.

AmazingSAR commented 8 months ago

@bhawkins-jpl Thanks again, I have tried setting mitigation_enabled: true option in my config file.

RFI

The RFI removing algorithm works but the task is too difficult.

@bhawkins-jpl Many thanks for your kind and warm help.