Quick start guide issue?

[cmray]

I am trying to follow the quick start guide, and failing.

I installed DeDop using the shell file DeDop-0.5.2-MacOSX-x86_64.sh, in the default location. The installation was smooth, and DeDop-Shell launches. I can "add" one of the Pacific_Patch L1B files. The file seems to be added to the default directory ~/.dedop/workspaces/default/inputs/CS_LTA__SIR1SAR_FR_20130109T132732_20130109T132816_C001.DBL.nc When I "run" the file DeDop chokes. See the following screen dump. I am guessing that I am misusing the software in my ignorance.

Begin Screen Dump.

Welcome to the DeDop Shell. Type dedop -h to get help.

(DeDop) $ dedop input add /Users/cray/Work2/DeDopData/CS_LTASIR1SAR_FR_20130109T132732_20130109T132816_C001.DBL.nc adding inputs: done
one input added (DeDop) $ dedop run /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CHD file contains unknown parameter: T0_h0_unit_conv_chd UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CHD file contains unknown parameter: pitch_bias_chd UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CHD file contains unknown parameter: yaw_bias_chd UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CHD file contains unknown parameter: N_bursts_cycle_chd UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CHD file contains unknown parameter: onboard_proc_sar_raw_chd UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CHD file contains unknown parameter: pri_T0_unit_conv_chd UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CHD file contains unknown parameter: cai_cor2_unit_conv_chd UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CHD file contains unknown parameter: h0_cor2_unit_conv_chd UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CHD file contains unknown parameter: i_sample_start_chd UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CHD file contains unknown parameter: rfu_rx_gain_ground_chd UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CHD file contains unknown parameter: roll_bias_chd UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CHD file contains unknown parameter: fai_shift_number_chd UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CHD file contains unknown parameter: pri_sar_chd UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: thermal_noise_first_range_bin_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: N_beams_sub_stack_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: step_avg_cal1_pulse_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: n_samples_fitting_rmc_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: zero_freq_sample_number_sar_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: thermal_noise_flag_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: num_lobes_int_cal1_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: min_num_contributing_looks_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: parameter_fitting_flag_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: sigma_z_seed_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: flag_l1bs_file_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: left_avoided_samples_cal2_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: zero_pad_fact_cal1_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: roughness_seed_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: zero_freq_sample_number_lrm_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: epoch_seed_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: flag_azimuth_weighting_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: num_samp_out_wfm_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: pu_seed_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: flag_cal1_source_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: sigma_alt_surf_th_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: thermal_noise_width_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: width_avg_cal1_pulse_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: flag_cal1_power_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: max_fitting_iterations_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: right_avoided_samples_cal2_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: n_samples_fitting_raw_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: decimation_fact_range_l1bs_cnf UnknownParameterWarning(name, cls._id) /Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py:48: UnknownParameterWarning: CNF file contains unknown parameter: k_fact_int_cal1_cnf UnknownParameterWarning(name, cls._id) Traceback (most recent call last): File "/Users/cray/dedop/bin/dedop", line 6, in sys.exit(dedop.ui.cli.main()) File "/Users/cray/dedop/lib/python3.5/site-packages/dedop/ui/cli.py", line 1268, in main status_and_message = args_obj.command_class().execute(args_obj) File "/Users/cray/dedop/lib/python3.5/site-packages/dedop/ui/cli.py", line 255, in execute skip_l1bs=skip_l1bs) File "/Users/cray/dedop/lib/python3.5/site-packages/dedop/ui/cli.py", line 79, in new_l1b_processor return L1BProcessor(name, cnf_file, cst_file, chd_file, output_dir, skip_l1bs) File "/Users/cray/dedop/lib/python3.5/site-packages/dedop/proc/sar/processor.py", line 91, in init self.gap_threshold = self.chd.bri_sar * 1.5 File "/Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/characterization.py", line 87, in bri_sar return 1. / self.brf_sar File "/Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_parameter.py", line 52, in get__ value = self._retreive_value(instance) File "/Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_parameter.py", line 69, in _retreive_value return instance[self.name] File "/Users/cray/dedop/lib/python3.5/site-packages/dedop/conf/auxiliary_file_reader.py", line 72, in getitem raise MissingParameterError(item, self._id) dedop.conf.auxiliary_errors.MissingParameterError: CHD file does not contain a definition for parameter: brf_sar_chd (DeDop) $ dedop output inspec L1B_default.nc usage: dedop output [-h] {clean,cl,list,ls,open,op,inspect,ins,compare,cmp} ... dedop: dedop output: error: invalid choice: 'inspec' (choose from 'clean', 'cl', 'list', 'ls', 'open', 'op', 'inspect', 'ins', 'compare', 'cmp')

(DeDop) $ dedop output inspect L1B_default.nc dedop: L1B product not found: /Users/cray/.dedop/workspaces/default/configs/default/outputs/L1B_default.nc (DeDop) $

[hans-permana]

Hi @cmray,

@mark-ep can correct me if I am wrong, but the input to the processor should at the moment be of L1A type. L1B and L1BS will then be generated with the dedop run command.

This may explain some errors about configurations not found in your stacktrace.

[cmray]

Hi Hans,

I ran the L1B command at the end to verify that the previous "dedop run" command had NOT generated any L1B results.

I was under the impression that the provided Pacific Patch data is L1A. For example CS_LTA__SIR1SAR_FR_20130109T132732_20130109T132816_C001.DBL.nc.

-Chris

[hans-permana]

In that case, I would like to invite @mark-ep to have a look at it.

Would it possible for you to share this input file? I just want to try to replicate this issue with the DeDop Shell in my local.

[cmray]

I got the input file from the http link that David sent via email on Nov 21. I get the same result with any of the Pacific-Patch files. I presume I would have the same results with other files too but have not checked.

I can send a file to you directly if you think I might have corrupted files.

[mark-ep]

Hi,

This looks like your CHD file is out of date. Perhaps you're using a workspace that was created in an earlier version of DeDop?

Could you please try creating a new workspace and processing the file in there?

[cmray]

Hi Mark, Thanks for taking a look. Ok, I will try. Though I am afraid I don't know what a workspace is or how to create a new one. -C

[hans-permana]

I am sorry for the lack of documentation on each command in ReadTheDocs. Right now, the easiest way to have information for DeDop Shell command is by using -h option, for example:

• dedop -h
• dedop w -h

For now, all you need to create a new workspace is dedop w add newWorkspaceName and it automatically changes the current workspace to this. To check your current workspace, use dedop w cur.

I have also verified that with the new set of configurations, the L1A file (I used CS_LTA__SIR1SAR_FR_20130101T101853_20130101T102423_C001.DBL.nc) can be processed successfully.

[cmray]

It worked!
I now also know the right place to look for documentation.
Thank you both for the help.