'list' object has no attribute 'shape'

[saberioon]

SEQ20210203T093935.zip trying processor on a short sequence and got this error !!

Processing sequence: /Users/mms/Projects/FROM_HY1/SEQ20210203T093935
Reading raw data...
seq /Users/mms/Projects/FROM_HY1/SEQ20210203T093935/metadata.txt
Failed: AttributeError("'list' object has no attribute 'shape'")
Traceback (most recent call last):
  File "/Users/mms/Projects/hypernets_processor/hypernets_processor/main/sequence_processor_main.py", line 126, in main
    sp.process_sequence(target_sequence)
  File "/Users/mms/Projects/hypernets_processor/hypernets_processor/sequence_processor.py", line 105, in process_sequence
    l0_irr,l0_rad,l0_bla,l0_swir_irr,l0_swir_rad,l0_swir_bla = reader.read_sequence(sequence_path,calibration_data_rad,calibration_data_irr,calibration_data_swir_rad,calibration_data_swir_irr)
  File "/Users/mms/Projects/hypernets_processor/hypernets_processor/data_io/hypernets_reader.py", line 906, in read_sequence
    l0_irr,l0_swir_irr = self.read_series_L(seq_dir,seriesIrr,lat,lon,
  File "/Users/mms/Projects/hypernets_processor/hypernets_processor/data_io/hypernets_reader.py", line 591, in read_series_L
    %(vnir.shape,swir.shape))
AttributeError: 'list' object has no attribute 'shape'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/mms/opt/miniconda3/envs/hyProcessor/bin/hypernets_sequence_processor", line 33, in <module>
    sys.exit(load_entry_point('hypernets-processor', 'console_scripts', 'hypernets_sequence_processor')())
  File "/Users/mms/Projects/hypernets_processor/hypernets_processor/cli/sequence_processor_cli.py", line 122, in cli
    main(processor_config_path=PROCESSOR_CONFIG_PATH, job_config_path=job_config_path, to_archive=False)
  File "/Users/mms/Projects/hypernets_processor/hypernets_processor/main/sequence_processor_main.py", line 132, in main
    context.anomaly_db.add_x_anomaly()
AttributeError: 'NoneType' object has no attribute 'add_x_anomaly'

[clemgoyens]

Hi Mehdi I transmit your question to NPL cause this is related to the land processing but I think this should be fixed easily (vnir seems to be a list and not a pandas array?). @pdevis ?

[pdevis]

Just had a look at this. There are a few issues with this sequence. The first is that there seems to be only one scan in each series. This was causing the problem with the list having not attribute shape. The particular problem with the list is easily solved, and is now fixed in my version (which I have not pushed to the master yet). However, having one scan will be a problem in later processing steps, as we take the std between scans to determine the random noise. We have been discussing the option to have the one-off processing possible without uncertainties. When we implement the changes necessary for this, we'll make sure that it works also when using only one scan. We'll keep you posted when this is implemented. The second issue is that there doesn't seem to be any radiance series. only blacks and irradiance. The processor currently expect there to be all three kinds. Is there a user need for running sequences with only radiance or only irradiance? In that case, it would be impossible to do the full processing regardless. But with some restructuring we could process to L1B if there is a significant user need for this.
For now, I think everything should work fine when using sequences that have more than one scan per series and have radiance, irradiance and black series.

[clemgoyens]

Thanks Pieter! Just two comments:

• To check the leveling of the instrument I do an irradiance measurements over the full azimuth. This sequence thus only has irradiance data but the only data I'm interested in is the L1A. So I think there is a need in having this option.
• See issue "Anomaly handling when anomaly_db is None #123", I have slightly changed the sequence_hypernets.py to allow such sequences to be handled (up to a certain level). If there is no radiance or irradiance data is just goes to the following sequence. Hope this helps!

[saberioon]

* To check the leveling of the instrument I do an irradiance measurements over the full azimuth. This sequence thus only has irradiance data but the only data I'm interested in is the L1A. So I think there is a need in having this option.

I also agree and would like to have this option.

[pdevis]

Ok, sounds good. I have now included this following the same approach for changes to sequence_hypernets as for the water network, combined with some edits to the reader for land (and some plotting edits which were necessary when only having a single scan). Using the latest master branch, it is now possible to process that sequence to L1a.