Open maabdelghaffar opened 6 years ago
Description: After executing Landsat8SearchOperator for many times using the new search criteria (startdate-enddate, etc...), repeated records appeared in the result-set as follows:
[....., ('LC08_L1TP_150037_20170723_20170729_01_T1', 'LC81500372017204LGN00', 'https://s3-us-west-2.amazonaws.com/landsat-pds/c1/L8/150/037/LC08_L1TP_150037_20170723_20170729_01_T1/index.html'), ('LC08_L1TP_150037_20170723_20170729_01_T1', 'LC81500372017204LGN00', 'https://s3-us-west-2.amazonaws.com/landsat-pds/c1/L8/150/037/LC08_L1TP_150037_20170723_20170729_01_T1/index.html'), .......]
Investigation: After exploring the issue, CSV file scene_list which is retrieved from AWS got some repeated lines.
Proposed solution: Update the UpdateSceneList plugin to make sure there is no repeated line in the CSV before importing it to the DB
Description: After executing Landsat8SearchOperator for many times using the new search criteria (startdate-enddate, etc...), repeated records appeared in the result-set as follows:
[....., ('LC08_L1TP_150037_20170723_20170729_01_T1', 'LC81500372017204LGN00', 'https://s3-us-west-2.amazonaws.com/landsat-pds/c1/L8/150/037/LC08_L1TP_150037_20170723_20170729_01_T1/index.html'), ('LC08_L1TP_150037_20170723_20170729_01_T1', 'LC81500372017204LGN00', 'https://s3-us-west-2.amazonaws.com/landsat-pds/c1/L8/150/037/LC08_L1TP_150037_20170723_20170729_01_T1/index.html'), .......]
Investigation: After exploring the issue, CSV file scene_list which is retrieved from AWS got some repeated lines.
Proposed solution: Update the UpdateSceneList plugin to make sure there is no repeated line in the CSV before importing it to the DB