Create test datasets for development and testing

linz / geospatial-data-lake-team

For Geospatial Data Lake team management

0 stars 0 forks source link

Create test datasets for development and testing #49

Closed imincik closed 3 years ago

imincik commented 3 years ago

Tasks

So that we can write quality automated tests we need test datasets.

Acceptance Criteria

[ ] Small test aerial imagery dataset (including STAC metadata) is created in S3 bucket
[ ] Large test aerial imagery dataset (including STAC metadata) is created in S3 bucket
[ ] Small historic imagery dataset (including STAC metadata) is created in S3 bucket

Additional context

Definition of Ready

[x] This story is ready to work on, according to the team's definition

Definition of Done

[ ] This story is done, according to the team's definition

billgeo commented 3 years ago

Sounds good. Is it possible to reuse the Aerial Imagery datasets from Data Lake 2? Also we don't really need the Aerial Imagery one for the first release. So could just do historic imagery for now?

imincik commented 3 years ago

Yes, exactly. We will try to re-use what's possible.

billgeo commented 3 years ago

Decided to use code derived test data instead.

jacobus commented 3 years ago

Just for clarity, if someone (like me ;-) needed sample STAC data sets, where will I find a good representative data set?

billgeo commented 3 years ago

Hey @jacobus. The 'Data Lake Lite' we built last year has some data and STAC metadata in an S3 bucket that might be of use. Or in two weeks we will have some test data as code (or whatever the proper term for that is).

palmerj commented 3 years ago

Very soon there will be a STAC implementation within https://github.com/linz/topo-processor