add notebook for preprocessing input datasets

GeoscienceAustralia / uncover-ml

Machine Learning system for Geoscience Australia uncover project

Apache License 2.0

30 stars 20 forks source link

add notebook for preprocessing input datasets #138

Closed sheecegardezi closed 3 years ago

sheecegardezi commented 3 years ago

remove any duplicate datasets in input
if a dataset contains more then one band then create a new dataset for each band
if a dataset in not in EPSG:3577; reproject it in EPSG:3577 projection
if a user has provided a smaller area of interest then crop the input datasets accordingly
create a vector csv file from the input datasets
drop no data values from csv
Save Normalized Dataset
- maximum absolute
- quantile
- standard scaler

sheecegardezi commented 3 years ago

Hi Sheece,

I just noticed that the original notebooks are deleted. I think they should be kept for reference

These notebooks are no longer maintained. I could not reproduce the results shared in these notebooks. But I'll revert this commit as per user interest.

RichardScottOZ commented 3 years ago

Hi Sheece,

I just noticed that the original notebooks are deleted. I think they should be kept for reference

I would agree with that, even if you put them in an 'older work' folder, etc. The methodology/approach/ideas can be valuable - and jumpstarts to other work/developers.