NRCan / geo-deep-learning

Deep learning applied to georeferenced datasets
https://geo-deep-learning.readthedocs.io/en/latest/
MIT License
150 stars 50 forks source link

dask_GDL #575

Open MarjanAsgari opened 3 months ago

MarjanAsgari commented 3 months ago

Thank you for submitting your PR and helping us with GDL. Please read the template below, fill it out as appropriate, and make additional changes to your code as needed. Please feel free to submit your PR even if it doesn't satisfy all of the requirements below. If you need assistance or review from a maintainer, add the label Status: Help Needed or Status: Review Needed respectively. After review, a maintainer will add the label Status: Revision Needed if further work is required for the PR to be merged.

Description

This PR intends to include dask for image contrast enhancement to speed up the pre-processing step. In addition, it has removed the download step of reading stack data and instead it is streaming data directly to arrays. Dependencies are:

Type of change

What types of changes does your code introduce? Put an x in all the boxes that apply:

How Has This Been Tested?

Testing part is an ongoing step.

Checklist:

If you're unsure about any of these, don't hesitate to ask. We're here to help!

If your PR does not fulfill all of the requirements in the checklist above, that's OK! If you need help, @-mention a maintainer and/or add the Status: Help Needed label.

MarjanAsgari commented 3 months ago

@remtav Thank you for the comments, for some odd reason, when I was adding the big log files to the .gitignore, it wasn't ignoring them and adding to the repo. I had to close that PR and open a new file with adding description to it as well. The clean commit for review https://github.com/NRCan/geo-deep-learning/pull/575/commits/b952c1a9589af7bbcd166a1e5087cfbfae2a3380