search

flaviostutz / sentinelloader

Sentinel-2 satellite tiles images downloader from Copernicus. Minimizes data download and combines multiple tiles to return a single area of interest.
MIT License
44 stars 18 forks source link

Docker space issue #7

Closed VJ-sts closed 3 years ago

VJ-sts commented 4 years ago

Hi Flavio Stutz Thanks for this very useful repository. I am using it with Docker. But when I go for a larger area of interest and months the docker space goes up and deleted all the output .tif files in Jupyter notebook. However the space in docker not released. I know it is more of docker related issue but have you come across this problem and found any solution? Also, I tried to clone the git code in the local system but I am having an issue with installing gdal versions (near black utility not available in recent gdal) but when I go older versions such as GDAL 2.2.2 I am facing FIONA incompatibility.

flaviostutz commented 4 years ago

I realized this error with recent releases of gdal (lacking near black utility) and didn’t find the utility elsewhere too... if you find, tell us so that we can update this image.

Regarding to free space, have you tried to “docker system prune” your OS after shutting down the container to see if unused instances are wiped out? Maybe some instances or volumes with huge caches are getting stuck.

Another try would be to mount the volumes to your host so that the tiff files won’t be neither on a local Docker volume nor in a Docker container instance and you will have more control over it. Enter the container using shell and discover where those files are and then mount that path as a volume mapping to some dir in your host.

Cache files can be safely deleted as sentinelloader will redownload them if not found.

Hope this helps.

Sent from my iPhone

On 9 Nov 2020, at 02:06, VJ-sts notifications@github.com wrote:

 Hi Flavio Stutz Thanks for this very useful repository. I am using it with Docker. But when I go for a larger area of interest and months the docker space goes up and deleted all the output .tif files in Jupyter notebook. However the space in docker not released. I know it is more of docker related issue but have you come across this problem and found any solution? Also, I tried to clone the git code in the local system but I am having an issue with installing gdal versions (near black utility not available in recent gdal) but when I go older versions such as GDAL 2.2.2 I am facing FIONA incompatibility.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

VJ-sts commented 3 years ago

Sure I guess QGIS has the near black utility function in recent versions too.

https://github.com/qgis/QGIS/blob/master/python/plugins/processing/algs/gdal/nearblack.py

The docker prune in windows is not helpful. I have to purge but it remove all the containers and data.

I will give a try on mounting the host dir.

Thanks