pytroll / tutorial-satpy-half-day

Jupyter Notebook-based tutorial providing an overview of Satpy's features lasting about 4 hours
Other
55 stars 20 forks source link

General questions about using Satpy #4

Closed gersmit closed 3 years ago

gersmit commented 4 years ago

Dear Mr. Hoese, hello DJ,

My name is Ger from the Netherlands and I follow your tutorial-satpy-half-day course via Github. All okay but a small problem question: When I run HRV via the scn_cropped I get the message "ValueError: Can't compare areas of different types". Is it possible to explain to me how to deal with HRV images.

Many thanks in advance for an answer

Regards Ger

djhoese commented 3 years ago

This is a cartopy/matplotlib issue, not a satpy one. Please look at cartopy's examples or my simple examples in this tutorial of using cartopy. My guess is you need to add a projection (or some other keyword argument) to add_subplot, but I'm not completely sure. You'll have to do your own research.

gersmit commented 3 years ago

@djhoese Thanks and i find out how its have to work. ax1 = fig_1.add_subplot(211) change to: ax1 = plt.subplot(211) 👍

gersmit commented 3 years ago

@djhoese An other small qustion/problem: When i use for cropping the coordinate: coor_box(-10,32,25,58) (Europe) new_scn = scn_cropped

Everything oke, when i increases the value 58 (nrd) to 64 (nrd) the image is totally black Even with areas.yaml te same problem.

What do i have to do to make the coordinatebox bigger withoud problem?

Thansk in advance for help.

djhoese commented 3 years ago

Just a guess, but I'm guessing your bounding box is outside or really close to the edge of the projection of the data. Meaning, the upper-right corner (25E, 64N) is outside the data. There is an existing bug report for this problem: https://github.com/pytroll/satpy/issues/1364

This issue talks about fixes that could be done in Satpy/Pyresample to get around this limitation, but in the end it is probably best to choose a different bounding box or define your own AreaDefinition if you need to.

gersmit commented 3 years ago

Dear @djhoese,

It all works fine and great. One small question: Can you tell me where to find the parameters for converting from native files to calculate cities locations and areas.

djhoese commented 3 years ago

native files

What do you mean by this? Converting anything to city locations is not something pytroll tools do right now as far as I know.

gersmit commented 3 years ago

With the separate MSG-4 files we can determine the lon / lat from the x.y. But I am wondering, can we also find this data within the native files? If so, how do I get this information.

djhoese commented 3 years ago

The same operations you do on SEVIRI data from one format should work on the other formats. Anything you do with one format's AreaDefinition you should be able to do with another format's AreaDefinition. If you are seeing something different let us know, but otherwise try out the same code and see if it works.

gersmit commented 3 years ago

@djhoese Everything is going good. One thing, when ik read artikle Enhancing the images, "After the composite is defined and created, it needs to be converted to an image", from url https://satpy.readthedocs.io/en/stable/composites.html. I wil ask you if it is possible to plot coastlines to it, or how or what to use as shape file? I hope you can explain it to me

Thanks Ger

djhoese commented 3 years ago

Yes, but it seems we are missing the direct documentation for that. In your save_datasets method call you can add an overlay keyword argument that looks like:

overlay={'coast_dir': '/path/to/coastlines/directory',
         'overlays': {'grid': grid_parameters, 'coasts': coast_parameters, 'borders': borders_parameters}}

There's a little more information about the options here: https://satpy.readthedocs.io/en/stable/api/satpy.writers.html#satpy.writers.add_overlay

It uses the pycoast package underneath. I'm not completely happy with the way it has to be used, but we haven't finalized what it should look like yet either.

gersmit commented 3 years ago

Thanks, it is something. The /path/to/coastlines/directory is that a self made directory where i put something in, or is it made by pycoast after i install that.

djhoese commented 3 years ago

https://pycoast.readthedocs.io/en/latest/installation.html

gersmit commented 3 years ago

I Install pycoast I make map with shapefiles oke = I put in Python ->from pycoast import ContourWriterAGG overlay={'coast_dir': '/GSHHS_DATA_ROOT', 'overlays': {'grid': grid_parameters, 'coasts': coast_parameters, 'borders': borders_parameters}}

I use it like this: img = get_enhanced_image(composite) img.invert([False, False, False]) img.stretch("linear") img.gamma(1.7) overlay={'coast_dir': 'home/pi/eumetview/GSHHS_DATA_ROOT/', 'overlays': {'grid': grid_parameters, 'coasts': coast_parameters, 'borders': borders_parameters}} img.show() img.save('image.tif')

But i think i forget something because i get error: Traceback (most recent call last): File "/home/pi/eumetview/maak_sneeuw_kaart.py", line 170, in 'overlays': {'grid': grid_parameters, 'coasts': coast_parameters, 'borders': borders_parameters}} NameError: name 'grid_parameters' is not defined

same for coast_parameters and borders_parameters. What do i wrong?

djhoese commented 3 years ago

Sorry, that was just an example. The grid_parameters (and the others) would be dictionaries of options that would get passed to the various "add_X" methods of pycoast. Sorry this isn't more clear. It is something we want to work on. For example, here is the add_borders method:

https://pycoast.readthedocs.io/en/latest/api/pycoast.html#pycoast.cw_agg.ContourWriterAGG.add_borders

So you could pass the keyword arguments for that method as a dictionary in borders_parameters.

gersmit commented 3 years ago

Thanks, It is working when i use : composite = 'VIS008' local_scn.save_dataset(composite, data_img_warmte + 'test.png', overlay={'coast_dir': '/home/pi/eumetview/GSHHS_DATA_ROOT/', 'color': (255,255,255)})

But when i use composite from result GenericCompositor, then ik wil not work. Can you explane to me how to handle to get the same scene to use from the GenericCompositor as from 'VIS008'. When that is possible then i am there where i want to be.

I hope you have some infromation for me wherefor thanks.

djhoese commented 3 years ago

All you've said is it will not work, how? Do you get an error? What does that error say? What does your full code look like when you get that error?

gersmit commented 3 years ago

========== DEZE WERKT ===OVERVIEW 3 LAGEN================

from satpy.writers import get_enhanced_image from satpy.writers import to_image from satpy.composites import GenericCompositor compositor = GenericCompositor("overview") local_scn = scn.resample('seviri_0deg_europe_crop_1km') composite = compositor([local_scn[0.8], local_scn[1.6], local_scn[3.9]])

img = to_image(composite)

img = get_enhanced_image(composite) img.invert([False, False, False]) img.stretch("linear") img.gamma(1.7) img.show() img.save('image.tif') composite = 'VIS008' local_scn.save_dataset(composite, data_img_warmte + 'test.png', overlay={'coast_dir': '/home/pi/eumetview/GSHHS_DATA_ROOT/', 'color': (255,255,255)}) print("coastline to image")

-------------------------------------

This is working, beautifull image with colors at the point img.show() Black and with with coastlines starting from composite = "VIS008".

========== DEZE WERKT NIET ===OVERVIEW 3 LAGEN COMPOSITE================

from satpy.writers import get_enhanced_image from satpy.writers import to_image from satpy.composites import GenericCompositor compositor = GenericCompositor("overview") local_scn = scn.resample('seviri_0deg_europe_crop_1km') composite = compositor([local_scn[0.8], local_scn[1.6], local_scn[3.9]])

img = to_image(composite)

img = get_enhanced_image(composite) img.invert([False, False, False]) img.stretch("linear") img.gamma(1.7) img.show() img.save('image.tif')

local_scn.save_dataset(composite, data_img_warmte + 'test.png', overlay={'coast_dir': '/home/pi/eumetview/GSHHS_DATA_ROOT/', 'color': (255,255,255)}) print("coastline to image")

-------------------------------------

This is not working because i use the composite from the result (3 layers) of compositor = GenericCompositor("overview") GET error: Traceback (most recent call last): File "/home/pi/eumetview/maak_sneeuw_kaart.py", line 170, in local_scn.save_dataset(composite, data_img_warmte + 'test.png', overlay={'coast_dir': '/home/pi/eumetview/GSHHS_DATA_ROOT/', 'color': (255,255,255)}) File "/home/pi/.local/lib/python3.7/site-packages/satpy/scene.py", line 1312, in save_dataset return writer.save_dataset(self[dataset_id], File "/home/pi/.local/lib/python3.7/site-packages/satpy/scene.py", line 690, in getitem return self._datasets[key] File "/home/pi/.local/lib/python3.7/site-packages/satpy/dataset/data_dict.py", line 169, in getitem return super(DatasetDict, self).getitem(item) TypeError: unhashable type: 'DataArray'

+++++++++++++++++++++++++++++++++++++++++++++++++++++

Same situation as above but now i try with IMG as result img = get_enhanced_image(composite): local_scn.save_dataset(img, data_img_warmte + 'test.png', overlay={'coast_dir': '/home/pi/eumetview/GSHHS_DATA_ROOT/', 'color': (255,255,255)}) Error is: Traceback (most recent call last): File "/home/pi/.local/lib/python3.7/site-packages/satpy/dataset/data_dict.py", line 169, in getitem return super(DatasetDict, self).getitem(item) KeyError: <trollimage.xrimage.XRImage object at 0x93466d50>

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/pi/.local/lib/python3.7/site-packages/satpy/dataset/dataid.py", line 705, in _create_id_dict_from_any_key ds_dict = dataset_key.to_dict() AttributeError: 'XRImage' object has no attribute 'to_dict'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/pi/eumetview/maak_sneeuw_kaart.py", line 170, in local_scn.save_dataset(img, data_img_warmte + 'test.png', overlay={'coast_dir': '/home/pi/eumetview/GSHHS_DATA_ROOT/', 'color': (255,255,255)}) File "/home/pi/.local/lib/python3.7/site-packages/satpy/scene.py", line 1312, in save_dataset return writer.save_dataset(self[dataset_id], File "/home/pi/.local/lib/python3.7/site-packages/satpy/scene.py", line 690, in getitem return self._datasets[key] File "/home/pi/.local/lib/python3.7/site-packages/satpy/dataset/data_dict.py", line 171, in getitem key = self.get_key(item) File "/home/pi/.local/lib/python3.7/site-packages/satpy/dataset/data_dict.py", line 159, in get_key best=best, **dfilter) File "/home/pi/.local/lib/python3.7/site-packages/satpy/dataset/data_dict.py", line 103, in get_key key = create_filtered_query(key, query) File "/home/pi/.local/lib/python3.7/site-packages/satpy/dataset/dataid.py", line 690, in create_filtered_query ds_dict = _create_id_dict_from_any_key(dataset_key) File "/home/pi/.local/lib/python3.7/site-packages/satpy/dataset/dataid.py", line 712, in _create_id_dict_from_any_key raise TypeError("Don't know how to interpret a dataset_key of type {}".format(type(dataset_key))) TypeError: Don't know how to interpret a dataset_key of type <class 'trollimage.xrimage.XRImage'>

Is there a possebility to transform the 3 band to 1 band after all colors are made, mabey than it wil work?

djhoese commented 3 years ago

This code:

local_scn.save_dataset(composite, ...

Is not how this is meant to be called when composite is a DataArray object. In your first code you are passing the string 'VIS008' in the second chunk of code you are providing the result of the compositor (a DataArray object). The .save_dataset method is meant to get the name of the product that is already contained in the local_scn object. You could assign your composite by doing:

local_scn['overview'] = composite
local_scn.save_dataset('overview', ...
gersmit commented 3 years ago

@djhoese , many thanks, beautifull. Sorry for the many questions. It is for some of us not easy to understand technical English. Google can translate but not the technical part. The last explanation is bright and give answer how to use some parts. Satpy give nice images and now we can enjoy it more with the coastlines and other arguments.

djhoese commented 3 years ago

Glad you were able to get things working.