ssec / polar2grid

Tools for reading, remapping, and writing satellite instrument data.
http://www.ssec.wisc.edu/software/polar2grid/
GNU General Public License v3.0
72 stars 34 forks source link

Error message returned when applying gamma correction? #352

Closed jpnIII closed 3 years ago

jpnIII commented 3 years ago

Hello, all -- We were able to get a new product to start execution, but received an error when trying to apply a gamma correction: TypeError: unsupported operand type(s) for /: 'float' and 'dict' There are several geo2grid.sh log files available on imaginator.ssec.wisc.edu that should help, including the following:

/scratch/tims/cases/2021/may/plumes/various_gammas/redo/abi_l1b_geotiff_20180301_220000.log

The output file from our script that runs geo2grid.sh is the following (see line #595 for the geo2grid.sh commandline): /scratch/tims/cases/2021/may/plumes/various_gammas/redo/xxout_2cases_plume_rgb_day_and_night5_20k_viaabiyaml_various_gammas_redo.txt

Thank you again for your help!

Sincerely,

Jim

djhoese commented 3 years ago

In the future please provide me with the relevant information in the github issue itself. I'd prefer not to have to search through log files and YAML files on a system that I don't normally use and am not familiar with. For example, here is the traceback from the log that would have been useful:

Traceback (most recent call last):
  File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/site-packages/polar2grid/glue.py", line 436, in <module>
    sys.exit(main())
  File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/site-packages/polar2grid/glue.py", line 423, in main
    to_save = write_scene(new_scn, args.writers, writer_args, resampled_products, to_save=to_save)
  File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/site-packages/polar2grid/glue.py", line 144, in write_scene
    res = scn.save_datasets(writer=writer_name, compute=False, datasets=datasets, **wargs)
  File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/site-packages/satpy/scene.py", line 1364, in save_datasets
    return writer.save_datasets(datasets, compute=compute, **save_kwargs)
  File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/site-packages/satpy/writers/__init__.py", line 697, in save_datasets
    results.append(self.save_dataset(ds, compute=False, **kwargs))
  File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/site-packages/satpy/writers/__init__.py", line 828, in save_dataset
    decorate=decorate, fill_value=fill_value)
  File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/site-packages/satpy/writers/__init__.py", line 466, in get_enhanced_image
    enhancer.apply(img, **dataset.attrs)
  File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/site-packages/satpy/writers/__init__.py", line 1057, in apply
    fun(img, *args, **kwargs)
  File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/site-packages/satpy/enhancements/__init__.py", line 35, in gamma
    return img.gamma(**kwargs)
  File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/site-packages/trollimage/xrimage.py", line 938, in gamma
    self.data **= 1.0 / gamma
TypeError: unsupported operand type(s) for /: 'float' and 'dict'

Based on the last couple lines of this it is pretty reasonable to assume something is wrong with the way the "gamma" is defined (as you have already realized based on your title). Since you are attempting to generate this plume product it is also safe to assume that the issue is related to that (or whatever the last change you made was). If we look at your custom YAML file at /home/shared/bin/geo2grid_v_1_0_2/etc/satpy/enhancements/abi.yaml, we can see in the entry for plume_diff_night_default you have:

      kwargs:
        gamma: {0.7, 1.0, 1.7}

The value of gamma should either be a single/scalar value or for RGBs a list of 3 numbers defined with square brackets []. The curly braces define a "dictionary". Changing it to the below should work:

      kwargs:
        gamma: [0.7, 1.0, 1.7]

I'm going to close this as I think this is the right solution. If it doesn't solve it feel free to comment again and we can reopen this issue.

jpnIII commented 3 years ago

Thank you, Dave – I will cut/paste into github in the future. My apologies for having you look on imaginator.

Changing from a dictionary back to a list does the trick. ☺

Later.

Sincerely,

Jim From: David Hoese @.> Sent: Tuesday, May 25, 2021 1:19 PM To: ssec/polar2grid @.> Cc: James P. Nelson @.>; Author @.> Subject: Re: [ssec/polar2grid] Error message returned when applying gamma correction? (#352)

In the future please provide me with the relevant information in the github issue itself. I'd prefer not to have to search through log files and YAML files on a system that I don't normally use and am not familiar with. For example, here is the traceback from the log that would have been useful:

Traceback (most recent call last):

File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/runpy.py", line 193, in _run_module_as_main

"__main__", mod_spec)

File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/runpy.py", line 85, in _run_code

exec(code, run_globals)

File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/site-packages/polar2grid/glue.py", line 436, in

sys.exit(main())

File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/site-packages/polar2grid/glue.py", line 423, in main

to_save = write_scene(new_scn, args.writers, writer_args, resampled_products, to_save=to_save)

File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/site-packages/polar2grid/glue.py", line 144, in write_scene

res = scn.save_datasets(writer=writer_name, compute=False, datasets=datasets, **wargs)

File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/site-packages/satpy/scene.py", line 1364, in save_datasets

return writer.save_datasets(datasets, compute=compute, **save_kwargs)

File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/site-packages/satpy/writers/init.py", line 697, in save_datasets

results.append(self.save_dataset(ds, compute=False, **kwargs))

File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/site-packages/satpy/writers/init.py", line 828, in save_dataset

decorate=decorate, fill_value=fill_value)

File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/site-packages/satpy/writers/init.py", line 466, in get_enhanced_image

enhancer.apply(img, **dataset.attrs)

File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/site-packages/satpy/writers/init.py", line 1057, in apply

fun(img, *args, **kwargs)

File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/site-packages/satpy/enhancements/init.py", line 35, in gamma

return img.gamma(**kwargs)

File "/home/shared/bin/geo2grid_v_1_0_2/lib/python3.7/site-packages/trollimage/xrimage.py", line 938, in gamma

self.data **= 1.0 / gamma

TypeError: unsupported operand type(s) for /: 'float' and 'dict'

Based on the last couple lines of this it is pretty reasonable to assume something is wrong with the way the "gamma" is defined (as you have already realized based on your title). Since you are attempting to generate this plume product it is also safe to assume that the issue is related to that (or whatever the last change you made was). If we look at your custom YAML file at /home/shared/bin/geo2grid_v_1_0_2/etc/satpy/enhancements/abi.yaml, we can see in the entry for plume_diff_night_default you have:

  kwargs:

    gamma: {0.7, 1.0, 1.7}

The value of gamma should either be a single/scalar value or for RGBs a list of 3 numbers defined with square brackets []. The curly braces define a "dictionary". Changing it to the below should work:

  kwargs:

    gamma: [0.7, 1.0, 1.7]

I'm going to close this as I think this is the right solution. If it doesn't solve it feel free to comment again and we can reopen this issue.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/ssec/polar2grid/issues/352#issuecomment-848104669, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AARELF3EWC3FUFWJQTFAACTTPPSYLANCNFSM45P3XETA.