poppy version incompatibility?

[zhutchens1]

The example given in the API documentation does not work for me, due to a poppy issue "forbidden control character" issue. Is webbpsf only compatible with certain versions of poppy? This is running with Python=3.12.2, webbpsf=1.3.0, poppy=1.1.1, synphot=1.4.0, astropy=6.1.0. Thanks in advance for any guidance!

Python 3.12.2 | packaged by conda-forge | (main, Feb 16 2024, 20:50:58) [GCC 12.3.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.25.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import webbpsf

In [2]: nc = webbpsf.NIRCam()

In [3]: nc.filter='F200W'

In [4]: nc.calc_psf(oversample=4)
/home/zhutchen/anaconda3/envs/astro2/lib/python3.12/site-packages/poppy/utils.py:1299: SyntaxWarning: invalid escape sequence '\*'
 """
---------------------------------------------------------------------------
ValueError                Traceback (most recent call last)
Cell In[4], line 1
----> 1 nc.calc_psf(oversample=4)

File ~/anaconda3/envs/astro2/lib/python3.12/site-packages/webbpsf/webbpsf_core.py:1198, in JWInstrument.calc_psf(self, outfile, source, nlambda, monochromatic, fov_arcsec, fov_pixels, oversample, detector_oversample, fft_oversample, overwrite, display, save_intermediates, return_intermediates, normalize, add_distortion, crop_psf)
  1195     self.pupil.update_opd()
  1197 # Run poppy calc_psf
-> 1198 psf = SpaceTelescopeInstrument.calc_psf(
  1199   self,
  1200   outfile=outfile,
  1201   source=source,
  1202   nlambda=nlambda,
  1203   monochromatic=monochromatic,
  1204   fov_arcsec=fov_arcsec,
  1205   fov_pixels=fov_pixels,
  1206   oversample=oversample,
  1207   detector_oversample=detector_oversample,
  1208   fft_oversample=fft_oversample,
  1209   overwrite=overwrite,
  1210   display=display,
  1211   save_intermediates=save_intermediates,
  1212   return_intermediates=return_intermediates,
  1213   normalize=normalize,
  1214 )
  1216 return psf

File ~/anaconda3/envs/astro2/lib/python3.12/site-packages/poppy/instrument.py:265, in Instrument.calc_psf(self, outfile, source, nlambda, monochromatic, fov_arcsec, fov_pixels, oversample, detector_oversample, fft_oversample, overwrite, display, save_intermediates, return_intermediates, normalize)
  263 self._check_for_aliasing(wavelens)
  264 # and use it to compute the PSF (the real work happens here, in code in poppy.py)
--> 265 result = self.optsys.calc_psf(wavelens, weights, display_intermediates=display, display=display,
  266                save_intermediates=save_intermediates, return_intermediates=return_intermediates,
  267                normalize=normalize)
  269 if return_intermediates: # this implies we got handed back a tuple, so split it apart
  270   result, intermediates = result

File ~/anaconda3/envs/astro2/lib/python3.12/site-packages/poppy/utils.py:1436, in BackCompatibleQuantityInput.__call__.<locals>.unit_check_wrapper(*func_args, **func_kwargs)
  1432 # Call the original function with any equivalencies in force.
  1433 with add_enabled_equivalencies(self.equivalencies):
  1434   # print("Args:  {}".format(bound_args.args))
  1435   # print("KWArgs: {}".format(bound_args.kwargs))
-> 1436   return wrapped_function(*bound_args.args, **bound_args.kwargs)

File ~/anaconda3/envs/astro2/lib/python3.12/site-packages/poppy/poppy_core.py:1714, in BaseOpticalSystem.calc_psf(self, wavelength, weight, save_intermediates, save_intermediates_what, display, return_intermediates, return_final, source, normalize, display_intermediates, inwave)
  1712   plt.clf()
  1713 for wlen, wave_weight in zip(wavelength, normwts):
-> 1714   mono_psf, mono_intermediate_wfs = self.propagate_mono(
  1715     wlen,
  1716     retain_intermediates=retain_intermediates,
  1717     retain_final=return_final,
  1718     display_intermediates=display_intermediates,
  1719     normalize=normalize,
  1720     inwave=inwave
  1721   )
  1723   if outfits is None:
  1724     # for the first wavelength processed, set up the arrays where we accumulate the output
  1725     outfits = mono_psf

File ~/anaconda3/envs/astro2/lib/python3.12/site-packages/poppy/utils.py:1436, in BackCompatibleQuantityInput.__call__.<locals>.unit_check_wrapper(*func_args, **func_kwargs)
  1432 # Call the original function with any equivalencies in force.
  1433 with add_enabled_equivalencies(self.equivalencies):
  1434   # print("Args:  {}".format(bound_args.args))
  1435   # print("KWArgs: {}".format(bound_args.kwargs))
-> 1436   return wrapped_function(*bound_args.args, **bound_args.kwargs)

File ~/anaconda3/envs/astro2/lib/python3.12/site-packages/poppy/poppy_core.py:1852, in BaseOpticalSystem.propagate_mono(self, wavelength, normalize, retain_intermediates, retain_final, display_intermediates, inwave)
  1850   wavefront, intermediate_wfs = self.propagate(wavefront, **kwargs)
  1851 else:
-> 1852   wavefront = self.propagate(wavefront, **kwargs)
  1853   intermediate_wfs = []
  1855 if (not retain_intermediates) & retain_final: # return the full complex wavefront of the last plane.

File ~/anaconda3/envs/astro2/lib/python3.12/site-packages/poppy/poppy_core.py:2200, in OpticalSystem.propagate(self, wavefront, normalize, return_intermediates, display_intermediates)
  2198 # The actual propagation:
  2199 wavefront.propagate_to(optic)
-> 2200 wavefront *= optic
  2202 # Normalize if appropriate:
  2203 if normalize.lower() == 'first' and wavefront.current_plane_index == 1: # set entrance plane to 1.

File ~/anaconda3/envs/astro2/lib/python3.12/site-packages/poppy/poppy_core.py:177, in BaseWavefront.__imul__(self, optic)
  174   self.location = 'at ' + optic.name
  175   return self
--> 177 phasor = optic.get_phasor(self)
  179 if not np.isscalar(phasor) and phasor.size > 1:
  180   assert self.wavefront.shape == phasor.shape, "Phasor shape {} does not match wavefront shape {}".format(
  181     phasor.shape, self.wavefront.shape)

File ~/anaconda3/envs/astro2/lib/python3.12/site-packages/poppy/poppy_core.py:2553, in OpticalElement.get_phasor(self, wave)
  2551   trans = self.get_transmission(wave)
  2552   opd = self.get_opd(wave)
-> 2553   self.phasor = ne.evaluate("trans * exp(1.j * opd * scale)")
  2554 else:
  2555   self.phasor = self.get_transmission(wave) * xp.exp(1.j * self.get_opd(wave) * scale)

File ~/anaconda3/envs/astro2/lib/python3.12/site-packages/numexpr/necompiler.py:975, in evaluate(ex, local_dict, global_dict, out, order, casting, sanitize, _frame_depth, **kwargs)
  973   return re_evaluate(local_dict=local_dict, _frame_depth=_frame_depth)
  974 else:
--> 975   raise e

File ~/anaconda3/envs/astro2/lib/python3.12/site-packages/numexpr/necompiler.py:872, in validate(ex, local_dict, global_dict, out, order, casting, _frame_depth, sanitize, **kwargs)
  870 expr_key = (ex, tuple(sorted(context.items())))
  871 if expr_key not in _names_cache:
--> 872   _names_cache[expr_key] = getExprNames(ex, context, sanitize=sanitize)
  873 names, ex_uses_vml = _names_cache[expr_key]
  874 arguments = getArguments(names, local_dict, global_dict, _frame_depth=_frame_depth)

File ~/anaconda3/envs/astro2/lib/python3.12/site-packages/numexpr/necompiler.py:721, in getExprNames(text, context, sanitize)
  720 def getExprNames(text, context, sanitize: bool=True):
--> 721   ex = stringToExpression(text, {}, context, sanitize)
  722   ast = expressionToAST(ex)
  723   input_order = getInputOrder(ast, None)

File ~/anaconda3/envs/astro2/lib/python3.12/site-packages/numexpr/necompiler.py:281, in stringToExpression(s, types, context, sanitize)
  279   no_whitespace = re.sub(r'\s+', '', s)
  280   if _blacklist_re.search(no_whitespace) is not None:
--> 281     raise ValueError(f'Expression {s} has forbidden control characters.')
  283 old_ctx = expressions._context.get_current_context()
  284 try:

ValueError: Expression trans * exp(1.j * opd * scale) has forbidden control characters.```

[mperrin]

Hi, and thanks for reporting this problem. This one looks very familiar: the "forbidden control characters" issue is due to a known bug in an older version of the math library numexpr which is used by poppy. You should be able to fix this by updating to a more recent version of numexpr:

pip install —upgrade numexpr

Please try that and hopefully it will work. If not please let us know!

[mperrin]

@BradleySappington @obi-wan76 This reminds me, we ought to update the requirements.txt to require numexpr > 2.9.0 to help folks avoid the buggy version 2.8.7 of that library... Hopefully that would help others avoid this problem in the future.

[zhutchens1]

Hey all, everything works perfectly after upgrading that package. Thanks very much for the quick response!

FWIW I also think it would be nice to list numexpr>2.9.0 on the Installation webpage.

[BradleySappington]

resolving in poppy PR 632: https://github.com/spacetelescope/poppy/pull/632