StudentT sampling

[najmehabiri]

Using sampling function on StudentT rise: NotImplementedError: Cannot convert a symbolic Tensor (gradients/stateless_random_gamma/StatelessRandomGammaV2_grad/sub:0) to a numpy array

I have tried few other distributions (Normal,..), and they work fine.

[najmehabiri]

Probably adding: loc = tf.convert_to_tensor(loc) scale = tf.convert_to_tensor(scale)

will fix the problem.

[ColCarroll]

Thanks for reporting! Any chance you can post the version you're using, and an example of when this happens? That's a really reasonable suggestion, but it looks like everything is getting converted to tensor already...

[najmehabiri]

Thank you for getting back to me. You are right; I didn't pay attention to the _n_sample function.

I am using: TensorFlow = 2.4.1 TFP = 0.12.1

This issue happened after upgrading TFP. Sampling is working for other distributions (the ones that are not dependent on Gamma).

[brianwa84]

Is it possible you're leaking a symbolic Tensor out of a tf.function context then using it outside the tf.function?

On Wed, Feb 3, 2021 at 11:20 AM Najmeh Abiri notifications@github.com wrote:

Thank you for getting back to me. You are right; I didn't pay attention to the _n_sample function.

I am using: TensorFlow = 2.4.1 TFP = 0.12.1

This issue happened after upgrading TFP. Sampling is working for other distributions (the ones that are not dependent on Gamma).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/tensorflow/probability/issues/1226#issuecomment-772632813, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFJFSI63W44AP54N4EZQVJDS5FZWBANCNFSM4W6L2UTQ .

[najmehabiri]

Running only these lines gives me the error.

t = tfd.StudentT(df=[2, 3], loc=[1, 2.], scale=[11, 22.]) 
t.sample()

And this is working perfectly:

n = tfd.Normal(loc=[1, 2.], scale=[11, 22.])
n.sample()

[brianwa84]

That code works for me on colab.research.google.com Something must be mixed up with your versions. Can you run pip freeze?

Brian Patton | Software Engineer | bjp@google.com

On Wed, Feb 3, 2021 at 11:55 AM Najmeh Abiri notifications@github.com wrote:

Running only these lines gives me the error.

t = tfd.StudentT(df=[2, 3], loc=[1, 2.], scale=[11, 22.]) t.sample()

And this is working perfectly:

n = tfd.Normal(loc=[1, 2.], scale=[11, 22.])n.sample()

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/tensorflow/probability/issues/1226#issuecomment-772659722, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFJFSI3OTWHE4PKVLOFSTSLS5F5ZHANCNFSM4W6L2UTQ .

[AngelBerihuete]

Hi @najmehabiri @brianwa84

Same problem here. When I try to execute the example in JointDistributionCorutine I have the NotImplementeError

Any update about this?

The example

tfd = tfp.distributions

# Consider the following generative model:
#     e ~ Exponential(rate=[100, 120])
#     g ~ Gamma(concentration=e[0], rate=e[1])
#     n ~ Normal(loc=0, scale=2.)
#     m ~ Normal(loc=n, scale=g)

# In TFP, we can write this as:
Root = tfd.JointDistributionCoroutine.Root  # Convenient alias.
def model():
  e = yield Root(tfd.Independent(tfd.Exponential(rate=[100, 120]), 1))
  g = yield tfd.Gamma(concentration=e[..., 0], rate=e[..., 1])
  n = yield Root(tfd.Normal(loc=0, scale=2.))
  m = yield tfd.Normal(loc=n, scale=g)

joint = tfd.JointDistributionCoroutine(model)

x = joint.sample()

And the result

/usr/lib/python3.9/site-packages/tensorflow/python/framework/ops.py in __array__(self)
    850
    851   def __array__(self):
--> 852     raise NotImplementedError(
    853         "Cannot convert a symbolic Tensor ({}) to a numpy array."
    854         " This error may indicate that you're trying to pass a Tensor to"

NotImplementedError: Cannot convert a symbolic Tensor (gradients/stateless_random_gamma/StatelessRandomGammaV2_grad/sub:0) to a numpy array. This error may indicate that you're trying to pass a Tensor to a NumPy call, which is not supported

pip freeze

absl-py==0.11.0
apparmor==3.0.1
appdirs==1.4.4
arandr==0.1.10
arviz==0.9.0
asciinema==2.0.2
asn1crypto==1.4.0
astor==0.8.1
astroid==2.4.2
astunparse==1.6.3
backcall==0.2.0
Beaker==1.11.0
beautifulsoup4==4.9.3
bleach==3.2.3
btrfsutil==5.10
CacheControl==0.12.6
cachetools==4.2.1
cffi==1.14.4
cftime==1.3.1
chardet==3.0.4
cloudpickle==1.6.0
colorama==0.4.4
configobj==5.1.0.dev0
contextlib2==0.6.0.post1
cryptography==3.3.1
cupshelpers==1.0
cycler==0.10.0
decorator==4.4.2
distlib==0.3.1
distro==1.5.0
dm-tree==0.1.5
docopt==0.6.2
flatbuffers==1.12.0
gast==0.3.3
google-auth==1.24.0
google-auth-oauthlib==0.4.2
google-pasta==0.2.0
greenlet==1.0.0
grpcio==1.35.0
h5py==3.1.0
html5lib==1.1
httplib2==0.18.1
idna==2.10
ipython==7.19.0
ipython-genutils==0.2.0
isc==2.0
isort==5.7.0
jedi==0.17.2
joblib==1.0.0
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.2
keyutils==0.6
kiwisolver==1.3.1
lazy-object-proxy==1.4.3
lensfun==0.3.95
LibAppArmor==3.0.1
lightdm-gtk-greeter-settings==1.2.2
louis==3.16.0
lxml==4.6.2
Mako==1.1.4
Markdown==3.3.3
MarkupSafe==1.1.1
matplotlib==3.3.4
mccabe==0.6.1
msgpack==1.0.2
neovim-remote==2.4.0
netCDF4==1.5.5.1
numpy==1.20.0
oauthlib==3.1.0
opt-einsum==3.3.0
ordered-set==4.0.2
packaging==20.8
pandas==1.2.1
parso==0.7.1
pep517==0.9.1
pexpect==4.8.0
pickleshare==0.7.5
Pillow==8.1.0
ply==3.11
powerline-status==2.8.1
progress==1.5
prompt-toolkit==3.0.14
protobuf==3.12.4
psutil==5.8.0
ptyprocess==0.7.0
pwquality==1.4.4
pyasn1==0.4.8
pyasn1-modules==0.2.8
pybind11==2.6.2
pycairo==1.20.0
pycparser==2.20
pycuda==2020.1
pycups==2.0.1
pycurl==7.43.0.6
Pygments==2.7.4
PyGObject==3.38.0
pylint==2.6.0
pynvim==0.4.2
pyOpenSSL==20.0.1
pyparsing==2.4.7
pyparted==3.11.7
python-dateutil==2.8.1
python-xapp==2.0.2
pytools==2021.1
pytz==2020.5
PyYAML==5.3.1
ranger-fm==1.9.3
Reflector==2021.1.10.0.6.34
requests==2.25.1
requests-oauthlib==1.3.0
resolvelib==0.5.4
retrying==1.3.3
rsa==4.7
scikit-learn==0.24.0
scipy==1.6.0
screeninfo==0.6.7
setproctitle==1.2.2
six==1.15.0
soupsieve==2.1
speedtest-cli==2.1.2
superpaper==2.1.0
system-hotkey==1.0.3
team==1.0
tensorboard==2.4.1
tensorboard-plugin-wit @ file:///build/python-tensorboard_plugin_wit/src/tensorboard_plugin_wit-1.7.0/wit-pip/release/dist/tensorboard_plugin_wit-1.7.0-py3-none-any.whl
tensorflow-estimator @ file:///build/python-tensorflow-estimator/src/estimator_pip/tensorflow_estimator-2.3.0-py2.py3-none-any.whl
tensorflow-gpu @ file:///build/tensorflow/src/tmpoptcuda/tensorflow_gpu-2.4.1-cp39-cp39-linux_x86_64.whl
tensorflow-probability==0.12.1
termcolor==1.1.0
threadpoolctl==2.1.0
toml==0.10.2
traitlets==5.0.5
udiskie==2.3.2
urllib3==1.26.3
validate==5.1.0.dev0
variety==0.8.5
wcwidth==0.2.5
webencodings==0.5.1
Werkzeug==1.0.1
wrapt==1.12.1
wxPython==4.0.7.post2
xarray==0.16.2
xcffib==0.11.1
xpybutil==0.0.6
youtube-dl==2021.1.24.1

[brianwa84]

I'm unable to reproduce with the latest code. Can you see if you can reproduce against tf-nightly and tfp-nightly?

[brianwa84]

I'm also unable to reproduce in colab, with tensorflow==2.4.1 and tensorflow-probability==0.12.1, but colab only has numpy==1.19.5, and py3.6 apparently loses support for numpy 1.20.0? What python are you running?

[AngelBerihuete]

I'm also unable to reproduce in colab, with tensorflow==2.4.1 and tensorflow-probability==0.12.1, but colab only has numpy==1.19.5, and py3.6 apparently loses support for numpy 1.20.0? What python are you running?

python 3.9

[AngelBerihuete]

I'm unable to reproduce with the latest code. Can you see if you can reproduce against tf-nightly and tfp-nightly?

Yes. Finally I've downgraded to py3.8 with tf-nightly and tfp-nightly. Everything works fine. Thanks!

[MedAbdelkaderAbba]

Same problem here. The following code gives errors: import tensorflow as tf import tensorflow_probability as tfp tfp.distributions.Gamma(1.,1.).sample() NotImplementedError: Cannot convert a symbolic Tensor (gradients/stateless_random_gamma/StatelessRandomGammaV2_grad/sub:0) to a numpy array. This error may indicate that you're trying to pass a Tensor to a NumPy call, which is not supported I have python 3.8.5 tensorflow 2.4.0 and tfp 0.12.1, here the pip freeze output:

pip freeze
absl-py==0.11.0
anyio==2.1.0
argon2-cffi==20.1.0
astunparse==1.6.3
async-generator==1.10
attrs==20.3.0
Babel==2.9.0
backcall==0.2.0
bleach==3.3.0
cachetools==4.2.1
certifi==2020.12.5
cffi==1.14.5
chardet==4.0.0
cloudpickle==1.6.0
cycler==0.10.0
decorator==4.4.2
defusedxml==0.6.0
dm-tree==0.1.5
entrypoints==0.3
flatbuffers==1.12
gast==0.3.3
google-auth==1.26.1
google-auth-oauthlib==0.4.2
google-pasta==0.2.0
grpcio==1.32.0
h5py==2.10.0
idna==2.10
ipykernel==5.4.3
ipython==7.20.0
ipython-genutils==0.2.0
ipywidgets==7.6.3
jedi==0.18.0
Jinja2==2.11.3
joblib==1.0.1
json5==0.9.5
jsonschema==3.2.0
jupyter==1.0.0
jupyter-client==6.1.11
jupyter-console==6.2.0
jupyter-core==4.7.1
jupyter-server==1.3.0
jupyterlab==3.0.7
jupyterlab-pygments==0.1.2
jupyterlab-server==2.2.0
jupyterlab-widgets==1.0.0
Keras-Preprocessing==1.1.2
kiwisolver==1.3.1
Markdown==3.3.3
MarkupSafe==1.1.1
matplotlib==3.3.4
mistune==0.8.4
nbclassic==0.2.6
nbclient==0.5.2
nbconvert==6.0.7
nbformat==5.1.2
nest-asyncio==1.5.1
notebook==6.2.0
numpy==1.20.1
oauthlib==3.1.0
opt-einsum==3.3.0
packaging==20.9
pandas==1.2.2
pandocfilters==1.4.3
parso==0.8.1
pexpect==4.8.0
pickleshare==0.7.5
Pillow==8.1.0
prometheus-client==0.9.0
prompt-toolkit==3.0.16
protobuf==3.14.0
ptyprocess==0.7.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.20
Pygments==2.7.4
pyparsing==2.4.7
pyrsistent==0.17.3
python-dateutil==2.8.1
pytz==2021.1
pyzmq==22.0.3
qtconsole==5.0.2
QtPy==1.9.0
requests==2.25.1
requests-oauthlib==1.3.0
rsa==4.7
scikit-learn==0.24.1
scipy==1.6.0
Send2Trash==1.5.0
six==1.15.0
sklearn==0.0
sniffio==1.2.0
tensorboard==2.4.1
tensorboard-plugin-wit==1.8.0
tensorflow==2.4.0
tensorflow-estimator==2.4.0
tensorflow-probability==0.12.0
termcolor==1.1.0
terminado==0.9.2
testpath==0.4.4
threadpoolctl==2.1.0
tornado==6.1
traitlets==5.0.5
typing-extensions==3.7.4.3
urllib3==1.26.3
wcwidth==0.2.5
webencodings==0.5.1
Werkzeug==1.0.1
widgetsnbextension==3.5.1
wrapt==1.12.1`

[fentonscode]

Bumping as this is still persistent in version 2.5 and makes sampling from a Gamma distribution highly unreliable when using the standard Distribution class object. Strangely, it only appears when running or debugging from a script - sampling a Gamma distribution from the Python console is fine. Trying to sample from a Gamma inside a .py file grenades instantly in Eager execution (and has occasionally resulted in graphs which hang indefinitely).

It appears the culprit is a @tf.function decorator buried down in the gamma PRNG routines which creates symbolic tensors that then bubble up to numpy.prod, which is called inside array_ops.py

The best workaround is to call tf.random.gamma directly passing concentration and rate to the alpha and beta parameters respectively

Python version is 3.7.4, on W10 Enterprise

pip freeze trace (filtered for relevance)

Keras-Applications==1.0.8 keras-nightly==2.5.0.dev2021032900 Keras-Preprocessing==1.1.2 numpy==1.20.3 pandas==1.2.4 protobuf==3.10.0 scikit-learn==0.23.1 scipy==1.6.3 tensorboard==2.5.0 tensorboard-data-server==0.6.1 tensorboard-plugin-wit==1.6.0.post3 tensorflow==2.5.0 tensorflow-estimator==2.5.0 tensorflow-probability==0.12.2