Internal error: pybind11::error_already_set called while Python error indicator not set.

[mikekgfb]

🐛 Describe the bug

https://github.com/pytorch/torchat/actions/runs/8643410590/job/23696389630?pr=136

when tracing into an operator, I get the following error. When code is inlined rather than called as operator, this passes (this is not a workaround, though, because it interferes with down-stream IR manipulation of an easy-to-recognize operator"

  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/torch/_subclasses/fake_tensor.py", line 918, in _cached_dispatch_impl
    output = self._dispatch_impl(func, types, args, kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/torch/_subclasses/fake_tensor.py", line 1402, in _dispatch_impl
    r = func(*args, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/torch/_ops.py", line 610, in __call__
    return self_._op(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/runner/work/torchat/torchat/quantized_ops.py", line 52, in embedding_int8
    rw_dtype = result_weights.to(dtype=scales.dtype)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Internal error: pybind11::error_already_set called while Python error indicator not set.

While executing %embedding_int8 : [num_users=3] = call_function[target=torch.ops.torchat.embedding_int8.default](args = (%arg72_1, %l__self___model_tok_embeddings_weight, %l__self___model_tok_embeddings_scales), kwargs = {})
Original traceback:
  File "/Users/runner/work/torchat/torchat/export.py", line 60, in forward
    logits = self.model(idx, input_pos)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/runner/work/torchat/torchat/model.py", line 208, in forward
    x = self.tok_embeddings(idx)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/Users/runner/work/torchat/torchat/quantize.py", line 599, in forward
    return torch.ops.torchat.embedding_int8(

Versions

This happens in the pytorch CI environment, with nightlies

cc @ezyang @msaroufim @bdhirsh @anijain2305 @chauhang

[albanD]

Could you share a repro here to help triage this?