Three-fiber OrbitControls and "R3F: Hooks can only be used within the Canvas component!" error

[Krande]

Hey!

I just discovered this library and think it looks really interesting!

I did a small experiment to see if ipyreact could replace some stuff I've made in the past using pythreejs. However I ran into a minor snag when trying to bring in an <OrbitControls/> component that I was wondering any of you might help me solve?

I should also mention that my frontend experience is limited, so I might (hopefully) be asking a dumb question and it's also likely that my question isn't really related to ipyreact itself:)

Anyways, I started by using the ESM>Playground where I created a minimal three-fiber example hoping I would be able to get this to work in my jupyterlab environment.

import React, { useRef, useState } from "react"
import { Canvas, useFrame, useThree } from 'https://esm.sh/@react-three/fiber@8.13.0'
import { OrbitControls } from "https://esm.sh/@react-three/drei@9.68.6";

function Box({position, color}) {
  const ref = useRef()
  useFrame(() => (ref.current.rotation.x = ref.current.rotation.y += 0.01))

  return (
    <mesh position={position} ref={ref}>
      <boxBufferGeometry args={[1, 1, 1]} attach="geometry" />
      <meshPhongMaterial color={color} attach="material" />
    </mesh>
  )
}

function App() {
  return (
    <Canvas>
      <Box color="#18a36e" position={[-1, 0, 3]} />
      <Box color="#f56f42" position={[1, 0, 3]} />
      <OrbitControls/>
      <directionalLight color="#ffffff" intensity={1} position={[-1, 2, 4]} />
    </Canvas>
  )
}

export default App

Okay, the controls and everything (except I was unable to get tailwind css working) seems to work as expected.

However, when copying this into my ipyreact environment tt seems that using the <OrbitControls/> component generates the error

R3F: Hooks can only be used within the Canvas component!

By simply removing the <OrbitControls/> element I was able to get the example working

When searching for an answer I found this but I'm not that familiar with the r3f hooks concept that I immediately understand how to fix it.

Anyone pointing me in the direction of solving this would be much appreciated!

Best Regards Kristoffer

[maartenbreddels]

Thank you.

The downside of this library is that we see all kinds of errors I know nothing about :) I will do my best to give pointers though :) Actually, interestingly I get a different error:

fs.RegExpParser is not a constructor

Too bad ipyreact doesn't show a tracktrace, I think we should fix that. I hope someone else has an idea.

[Krande]

@maartenbreddels Thanks for responding so quickly :)

I'll continue to do some digging and update this issue if I find anything!

Btw: If it can help debugging why you're seeing a different error here is the environment I tested it on:

(I'm using JupyterLab btw)

# Name                    Version                   Build  Channel
anyio                     3.6.2              pyhd8ed1ab_0    conda-forge
anywidget                 0.3.0                    pypi_0    pypi
argon2-cffi               21.3.0             pyhd8ed1ab_0    conda-forge
argon2-cffi-bindings      21.2.0          py311ha68e1ae_3    conda-forge
asttokens                 2.2.1              pyhd8ed1ab_0    conda-forge
async-lru                 2.0.2              pyhd8ed1ab_0    conda-forge
attrs                     23.1.0             pyh71513ae_1    conda-forge
babel                     2.12.1             pyhd8ed1ab_1    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.functools_lru_cache 1.6.4              pyhd8ed1ab_0    conda-forge
beautifulsoup4            4.12.2             pyha770c72_0    conda-forge
bleach                    6.0.0              pyhd8ed1ab_0    conda-forge
brotli                    1.0.9                hcfcfb64_8    conda-forge
brotli-bin                1.0.9                hcfcfb64_8    conda-forge
bzip2                     1.0.8                h8ffe710_4    conda-forge
ca-certificates           2023.5.7             h56e8100_0    conda-forge
certifi                   2023.5.7           pyhd8ed1ab_0    conda-forge
cffi                      1.15.1          py311h7d9ee11_3    conda-forge
charset-normalizer        3.1.0              pyhd8ed1ab_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
comm                      0.1.3              pyhd8ed1ab_0    conda-forge
debugpy                   1.6.7           py311h12c1d0e_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
entrypoints               0.4                pyhd8ed1ab_0    conda-forge
executing                 1.2.0              pyhd8ed1ab_0    conda-forge
flit-core                 3.9.0              pyhd8ed1ab_0    conda-forge
idna                      3.4                pyhd8ed1ab_0    conda-forge
importlib-metadata        6.6.0              pyha770c72_0    conda-forge
importlib_metadata        6.6.0                hd8ed1ab_0    conda-forge
importlib_resources       5.12.0             pyhd8ed1ab_0    conda-forge
ipykernel                 6.23.1             pyh025b116_0    conda-forge
ipyreact                  0.3.0                    pypi_0    pypi
ipython                   8.13.2             pyh08f2357_0    conda-forge
ipywidgets                8.0.6                    pypi_0    pypi
jedi                      0.18.2             pyhd8ed1ab_0    conda-forge
jinja2                    3.1.2              pyhd8ed1ab_1    conda-forge
json5                     0.9.5              pyh9f0ad1d_0    conda-forge
jsonschema                4.17.3             pyhd8ed1ab_0    conda-forge
jupyter-lsp               2.1.0              pyhd8ed1ab_0    conda-forge
jupyter_client            8.2.0              pyhd8ed1ab_0    conda-forge
jupyter_core              5.3.0           py311h1ea47a8_0    conda-forge
jupyter_events            0.6.3              pyhd8ed1ab_0    conda-forge
jupyter_server            2.5.0              pyhd8ed1ab_0    conda-forge
jupyter_server_terminals  0.4.4              pyhd8ed1ab_1    conda-forge
jupyterlab                4.0.0              pyhd8ed1ab_1    conda-forge
jupyterlab-widgets        3.0.7                    pypi_0    pypi
jupyterlab_pygments       0.2.2              pyhd8ed1ab_0    conda-forge
jupyterlab_server         2.22.1             pyhd8ed1ab_0    conda-forge
libbrotlicommon           1.0.9                hcfcfb64_8    conda-forge
libbrotlidec              1.0.9                hcfcfb64_8    conda-forge
libbrotlienc              1.0.9                hcfcfb64_8    conda-forge
libexpat                  2.5.0                h63175ca_1    conda-forge
libffi                    3.4.2                h8ffe710_5    conda-forge
libsodium                 1.0.18               h8d14728_1    conda-forge
libsqlite                 3.42.0               hcfcfb64_0    conda-forge
libzlib                   1.2.13               hcfcfb64_4    conda-forge
markupsafe                2.1.2           py311ha68e1ae_0    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
mistune                   2.0.5              pyhd8ed1ab_0    conda-forge
mypy-extensions           1.0.0                    pypi_0    pypi
nbclient                  0.8.0              pyhd8ed1ab_0    conda-forge
nbconvert-core            7.4.0              pyhd8ed1ab_0    conda-forge
nbformat                  5.8.0              pyhd8ed1ab_0    conda-forge
nest-asyncio              1.5.6              pyhd8ed1ab_0    conda-forge
nodejs                    18.15.0              h57928b3_0    conda-forge
notebook-shim             0.2.3              pyhd8ed1ab_0    conda-forge
openssl                   3.1.0                hcfcfb64_3    conda-forge
packaging                 23.1               pyhd8ed1ab_0    conda-forge
pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pip                       23.1.2             pyhd8ed1ab_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_0    conda-forge
platformdirs              3.5.1              pyhd8ed1ab_0    conda-forge
prometheus_client         0.16.0             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.38             pyha770c72_0    conda-forge
prompt_toolkit            3.0.38               hd8ed1ab_0    conda-forge
psutil                    5.9.5           py311ha68e1ae_0    conda-forge
psygnal                   0.9.0                    pypi_0    pypi
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pygments                  2.15.1             pyhd8ed1ab_0    conda-forge
pyrsistent                0.19.3          py311ha68e1ae_0    conda-forge
pysocks                   1.7.1              pyh0701188_6    conda-forge
python                    3.11.3          h2628c8c_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-fastjsonschema     2.17.1             pyhd8ed1ab_0    conda-forge
python-json-logger        2.0.7              pyhd8ed1ab_0    conda-forge
python_abi                3.11                    3_cp311    conda-forge
pytz                      2023.3             pyhd8ed1ab_0    conda-forge
pywin32                   306                      pypi_0    pypi
pywinpty                  2.0.10          py311h12c1d0e_0    conda-forge
pyyaml                    6.0             py311ha68e1ae_5    conda-forge
pyzmq                     25.0.2          py311h7b3f143_0    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
rfc3339-validator         0.1.4              pyhd8ed1ab_0    conda-forge
rfc3986-validator         0.1.1              pyh9f0ad1d_0    conda-forge
send2trash                1.8.2              pyh08f2357_0    conda-forge
setuptools                67.7.2             pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
sniffio                   1.3.0              pyhd8ed1ab_0    conda-forge
soupsieve                 2.3.2.post1        pyhd8ed1ab_0    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
terminado                 0.17.0             pyh08f2357_0    conda-forge
tinycss2                  1.2.1              pyhd8ed1ab_0    conda-forge
tk                        8.6.12               h8ffe710_0    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
tornado                   6.3.2           py311ha68e1ae_0    conda-forge
traitlets                 5.9.0              pyhd8ed1ab_0    conda-forge
typing-extensions         4.6.1                    pypi_0    pypi
typing_extensions         4.6.0              pyha770c72_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
ucrt                      10.0.22621.0         h57928b3_0    conda-forge
urllib3                   2.0.2              pyhd8ed1ab_0    conda-forge
vc                        14.3                hb25d44b_16    conda-forge
vc14_runtime              14.34.31931         h5081d32_16    conda-forge
vs2015_runtime            14.34.31931         hed1258a_16    conda-forge
watchfiles                0.19.0                   pypi_0    pypi
wcwidth                   0.2.6              pyhd8ed1ab_0    conda-forge
webencodings              0.5.1                      py_1    conda-forge
websocket-client          1.5.2              pyhd8ed1ab_0    conda-forge
wheel                     0.40.0             pyhd8ed1ab_0    conda-forge
widgetsnbextension        4.0.7                    pypi_0    pypi
win_inet_pton             1.1.0              pyhd8ed1ab_6    conda-forge
winpty                    0.4.3                         4    conda-forge
xz                        5.2.6                h8d14728_0    conda-forge
yaml                      0.2.5                h8ffe710_2    conda-forge
yarn                      3.5.1                h08f2357_0    conda-forge
zeromq                    4.3.4                h0e60522_1    conda-forge
zipp                      3.15.0             pyhd8ed1ab_0    conda-forge

[kolibril13]

hi @Krande , I've recently built a React template for anywidget, and just had some time to take a look at this OrbitControls issue, and got it working, here you can find a minimal example: https://github.com/kolibril13/ipy-react-three-fiber

You can clone the repo and type npm install and thennpm run dev in order to get it running:

https://github.com/widgetti/ipyreact/assets/44469195/768da6d7-c3df-422b-8a83-29895e9c308e

[maartenbreddels]

Opening this so I don't forget to get back to this, this can now work really well with ipyreact, will follow up later!

[maartenbreddels]

I've added an example notebook: https://github.com/widgetti/ipyreact?tab=readme-ov-file#bundled-es-modules-for-threejs

This shows how to build a bundle, and finally make a composable app with it using solara: