facebookresearch / vfusion3d

[ECCV 2024] Code for VFusion3D: Learning Scalable 3D Generative Models from Video Diffusion Models
Other
386 stars 26 forks source link

TypeError: _native_multi_head_attention(): argument 'qkv_bias' (position 7) must be Tensor, not NoneType #8

Closed ehsanwwe closed 1 month ago

ehsanwwe commented 1 month ago

Hi whene i test Gradio Demo i got this error

image load and i was click on generate mesh btn but after about 60 seconds i got below error

python gradio_app.py /home/ehsan/anaconda3/lib/python3.9/site-packages/numba/np/ufunc/parallel.py:371: NumbaWarning: The TBB threading layer requires TBB version 2021 update 6 or later i.e., TBB_INTERFACE_VERSION >= 12060. Found TBB_INTERFACE_VERSION = 12050. The TBB threading layer is disabled. warnings.warn(problem) Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch(). Exception in thread Thread-5: Traceback (most recent call last): File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpx/_transports/default.py", line 69, in map_httpcore_exceptions Exception in thread Thread-3: Traceback (most recent call last): File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpx/_transports/default.py", line 69, in map_httpcore_exceptions yield File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpx/_transports/default.py", line 233, in handle_request yield File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpx/_transports/default.py", line 233, in handle_request resp = self._pool.handle_request(req) resp = self._pool.handle_request(req) File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpcore/_sync/connection_pool.py", line 216, in handle_request File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpcore/_sync/connection_pool.py", line 216, in handle_request raise exc from None File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpcore/_sync/connection_pool.py", line 196, in handle_request raise exc from None File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpcore/_sync/connection_pool.py", line 196, in handle_request response = connection.handle_request( File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpcore/_sync/connection.py", line 99, in handle_request response = connection.handle_request( File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpcore/_sync/connection.py", line 99, in handle_request raise exc File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpcore/_sync/connection.py", line 76, in handle_request raise exc File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpcore/_sync/connection.py", line 76, in handle_request stream = self._connect(request) stream = self._connect(request) File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpcore/_sync/connection.py", line 154, in _connect File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpcore/_sync/connection.py", line 154, in _connect stream = stream.start_tls(kwargs) File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpcore/_backends/sync.py", line 168, in start_tls raise exc File "/home/ehsan/anaconda3/lib/python3.9/contextlib.py", line 137, in exit self.gen.throw(typ, value, traceback) File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions raise to_exc(exc) from exc stream = stream.start_tls(kwargs) File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpcore/_backends/sync.py", line 168, in start_tls httpcore.ConnectTimeout: _ssl.c:1112: The handshake operation timed out raise exc File "/home/ehsan/anaconda3/lib/python3.9/contextlib.py", line 137, in exit

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/ehsan/anaconda3/lib/python3.9/threading.py", line 973, in _bootstrap_inner self.gen.throw(typ, value, traceback) File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions raise to_exc(exc) from exc httpcore.ConnectTimeout: _ssl.c:1112: The handshake operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/ehsan/anaconda3/lib/python3.9/threading.py", line 973, in _bootstrap_inner self.run() File "/home/ehsan/anaconda3/lib/python3.9/threading.py", line 910, in run self.run() self._target(*self._args, self._kwargs) File "/home/ehsan/anaconda3/lib/python3.9/site-packages/gradio/analytics.py", line 70, in _do_normal_analytics_request File "/home/ehsan/anaconda3/lib/python3.9/threading.py", line 910, in run data["ip_address"] = get_local_ip_address() File "/home/ehsan/anaconda3/lib/python3.9/site-packages/gradio/analytics.py", line 131, in get_local_ip_address self._target(*self._args, *self._kwargs) File "/home/ehsan/anaconda3/lib/python3.9/site-packages/gradio/analytics.py", line 70, in _do_normal_analytics_request ip_address = httpx.get( File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpx/_api.py", line 198, in get data["ip_address"] = get_local_ip_address() File "/home/ehsan/anaconda3/lib/python3.9/site-packages/gradio/analytics.py", line 131, in get_local_ip_address return request( File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpx/_api.py", line 106, in request ip_address = httpx.get( File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpx/_api.py", line 198, in get return client.request( File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpx/_client.py", line 827, in request return request( File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpx/_api.py", line 106, in request return client.request( File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpx/_client.py", line 827, in request return self.send(request, auth=auth, follow_redirects=follow_redirects) File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpx/_client.py", line 914, in send return self.send(request, auth=auth, follow_redirects=follow_redirects) File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpx/_client.py", line 914, in send response = self._send_handling_auth( File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpx/_client.py", line 942, in _send_handling_auth response = self._send_handling_auth( File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpx/_client.py", line 942, in _send_handling_auth response = self._send_handling_redirects( File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpx/_client.py", line 979, in _send_handling_redirects response = self._send_handling_redirects( response = self._send_single_request(request) File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpx/_client.py", line 1015, in _send_single_request File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpx/_client.py", line 979, in _send_handling_redirects response = transport.handle_request(request) File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpx/_transports/default.py", line 233, in handle_request response = self._send_single_request(request) File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpx/_client.py", line 1015, in _send_single_request resp = self._pool.handle_request(req) File "/home/ehsan/anaconda3/lib/python3.9/contextlib.py", line 137, in exit self.gen.throw(typ, value, traceback) File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpx/_transports/default.py", line 86, in map_httpcore_exceptions raise mapped_exc(message) from exc httpx.ConnectTimeout: _ssl.c:1112: The handshake operation timed out response = transport.handle_request(request) File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpx/_transports/default.py", line 233, in handle_request resp = self._pool.handle_request(req) File "/home/ehsan/anaconda3/lib/python3.9/contextlib.py", line 137, in exit self.gen.throw(typ, value, traceback) File "/home/ehsan/anaconda3/lib/python3.9/site-packages/httpx/_transports/default.py", line 86, in map_httpcore_exceptions raise mapped_exc(message) from exc httpx.ConnectTimeout: _ssl.c:1112: The handshake operation timed out Traceback (most recent call last): File "/home/ehsan/anaconda3/lib/python3.9/site-packages/gradio/queueing.py", line 528, in process_events response = await route_utils.call_process_api( File "/home/ehsan/anaconda3/lib/python3.9/site-packages/gradio/route_utils.py", line 270, in call_process_api output = await app.get_blocks().process_api( File "/home/ehsan/anaconda3/lib/python3.9/site-packages/gradio/blocks.py", line 1908, in process_api result = await self.call_function( File "/home/ehsan/anaconda3/lib/python3.9/site-packages/gradio/blocks.py", line 1485, in call_function prediction = await anyio.to_thread.run_sync( File "/home/ehsan/anaconda3/lib/python3.9/site-packages/anyio/to_thread.py", line 28, in run_sync return await get_asynclib().run_sync_in_worker_thread(func, args, cancellable=cancellable, File "/home/ehsan/anaconda3/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 818, in run_sync_in_worker_thread return await future File "/home/ehsan/anaconda3/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 754, in run result = context.run(func, args) File "/home/ehsan/anaconda3/lib/python3.9/site-packages/gradio/utils.py", line 808, in wrapper response = f(args, kwargs) File "/home/ehsan/PycharmProject/vfusion3d/VFusion3D/gradio_app.py", line 212, in step_2_generatevideo , video_path = generate_mesh(image, export_video=True) File "/home/ehsan/PycharmProject/vfusion3d/VFusion3D/gradio_app.py", line 165, in generate_mesh planes = model_wrapper.forward(image, source_camera) File "/home/ehsan/PycharmProject/vfusion3d/VFusion3D/gradio_app.py", line 32, in forward return self.model(image, camera) File "/home/ehsan/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, kwargs) File "/home/ehsan/.cache/huggingface/modules/transformers_modules/jadechoghari/vfusion3d/142391d24150f1a4457b52c83970b23a0e3417b6/modeling.py", line 88, in forward planes = self.transformer(image_feats, camera_embeddings) File "/home/ehsan/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, *kwargs) File "/home/ehsan/.cache/huggingface/modules/transformers_modules/jadechoghari/vfusion3d/142391d24150f1a4457b52c83970b23a0e3417b6/transformer.py", line 118, in forward x = layer(x, image_feats, camera_embeddings) File "/home/ehsan/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(args, kwargs) File "/home/ehsan/.cache/huggingface/modules/transformers_modules/jadechoghari/vfusion3d/142391d24150f1a4457b52c83970b23a0e3417b6/transformer.py", line 71, in forward x = x + self.self_attn(before_sa, before_sa, before_sa, need_weights=False)[0] File "/home/ehsan/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/home/ehsan/anaconda3/lib/python3.9/site-packages/torch/nn/modules/activation.py", line 1160, in forward return torch._native_multi_head_attention( TypeError: _native_multi_head_attention(): argument 'qkv_bias' (position 7) must be Tensor, not NoneType

ehsanwwe commented 1 month ago

fortunetly this command works like charm

python -m lrm.inferrer --export_mesh --resume ./checkpoints/vfusion3dckpt --render_size 256 --source_path ./assets/man