Google Colab

[scatterp2]

I started work on a google colab anyone got any idea how to finalize this ? https://colab.research.google.com/drive/1Sv5Z1uVM5ZCGJdJSc67qGzEnyA3mlpeF?authuser=3#scrollTo=lzVAqpGLoESp

[j0seo]

Hi, @scatterp2 Can you try the following setup for Colab? I'm still preparing the Colab demo, but I confirmed this env. setup code works well.

import sys
import torch
pyt_version_str=torch.__version__.split("+")[0].replace(".", "")
version_str="".join([
    f"py3{sys.version_info.minor}_cu",
    torch.version.cuda.replace(".",""),
    f"_pyt{pyt_version_str}"
])
!pip install fvcore iopath
!pip install --no-index --no-cache-dir pytorch3d -f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/{version_str}/download.html
!git clone https://github.com/j0seo/3DFuse_Colab
!mv 3DFuse_Colab/* ./
!pip install -r requirements.txt
!mkdir weights
!wget -P ./weights https://huggingface.co/jyseo/3DFuse_weights/resolve/main/models/3DFuse_sparse_depth_injector.ckpt
!pip install pytorch-lightning==1.8.3.post0
!pip install 'prettytable<1'
!pip install 'setuptools==62.3.2'
!pip install 'jedi>=0.16'

[scatterp2]

Hi

With the following code: https://colab.research.google.com/drive/1uNgcLy6h8E1dki6AHjEhrGKDa-gGuAlt?usp=sharing and keeping the settings on default (a bed) after generating the image when trying to create the 3d result it crashes below is the output: note: I found this "Ya, currently pypi version 0.3.0 is broken for the latest TF 2.6.0:"

Collapsing Lin Lora in q_proj Collapsing Lin Lora in out_proj Reshaping encoder.mid.attn_1.q.weight for SD format Reshaping encoder.mid.attn_1.k.weight for SD format Reshaping encoder.mid.attn_1.v.weight for SD format Reshaping encoder.mid.attn_1.proj_out.weight for SD format Reshaping decoder.mid.attn_1.q.weight for SD format Reshaping decoder.mid.attn_1.k.weight for SD format Reshaping decoder.mid.attn_1.v.weight for SD format Reshaping decoder.mid.attn_1.proj_out.weight for SD format Added Token: ['<0>'] a comfortable <0> 0% 0/10000 [00:00<?, ?it/s]/usr/local/lib/python3.9/dist-packages/torch/nn/functional.py:4236: UserWarning:

Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.

0% 0/10000 [00:00<?, ?it/s] Traceback (most recent call last): File "/usr/local/lib/python3.9/dist-packages/gradio/routes.py", line 337, in run_predict output = await app.get_blocks().process_api( File "/usr/local/lib/python3.9/dist-packages/gradio/blocks.py", line 1015, in process_api result = await self.call_function( File "/usr/local/lib/python3.9/dist-packages/gradio/blocks.py", line 847, in call_function prediction = await anyio.to_thread.run_sync( File "/usr/local/lib/python3.9/dist-packages/anyio/to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/usr/local/lib/python3.9/dist-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "/usr/local/lib/python3.9/dist-packages/anyio/_backends/_asyncio.py", line 867, in run result = context.run(func, *args) File "/usr/local/lib/python3.9/dist-packages/gradio/utils.py", line 410, in async_iteration return next(iterator) File "/content/gradio_app.py", line 41, in gen_3d yield from model.run_gradio(points) File "/content/run_3DFuse.py", line 147, in run_gradio yield from fuse_3d(cfgs, poser=poser,model=model,vox=vox,exp_dir=exp_dir, points=points, is_gradio=True) File "/content/run_3DFuse.py", line 195, in fuse_3d y, depth, ws = render_oneview(vox, aabb, H, W, Ks[i], poses_[i], return_w=True) File "/content/run_3DFuse.py", line 325, in render_one_view rgbs = rearrange(rgbs, "(h w) c -> 1 c h w", h=H, w=W) File "/usr/local/lib/python3.9/dist-packages/einops/einops.py", line 424, in rearrange return reduce(tensor, pattern, reduction='rearrange', axes_lengths) File "/usr/local/lib/python3.9/dist-packages/einops/einops.py", line 368, in reduce return recipe.apply(tensor) File "/usr/local/lib/python3.9/dist-packages/einops/einops.py", line 203, in apply backend = get_backend(tensor) File "/usr/local/lib/python3.9/dist-packages/einops/_backends.py", line 49, in get_backend if backend.is_appropriate_type(tensor): File "/usr/local/lib/python3.9/dist-packages/einops/_backends.py", line 513, in is_appropriate_type return self.K.is_tensor(tensor) and self.K.is_keras_tensor(tensor) AttributeError: module 'keras.backend' has no attribute 'is_tensor'