The result doesn't look good in stage 1

[MnKnight1]

The commands I run are as follows：

cd ./prolificdreamer-main
CUDA_VISIBLE_DEVICES=0 python main.py --text "Albert Einstein is playing the guitar." --iters 25000 --lambda_entropy 100 --scale 7.5 --n_particles 1 --h 512  --w 512 --workspace exp-nerf-stage1/”

But the output video seems meaningless. What's the reason? df_ep0250_00_textureless_rgb.mp4

[thuwzy]

How about the results with rgb? The this is the video of textureless result.

[MnKnight1]

How about the results with rgb? The this is the video of textureless result. I have also tried the “the pineapple”in the readme，but he result, like it, is a meaningless floating object. df_ep0250_00_albedo_rgb.mp4

[MnKnight1]

Is it due to environmental problems？ but the program can run, the result is wrong, I do not understand why. pip list

Package                     Version
--------------------------- ------------
absl-py                     2.0.0
accelerate                  0.25.0
aiofiles                    23.2.1
aiohttp                     3.9.1
aiosignal                   1.3.1
annotated-types             0.6.0
antlr4-python3-runtime      4.9.3
anyio                       3.7.1
async-timeout               4.0.3
attrs                       23.1.0
beautifulsoup4              4.12.2
Brotli                      1.0.9
cachetools                  5.3.2
carvekit-colab              4.1.0
certifi                     2023.11.17
cffi                        1.15.0
chardet                     4.0.0
charset-normalizer          2.0.4
click                       8.1.7
clip                        1.0
contourpy                   1.1.1
cryptography                41.0.3
cycler                      0.12.1
dearpygui                   1.10.1
debugpy-run                 1.8
diffusers                   0.15.0
einops                      0.7.0
exceptiongroup              1.2.0
fastapi                     0.105.0
filelock                    3.13.1
fonttools                   4.46.0
frozenlist                  1.4.0
fsspec                      2023.12.2
ftfy                        6.1.3
gdown                       4.7.1
gmpy2                       2.1.2
google-auth                 2.25.2
google-auth-oauthlib        1.0.0
grpcio                      1.60.0
h11                         0.14.0
huggingface-hub             0.19.4
idna                        3.4
imageio                     2.33.1
imageio-ffmpeg              0.4.9
importlib-metadata          7.0.0
importlib-resources         6.1.1
Jinja2                      3.1.2
joblib                      1.3.2
kiwisolver                  1.4.5
kornia                      0.7.0
lightning-utilities         0.10.0
loguru                      0.7.2
Markdown                    3.5.1
markdown-it-py              3.0.0
MarkupSafe                  2.1.1
matplotlib                  3.7.4
mdurl                       0.1.2
mkl-fft                     1.3.1
mkl-random                  1.2.2
mkl-service                 2.4.0
mpmath                      1.3.0
multidict                   6.0.4
networkx                    3.1
ninja                       1.11.1.1
numpy                       1.24.3
nvdiffrast                  0.3.1
nvidia-cublas-cu12          12.1.3.1
nvidia-cuda-cupti-cu12      12.1.105
nvidia-cuda-nvrtc-cu12      12.1.105
nvidia-cuda-runtime-cu12    12.1.105
nvidia-cudnn-cu12           8.9.2.26
nvidia-cufft-cu12           11.0.2.54
nvidia-curand-cu12          10.3.2.106
nvidia-cusolver-cu12        11.4.5.107
nvidia-cusparse-cu12        12.1.0.106
nvidia-nccl-cu12            2.18.1
nvidia-nvjitlink-cu12       12.3.101
nvidia-nvtx-cu12            12.1.105
oauthlib                    3.2.2
omegaconf                   2.3.0
opencv-python               4.8.1.78
packaging                   23.2
pandas                      2.0.3
Pillow                      10.0.1
pip                         23.3.1
protobuf                    4.25.1
psutil                      5.9.6
pyasn1                      0.5.1
pyasn1-modules              0.3.0
pycparser                   2.21
pydantic                    2.5.2
pydantic_core               2.14.5
Pygments                    2.17.2
PyMCubes                    0.1.4
pymeshlab                   2022.2.post3
pyOpenSSL                   23.2.0
pyparsing                   3.1.1
PySocks                     1.7.1
python-dateutil             2.8.2
python-multipart            0.0.6
pytorch-lightning           2.1.2
pytz                        2023.3.post1
PyYAML                      6.0.1
regex                       2023.10.3
requests                    2.31.0
requests-oauthlib           1.3.1
rich                        13.7.0
rsa                         4.9
safetensors                 0.4.1
scikit-learn                1.3.2
scipy                       1.10.1
sentencepiece               0.1.99
setuptools                  68.0.0
six                         1.16.0
sniffio                     1.3.0
soupsieve                   2.5
starlette                   0.27.0
sympy                       1.12
taming-transformers-rom1504 0.0.6
tensorboard                 2.14.0
tensorboard-data-server     0.7.2
tensorboardX                2.6.2.2
threadpoolctl               3.2.0
timm                        0.9.12
tokenizers                  0.15.0
torch                       1.12.1+cu113
torch-ema                   0.3
torchaudio                  2.0.2
torchmetrics                1.2.1
torchvision                 0.13.1+cu113
tqdm                        4.66.1
transformers                4.36.0
trimesh                     4.0.5
triton                      2.0.0
typing_extensions           4.7.1
tzdata                      2023.3
urllib3                     1.26.18
uvicorn                     0.24.0.post1
wcwidth                     0.2.12
Werkzeug                    3.0.1
wheel                       0.41.2
xatlas                      0.0.8
yarl                        1.9.4
zipp                        3.17.0

[thuwzy]

(1) The lambda_entropy is a critical parameter. How about setting --lambda_entropy 10? (2) This prompt may be too hard for stable diffusion, maybe you can try a pineapple first and figure out if a simple prompt can work.

[MnKnight1]

yes，if I set lambda_entropy to 100,the pineapple can be generated. But some sentence like“Albert Einstein is playing the guitar” still can not be generated ,when lambda_entropy equals 100. Do you think this can be generated by adjusting the parameters？ Because I see that some imaginative sentences in the paper can be generated, such as "Michelangelo style statue ofdog reading news on a cellphone"

[thuwzy]

In general, complex prompts require a more careful choice of the parameters. It is possible that this may be generated by adjusting the parameters or even by adding new regularization loss.