I keep getting the same output.

[downysoftware]

Everything appears to be working, but I keep getting a vertical Chinese scroll painting surrounded by other paintings, regardless of image input, prompts, or seed. The only thing that makes any difference is camera path.

Result I've been getting: https://files.catbox.moe/jsdy2w.mp4

Note that this result only made sense the very first time I ran LucidDreamer. Since that time, I've tried many different prompts that should not produce anything like this.

EDIT: I tried outputting text from the command prompt window to a log file, which yielded error messages. Log: https://pastebin.com/cX5ZzsBc

EDIT: Shortened and clarified initial post.

EDIT: The log mentioned "Using cache found in C:\Users\stupi/.cache\torch\hub\intel-isl_MiDaS_master" so I tried deleting that cache. No effect.

EDIT: Tried deleting all the pycache directories. No effect.

[robot0321]

Did you try our radio demo here? The code of this demo is placed on the same page here, which is almost the same as the code in this GitHub repo. Please check if there is any issue with your environment!

[PaulCollingwood]

I also have this. It's the same for me also.

[bably7]

I have this problem on both Windows and WSL2 Ubuntu, there are no other errors, and it runs fine the first time.

[esw0116]

Hello, I have tested on Windows 11, and I cannot observe that phenomenon in my environment. To make the problem, I would like to ask you some questions about the issue.

1. The output video does not change even if you change the initial input image?
2. Which method you are generating the video, command line (python run.py) or gradio app (python app.py)?

[downysoftware]

1. Yes.
2. Gradio app, both regular and mini. I haven't tried command line.

[esw0116]

Thank you for the answer. I will check the cause of the problem.

[PaulCollingwood]

1, yes 2: both

[downysoftware]

Did people having this problem do a completely clean install, or were there existing Python versions and libraries on the machine?

I see everything installed under anaconda3\envs\lucid\Lib\site-packages, but I wonder if somehow something outside the environment is causing this problem. I don't know much about how conda works.

I run the app by first activating the conda environment, then launching app.py from D:\Program Files\Python\Python39\LucidDreamer. Is that wrong? Or maybe I should delete that and clone the repository again.

I was unable to launch the gradio app at all, at first, so I deleted everything luciddreamer-related (I thought) and reinstalled. My next step would be reinstall everything Python-related, but I have too many working installations on this machine to just start over. Maybe I'll install on a virtual machine. But, if that works, it won't tell anyone why this strange behavior occurs.

It seems like a file was created on first run, and then that file keeps getting reused. An image or video file, or maybe another type of file that's created from the input image. If that's the case, maybe code that's supposed to delete that file is malfunctioning.

Anyway, I have all day to mess around with this. Maybe I'll figure it out.

EDIT: I deleted all files in AppData\Local\Temp\gradio, no change.

Deleted tons more temp files, no change.

Deleted \Python39\LucidDreamer and cloned it from Github again. After doing that, it worked once, and now it's giving the same result again.

I'm leaning towards some file being created, and then not getting deleted, as the culprit.

⭐ I found it: \Python39\LucidDreamer\gsplat.ply

Now, what script to edit, and where?

[downysoftware]

That was definitely it. It was reusing a file that should have been deleted, thus repeating the same result every time.

Where to clone the LucidDreamer repo isn't mentioned in the readme, but is it possible I put it in an unexpected location? Maybe it's supposed to be in anaconda3\envs\lucid.

This may be related to my other problem, which is that results look nothing like the input image. I thought it was supposed to do inpainting only, to provide depth and motion, but it's generating a completely new image that's only vaguely similar to the input. Trying the huggingface demo now, to see if it does better.

[esw0116]

@downysoftware I really appreciate for sharing your valuable experience related to the issue. We have fixed the code to remove gsplat.py when pressing the run button. Would you please run a new experiment after updating the repository? I think the problem would be resolved.

[downysoftware]

Now it just errors out.

  File "D:\Program Files\Python\Python39\LucidDreamer\scene\dataset_readers.py", line 376, in loadCameraPreset
    W, H = traindata["frames"][0]["image"].size
IndexError: list index out of range

I think there must be something up with my environment. Either from preexisting Python installations or installing something incorrectly. I give up.

[PaulCollingwood]

I noticed that if len(valid_idxj) == 0: continue always continues because len(valid_idxj) is always falsy and therefore traindata['frames'] is never populated. This is then the source of the last error noted by downysoftware.

I also give up.

[jojokic]

@downysoftware @PaulCollingwood https://github.com/luciddreamer-cvlab/LucidDreamer/issues/47