search

TencentARC / InstantMesh

InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models
Apache License 2.0
3.32k stars 353 forks source link

Number of epochs before seeing a decrease in the loss #113

Open Kev1MSL opened 4 months ago

Kev1MSL commented 4 months ago

Hi, I am currently training the instant-mesh model but after 3 epochs I am not seeing convergence and the loss does not seem to decrease. I am using the same config as you set (DDP strategy) and also I added grad_accumulation. I am not sure if I am doing something wrong or if I should be more patient and wait for more epochs. Not sure if I should train the instant-nerf first and then instant-mesh.

Here some plots if it helps: image image

Thanks for your help!

bluestyle97 commented 4 months ago

You need to train instant-nerf first. The mesh-based rendering can only provide gradients at near-surface area, making the network hard to converge. Our mesh model is finetuned from the nerf model.

Kev1MSL commented 4 months ago

Alright makes sense thank you! I have tried the training of instant-nerf, however I am starting with cubes as first object rendered and I was wondering if the default code starts with pretrained weights for the LRM since I am starting with cubes?

Sri-vatsa commented 3 months ago

@Kev1MSL According to the paper, the training is initialized using openlrm weights. @bluestyle97 How many training steps have you run the training of instant-nerf for using your training data?

Mrguanglei commented 3 months ago

@Kev1MSL Hello, please take a look at your dataset structure and training configuration file？ This will help you better

Mrguanglei commented 3 months ago

@Sri-vatsa Hello, could you take a look at the structure of your dataset and the configuration file used to train nerf? I have encountered a problem in this regard, which I cannot solve, and I need your help

HaFred commented 3 months ago

Alright makes sense thank you! I have tried the training of instant-nerf, however I am starting with cubes as first object rendered and I was wondering if the default code starts with pretrained weights for the LRM since I am starting with cubes?

Hi based on your graph, it seems like your training set contains only 3.5k instances, is that true? The paper says 270k instances, so do you know how to curate the dataset correctly for training instantnerf? Thanks a lot!