JHnvidia
commented
9 months ago Hi,
If you're looking primarily for geometric quality it might be a good idea to look into something like NeuralAngelo. Our strong points are being able to jointly optimize lighting, material, and geometry, while other projects may be more suited for geometry extraction.
Unfortunately, the batching parameter is probably the most cruicial to quality, but it also increases memory overhead. Other than that, learning rate, scheduling, and grid resolution all affect quality, but they all affect each other and are complex to tweak.
Below are some short descriptions of the parameters: "ref_mesh": "data/nerf_synthetic/drums", -- Dataset (llff/nerf folder or obj mesh file) "random_textures": true, -- Intitialize trainable textures to random noise (use constant value if false) "iter": 5000, --iterations, can help mesh quality, but we've tweaked the learning rate schedule mainly for 5k iters, some codechanges may be required "save_interval": 100, -- Dump debug images every 100 iters "texture_res": [ 2048, 2048 ], -- Resolutions for trainable textures. Textures can have a lot of noise if much higher than rendering resolution due to mip-mapping. Rule of thumb is to approximately match rendering resolution, but keep as power of two. "train_res": [800, 800], --Must match png size for nerf. Can use any resolution when using .obj meshfile "batch": 2, -- Gradients will be averaged for a batch, usually leading to better quality with higher number. Probably largest impact on quality, but also costs a lot of memory :(. Raising the number of iterations is not equivalent due to noisier gradients "learning_rate": [0.03, 0.01], First, second pass, controls "learning speed". Higher value is noisier training that converges quicker in the beginning, and can lead to lower quality meshes. Lower values have slow convergence and can get stuck in bad local minimas. "ks_min" : [0, 0.1, 0.0], -- ks_min, ks_max, kd_min, kd_max are hints to the optimizer to limit the range of material parameters. The paper / supplemental details the parameters in greater detail. E.g. ks_min = [0,1,0] would indicate a diffuse only-material where roughness = 1 "dmtet_grid" : 128, -- Sets the geometric complexity (number of cells in dmtet). Higher value means more detailed geometry can represented. More details also makes training harder, so it's not a guarantee you get better quality with higher value. Set to lower for simpler/smoother surfaces. "mesh_scale" : 2.3, -- The scale for the dmtet_grid (i.e. scale of your scene). Geometry will be optimized only within the bounds given by mesh_scale. You should see that the random geometry in the first debug image gets larger if you increase this value. Also connected to the dmtet_grid parameter. Dmtet_grid controls the number of triangles, and mesh_scale controls the size of the triangles. "laplace_scale" : 3000, -- Scale of the laplacian loss used in the second pass, when optimizing vertex positions. The laplacian term is used to smooth each training iteration, as described in the supplemental material of the paper. "display": [{"latlong" : true}, {"bsdf" : "kd"}, {"bsdf" : "ks"}, {"bsdf" : "normal"}], --different images rendered for the save_interval debug images "layers" : 4, -- Number of depth-peeling layers for transparent geometry. Active only during the second optimization pass. Delete this line for opaque geometry. "background" : "white", -- just cosmetic, background color for save_interval images "out_dir": "nerf_drums"--change output directory name.
cthulhu-rises
Hello, I'm struggling to understand the formatting of some of the JSON paramters, and from the other comments clearly others can be added.
If I were trying to run the Nerf_Drums test, in what ways would I modify the paramters in the JSON file if I:
Also, are there any parameters I should be adding to the JSON?
If you can answer this for this test, I will understand enough (I believe) to keep pushing forward with my project.
nerf_drums JSON parameters below for reference --With my notes on the right "ref_mesh": "data/nerf_synthetic/drums", --directory path for the images the Nerf needs? "random_textures": true, -- no idea "iter": 5000, --iterations, will cranking this up give better meshes, I have gotten mixed results toying with this myself "save_interval": 100, -- how many iterations between each saved progress image? not sure if progress is actually saved "texture_res": [ 2048, 2048 ], --I should probably turn this down to save time? "train_res": [800, 800], --THis appears to need to match the training PNG size? "batch": 2, --I believ I turned this down to save RAM, can I increase iterations to compensate? "learning_rate": [0.03, 0.01], --first pass, second pass? no idea howe these impact mesh quality yet. "ks_min" : [0, 0.1, 0.0], -- absolutely no idea "dmtet_grid" : 128, -- I bet this is involved with mesh quality, not sure which way I should adjust "mesh_scale" : 2.3, -- I made this way bigger and it got much worse, so If this is triangle scaling I should probably lower this?? "laplace_scale" : 3000, -- No idea "display": [{"latlong" : true}, {"bsdf" : "kd"}, {"bsdf" : "ks"}, {"bsdf" : "normal"}], --no idea "layers" : 4, --No idea, layer depth of neural net somewhere? "background" : "white", -- have not touched this "out_dir": "nerf_drums"--change output directory name.