search

NVlabs / nvdiffrecmc

Official code for the NeurIPS 2022 paper "Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoising".
Other
363 stars 28 forks source link

texture map doesn't look right #11

Closed yezifeiafei closed 1 year ago

yezifeiafei commented 1 year ago

I run the example with default configuartion python train.py --config configs/bob.json but texture map doesn't look right. image How does that happen?

jmunkberg commented 1 year ago

Hello,

This is the expected look of the textures after optimization for the bob example:

image

yezifeiafei commented 1 year ago

This is my log

Using /root/.cache/torch_extensions/py38_cu111 as PyTorch extensions root...
Detected CUDA files, patching ldflags
Emitting ninja build file /root/.cache/torch_extensions/py38_cu111/optixutils_plugin/build.ninja...
Building extension module optixutils_plugin...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module optixutils_plugin...
Config / Flags:
---------
iter 3000
batch 8
spp 1
layers 1
train_res [512, 512]
display_res [512, 512]
texture_res [1024, 1024]
display_interval 0
save_interval 100
learning_rate [0.03, 0.01]
custom_mip False
background white
loss logl1
out_dir out/bob
config configs/bob.json
ref_mesh data/bob/bob_tri.obj
base_mesh None
validate True
n_samples 4
bsdf pbr
denoiser bilateral
denoiser_demodulate True
mtl_override None
dmtet_grid 64
mesh_scale 2.1
envlight data/irrmaps/dreifaltigkeitsberg_2k.hdr
env_scale 1.0
probe_res 256
learn_lighting True
display [{'latlong': True}]
transparency False
lock_light False
lock_pos True
sdf_regularizer 0.2
laplace relative
laplace_scale 3000.0
pre_load True
no_perturbed_nrm False
decorrelated False
kd_min [0.0, 0.0, 0.0, 0.0]
kd_max [1.0, 1.0, 1.0, 1.0]
ks_min [0, 0.1, 0.0]
ks_max [0, 1.0, 1.0]
nrm_min [-1.0, -1.0, 0.0]
nrm_max [1.0, 1.0, 1.0]
clip_max_norm 0.0
cam_near_far [0.1, 1000.0]
lambda_kd 0.1
lambda_ks 0.05
lambda_nrm 0.025
lambda_nrm2 0.25
lambda_chroma 0.0
lambda_diffuse 0.15
lambda_specular 0.0025
random_textures True
---------
DatasetMesh: ref mesh has 10688 triangles and 5344 vertices
Build Optix bvh
Cuda path /usr/local/cuda
End of OptiXStateWrapper 
Done building OptiX bvh
---> WARNING: Picked a texture resolution lower than the reference mesh [1024, 1024] < [2048, 2048]
EnvProbe, torch.Size([1024, 2048, 3]) , min/max 0.0 113664.0
/opt/conda/lib/python3.8/site-packages/torch/functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at  ../aten/src/ATen/native/TensorShape.cpp:2157.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
DatasetMesh: ref mesh has 10688 triangles and 5344 vertices
Build Optix bvh
Cuda path /usr/local/cuda
End of OptiXStateWrapper 
Done building OptiX bvh
---> WARNING: Picked a texture resolution lower than the reference mesh [1024, 1024] < [2048, 2048]
EnvProbe, torch.Size([1024, 2048, 3]) , min/max 0.0 113664.0
Cuda path /usr/local/cuda
End of OptiXStateWrapper 
Encoder output: 32 dims
Using /root/.cache/torch_extensions/py38_cu111 as PyTorch extensions root...
Detected CUDA files, patching ldflags
Emitting ninja build file /root/.cache/torch_extensions/py38_cu111/renderutils_plugin/build.ninja...
Building extension module renderutils_plugin...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module renderutils_plugin...
iter=    0, img_loss=0.442193, reg_loss=0.335910, lr=0.02999, time=559.1 ms, rem=27.95 m
iter=   10, img_loss=0.146829, reg_loss=0.327695, lr=0.02985, time=577.1 ms, rem=28.76 m
iter=   20, img_loss=0.055874, reg_loss=0.310453, lr=0.02971, time=579.8 ms, rem=28.80 m
iter=   30, img_loss=0.037663, reg_loss=0.290498, lr=0.02957, time=578.0 ms, rem=28.61 m
iter=   40, img_loss=0.027784, reg_loss=0.282415, lr=0.02944, time=580.1 ms, rem=28.62 m
iter=   50, img_loss=0.020137, reg_loss=0.276904, lr=0.02930, time=582.7 ms, rem=28.65 m
iter=   60, img_loss=0.017593, reg_loss=0.271364, lr=0.02917, time=580.4 ms, rem=28.44 m
iter=   70, img_loss=0.015929, reg_loss=0.266382, lr=0.02903, time=581.0 ms, rem=28.37 m
iter=   80, img_loss=0.015943, reg_loss=0.261982, lr=0.02890, time=581.7 ms, rem=28.31 m
iter=   90, img_loss=0.015313, reg_loss=0.257971, lr=0.02877, time=584.1 ms, rem=28.33 m
iter=  100, img_loss=0.014274, reg_loss=0.254378, lr=0.02864, time=582.9 ms, rem=28.17 m
iter=  110, img_loss=0.016354, reg_loss=0.250519, lr=0.02851, time=581.4 ms, rem=28.01 m
iter=  120, img_loss=0.011690, reg_loss=0.246711, lr=0.02837, time=585.2 ms, rem=28.09 m
iter=  130, img_loss=0.014356, reg_loss=0.242824, lr=0.02824, time=582.7 ms, rem=27.87 m
iter=  140, img_loss=0.011433, reg_loss=0.238958, lr=0.02811, time=585.4 ms, rem=27.90 m
iter=  150, img_loss=0.013802, reg_loss=0.235212, lr=0.02798, time=584.0 ms, rem=27.74 m
iter=  160, img_loss=0.013134, reg_loss=0.231555, lr=0.02786, time=583.8 ms, rem=27.63 m
iter=  170, img_loss=0.012153, reg_loss=0.227694, lr=0.02773, time=585.7 ms, rem=27.62 m
iter=  180, img_loss=0.013868, reg_loss=0.223820, lr=0.02760, time=582.4 ms, rem=27.37 m
iter=  190, img_loss=0.012241, reg_loss=0.219769, lr=0.02747, time=585.4 ms, rem=27.42 m
iter=  200, img_loss=0.014504, reg_loss=0.216222, lr=0.02735, time=582.8 ms, rem=27.20 m
iter=  210, img_loss=0.011462, reg_loss=0.212162, lr=0.02722, time=587.2 ms, rem=27.30 m
iter=  220, img_loss=0.012269, reg_loss=0.208446, lr=0.02710, time=581.6 ms, rem=26.95 m
iter=  230, img_loss=0.010893, reg_loss=0.204448, lr=0.02697, time=584.7 ms, rem=27.00 m
iter=  240, img_loss=0.014152, reg_loss=0.200513, lr=0.02685, time=581.7 ms, rem=26.76 m
iter=  250, img_loss=0.014475, reg_loss=0.196811, lr=0.02673, time=579.1 ms, rem=26.54 m
iter=  260, img_loss=0.013139, reg_loss=0.193116, lr=0.02660, time=580.4 ms, rem=26.51 m
iter=  270, img_loss=0.011324, reg_loss=0.189473, lr=0.02648, time=582.4 ms, rem=26.50 m
iter=  280, img_loss=0.012877, reg_loss=0.185701, lr=0.02636, time=582.4 ms, rem=26.40 m
iter=  290, img_loss=0.010882, reg_loss=0.182141, lr=0.02624, time=583.4 ms, rem=26.35 m
iter=  300, img_loss=0.013996, reg_loss=0.178586, lr=0.02612, time=580.9 ms, rem=26.14 m
iter=  310, img_loss=0.012277, reg_loss=0.174882, lr=0.02600, time=582.5 ms, rem=26.12 m
iter=  320, img_loss=0.013393, reg_loss=0.171271, lr=0.02588, time=580.3 ms, rem=25.92 m
iter=  330, img_loss=0.013980, reg_loss=0.167748, lr=0.02576, time=580.1 ms, rem=25.82 m
iter=  340, img_loss=0.011935, reg_loss=0.164093, lr=0.02564, time=581.0 ms, rem=25.76 m
iter=  350, img_loss=0.010579, reg_loss=0.160335, lr=0.02552, time=586.3 ms, rem=25.89 m
iter=  360, img_loss=0.014369, reg_loss=0.156834, lr=0.02541, time=620.1 ms, rem=27.29 m
iter=  370, img_loss=0.011845, reg_loss=0.153188, lr=0.02529, time=622.4 ms, rem=27.28 m
iter=  380, img_loss=0.012581, reg_loss=0.149554, lr=0.02517, time=623.4 ms, rem=27.22 m
iter=  390, img_loss=0.012658, reg_loss=0.145915, lr=0.02506, time=622.2 ms, rem=27.06 m
iter=  400, img_loss=0.010484, reg_loss=0.142267, lr=0.02494, time=621.6 ms, rem=26.94 m
iter=  410, img_loss=0.011218, reg_loss=0.138732, lr=0.02483, time=621.9 ms, rem=26.85 m
iter=  420, img_loss=0.013577, reg_loss=0.135173, lr=0.02471, time=619.7 ms, rem=26.65 m
iter=  430, img_loss=0.010858, reg_loss=0.131550, lr=0.02460, time=623.7 ms, rem=26.72 m
iter=  440, img_loss=0.014268, reg_loss=0.128080, lr=0.02449, time=619.6 ms, rem=26.44 m
iter=  450, img_loss=0.010617, reg_loss=0.124458, lr=0.02437, time=621.3 ms, rem=26.40 m
iter=  460, img_loss=0.012457, reg_loss=0.120825, lr=0.02426, time=622.2 ms, rem=26.34 m
iter=  470, img_loss=0.009664, reg_loss=0.117194, lr=0.02415, time=625.7 ms, rem=26.38 m
iter=  480, img_loss=0.012927, reg_loss=0.113602, lr=0.02404, time=621.7 ms, rem=26.11 m
iter=  490, img_loss=0.010381, reg_loss=0.109960, lr=0.02393, time=622.6 ms, rem=26.05 m
iter=  500, img_loss=0.011866, reg_loss=0.106287, lr=0.02382, time=621.5 ms, rem=25.90 m
iter=  510, img_loss=0.010176, reg_loss=0.102694, lr=0.02371, time=623.7 ms, rem=25.88 m
iter=  520, img_loss=0.011497, reg_loss=0.099082, lr=0.02360, time=621.0 ms, rem=25.67 m
iter=  530, img_loss=0.010816, reg_loss=0.095516, lr=0.02349, time=623.2 ms, rem=25.66 m
iter=  540, img_loss=0.010517, reg_loss=0.091894, lr=0.02338, time=622.2 ms, rem=25.51 m
iter=  550, img_loss=0.012197, reg_loss=0.088242, lr=0.02328, time=622.5 ms, rem=25.42 m
iter=  560, img_loss=0.010524, reg_loss=0.084658, lr=0.02317, time=623.2 ms, rem=25.34 m
iter=  570, img_loss=0.011565, reg_loss=0.081141, lr=0.02306, time=620.9 ms, rem=25.14 m
iter=  580, img_loss=0.010778, reg_loss=0.077505, lr=0.02296, time=622.1 ms, rem=25.09 m
iter=  590, img_loss=0.010357, reg_loss=0.073815, lr=0.02285, time=622.9 ms, rem=25.02 m
iter=  600, img_loss=0.008662, reg_loss=0.070221, lr=0.02275, time=623.0 ms, rem=24.92 m
iter=  610, img_loss=0.011477, reg_loss=0.066642, lr=0.02264, time=622.6 ms, rem=24.80 m
iter=  620, img_loss=0.009614, reg_loss=0.063039, lr=0.02254, time=620.8 ms, rem=24.63 m
iter=  630, img_loss=0.008545, reg_loss=0.059399, lr=0.02243, time=627.4 ms, rem=24.78 m
iter=  640, img_loss=0.009713, reg_loss=0.055807, lr=0.02233, time=622.5 ms, rem=24.48 m
iter=  650, img_loss=0.010997, reg_loss=0.052195, lr=0.02223, time=621.6 ms, rem=24.35 m
iter=  660, img_loss=0.011146, reg_loss=0.048617, lr=0.02213, time=620.9 ms, rem=24.22 m
iter=  670, img_loss=0.011438, reg_loss=0.045006, lr=0.02203, time=621.8 ms, rem=24.15 m
iter=  680, img_loss=0.009305, reg_loss=0.041378, lr=0.02192, time=624.0 ms, rem=24.13 m
iter=  690, img_loss=0.011968, reg_loss=0.037771, lr=0.02182, time=621.2 ms, rem=23.92 m
iter=  700, img_loss=0.008771, reg_loss=0.034149, lr=0.02172, time=630.8 ms, rem=24.18 m
iter=  710, img_loss=0.009587, reg_loss=0.030542, lr=0.02162, time=681.3 ms, rem=26.00 m
iter=  720, img_loss=0.010333, reg_loss=0.026927, lr=0.02152, time=681.1 ms, rem=25.88 m
iter=  730, img_loss=0.010205, reg_loss=0.023324, lr=0.02143, time=682.2 ms, rem=25.81 m
iter=  740, img_loss=0.010298, reg_loss=0.019731, lr=0.02133, time=680.3 ms, rem=25.63 m
iter=  750, img_loss=0.009012, reg_loss=0.016120, lr=0.02123, time=678.7 ms, rem=25.45 m
iter=  760, img_loss=0.011420, reg_loss=0.014504, lr=0.02113, time=681.8 ms, rem=25.45 m
iter=  770, img_loss=0.009829, reg_loss=0.014506, lr=0.02103, time=679.6 ms, rem=25.26 m
iter=  780, img_loss=0.010248, reg_loss=0.014493, lr=0.02094, time=681.3 ms, rem=25.21 m
iter=  790, img_loss=0.008961, reg_loss=0.014491, lr=0.02084, time=682.0 ms, rem=25.12 m
iter=  800, img_loss=0.013093, reg_loss=0.014500, lr=0.02075, time=678.0 ms, rem=24.86 m
iter=  810, img_loss=0.008850, reg_loss=0.014484, lr=0.02065, time=684.3 ms, rem=24.98 m
iter=  820, img_loss=0.010849, reg_loss=0.014484, lr=0.02056, time=679.1 ms, rem=24.68 m
iter=  830, img_loss=0.009572, reg_loss=0.014474, lr=0.02046, time=682.6 ms, rem=24.69 m
iter=  840, img_loss=0.010045, reg_loss=0.014472, lr=0.02037, time=682.7 ms, rem=24.58 m
iter=  850, img_loss=0.009613, reg_loss=0.014486, lr=0.02027, time=680.5 ms, rem=24.39 m
iter=  860, img_loss=0.008765, reg_loss=0.014483, lr=0.02018, time=679.7 ms, rem=24.24 m
iter=  870, img_loss=0.009359, reg_loss=0.014483, lr=0.02009, time=681.1 ms, rem=24.18 m
iter=  880, img_loss=0.009724, reg_loss=0.014474, lr=0.01999, time=681.1 ms, rem=24.06 m
iter=  890, img_loss=0.012144, reg_loss=0.014478, lr=0.01990, time=680.0 ms, rem=23.91 m
iter=  900, img_loss=0.009993, reg_loss=0.014477, lr=0.01981, time=680.7 ms, rem=23.83 m
iter=  910, img_loss=0.007951, reg_loss=0.014477, lr=0.01972, time=684.8 ms, rem=23.85 m
iter=  920, img_loss=0.010471, reg_loss=0.014472, lr=0.01963, time=680.8 ms, rem=23.60 m
iter=  930, img_loss=0.009050, reg_loss=0.014479, lr=0.01954, time=681.8 ms, rem=23.52 m
iter=  940, img_loss=0.010412, reg_loss=0.014482, lr=0.01945, time=679.4 ms, rem=23.33 m
iter=  950, img_loss=0.008318, reg_loss=0.014479, lr=0.01936, time=682.7 ms, rem=23.32 m
iter=  960, img_loss=0.008518, reg_loss=0.014480, lr=0.01927, time=681.9 ms, rem=23.19 m
iter=  970, img_loss=0.009346, reg_loss=0.014483, lr=0.01918, time=680.7 ms, rem=23.03 m
iter=  980, img_loss=0.010098, reg_loss=0.014481, lr=0.01910, time=681.1 ms, rem=22.93 m
iter=  990, img_loss=0.008483, reg_loss=0.014469, lr=0.01901, time=684.5 ms, rem=22.93 m
iter= 1000, img_loss=0.007748, reg_loss=0.014466, lr=0.01892, time=681.8 ms, rem=22.73 m
iter= 1010, img_loss=0.011127, reg_loss=0.014461, lr=0.01883, time=682.1 ms, rem=22.62 m
iter= 1020, img_loss=0.008088, reg_loss=0.014456, lr=0.01875, time=680.8 ms, rem=22.47 m
iter= 1030, img_loss=0.010410, reg_loss=0.014461, lr=0.01866, time=680.1 ms, rem=22.33 m
iter= 1040, img_loss=0.010512, reg_loss=0.014465, lr=0.01857, time=681.2 ms, rem=22.25 m
iter= 1050, img_loss=0.008644, reg_loss=0.014459, lr=0.01849, time=690.3 ms, rem=22.43 m
iter= 1060, img_loss=0.009815, reg_loss=0.014467, lr=0.01840, time=757.3 ms, rem=24.49 m
iter= 1070, img_loss=0.008719, reg_loss=0.014469, lr=0.01832, time=755.7 ms, rem=24.31 m
iter= 1080, img_loss=0.008948, reg_loss=0.014466, lr=0.01824, time=755.4 ms, rem=24.17 m
iter= 1090, img_loss=0.008593, reg_loss=0.014464, lr=0.01815, time=756.3 ms, rem=24.08 m
iter= 1100, img_loss=0.010967, reg_loss=0.014461, lr=0.01807, time=753.5 ms, rem=23.86 m
iter= 1110, img_loss=0.010124, reg_loss=0.014461, lr=0.01799, time=756.7 ms, rem=23.84 m
iter= 1120, img_loss=0.009360, reg_loss=0.014458, lr=0.01790, time=756.6 ms, rem=23.71 m
iter= 1130, img_loss=0.010114, reg_loss=0.014464, lr=0.01782, time=756.4 ms, rem=23.58 m
iter= 1140, img_loss=0.006399, reg_loss=0.014452, lr=0.01774, time=758.0 ms, rem=23.50 m
iter= 1150, img_loss=0.008507, reg_loss=0.014467, lr=0.01766, time=755.5 ms, rem=23.30 m
iter= 1160, img_loss=0.009420, reg_loss=0.014465, lr=0.01758, time=756.2 ms, rem=23.19 m
iter= 1170, img_loss=0.009444, reg_loss=0.014467, lr=0.01750, time=755.9 ms, rem=23.06 m
iter= 1180, img_loss=0.008323, reg_loss=0.014476, lr=0.01741, time=756.9 ms, rem=22.96 m
iter= 1190, img_loss=0.009264, reg_loss=0.014473, lr=0.01733, time=756.3 ms, rem=22.81 m
iter= 1200, img_loss=0.007992, reg_loss=0.014464, lr=0.01726, time=758.4 ms, rem=22.75 m
iter= 1210, img_loss=0.008709, reg_loss=0.014466, lr=0.01718, time=756.4 ms, rem=22.57 m
iter= 1220, img_loss=0.009796, reg_loss=0.014477, lr=0.01710, time=753.8 ms, rem=22.36 m
iter= 1230, img_loss=0.008448, reg_loss=0.014477, lr=0.01702, time=756.8 ms, rem=22.32 m
iter= 1240, img_loss=0.006870, reg_loss=0.014469, lr=0.01694, time=757.3 ms, rem=22.21 m
iter= 1250, img_loss=0.009921, reg_loss=0.014475, lr=0.01686, time=756.3 ms, rem=22.06 m
iter= 1260, img_loss=0.009362, reg_loss=0.014478, lr=0.01678, time=756.9 ms, rem=21.95 m
iter= 1270, img_loss=0.010447, reg_loss=0.014464, lr=0.01671, time=756.0 ms, rem=21.80 m
iter= 1280, img_loss=0.011892, reg_loss=0.014471, lr=0.01663, time=753.3 ms, rem=21.60 m
iter= 1290, img_loss=0.009588, reg_loss=0.014459, lr=0.01655, time=756.5 ms, rem=21.56 m
iter= 1300, img_loss=0.011898, reg_loss=0.014465, lr=0.01648, time=752.9 ms, rem=21.33 m
iter= 1310, img_loss=0.009074, reg_loss=0.014451, lr=0.01640, time=755.8 ms, rem=21.29 m
iter= 1320, img_loss=0.010703, reg_loss=0.014460, lr=0.01633, time=754.3 ms, rem=21.12 m
iter= 1330, img_loss=0.008229, reg_loss=0.014460, lr=0.01625, time=755.8 ms, rem=21.04 m
iter= 1340, img_loss=0.009028, reg_loss=0.014456, lr=0.01618, time=756.5 ms, rem=20.93 m
iter= 1350, img_loss=0.007767, reg_loss=0.014455, lr=0.01610, time=758.4 ms, rem=20.86 m
iter= 1360, img_loss=0.010652, reg_loss=0.014476, lr=0.01603, time=753.9 ms, rem=20.61 m
iter= 1370, img_loss=0.011317, reg_loss=0.014470, lr=0.01596, time=755.9 ms, rem=20.54 m
iter= 1380, img_loss=0.008202, reg_loss=0.014458, lr=0.01588, time=757.9 ms, rem=20.46 m
iter= 1390, img_loss=0.007516, reg_loss=0.014471, lr=0.01581, time=758.3 ms, rem=20.35 m
iter= 1400, img_loss=0.008175, reg_loss=0.014464, lr=0.01574, time=767.0 ms, rem=20.45 m
iter= 1410, img_loss=0.010430, reg_loss=0.014469, lr=0.01566, time=848.1 ms, rem=22.47 m
iter= 1420, img_loss=0.011582, reg_loss=0.014480, lr=0.01559, time=847.4 ms, rem=22.32 m
iter= 1430, img_loss=0.009125, reg_loss=0.014468, lr=0.01552, time=848.5 ms, rem=22.20 m
iter= 1440, img_loss=0.008565, reg_loss=0.014472, lr=0.01545, time=851.2 ms, rem=22.13 m
iter= 1450, img_loss=0.009911, reg_loss=0.014477, lr=0.01538, time=847.7 ms, rem=21.90 m
iter= 1460, img_loss=0.008104, reg_loss=0.014467, lr=0.01531, time=851.4 ms, rem=21.85 m
iter= 1470, img_loss=0.007395, reg_loss=0.014466, lr=0.01524, time=849.9 ms, rem=21.67 m
iter= 1480, img_loss=0.009403, reg_loss=0.014466, lr=0.01517, time=848.5 ms, rem=21.49 m
iter= 1490, img_loss=0.007911, reg_loss=0.014453, lr=0.01510, time=850.2 ms, rem=21.40 m
iter= 1500, img_loss=0.007653, reg_loss=0.014456, lr=0.01503, time=848.7 ms, rem=21.22 m
iter= 1510, img_loss=0.012282, reg_loss=0.014458, lr=0.01496, time=850.0 ms, rem=21.11 m
iter= 1520, img_loss=0.010439, reg_loss=0.014455, lr=0.01489, time=846.3 ms, rem=20.87 m
iter= 1530, img_loss=0.007966, reg_loss=0.014456, lr=0.01482, time=848.4 ms, rem=20.78 m
iter= 1540, img_loss=0.008378, reg_loss=0.014451, lr=0.01475, time=848.6 ms, rem=20.65 m
iter= 1550, img_loss=0.009332, reg_loss=0.014455, lr=0.01469, time=846.3 ms, rem=20.45 m
iter= 1560, img_loss=0.010327, reg_loss=0.014456, lr=0.01462, time=847.3 ms, rem=20.34 m
iter= 1570, img_loss=0.010127, reg_loss=0.014458, lr=0.01455, time=846.6 ms, rem=20.18 m
iter= 1580, img_loss=0.011856, reg_loss=0.014452, lr=0.01449, time=847.4 ms, rem=20.06 m
iter= 1590, img_loss=0.011447, reg_loss=0.014459, lr=0.01442, time=845.9 ms, rem=19.88 m
iter= 1600, img_loss=0.009099, reg_loss=0.014454, lr=0.01435, time=849.6 ms, rem=19.82 m
iter= 1610, img_loss=0.010824, reg_loss=0.014454, lr=0.01429, time=848.1 ms, rem=19.65 m
iter= 1620, img_loss=0.008832, reg_loss=0.014464, lr=0.01422, time=850.4 ms, rem=19.56 m
iter= 1630, img_loss=0.010439, reg_loss=0.014459, lr=0.01416, time=846.8 ms, rem=19.33 m
iter= 1640, img_loss=0.007504, reg_loss=0.014462, lr=0.01409, time=851.3 ms, rem=19.30 m
iter= 1650, img_loss=0.009812, reg_loss=0.014469, lr=0.01403, time=848.3 ms, rem=19.09 m
iter= 1660, img_loss=0.008019, reg_loss=0.014475, lr=0.01396, time=849.8 ms, rem=18.98 m
iter= 1670, img_loss=0.009799, reg_loss=0.014478, lr=0.01390, time=848.4 ms, rem=18.81 m
iter= 1680, img_loss=0.007455, reg_loss=0.014462, lr=0.01383, time=849.5 ms, rem=18.69 m
iter= 1690, img_loss=0.009553, reg_loss=0.014458, lr=0.01377, time=848.9 ms, rem=18.53 m
iter= 1700, img_loss=0.009989, reg_loss=0.014461, lr=0.01371, time=848.7 ms, rem=18.39 m
iter= 1710, img_loss=0.009769, reg_loss=0.014458, lr=0.01364, time=846.2 ms, rem=18.19 m
iter= 1720, img_loss=0.008240, reg_loss=0.014460, lr=0.01358, time=847.4 ms, rem=18.08 m
iter= 1730, img_loss=0.010724, reg_loss=0.014458, lr=0.01352, time=849.0 ms, rem=17.97 m
iter= 1740, img_loss=0.008731, reg_loss=0.014450, lr=0.01346, time=851.3 ms, rem=17.88 m
iter= 1750, img_loss=0.010559, reg_loss=0.014461, lr=0.01339, time=848.0 ms, rem=17.67 m
iter= 1760, img_loss=0.009650, reg_loss=0.014462, lr=0.01333, time=849.3 ms, rem=17.55 m
iter= 1770, img_loss=0.009461, reg_loss=0.014453, lr=0.01327, time=847.6 ms, rem=17.38 m
iter= 1780, img_loss=0.009132, reg_loss=0.014443, lr=0.01321, time=849.8 ms, rem=17.28 m
iter= 1790, img_loss=0.009346, reg_loss=0.014448, lr=0.01315, time=846.2 ms, rem=17.07 m
iter= 1800, img_loss=0.010775, reg_loss=0.014461, lr=0.01309, time=848.4 ms, rem=16.97 m
iter= 1810, img_loss=0.010819, reg_loss=0.014466, lr=0.01303, time=847.8 ms, rem=16.81 m
iter= 1820, img_loss=0.010020, reg_loss=0.014462, lr=0.01297, time=847.2 ms, rem=16.66 m
iter= 1830, img_loss=0.009122, reg_loss=0.014457, lr=0.01291, time=847.8 ms, rem=16.53 m
iter= 1840, img_loss=0.008336, reg_loss=0.014455, lr=0.01285, time=850.6 ms, rem=16.45 m
iter= 1850, img_loss=0.008878, reg_loss=0.014465, lr=0.01279, time=849.1 ms, rem=16.27 m
iter= 1860, img_loss=0.008929, reg_loss=0.014462, lr=0.01273, time=848.0 ms, rem=16.11 m
iter= 1870, img_loss=0.008502, reg_loss=0.014466, lr=0.01267, time=846.6 ms, rem=15.94 m
iter= 1880, img_loss=0.009513, reg_loss=0.014466, lr=0.01262, time=847.9 ms, rem=15.83 m
iter= 1890, img_loss=0.008000, reg_loss=0.014461, lr=0.01256, time=848.9 ms, rem=15.71 m
iter= 1900, img_loss=0.010369, reg_loss=0.014462, lr=0.01250, time=845.9 ms, rem=15.51 m
iter= 1910, img_loss=0.009403, reg_loss=0.014466, lr=0.01244, time=849.7 ms, rem=15.44 m
iter= 1920, img_loss=0.009161, reg_loss=0.014457, lr=0.01239, time=849.0 ms, rem=15.28 m
iter= 1930, img_loss=0.007488, reg_loss=0.014455, lr=0.01233, time=851.3 ms, rem=15.18 m
iter= 1940, img_loss=0.009218, reg_loss=0.014463, lr=0.01227, time=848.6 ms, rem=14.99 m
iter= 1950, img_loss=0.008720, reg_loss=0.014451, lr=0.01222, time=849.4 ms, rem=14.86 m
iter= 1960, img_loss=0.009966, reg_loss=0.014456, lr=0.01216, time=847.9 ms, rem=14.70 m
iter= 1970, img_loss=0.008971, reg_loss=0.014447, lr=0.01210, time=848.0 ms, rem=14.56 m
iter= 1980, img_loss=0.007675, reg_loss=0.014454, lr=0.01205, time=848.7 ms, rem=14.43 m
iter= 1990, img_loss=0.012464, reg_loss=0.014460, lr=0.01199, time=847.1 ms, rem=14.26 m
iter= 2000, img_loss=0.009461, reg_loss=0.014452, lr=0.01194, time=848.1 ms, rem=14.14 m
iter= 2010, img_loss=0.008253, reg_loss=0.014446, lr=0.01188, time=849.7 ms, rem=14.02 m
iter= 2020, img_loss=0.008212, reg_loss=0.014446, lr=0.01183, time=849.0 ms, rem=13.87 m
iter= 2030, img_loss=0.009013, reg_loss=0.014440, lr=0.01177, time=846.8 ms, rem=13.69 m
iter= 2040, img_loss=0.009250, reg_loss=0.014443, lr=0.01172, time=848.4 ms, rem=13.57 m
iter= 2050, img_loss=0.008458, reg_loss=0.014438, lr=0.01167, time=850.5 ms, rem=13.47 m
iter= 2060, img_loss=0.009616, reg_loss=0.014435, lr=0.01161, time=848.6 ms, rem=13.29 m
iter= 2070, img_loss=0.009910, reg_loss=0.014441, lr=0.01156, time=846.4 ms, rem=13.12 m
iter= 2080, img_loss=0.010229, reg_loss=0.014435, lr=0.01151, time=848.0 ms, rem=13.00 m
iter= 2090, img_loss=0.010049, reg_loss=0.014445, lr=0.01145, time=849.4 ms, rem=12.88 m
iter= 2100, img_loss=0.010337, reg_loss=0.014443, lr=0.01140, time=847.1 ms, rem=12.71 m
iter= 2110, img_loss=0.008319, reg_loss=0.014438, lr=0.01135, time=849.7 ms, rem=12.60 m
iter= 2120, img_loss=0.011219, reg_loss=0.014448, lr=0.01130, time=848.4 ms, rem=12.44 m
iter= 2130, img_loss=0.011242, reg_loss=0.014446, lr=0.01124, time=847.8 ms, rem=12.29 m
iter= 2140, img_loss=0.007954, reg_loss=0.014438, lr=0.01119, time=850.5 ms, rem=12.19 m
iter= 2150, img_loss=0.009442, reg_loss=0.014430, lr=0.01114, time=848.4 ms, rem=12.02 m
iter= 2160, img_loss=0.007715, reg_loss=0.014429, lr=0.01109, time=848.9 ms, rem=11.88 m
iter= 2170, img_loss=0.010130, reg_loss=0.014439, lr=0.01104, time=846.5 ms, rem=11.71 m
iter= 2180, img_loss=0.011053, reg_loss=0.014438, lr=0.01099, time=847.9 ms, rem=11.59 m
iter= 2190, img_loss=0.011355, reg_loss=0.014440, lr=0.01094, time=848.8 ms, rem=11.46 m
iter= 2200, img_loss=0.010582, reg_loss=0.014431, lr=0.01089, time=845.6 ms, rem=11.28 m
iter= 2210, img_loss=0.009530, reg_loss=0.014433, lr=0.01084, time=849.2 ms, rem=11.18 m
iter= 2220, img_loss=0.008807, reg_loss=0.014435, lr=0.01079, time=848.6 ms, rem=11.03 m
iter= 2230, img_loss=0.008006, reg_loss=0.014434, lr=0.01074, time=848.8 ms, rem=10.89 m
iter= 2240, img_loss=0.007685, reg_loss=0.014435, lr=0.01069, time=850.2 ms, rem=10.77 m
iter= 2250, img_loss=0.010871, reg_loss=0.014441, lr=0.01064, time=846.5 ms, rem=10.58 m
iter= 2260, img_loss=0.009798, reg_loss=0.014432, lr=0.01059, time=848.2 ms, rem=10.46 m
iter= 2270, img_loss=0.008395, reg_loss=0.014426, lr=0.01054, time=847.7 ms, rem=10.31 m
iter= 2280, img_loss=0.009625, reg_loss=0.014430, lr=0.01049, time=848.5 ms, rem=10.18 m
iter= 2290, img_loss=0.010778, reg_loss=0.014432, lr=0.01045, time=846.4 ms, rem=10.02 m
iter= 2300, img_loss=0.010616, reg_loss=0.014448, lr=0.01040, time=848.7 ms, rem=9.90 m
iter= 2310, img_loss=0.008304, reg_loss=0.014448, lr=0.01035, time=849.0 ms, rem=9.76 m
iter= 2320, img_loss=0.010317, reg_loss=0.014447, lr=0.01030, time=847.7 ms, rem=9.61 m
iter= 2330, img_loss=0.007938, reg_loss=0.014447, lr=0.01025, time=849.1 ms, rem=9.48 m
iter= 2340, img_loss=0.009349, reg_loss=0.014446, lr=0.01021, time=847.2 ms, rem=9.32 m
iter= 2350, img_loss=0.009260, reg_loss=0.014439, lr=0.01016, time=847.3 ms, rem=9.18 m
iter= 2360, img_loss=0.009064, reg_loss=0.014426, lr=0.01011, time=847.0 ms, rem=9.03 m
iter= 2370, img_loss=0.010249, reg_loss=0.014423, lr=0.01007, time=847.6 ms, rem=8.90 m
iter= 2380, img_loss=0.011447, reg_loss=0.014415, lr=0.01002, time=847.9 ms, rem=8.76 m
iter= 2390, img_loss=0.009898, reg_loss=0.014406, lr=0.00998, time=846.8 ms, rem=8.61 m
iter= 2400, img_loss=0.008722, reg_loss=0.014415, lr=0.00993, time=848.8 ms, rem=8.49 m
iter= 2410, img_loss=0.011140, reg_loss=0.014423, lr=0.00988, time=845.2 ms, rem=8.31 m
iter= 2420, img_loss=0.007737, reg_loss=0.014418, lr=0.00984, time=850.2 ms, rem=8.22 m
iter= 2430, img_loss=0.009299, reg_loss=0.014423, lr=0.00979, time=852.5 ms, rem=8.10 m
iter= 2440, img_loss=0.010766, reg_loss=0.014429, lr=0.00975, time=847.6 ms, rem=7.91 m
iter= 2450, img_loss=0.009192, reg_loss=0.014428, lr=0.00970, time=847.4 ms, rem=7.77 m
iter= 2460, img_loss=0.010136, reg_loss=0.014431, lr=0.00966, time=847.2 ms, rem=7.62 m
iter= 2470, img_loss=0.009630, reg_loss=0.014427, lr=0.00961, time=847.9 ms, rem=7.49 m
iter= 2480, img_loss=0.009872, reg_loss=0.014419, lr=0.00957, time=847.4 ms, rem=7.34 m
iter= 2490, img_loss=0.011706, reg_loss=0.014425, lr=0.00953, time=844.6 ms, rem=7.18 m
iter= 2500, img_loss=0.008658, reg_loss=0.014416, lr=0.00948, time=850.5 ms, rem=7.09 m
iter= 2510, img_loss=0.008001, reg_loss=0.014413, lr=0.00944, time=849.1 ms, rem=6.93 m
iter= 2520, img_loss=0.010480, reg_loss=0.014413, lr=0.00940, time=849.6 ms, rem=6.80 m
iter= 2530, img_loss=0.009230, reg_loss=0.014418, lr=0.00935, time=846.6 ms, rem=6.63 m
iter= 2540, img_loss=0.009982, reg_loss=0.014414, lr=0.00931, time=851.1 ms, rem=6.53 m
iter= 2550, img_loss=0.007620, reg_loss=0.014419, lr=0.00927, time=849.8 ms, rem=6.37 m
iter= 2560, img_loss=0.007744, reg_loss=0.014413, lr=0.00922, time=850.7 ms, rem=6.24 m
iter= 2570, img_loss=0.009443, reg_loss=0.014417, lr=0.00918, time=845.3 ms, rem=6.06 m
iter= 2580, img_loss=0.009737, reg_loss=0.014413, lr=0.00914, time=849.6 ms, rem=5.95 m
iter= 2590, img_loss=0.007492, reg_loss=0.014410, lr=0.00910, time=851.2 ms, rem=5.82 m
iter= 2600, img_loss=0.009303, reg_loss=0.014416, lr=0.00906, time=849.0 ms, rem=5.66 m
iter= 2610, img_loss=0.009185, reg_loss=0.014417, lr=0.00901, time=847.1 ms, rem=5.51 m
iter= 2620, img_loss=0.008498, reg_loss=0.014416, lr=0.00897, time=849.3 ms, rem=5.38 m
iter= 2630, img_loss=0.008600, reg_loss=0.014420, lr=0.00893, time=849.8 ms, rem=5.24 m
iter= 2640, img_loss=0.009036, reg_loss=0.014419, lr=0.00889, time=847.9 ms, rem=5.09 m
iter= 2650, img_loss=0.009799, reg_loss=0.014418, lr=0.00885, time=849.3 ms, rem=4.95 m
iter= 2660, img_loss=0.008498, reg_loss=0.014416, lr=0.00881, time=848.3 ms, rem=4.81 m
iter= 2670, img_loss=0.009013, reg_loss=0.014424, lr=0.00877, time=847.5 ms, rem=4.66 m
iter= 2680, img_loss=0.009685, reg_loss=0.014422, lr=0.00873, time=847.1 ms, rem=4.52 m
iter= 2690, img_loss=0.007085, reg_loss=0.014412, lr=0.00869, time=853.9 ms, rem=4.41 m
iter= 2700, img_loss=0.009471, reg_loss=0.014417, lr=0.00865, time=849.2 ms, rem=4.25 m
iter= 2710, img_loss=0.013013, reg_loss=0.014419, lr=0.00861, time=846.3 ms, rem=4.09 m
iter= 2720, img_loss=0.010403, reg_loss=0.014427, lr=0.00857, time=845.5 ms, rem=3.95 m
iter= 2730, img_loss=0.010199, reg_loss=0.014415, lr=0.00853, time=849.5 ms, rem=3.82 m
iter= 2740, img_loss=0.009712, reg_loss=0.014400, lr=0.00849, time=847.9 ms, rem=3.67 m
iter= 2750, img_loss=0.009893, reg_loss=0.014397, lr=0.00845, time=847.3 ms, rem=3.53 m
iter= 2760, img_loss=0.010275, reg_loss=0.014399, lr=0.00841, time=848.2 ms, rem=3.39 m
iter= 2770, img_loss=0.011155, reg_loss=0.014399, lr=0.00837, time=845.5 ms, rem=3.24 m
iter= 2780, img_loss=0.011202, reg_loss=0.014398, lr=0.00834, time=847.4 ms, rem=3.11 m
iter= 2790, img_loss=0.009078, reg_loss=0.014403, lr=0.00830, time=846.7 ms, rem=2.96 m
iter= 2800, img_loss=0.007366, reg_loss=0.014406, lr=0.00826, time=851.1 ms, rem=2.84 m
iter= 2810, img_loss=0.009697, reg_loss=0.014416, lr=0.00822, time=849.2 ms, rem=2.69 m
iter= 2820, img_loss=0.008554, reg_loss=0.014411, lr=0.00818, time=849.2 ms, rem=2.55 m
iter= 2830, img_loss=0.008310, reg_loss=0.014410, lr=0.00815, time=850.7 ms, rem=2.41 m
iter= 2840, img_loss=0.009761, reg_loss=0.014413, lr=0.00811, time=848.6 ms, rem=2.26 m
iter= 2850, img_loss=0.009872, reg_loss=0.014406, lr=0.00807, time=847.2 ms, rem=2.12 m
iter= 2860, img_loss=0.010373, reg_loss=0.014407, lr=0.00803, time=848.6 ms, rem=1.98 m
iter= 2870, img_loss=0.009383, reg_loss=0.014405, lr=0.00800, time=848.8 ms, rem=1.84 m
iter= 2880, img_loss=0.008084, reg_loss=0.014408, lr=0.00796, time=848.0 ms, rem=1.70 m
iter= 2890, img_loss=0.009452, reg_loss=0.014409, lr=0.00792, time=848.2 ms, rem=1.55 m
iter= 2900, img_loss=0.008998, reg_loss=0.014403, lr=0.00789, time=847.6 ms, rem=1.41 m
iter= 2910, img_loss=0.008265, reg_loss=0.014400, lr=0.00785, time=850.9 ms, rem=1.28 m
iter= 2920, img_loss=0.011217, reg_loss=0.014413, lr=0.00781, time=848.2 ms, rem=1.13 m
iter= 2930, img_loss=0.010852, reg_loss=0.014415, lr=0.00778, time=846.0 ms, rem=59.22 s
iter= 2940, img_loss=0.008579, reg_loss=0.014406, lr=0.00774, time=850.7 ms, rem=51.04 s
iter= 2950, img_loss=0.008460, reg_loss=0.014402, lr=0.00771, time=847.6 ms, rem=42.38 s
iter= 2960, img_loss=0.007280, reg_loss=0.014396, lr=0.00767, time=851.1 ms, rem=34.04 s
iter= 2970, img_loss=0.007097, reg_loss=0.014401, lr=0.00764, time=850.0 ms, rem=25.50 s
iter= 2980, img_loss=0.011145, reg_loss=0.014408, lr=0.00760, time=846.4 ms, rem=16.93 s
iter= 2990, img_loss=0.009313, reg_loss=0.014402, lr=0.00757, time=849.2 ms, rem=8.49 s
Running validation
MSE,      PSNR
0.00084232, 30.788
Cuda path /usr/local/cuda
End of OptiXStateWrapper 
Base mesh has 10612 triangles and 5308 vertices.
Avg edge length: 0.037852
Writing mesh:  out/bob/dmtet_mesh/mesh.obj
    writing 5308 vertices
    writing 7932 texcoords
    writing 5308 normals
    writing 10612 faces
Writing material:  out/bob/dmtet_mesh/mesh.mtl
OptiXStateWrapper destructor 
Done exporting mesh
iter=    0, img_loss=0.007739, reg_loss=0.015051, lr=0.00010, time=409.2 ms, rem=20.46 m
iter=   10, img_loss=0.008252, reg_loss=0.014887, lr=0.00110, time=406.8 ms, rem=20.27 m
iter=   20, img_loss=0.008838, reg_loss=0.015425, lr=0.00210, time=406.7 ms, rem=20.20 m
iter=   30, img_loss=0.010698, reg_loss=0.015444, lr=0.00310, time=406.8 ms, rem=20.14 m
iter=   40, img_loss=0.008559, reg_loss=0.015010, lr=0.00410, time=406.8 ms, rem=20.07 m
iter=   50, img_loss=0.007864, reg_loss=0.014362, lr=0.00510, time=406.9 ms, rem=20.00 m
iter=   60, img_loss=0.012201, reg_loss=0.015895, lr=0.00610, time=406.9 ms, rem=19.94 m
iter=   70, img_loss=0.012125, reg_loss=0.015467, lr=0.00710, time=406.7 ms, rem=19.86 m
iter=   80, img_loss=0.010982, reg_loss=0.014836, lr=0.00810, time=407.1 ms, rem=19.81 m
iter=   90, img_loss=0.011356, reg_loss=0.015559, lr=0.00910, time=407.4 ms, rem=19.76 m
iter=  100, img_loss=0.010647, reg_loss=0.015061, lr=0.01000, time=407.4 ms, rem=19.69 m
iter=  110, img_loss=0.008678, reg_loss=0.014965, lr=0.00995, time=407.4 ms, rem=19.63 m
iter=  120, img_loss=0.006168, reg_loss=0.014371, lr=0.00990, time=407.5 ms, rem=19.56 m
iter=  130, img_loss=0.010101, reg_loss=0.015366, lr=0.00986, time=407.5 ms, rem=19.49 m
iter=  140, img_loss=0.007696, reg_loss=0.014999, lr=0.00981, time=407.5 ms, rem=19.42 m
iter=  150, img_loss=0.009253, reg_loss=0.014866, lr=0.00977, time=407.3 ms, rem=19.35 m
iter=  160, img_loss=0.009489, reg_loss=0.015215, lr=0.00972, time=407.4 ms, rem=19.28 m
iter=  170, img_loss=0.010048, reg_loss=0.014848, lr=0.00968, time=407.4 ms, rem=19.21 m
iter=  180, img_loss=0.008477, reg_loss=0.015122, lr=0.00963, time=407.4 ms, rem=19.15 m
iter=  190, img_loss=0.009917, reg_loss=0.015398, lr=0.00959, time=407.3 ms, rem=19.08 m
iter=  200, img_loss=0.008188, reg_loss=0.014912, lr=0.00955, time=407.5 ms, rem=19.02 m
iter=  210, img_loss=0.008850, reg_loss=0.014637, lr=0.00950, time=407.4 ms, rem=18.95 m
iter=  220, img_loss=0.007849, reg_loss=0.014485, lr=0.00946, time=407.6 ms, rem=18.88 m
iter=  230, img_loss=0.007735, reg_loss=0.014306, lr=0.00941, time=407.6 ms, rem=18.82 m
iter=  240, img_loss=0.010005, reg_loss=0.015319, lr=0.00937, time=407.4 ms, rem=18.74 m
iter=  250, img_loss=0.010243, reg_loss=0.015571, lr=0.00933, time=407.4 ms, rem=18.67 m
iter=  260, img_loss=0.008892, reg_loss=0.014903, lr=0.00929, time=407.4 ms, rem=18.60 m
iter=  270, img_loss=0.008839, reg_loss=0.014815, lr=0.00924, time=407.5 ms, rem=18.54 m
iter=  280, img_loss=0.008435, reg_loss=0.014836, lr=0.00920, time=407.3 ms, rem=18.47 m
iter=  290, img_loss=0.008693, reg_loss=0.014498, lr=0.00916, time=407.5 ms, rem=18.40 m
iter=  300, img_loss=0.008182, reg_loss=0.014562, lr=0.00912, time=407.5 ms, rem=18.34 m
iter=  310, img_loss=0.008483, reg_loss=0.014985, lr=0.00907, time=407.4 ms, rem=18.27 m
iter=  320, img_loss=0.009305, reg_loss=0.014906, lr=0.00903, time=407.3 ms, rem=18.19 m
iter=  330, img_loss=0.010036, reg_loss=0.015837, lr=0.00899, time=407.5 ms, rem=18.13 m
iter=  340, img_loss=0.010366, reg_loss=0.015079, lr=0.00895, time=407.4 ms, rem=18.06 m
iter=  350, img_loss=0.008543, reg_loss=0.014703, lr=0.00891, time=407.5 ms, rem=18.00 m
iter=  360, img_loss=0.009988, reg_loss=0.015222, lr=0.00887, time=407.5 ms, rem=17.93 m
iter=  370, img_loss=0.008943, reg_loss=0.015021, lr=0.00883, time=407.4 ms, rem=17.86 m
iter=  380, img_loss=0.009047, reg_loss=0.015235, lr=0.00879, time=407.4 ms, rem=17.79 m
iter=  390, img_loss=0.008271, reg_loss=0.014787, lr=0.00875, time=407.4 ms, rem=17.72 m
iter=  400, img_loss=0.009400, reg_loss=0.014970, lr=0.00871, time=407.5 ms, rem=17.66 m
iter=  410, img_loss=0.009935, reg_loss=0.015052, lr=0.00867, time=407.5 ms, rem=17.59 m
iter=  420, img_loss=0.010835, reg_loss=0.015393, lr=0.00863, time=407.5 ms, rem=17.52 m
iter=  430, img_loss=0.009245, reg_loss=0.015241, lr=0.00859, time=407.5 ms, rem=17.45 m
iter=  440, img_loss=0.011414, reg_loss=0.015453, lr=0.00855, time=407.4 ms, rem=17.38 m
iter=  450, img_loss=0.011614, reg_loss=0.014865, lr=0.00851, time=407.3 ms, rem=17.31 m
iter=  460, img_loss=0.010018, reg_loss=0.014520, lr=0.00847, time=407.5 ms, rem=17.25 m
iter=  470, img_loss=0.007079, reg_loss=0.014679, lr=0.00843, time=407.4 ms, rem=17.18 m
iter=  480, img_loss=0.009070, reg_loss=0.014633, lr=0.00839, time=407.6 ms, rem=17.12 m
iter=  490, img_loss=0.009029, reg_loss=0.015475, lr=0.00835, time=407.4 ms, rem=17.04 m
iter=  500, img_loss=0.009690, reg_loss=0.015323, lr=0.00831, time=407.5 ms, rem=16.98 m
iter=  510, img_loss=0.008074, reg_loss=0.014658, lr=0.00828, time=407.5 ms, rem=16.91 m
iter=  520, img_loss=0.009335, reg_loss=0.014920, lr=0.00824, time=407.5 ms, rem=16.84 m
iter=  530, img_loss=0.010342, reg_loss=0.015114, lr=0.00820, time=407.6 ms, rem=16.78 m
iter=  540, img_loss=0.008367, reg_loss=0.014722, lr=0.00816, time=407.6 ms, rem=16.71 m
iter=  550, img_loss=0.008259, reg_loss=0.014919, lr=0.00812, time=407.5 ms, rem=16.64 m
iter=  560, img_loss=0.008954, reg_loss=0.014932, lr=0.00809, time=407.0 ms, rem=16.55 m
iter=  570, img_loss=0.009969, reg_loss=0.015331, lr=0.00805, time=406.7 ms, rem=16.47 m
iter=  580, img_loss=0.008359, reg_loss=0.014919, lr=0.00801, time=406.9 ms, rem=16.41 m
iter=  590, img_loss=0.010708, reg_loss=0.015288, lr=0.00798, time=406.8 ms, rem=16.34 m
iter=  600, img_loss=0.009129, reg_loss=0.015189, lr=0.00794, time=406.8 ms, rem=16.27 m
iter=  610, img_loss=0.012037, reg_loss=0.016024, lr=0.00790, time=406.7 ms, rem=16.20 m
iter=  620, img_loss=0.009734, reg_loss=0.014766, lr=0.00787, time=406.8 ms, rem=16.14 m
iter=  630, img_loss=0.007275, reg_loss=0.014570, lr=0.00783, time=406.9 ms, rem=16.07 m
iter=  640, img_loss=0.009476, reg_loss=0.015726, lr=0.00779, time=406.7 ms, rem=16.00 m
iter=  650, img_loss=0.009401, reg_loss=0.014888, lr=0.00776, time=406.8 ms, rem=15.93 m
iter=  660, img_loss=0.011574, reg_loss=0.015976, lr=0.00772, time=406.8 ms, rem=15.86 m
iter=  670, img_loss=0.009050, reg_loss=0.014969, lr=0.00769, time=407.0 ms, rem=15.80 m
iter=  680, img_loss=0.013896, reg_loss=0.016032, lr=0.00765, time=406.7 ms, rem=15.73 m
iter=  690, img_loss=0.008351, reg_loss=0.015171, lr=0.00762, time=406.8 ms, rem=15.66 m
iter=  700, img_loss=0.008718, reg_loss=0.014760, lr=0.00758, time=406.9 ms, rem=15.60 m
iter=  710, img_loss=0.010009, reg_loss=0.015155, lr=0.00755, time=408.6 ms, rem=15.60 m
iter=  720, img_loss=0.008509, reg_loss=0.014706, lr=0.00751, time=407.8 ms, rem=15.49 m
iter=  730, img_loss=0.009829, reg_loss=0.015245, lr=0.00748, time=407.5 ms, rem=15.42 m
iter=  740, img_loss=0.007645, reg_loss=0.014563, lr=0.00744, time=407.6 ms, rem=15.35 m
iter=  750, img_loss=0.008031, reg_loss=0.015134, lr=0.00741, time=407.3 ms, rem=15.27 m
iter=  760, img_loss=0.009054, reg_loss=0.015254, lr=0.00738, time=406.7 ms, rem=15.18 m
iter=  770, img_loss=0.008360, reg_loss=0.015037, lr=0.00734, time=406.8 ms, rem=15.12 m
iter=  780, img_loss=0.008961, reg_loss=0.014816, lr=0.00731, time=406.8 ms, rem=15.05 m
iter=  790, img_loss=0.009092, reg_loss=0.014877, lr=0.00727, time=406.8 ms, rem=14.99 m
iter=  800, img_loss=0.010296, reg_loss=0.015042, lr=0.00724, time=406.8 ms, rem=14.92 m
iter=  810, img_loss=0.011958, reg_loss=0.015654, lr=0.00721, time=406.7 ms, rem=14.85 m
iter=  820, img_loss=0.011974, reg_loss=0.016124, lr=0.00717, time=406.7 ms, rem=14.78 m
iter=  830, img_loss=0.008973, reg_loss=0.015059, lr=0.00714, time=406.8 ms, rem=14.71 m
iter=  840, img_loss=0.009596, reg_loss=0.015078, lr=0.00711, time=406.8 ms, rem=14.64 m
iter=  850, img_loss=0.010839, reg_loss=0.015093, lr=0.00708, time=406.7 ms, rem=14.58 m
iter=  860, img_loss=0.009156, reg_loss=0.015456, lr=0.00704, time=406.7 ms, rem=14.51 m
iter=  870, img_loss=0.009741, reg_loss=0.014557, lr=0.00701, time=406.8 ms, rem=14.44 m
iter=  880, img_loss=0.010782, reg_loss=0.015376, lr=0.00698, time=406.9 ms, rem=14.38 m
iter=  890, img_loss=0.008434, reg_loss=0.014654, lr=0.00695, time=408.3 ms, rem=14.36 m
iter=  900, img_loss=0.006887, reg_loss=0.014638, lr=0.00692, time=407.5 ms, rem=14.26 m
iter=  910, img_loss=0.010423, reg_loss=0.015546, lr=0.00688, time=407.0 ms, rem=14.18 m
iter=  920, img_loss=0.010221, reg_loss=0.015528, lr=0.00685, time=407.0 ms, rem=14.11 m
iter=  930, img_loss=0.012177, reg_loss=0.015517, lr=0.00682, time=406.9 ms, rem=14.04 m
iter=  940, img_loss=0.009098, reg_loss=0.015082, lr=0.00679, time=406.8 ms, rem=13.97 m
iter=  950, img_loss=0.010713, reg_loss=0.014952, lr=0.00676, time=406.9 ms, rem=13.90 m
iter=  960, img_loss=0.009853, reg_loss=0.014605, lr=0.00673, time=406.9 ms, rem=13.84 m
iter=  970, img_loss=0.006835, reg_loss=0.014719, lr=0.00670, time=406.9 ms, rem=13.77 m
iter=  980, img_loss=0.011110, reg_loss=0.015557, lr=0.00666, time=406.9 ms, rem=13.70 m
iter=  990, img_loss=0.008628, reg_loss=0.014660, lr=0.00663, time=408.5 ms, rem=13.68 m
iter= 1000, img_loss=0.008356, reg_loss=0.015066, lr=0.00660, time=406.9 ms, rem=13.56 m
iter= 1010, img_loss=0.008239, reg_loss=0.014869, lr=0.00657, time=407.7 ms, rem=13.52 m
iter= 1020, img_loss=0.008200, reg_loss=0.014894, lr=0.00654, time=408.0 ms, rem=13.46 m
iter= 1030, img_loss=0.009477, reg_loss=0.015142, lr=0.00651, time=406.9 ms, rem=13.36 m
iter= 1040, img_loss=0.007450, reg_loss=0.014749, lr=0.00648, time=407.1 ms, rem=13.30 m
iter= 1050, img_loss=0.010156, reg_loss=0.015091, lr=0.00645, time=407.0 ms, rem=13.23 m
iter= 1060, img_loss=0.008511, reg_loss=0.014448, lr=0.00642, time=407.1 ms, rem=13.16 m
iter= 1070, img_loss=0.009341, reg_loss=0.015807, lr=0.00639, time=406.9 ms, rem=13.09 m
iter= 1080, img_loss=0.008963, reg_loss=0.014701, lr=0.00637, time=406.9 ms, rem=13.02 m
iter= 1090, img_loss=0.011254, reg_loss=0.015469, lr=0.00634, time=407.1 ms, rem=12.96 m
iter= 1100, img_loss=0.009496, reg_loss=0.014868, lr=0.00631, time=407.0 ms, rem=12.89 m
iter= 1110, img_loss=0.009539, reg_loss=0.015428, lr=0.00628, time=407.0 ms, rem=12.82 m
iter= 1120, img_loss=0.010408, reg_loss=0.015199, lr=0.00625, time=407.0 ms, rem=12.75 m
iter= 1130, img_loss=0.007351, reg_loss=0.014606, lr=0.00622, time=407.1 ms, rem=12.69 m
iter= 1140, img_loss=0.010855, reg_loss=0.015806, lr=0.00619, time=406.9 ms, rem=12.62 m
iter= 1150, img_loss=0.010882, reg_loss=0.015032, lr=0.00616, time=407.0 ms, rem=12.55 m
iter= 1160, img_loss=0.009784, reg_loss=0.015677, lr=0.00613, time=407.0 ms, rem=12.48 m
iter= 1170, img_loss=0.009106, reg_loss=0.015358, lr=0.00611, time=407.0 ms, rem=12.41 m
iter= 1180, img_loss=0.010929, reg_loss=0.015628, lr=0.00608, time=406.9 ms, rem=12.34 m
iter= 1190, img_loss=0.007717, reg_loss=0.014997, lr=0.00605, time=407.0 ms, rem=12.28 m
iter= 1200, img_loss=0.010000, reg_loss=0.015208, lr=0.00602, time=407.1 ms, rem=12.21 m
iter= 1210, img_loss=0.008605, reg_loss=0.014845, lr=0.00600, time=407.0 ms, rem=12.14 m
iter= 1220, img_loss=0.009073, reg_loss=0.014947, lr=0.00597, time=407.0 ms, rem=12.08 m
iter= 1230, img_loss=0.009803, reg_loss=0.015415, lr=0.00594, time=407.0 ms, rem=12.01 m
iter= 1240, img_loss=0.009536, reg_loss=0.014613, lr=0.00591, time=407.1 ms, rem=11.94 m
iter= 1250, img_loss=0.011300, reg_loss=0.015828, lr=0.00589, time=407.1 ms, rem=11.87 m
iter= 1260, img_loss=0.009315, reg_loss=0.014920, lr=0.00586, time=406.9 ms, rem=11.80 m
iter= 1270, img_loss=0.008950, reg_loss=0.015451, lr=0.00583, time=407.0 ms, rem=11.73 m
iter= 1280, img_loss=0.007091, reg_loss=0.014694, lr=0.00580, time=407.1 ms, rem=11.67 m
iter= 1290, img_loss=0.008777, reg_loss=0.014789, lr=0.00578, time=407.1 ms, rem=11.60 m
iter= 1300, img_loss=0.009829, reg_loss=0.015398, lr=0.00575, time=407.0 ms, rem=11.53 m
iter= 1310, img_loss=0.012329, reg_loss=0.015724, lr=0.00573, time=407.0 ms, rem=11.46 m
iter= 1320, img_loss=0.008237, reg_loss=0.014890, lr=0.00570, time=407.1 ms, rem=11.40 m
iter= 1330, img_loss=0.008557, reg_loss=0.014683, lr=0.00567, time=407.1 ms, rem=11.33 m
iter= 1340, img_loss=0.009938, reg_loss=0.015547, lr=0.00565, time=407.0 ms, rem=11.26 m
iter= 1350, img_loss=0.012449, reg_loss=0.015342, lr=0.00562, time=407.0 ms, rem=11.19 m
iter= 1360, img_loss=0.010474, reg_loss=0.015423, lr=0.00559, time=407.0 ms, rem=11.12 m
iter= 1370, img_loss=0.009211, reg_loss=0.014849, lr=0.00557, time=407.1 ms, rem=11.06 m
iter= 1380, img_loss=0.011384, reg_loss=0.015553, lr=0.00554, time=406.9 ms, rem=10.99 m
iter= 1390, img_loss=0.008227, reg_loss=0.014636, lr=0.00552, time=407.0 ms, rem=10.92 m
iter= 1400, img_loss=0.012788, reg_loss=0.015944, lr=0.00549, time=407.0 ms, rem=10.85 m
iter= 1410, img_loss=0.008401, reg_loss=0.014748, lr=0.00547, time=407.0 ms, rem=10.79 m
iter= 1420, img_loss=0.009093, reg_loss=0.014857, lr=0.00544, time=407.0 ms, rem=10.72 m
iter= 1430, img_loss=0.010172, reg_loss=0.015228, lr=0.00542, time=406.9 ms, rem=10.65 m
iter= 1440, img_loss=0.008369, reg_loss=0.014759, lr=0.00539, time=407.0 ms, rem=10.58 m
iter= 1450, img_loss=0.007941, reg_loss=0.014573, lr=0.00537, time=407.1 ms, rem=10.52 m
iter= 1460, img_loss=0.010361, reg_loss=0.015783, lr=0.00534, time=406.9 ms, rem=10.44 m
iter= 1470, img_loss=0.007646, reg_loss=0.014555, lr=0.00532, time=407.1 ms, rem=10.38 m
iter= 1480, img_loss=0.008010, reg_loss=0.014247, lr=0.00529, time=407.0 ms, rem=10.31 m
iter= 1490, img_loss=0.011778, reg_loss=0.015825, lr=0.00527, time=407.0 ms, rem=10.24 m
iter= 1500, img_loss=0.009213, reg_loss=0.014935, lr=0.00525, time=407.0 ms, rem=10.18 m
iter= 1510, img_loss=0.011548, reg_loss=0.015856, lr=0.00522, time=406.9 ms, rem=10.11 m
iter= 1520, img_loss=0.009872, reg_loss=0.015453, lr=0.00520, time=407.0 ms, rem=10.04 m
iter= 1530, img_loss=0.010076, reg_loss=0.014914, lr=0.00517, time=407.0 ms, rem=9.97 m
iter= 1540, img_loss=0.009345, reg_loss=0.015079, lr=0.00515, time=407.0 ms, rem=9.90 m
iter= 1550, img_loss=0.009380, reg_loss=0.014899, lr=0.00513, time=407.0 ms, rem=9.84 m
iter= 1560, img_loss=0.008547, reg_loss=0.015138, lr=0.00510, time=407.0 ms, rem=9.77 m
iter= 1570, img_loss=0.009960, reg_loss=0.015277, lr=0.00508, time=406.9 ms, rem=9.70 m
iter= 1580, img_loss=0.010470, reg_loss=0.015421, lr=0.00506, time=407.0 ms, rem=9.63 m
iter= 1590, img_loss=0.009132, reg_loss=0.014716, lr=0.00503, time=407.1 ms, rem=9.57 m
iter= 1600, img_loss=0.009614, reg_loss=0.015146, lr=0.00501, time=407.0 ms, rem=9.50 m
iter= 1610, img_loss=0.009737, reg_loss=0.015195, lr=0.00499, time=407.1 ms, rem=9.43 m
iter= 1620, img_loss=0.008912, reg_loss=0.014913, lr=0.00496, time=407.0 ms, rem=9.36 m
iter= 1630, img_loss=0.010092, reg_loss=0.015424, lr=0.00494, time=406.9 ms, rem=9.29 m
iter= 1640, img_loss=0.010371, reg_loss=0.015103, lr=0.00492, time=407.0 ms, rem=9.22 m
iter= 1650, img_loss=0.009225, reg_loss=0.015005, lr=0.00490, time=406.9 ms, rem=9.15 m
iter= 1660, img_loss=0.010356, reg_loss=0.015305, lr=0.00487, time=407.0 ms, rem=9.09 m
iter= 1670, img_loss=0.009658, reg_loss=0.015340, lr=0.00485, time=406.9 ms, rem=9.02 m
iter= 1680, img_loss=0.008623, reg_loss=0.014647, lr=0.00483, time=407.0 ms, rem=8.95 m
iter= 1690, img_loss=0.009447, reg_loss=0.015597, lr=0.00481, time=406.9 ms, rem=8.88 m
iter= 1700, img_loss=0.007376, reg_loss=0.014662, lr=0.00478, time=407.0 ms, rem=8.82 m
iter= 1710, img_loss=0.011079, reg_loss=0.015554, lr=0.00476, time=407.0 ms, rem=8.75 m
iter= 1720, img_loss=0.008875, reg_loss=0.014773, lr=0.00474, time=407.1 ms, rem=8.68 m
iter= 1730, img_loss=0.009600, reg_loss=0.014852, lr=0.00472, time=406.9 ms, rem=8.61 m
iter= 1740, img_loss=0.009099, reg_loss=0.014722, lr=0.00470, time=407.0 ms, rem=8.55 m
iter= 1750, img_loss=0.009397, reg_loss=0.015188, lr=0.00468, time=407.1 ms, rem=8.48 m
iter= 1760, img_loss=0.008931, reg_loss=0.015335, lr=0.00465, time=407.0 ms, rem=8.41 m
iter= 1770, img_loss=0.008955, reg_loss=0.014897, lr=0.00463, time=407.1 ms, rem=8.34 m
iter= 1780, img_loss=0.008624, reg_loss=0.014742, lr=0.00461, time=407.3 ms, rem=8.28 m
iter= 1790, img_loss=0.010281, reg_loss=0.014906, lr=0.00459, time=407.0 ms, rem=8.21 m
iter= 1800, img_loss=0.009912, reg_loss=0.015410, lr=0.00457, time=407.0 ms, rem=8.14 m
iter= 1810, img_loss=0.009018, reg_loss=0.015245, lr=0.00455, time=407.6 ms, rem=8.08 m
iter= 1820, img_loss=0.008431, reg_loss=0.014609, lr=0.00453, time=407.1 ms, rem=8.01 m
iter= 1830, img_loss=0.008668, reg_loss=0.014712, lr=0.00451, time=407.0 ms, rem=7.94 m
iter= 1840, img_loss=0.011204, reg_loss=0.015777, lr=0.00449, time=407.0 ms, rem=7.87 m
iter= 1850, img_loss=0.012956, reg_loss=0.016749, lr=0.00446, time=406.9 ms, rem=7.80 m
iter= 1860, img_loss=0.008784, reg_loss=0.014769, lr=0.00444, time=407.0 ms, rem=7.73 m
iter= 1870, img_loss=0.009590, reg_loss=0.015136, lr=0.00442, time=407.0 ms, rem=7.66 m
iter= 1880, img_loss=0.005877, reg_loss=0.014300, lr=0.00440, time=407.1 ms, rem=7.60 m
iter= 1890, img_loss=0.011716, reg_loss=0.015836, lr=0.00438, time=407.0 ms, rem=7.53 m
iter= 1900, img_loss=0.007833, reg_loss=0.014310, lr=0.00436, time=407.1 ms, rem=7.46 m
iter= 1910, img_loss=0.010490, reg_loss=0.014949, lr=0.00434, time=407.0 ms, rem=7.39 m
iter= 1920, img_loss=0.008579, reg_loss=0.014567, lr=0.00432, time=407.0 ms, rem=7.33 m
iter= 1930, img_loss=0.007007, reg_loss=0.014380, lr=0.00430, time=407.2 ms, rem=7.26 m
iter= 1940, img_loss=0.010700, reg_loss=0.015219, lr=0.00428, time=407.0 ms, rem=7.19 m
iter= 1950, img_loss=0.008173, reg_loss=0.014463, lr=0.00426, time=407.0 ms, rem=7.12 m
iter= 1960, img_loss=0.008136, reg_loss=0.014851, lr=0.00424, time=407.1 ms, rem=7.06 m
iter= 1970, img_loss=0.007937, reg_loss=0.015112, lr=0.00422, time=407.0 ms, rem=6.99 m
iter= 1980, img_loss=0.007697, reg_loss=0.014879, lr=0.00421, time=407.0 ms, rem=6.92 m
iter= 1990, img_loss=0.007201, reg_loss=0.014672, lr=0.00419, time=407.1 ms, rem=6.85 m
iter= 2000, img_loss=0.008639, reg_loss=0.014791, lr=0.00417, time=407.2 ms, rem=6.79 m
iter= 2010, img_loss=0.008423, reg_loss=0.014923, lr=0.00415, time=407.1 ms, rem=6.72 m
iter= 2020, img_loss=0.009426, reg_loss=0.015191, lr=0.00413, time=407.0 ms, rem=6.65 m
iter= 2030, img_loss=0.009122, reg_loss=0.014740, lr=0.00411, time=407.0 ms, rem=6.58 m
iter= 2040, img_loss=0.009472, reg_loss=0.014907, lr=0.00409, time=407.0 ms, rem=6.51 m
iter= 2050, img_loss=0.010248, reg_loss=0.015486, lr=0.00407, time=407.1 ms, rem=6.44 m
iter= 2060, img_loss=0.007725, reg_loss=0.014629, lr=0.00405, time=407.1 ms, rem=6.38 m
iter= 2070, img_loss=0.008398, reg_loss=0.014908, lr=0.00403, time=407.0 ms, rem=6.31 m
iter= 2080, img_loss=0.009458, reg_loss=0.015289, lr=0.00402, time=407.0 ms, rem=6.24 m
iter= 2090, img_loss=0.009719, reg_loss=0.015202, lr=0.00400, time=407.0 ms, rem=6.17 m
iter= 2100, img_loss=0.012465, reg_loss=0.015576, lr=0.00398, time=407.1 ms, rem=6.11 m
iter= 2110, img_loss=0.007863, reg_loss=0.014985, lr=0.00396, time=407.0 ms, rem=6.04 m
iter= 2120, img_loss=0.008178, reg_loss=0.014701, lr=0.00394, time=407.0 ms, rem=5.97 m
iter= 2130, img_loss=0.010850, reg_loss=0.015746, lr=0.00392, time=407.0 ms, rem=5.90 m
iter= 2140, img_loss=0.007991, reg_loss=0.014971, lr=0.00391, time=407.0 ms, rem=5.83 m
iter= 2150, img_loss=0.009150, reg_loss=0.015141, lr=0.00389, time=407.1 ms, rem=5.77 m
iter= 2160, img_loss=0.007894, reg_loss=0.014658, lr=0.00387, time=407.0 ms, rem=5.70 m
iter= 2170, img_loss=0.008405, reg_loss=0.014824, lr=0.00385, time=407.1 ms, rem=5.63 m
iter= 2180, img_loss=0.011127, reg_loss=0.015509, lr=0.00384, time=407.0 ms, rem=5.56 m
iter= 2190, img_loss=0.010164, reg_loss=0.015025, lr=0.00382, time=407.0 ms, rem=5.49 m
iter= 2200, img_loss=0.008510, reg_loss=0.014700, lr=0.00380, time=407.0 ms, rem=5.43 m
iter= 2210, img_loss=0.011219, reg_loss=0.015170, lr=0.00378, time=407.0 ms, rem=5.36 m
iter= 2220, img_loss=0.009995, reg_loss=0.015571, lr=0.00377, time=407.1 ms, rem=5.29 m
iter= 2230, img_loss=0.007612, reg_loss=0.014045, lr=0.00375, time=407.1 ms, rem=5.22 m
iter= 2240, img_loss=0.010648, reg_loss=0.015685, lr=0.00373, time=406.9 ms, rem=5.15 m
iter= 2250, img_loss=0.011509, reg_loss=0.015775, lr=0.00371, time=407.1 ms, rem=5.09 m
iter= 2260, img_loss=0.008520, reg_loss=0.014986, lr=0.00370, time=407.1 ms, rem=5.02 m
iter= 2270, img_loss=0.010904, reg_loss=0.015603, lr=0.00368, time=407.0 ms, rem=4.95 m
iter= 2280, img_loss=0.011532, reg_loss=0.015314, lr=0.00366, time=407.1 ms, rem=4.88 m
iter= 2290, img_loss=0.008373, reg_loss=0.015001, lr=0.00365, time=407.0 ms, rem=4.82 m
iter= 2300, img_loss=0.007704, reg_loss=0.014555, lr=0.00363, time=407.1 ms, rem=4.75 m
iter= 2310, img_loss=0.010582, reg_loss=0.015983, lr=0.00361, time=406.9 ms, rem=4.68 m
iter= 2320, img_loss=0.009782, reg_loss=0.015556, lr=0.00360, time=407.0 ms, rem=4.61 m
iter= 2330, img_loss=0.006865, reg_loss=0.014423, lr=0.00358, time=407.0 ms, rem=4.55 m
iter= 2340, img_loss=0.009543, reg_loss=0.015297, lr=0.00356, time=407.0 ms, rem=4.48 m
iter= 2350, img_loss=0.010120, reg_loss=0.015032, lr=0.00355, time=406.9 ms, rem=4.41 m
iter= 2360, img_loss=0.009759, reg_loss=0.015206, lr=0.00353, time=407.1 ms, rem=4.34 m
iter= 2370, img_loss=0.008980, reg_loss=0.014785, lr=0.00351, time=407.0 ms, rem=4.27 m
iter= 2380, img_loss=0.009721, reg_loss=0.015075, lr=0.00350, time=407.0 ms, rem=4.21 m
iter= 2390, img_loss=0.008101, reg_loss=0.014872, lr=0.00348, time=407.0 ms, rem=4.14 m
iter= 2400, img_loss=0.009240, reg_loss=0.014684, lr=0.00347, time=407.1 ms, rem=4.07 m
iter= 2410, img_loss=0.010046, reg_loss=0.015063, lr=0.00345, time=407.0 ms, rem=4.00 m
iter= 2420, img_loss=0.009277, reg_loss=0.015181, lr=0.00343, time=407.0 ms, rem=3.93 m
iter= 2430, img_loss=0.011673, reg_loss=0.015576, lr=0.00342, time=407.0 ms, rem=3.87 m
iter= 2440, img_loss=0.009068, reg_loss=0.015679, lr=0.00340, time=407.0 ms, rem=3.80 m
iter= 2450, img_loss=0.008261, reg_loss=0.014832, lr=0.00339, time=406.9 ms, rem=3.73 m
iter= 2460, img_loss=0.009171, reg_loss=0.015058, lr=0.00337, time=407.0 ms, rem=3.66 m
iter= 2470, img_loss=0.008114, reg_loss=0.014505, lr=0.00336, time=407.1 ms, rem=3.60 m
iter= 2480, img_loss=0.009978, reg_loss=0.014931, lr=0.00334, time=407.0 ms, rem=3.53 m
iter= 2490, img_loss=0.009212, reg_loss=0.015097, lr=0.00333, time=407.0 ms, rem=3.46 m
iter= 2500, img_loss=0.010375, reg_loss=0.015736, lr=0.00331, time=407.0 ms, rem=3.39 m
iter= 2510, img_loss=0.012177, reg_loss=0.016041, lr=0.00329, time=407.0 ms, rem=3.32 m
iter= 2520, img_loss=0.009096, reg_loss=0.014903, lr=0.00328, time=407.0 ms, rem=3.26 m
iter= 2530, img_loss=0.012927, reg_loss=0.015745, lr=0.00326, time=406.9 ms, rem=3.19 m
iter= 2540, img_loss=0.012855, reg_loss=0.015738, lr=0.00325, time=406.9 ms, rem=3.12 m
iter= 2550, img_loss=0.009594, reg_loss=0.015253, lr=0.00323, time=407.0 ms, rem=3.05 m
iter= 2560, img_loss=0.007538, reg_loss=0.014566, lr=0.00322, time=407.1 ms, rem=2.99 m
iter= 2570, img_loss=0.008662, reg_loss=0.014931, lr=0.00320, time=407.0 ms, rem=2.92 m
iter= 2580, img_loss=0.010376, reg_loss=0.015117, lr=0.00319, time=407.1 ms, rem=2.85 m
iter= 2590, img_loss=0.009226, reg_loss=0.015505, lr=0.00318, time=406.9 ms, rem=2.78 m
iter= 2600, img_loss=0.009376, reg_loss=0.015107, lr=0.00316, time=407.1 ms, rem=2.71 m
iter= 2610, img_loss=0.010097, reg_loss=0.015165, lr=0.00315, time=407.1 ms, rem=2.65 m
iter= 2620, img_loss=0.007083, reg_loss=0.014524, lr=0.00313, time=407.2 ms, rem=2.58 m
iter= 2630, img_loss=0.010213, reg_loss=0.015209, lr=0.00312, time=406.9 ms, rem=2.51 m
iter= 2640, img_loss=0.009429, reg_loss=0.015250, lr=0.00310, time=406.9 ms, rem=2.44 m
iter= 2650, img_loss=0.009624, reg_loss=0.015169, lr=0.00309, time=407.1 ms, rem=2.37 m
iter= 2660, img_loss=0.006756, reg_loss=0.014507, lr=0.00307, time=407.0 ms, rem=2.31 m
iter= 2670, img_loss=0.009333, reg_loss=0.014813, lr=0.00306, time=407.1 ms, rem=2.24 m
iter= 2680, img_loss=0.007750, reg_loss=0.014311, lr=0.00305, time=407.0 ms, rem=2.17 m
iter= 2690, img_loss=0.010960, reg_loss=0.016005, lr=0.00303, time=406.9 ms, rem=2.10 m
iter= 2700, img_loss=0.006356, reg_loss=0.013870, lr=0.00302, time=407.2 ms, rem=2.04 m
iter= 2710, img_loss=0.011482, reg_loss=0.015845, lr=0.00300, time=407.0 ms, rem=1.97 m
iter= 2720, img_loss=0.010263, reg_loss=0.014706, lr=0.00299, time=407.1 ms, rem=1.90 m
iter= 2730, img_loss=0.010096, reg_loss=0.015252, lr=0.00298, time=407.0 ms, rem=1.83 m
iter= 2740, img_loss=0.012526, reg_loss=0.015981, lr=0.00296, time=407.0 ms, rem=1.76 m
iter= 2750, img_loss=0.008390, reg_loss=0.014653, lr=0.00295, time=407.0 ms, rem=1.70 m
iter= 2760, img_loss=0.010298, reg_loss=0.015127, lr=0.00294, time=407.0 ms, rem=1.63 m
iter= 2770, img_loss=0.010271, reg_loss=0.015239, lr=0.00292, time=407.0 ms, rem=1.56 m
iter= 2780, img_loss=0.009349, reg_loss=0.014926, lr=0.00291, time=407.0 ms, rem=1.49 m
iter= 2790, img_loss=0.008722, reg_loss=0.014790, lr=0.00290, time=407.0 ms, rem=1.42 m
iter= 2800, img_loss=0.011304, reg_loss=0.015881, lr=0.00288, time=407.0 ms, rem=1.36 m
iter= 2810, img_loss=0.010129, reg_loss=0.015735, lr=0.00287, time=407.0 ms, rem=1.29 m
iter= 2820, img_loss=0.008980, reg_loss=0.015045, lr=0.00286, time=407.0 ms, rem=1.22 m
iter= 2830, img_loss=0.009518, reg_loss=0.015004, lr=0.00284, time=406.9 ms, rem=1.15 m
iter= 2840, img_loss=0.009727, reg_loss=0.015398, lr=0.00283, time=407.0 ms, rem=1.09 m
iter= 2850, img_loss=0.011379, reg_loss=0.015749, lr=0.00282, time=406.9 ms, rem=1.02 m
iter= 2860, img_loss=0.008908, reg_loss=0.015067, lr=0.00280, time=407.1 ms, rem=56.99 s
iter= 2870, img_loss=0.008287, reg_loss=0.014838, lr=0.00279, time=407.0 ms, rem=52.91 s
iter= 2880, img_loss=0.009854, reg_loss=0.015301, lr=0.00278, time=407.0 ms, rem=48.84 s
iter= 2890, img_loss=0.008868, reg_loss=0.015062, lr=0.00277, time=407.0 ms, rem=44.77 s
iter= 2900, img_loss=0.009114, reg_loss=0.014884, lr=0.00275, time=407.1 ms, rem=40.71 s
iter= 2910, img_loss=0.009355, reg_loss=0.014764, lr=0.00274, time=407.0 ms, rem=36.63 s
iter= 2920, img_loss=0.008417, reg_loss=0.015064, lr=0.00273, time=407.0 ms, rem=32.56 s
iter= 2930, img_loss=0.008553, reg_loss=0.015007, lr=0.00272, time=407.0 ms, rem=28.49 s
iter= 2940, img_loss=0.010586, reg_loss=0.014963, lr=0.00270, time=406.9 ms, rem=24.42 s
iter= 2950, img_loss=0.010213, reg_loss=0.015117, lr=0.00269, time=407.0 ms, rem=20.35 s
iter= 2960, img_loss=0.007876, reg_loss=0.014600, lr=0.00268, time=407.1 ms, rem=16.28 s
iter= 2970, img_loss=0.009897, reg_loss=0.015207, lr=0.00267, time=406.9 ms, rem=12.21 s
iter= 2980, img_loss=0.009387, reg_loss=0.014941, lr=0.00265, time=407.0 ms, rem=8.14 s
iter= 2990, img_loss=0.009824, reg_loss=0.015365, lr=0.00264, time=406.9 ms, rem=4.07 s
Running validation
MSE,      PSNR
0.00084232, 30.788
Writing mesh:  out/bob/mesh/mesh.obj
    writing 5308 vertices
    writing 7932 texcoords
    writing 5308 normals
    writing 10612 faces
Writing material:  out/bob/mesh/mesh.mtl
Done exporting mesh
OptiXStateWrapper destructor 
OptiXStateWrapper destructor 
OptiXStateWrapper destructor

Is it because I did something wrong？

jmunkberg commented 1 year ago

The log looks reasonable, but I'm getting higher PSNR values

MSE,      PSNR
0.00041472, 33.860

In what folder are you looking at the textures? I'm a bit surprised the textures are not exported correctly if you have a PSNR of 30 dB.

What Cuda version are you using? We require Cuda 11.3 or newer.

yezifeiafei commented 1 year ago

The texturs showed above were found in folder out/bob/mesh/. I use docker/make_image.sh to setup my environment. The cuda version of my host machine is 11.7.

jmunkberg commented 1 year ago

Sorry, I cannot reproduce your issue here. We have tested on both Windows and Linux.

Two additional questions.

  1. What do the images from the optimization look like? For example, below is img_mesh_pass_000029.png

  2. Do you get good textures after the first pass (check the dmtet_mesh folder)?

img_mesh_pass_000029

jmunkberg commented 1 year ago

Perhaps there is some permission rights problem on that folder, and you have some old, stale, data that we cannot overwrite? Perhaps delete the out/bob/ folder and rerun the optimization.

yezifeiafei commented 1 year ago
  1. This is my img_mesh_pass_000029.png img_mesh_pass_000029 It doesn't seem right.
  2. No. These are my textures in out/bob/dmtet_mesh/ image
jmunkberg commented 1 year ago

It looks like there is an issue with NVIDIA OptiX on your machine. Shading is all black, so I suspect this call failed: https://github.com/NVlabs/nvdiffrecmc/blob/main/render/render.py#L113

You can try replacing the line at: https://github.com/NVlabs/nvdiffrecmc/blob/main/render/render.py#L113 e.g., diffuse_accum, specular_accum = ou.optix_env_shade(...) with diffuse_accum, specular_accum = (gb_normal + 1.0)*0.5, torch.zeros_like(gb_normal)

And see if you get any colors out.

jmunkberg commented 1 year ago

You can also try our previous reconstruction paper, which does not require OptiX https://github.com/NVlabs/nvdiffrec . There is a bob example there as well.

yezifeiafei commented 1 year ago

I have try the https://github.com/NVlabs/nvdiffrec and the result seems right. If I replace the line as you mentioned above, does that mean the nvdiffrecmc I use will degenerate into nvdiffrec?

jmunkberg commented 1 year ago

No.

yezifeiafei commented 1 year ago

Thanks for your reply! Can I take the code replacement as the final solution, and ignore the issue about my OptiX? Are there any side effects?

jmunkberg commented 1 year ago

No, that replacement was just a quick debugging test to see if OptiX was the issue on your machine. The replacement line simply replaces the real shading with a simple shading based on the surface normal.

The best solution would be to verify that OptiX works fine on your machine, as that is crucial for our approach (we use OptiX to trace shadow rays). If that doesn't work, then you can perhaps use our previous code: https://github.com/NVlabs/nvdiffrec

MartinSmeyer commented 1 year ago

Had the same problem on a 1080Ti but it worked on a RTX 3090 with OptiX support using the same packages.