Hi,
This issue is quite common during model training. It may manifest at different times, sometimes only a few hours into the program, and other times not until sevaral hours have passed.
The detailed error log is below.
[17230] File "/opt/conda/envs/suds_my/lib/python3.8/site-packages/nerfstudio/utils/decorators.py", line 82, in wrapper
[17231] ret = func(*args, **kwargs)
[17232] File "/opt/conda/envs/suds_my/lib/python3.8/site-packages/nerfstudio/utils/writer.py", line 160, in write_out_storage
[17233] writer.write_stats_log(EVENT_STORAGE[0]["step"])
[17234] File "/opt/conda/envs/suds_my/lib/python3.8/site-packages/nerfstudio/utils/decorators.py", line 82, in wrapper
[17235] ret = func(*args, **kwargs)
[17236] File "/opt/conda/envs/suds_my/lib/python3.8/site-packages/nerfstudio/utils/writer.py", line 395, in write_stats_log
[17237] self._print_stats(latest_map)
[17238] File "/opt/conda/envs/suds_my/lib/python3.8/site-packages/nerfstudio/utils/decorators.py", line 82, in wrapper
[17239] ret = func(*args, **kwargs)
[17240] File "/opt/conda/envs/suds_my/lib/python3.8/site-packages/nerfstudio/utils/writer.py", line 452, in _print_stats
[17241] v = human_format(v)
[17242] File "/opt/conda/envs/suds_my/lib/python3.8/site-packages/nerfstudio/utils/printing.py", line 44, in human_format
[17243] magnitude = int(floor(log(num, k)))
[17244] ValueError: math domain error
Hi, This issue is quite common during model training. It may manifest at different times, sometimes only a few hours into the program, and other times not until sevaral hours have passed. The detailed error log is below.