pyparsing.exceptions.ParseException: Expected '}', found '='

[M3LLI55X]

A weird problem I met when I was trying to do geometry reconstruction, but I have never found anything about '}' and '='

Traceback (most recent call last):
  File "/dtu/blackhole/11/180913/material/vqnfr_pro_release/geo/NeuS-ours2/dtu_runner.py", line 424, in <module>
    args.conf = conf_dict[args.case]
                ~~~~~~~~~^^^^^^^^^^^
KeyError: 'dtu_split2'
(xd) /dtu/blackhole/11/180913/material/vqnfr_pro_release/geo/NeuS-ours2
n-62-11-47(s222398) $ python dtu_runner.py --case dtu_scan24
Hello Wooden
/dtu/blackhole/11/180913/X-Dreamer/xd/lib/python3.11/site-packages/torch/__init__.py:696: UserWarning: torch.set_default_tensor_type() is deprecated as of PyTorch 2.1, please use torch.set_default_dtype() and torch.set_default_device() as alternatives. (Triggered internally at ../torch/csrc/tensor/python_tensor.cpp:451.)
  _C._set_default_tensor_type(t)
pyparsing.exceptions.ParseException: Expected '}', found '='  (at char 892), (line:55, col:14)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/dtu/blackhole/11/180913/material/vqnfr_pro_release/geo/NeuS-ours2/dtu_runner.py", line 427, in <module>
    runner = Runner(args.conf, args.mode, args.case, args.is_continue)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dtu/blackhole/11/180913/material/vqnfr_pro_release/geo/NeuS-ours2/dtu_runner.py", line 32, in __init__
    self.conf = ConfigFactory.parse_string(conf_text)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dtu/blackhole/11/180913/X-Dreamer/xd/lib/python3.11/site-packages/pyhocon/config_parser.py", line 192, in parse_string
    return ConfigParser().parse(content, basedir, resolve, unresolved_value)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dtu/blackhole/11/180913/X-Dreamer/xd/lib/python3.11/site-packages/pyhocon/config_parser.py", line 455, in parse
    config = config_expr.parseString(content, parseAll=True)[0]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dtu/blackhole/11/180913/X-Dreamer/xd/lib/python3.11/site-packages/pyparsing/util.py", line 256, in _inner
    return fn(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dtu/blackhole/11/180913/X-Dreamer/xd/lib/python3.11/site-packages/pyparsing/core.py", line 1200, in parse_string
    raise exc.with_traceback(None)
pyparsing.exceptions.ParseSyntaxException: Expected '}', found '='  (at char 892), (line:55, col:14)

[JiuTongBro]

Hi. Thanks for your interest in our work!

First, we found that we made a typo in our code releasement. The running configs for dtu scenes shall be confs/dtu.conf, but not confs/nerf.conf which is currently set in models/helpers.py. We have corrected it, please update the file models/helpers.py from the current-version code repo.

Before: https://github.com/JiuTongBro/vqnerf_release/blob/faa0945e0427206c406ad6bd043ed4a64309cd67/geo/NeuS-ours2/models/helpers.py#L17-L19

Now (Corrected): https://github.com/JiuTongBro/vqnerf_release/blob/a2425af8535a3c37e63a1217259bddfaa4b436c0/geo/NeuS-ours2/models/helpers.py#L17-L19

We apologize for our careless.😀

As for your reported bug, we tested our code, but didn't find this error. We wonder, have you modified the code or the config file? There are two suspecssions we assume may cause the bug:

1. Traceback (most recent call last):
   File "/dtu/blackhole/11/180913/material/vqnfr_pro_release/geo/NeuS-ours2/dtu_runner.py", line 424, in <module>
   args.conf = conf_dict[args.case]
               ~~~~~~~~~^^^^^^^^^^^
   KeyError: 'dtu_split2'

   According to your log, the args.case in line 424 of dtu_runner.py is dtu_split2. But it shall be (and in our test it is) dtu_scan24. The dtu_split2 is the parent directory/data root of the dtu_scan24 in data path, i.e. <your_data_root>/dtu_split2/dtu_scan24. It seems your code mistaken the name of the data root as the scene/case name. Please check this part of the code.

2. pyparsing.exceptions.ParseException: Expected '}', found '='  (at char 892), (line:55, col:14)

   We assume it is caused by the unclosed parentheses (inconsistency between { and }) in the config file, i.e. .../confs/xxx.conf. The parameters in the configs file is formated like json. Please check the config format.

Please contact us if there is any other problem.

[M3LLI55X]

Hi. Thanks for your interest in our work!

First, we found that we made a typo in our code releasement. The running configs for dtu scenes shall be confs/dtu.conf, but not confs/nerf.conf which is currently set in models/helpers.py. We have corrected it, please update the file models/helpers.py from the current-version code repo.

Before:

https://github.com/JiuTongBro/vqnerf_release/blob/faa0945e0427206c406ad6bd043ed4a64309cd67/geo/NeuS-ours2/models/helpers.py#L17-L19

Now (Corrected):

https://github.com/JiuTongBro/vqnerf_release/blob/a2425af8535a3c37e63a1217259bddfaa4b436c0/geo/NeuS-ours2/models/helpers.py#L17-L19

We apologize for our careless.😀

As for your reported bug, we tested our code, but didn't find this error. We wonder, have you modified the code or the config file? There are two suspecssions we assume may cause the bug:

Traceback (most recent call last):
  File "/dtu/blackhole/11/180913/material/vqnfr_pro_release/geo/NeuS-ours2/dtu_runner.py", line 424, in <module>
    args.conf = conf_dict[args.case]
                ~~~~~~~~~^^^^^^^^^^^
KeyError: 'dtu_split2'

According to your log, the args.case in line 424 of dtu_runner.py is dtu_split2. But it shall be (and in our test it is) dtu_scan24. The dtu_split2 is the parent directory/data root of the dtu_scan24 in data path, i.e. <your_data_root>/dtu_split2/dtu_scan24. It seems your code mistaken the name of the data root as the scene/case name. Please check this part of the code.

pyparsing.exceptions.ParseException: Expected '}', found '='  (at char 892), (line:55, col:14)

We assume it is caused by the unclosed parentheses (inconsistency between { and }) in the config file, i.e. .../confs/xxx.conf. The parameters in the configs file is formated like json. Please check the config format.

Please contact us if there is any other problem.

Thanks for your reply. I solved the problem except for the problem about } and = , I try to run python dtu_runner.py --case dtu_scan24 but still got that error report. I checked the dtu.conf and I am sure there is nothing wrong with the { and } , so weird.😢

[JiuTongBro]

Sorry. Could you please provide me the content of your ./confs/nerf.conf😀

I am not able to reproduce this bug in my env. I cloned this repo to my local env and ran the code smoothly:

(/home/zhonghongliang/vqnfr_pro_release/geo/geo_env) zhonghongliang@ss420d:~/vqnfr_pro_release/geo/NeuS-ours2$ python dtu_runner.py --case dtu_scan24
Hello Wooden
Load data: Begin
hw:  512 682
near:  1.7560128151672183   far:  3.9673853830902273   radius:  1.0
Load data: End
  0%|                                                                          | 15/100000 [00:01<2:33:47, 10.84it/s]  0%|                                                                          | 15/100000 [00:01<2:41:46, 10.30it/s]

Please also pay attention to the comma in: https://github.com/JiuTongBro/vqnerf_release/blob/a2425af8535a3c37e63a1217259bddfaa4b436c0/geo/NeuS-ours2/confs/dtu.conf#L36-L47

You can also refer the the official NeuS repo to check their config format. Our config is in the same format as theirs.

[JiuTongBro]

Sorry. I mean ./confs/dtu.conf.

[M3LLI55X]

general {
    base_exp_dir = ./exp/CASE_NAME/dtu
    scene_out_dir = ./surf/dtu_surf/CASE_NAME
    recording = [
        ./,
        ./models
    ]
}

dataset {
    data_dir = ./data/dtu_split2/CASE_NAME/
    new_h = 512
}

train {
    learning_rate = 5e-4
    learning_rate_alpha = 0.05
    end_iter = 100000
    lr_end_iter = 300000

    batch_size = 512
    validate_resolution_level = 1
    warm_up_end = 5000
    anneal_end = 0
    use_white_bkgd = False

    save_freq = 10000
    val_freq = 2500
    val_mesh_freq = 5000
    report_freq = 100

    igr_weight = 0.1
    mask_weight = 0.1
}

model {
    nerf {
        D = 8,
        d_in = 4,
        d_in_view = 3,
        W = 256,
        multires = 10,
        multires_view = 4,
        output_ch = 4,
        skips=[4],
        use_viewdirs=True
    }

    sdf_network {
        d_out = 257
        d_in = 3
        d_hidden = 256
        n_layers = 8
        skip_in = [4]
        multires = 6
        bias = 0.5
        scale = 1.0
        geometric_init = True
        weight_norm = True
    }

    variance_network {
        init_val = 0.3
    }

    rendering_network {
        d_feature = 256
        mode = idr
        d_in = 9
        d_out = 3
        d_hidden = 256
        n_layers = 4
        weight_norm = True
        multires_view = 4
        squeeze_out = True
    }

    neus_renderer {
        n_samples = 64
        n_importance = 64
        n_outside = 0
        up_sample_steps = 4     # 1 for simple coarse-to-fine sampling
        perturb = 1.0
    }
}

Actually, I didn't edit anything in dtu.conf

[JiuTongBro]

Wow! I got it! This is caused by the wrong pyhocon version. Please install pyhocon==0.3.57： pip install pyhocon==0.3.57

We have updated the instructions in README.

Please contact us if there is any other problems.

[JiuTongBro]

Here we provide a full list of our installed env for the geo stage:

Package                 Version
----------------------- -----------
absl-py                 1.4.0
asttokens               2.4.1
cachetools              4.2.4
certifi                 2021.5.30
charset-normalizer      2.0.12
colorama                0.4.5
cycler                  0.11.0
dataclasses             0.8
decorator               4.4.2
executing               2.0.1
google-auth             2.22.0
google-auth-oauthlib    0.4.6
grpcio                  1.48.2
icecream                2.1.0
idna                    3.7
imageio                 2.15.0
importlib-metadata      4.8.3
kiwisolver              1.3.1
Markdown                3.3.7
matplotlib              3.3.4
networkx                2.5.1
numpy                   1.19.2
oauthlib                3.2.2
opencv-python           4.5.2.52
Pillow                  8.4.0
pip                     21.2.2
protobuf                3.19.6
pyasn1                  0.5.1
pyasn1-modules          0.3.0
Pygments                2.14.0
pyhocon                 0.3.57
PyMCubes                0.1.2
pyparsing               3.1.2
python-dateutil         2.9.0.post0
PyWavelets              1.1.1
requests                2.27.1
requests-oauthlib       2.0.0
rsa                     4.9
scikit-image            0.17.2
scipy                   1.5.4
setuptools              58.0.4
six                     1.16.0
tensorboard             2.10.1
tensorboard-data-server 0.6.1
tensorboard-plugin-wit  1.8.1
tifffile                2020.9.3
torch                   1.8.0+cu111
torchaudio              0.8.0
torchvision             0.9.0+cu111
tqdm                    4.50.2
trimesh                 3.9.8
typing_extensions       4.1.1
urllib3                 1.26.18
Werkzeug                2.0.3
wheel                   0.37.1
zipp                    3.6.0

[M3LLI55X]

Bravo! I changed from pyhocon 0.3.60 to pyhocon-0.3.57, it works. Thank you!