search

XinWu98 / SC-wLS

MIT License
12 stars 1 forks source link

Different sesults on 7scenes dataset, chess #2

Open MisEty opened 1 year ago

MisEty commented 1 year ago

I want to reproduce the results and test the released code with the following command: python test_e2e_DLT.py <dataset path> --dataset 7Scenes --scene chess --config_file configs/single_frame.ini --model DenseSC_GNN --pretrained-model checkpoints/7scenes_unsup_model/chess/model.pkl The code run successfuly, and the output is:

val loader: 2000 median t,r error for DLT: 0.021533024 0.6407075225990593 ptc5 for DLT: 0.886

I noticed that the result in the paper is "Ours (dlt) 0.029/0.78". The results are differen. Can you show the right command of the paper's results?

Update: I also test another command and it seems that the results of end-to-end model is OK: python test_e2e_DLT.py /home/pengzhexi/camera_relocalization/dataset/7scenes --dataset 7Scenes --scene pumpkin -b 1 --config_file configs/single_frame.ini --model DenseSC_GNN --pretrained-model checkpoints/7scenes_e2e_model/pumpkin/model.pkl The outputs are equal to paper

Update 4.18: I try to run SC-wLS on my dataset and I have processed my data in 7Scene format. I want to use the 7Scene dataloader but the code need coords_mean.txt file. Can you tell me the influence if I use zero instead, and how can I compute this file?

Update 4.19: I try to use 7Scene dataloader to run my data and I have done the following steps: first, resize my rgb images to 640*480; second, change the camera K to my resized camera ([[373.3, 0, 308.67], [0, 497.7, 249.2], [0, 0, 1]]); third, compute the mean of gt camera pose translation as scene_mean. I only use 50 reference images as a simple test but the result is not good. Here scene coordinate initialization network best output: t_median q_median lr 3.5635636 1.4805614300851522 0.0001 I also test 50 reference in 7scene chess dataset: t_median q_median lr 0.05904792 0.815205435638383 0.0001 Are there other important parameters that I should change?

Sincerely, thank you.

muzi-Lu commented 1 year ago

你好，我想问一下我在执行setup.py这个文件时出现 `FAILED: /home/benben/code/SC-wLS/dsacstar/build/temp.linux-x86_64-cpython-38/dsacstar.o c++ -MMD -MF /home/benben/code/SC-wLS/dsacstar/build/temp.linux-x86_64-cpython-38/dsacstar.o.d -pthread -B /home/benben/anaconda3/envs/SCwLS/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/opencv3.4/include -I/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/torch/include -I/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/torch/include/TH -I/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/torch/include/THC -I/home/benben/anaconda3/envs/SCwLS/include/python3.8 -c -c /home/benben/code/SC-wLS/dsacstar/dsacstar.cpp -o /home/benben/code/SC-wLS/dsacstar/build/temp.linux-x86_64-cpython-38/dsacstar.o -fopenmp -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=dsacstar -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14 cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ In file included from /home/benben/code/SC-wLS/dsacstar/dsacstar_util.h:37:0, from /home/benben/code/SC-wLS/dsacstar/dsacstar.cpp:42: /home/benben/code/SC-wLS/dsacstar/dsacstar_util_rgbd.h: In function ‘void matMulDerivWrapper(cv::InputArray, cv::InputArray, cv::OutputArray, cv::OutputArray)’: /home/benben/code/SC-wLS/dsacstar/dsacstar_util_rgbd.h:222:22: error: cannot convert ‘cv::Mat’ to ‘int’ for argument ‘1’ to ‘CvMat cvMat(int, int, int, void*)’ CvMat matA = cvMat(A), matB = cvMat(B), c_dABdA = cvMat(_dABdA.getMat()), c_dABdB = cvMat(_dABdB.getMat()); ^ /home/benben/code/SC-wLS/dsacstar/dsacstar_util_rgbd.h:223:28: error: ‘matB’ was not declared in this scope cvCalcMatMulDeriv(&matA, &matB, _dABdA.needed() ? &c_dABdA : 0, _dABdB.needed() ? &c_dABdB : 0); ^~~~ /home/benben/code/SC-wLS/dsacstar/dsacstar_util_rgbd.h:223:28: note: suggested alternative: ‘matA’ cvCalcMatMulDeriv(&matA, &matB, _dABdA.needed() ? &c_dABdA : 0, _dABdB.needed() ? &c_dABdB : 0); ^~~~ matA /home/benben/code/SC-wLS/dsacstar/dsacstar_util_rgbd.h:223:53: error: ‘c_dABdA’ was not declared in this scope cvCalcMatMulDeriv(&matA, &matB, _dABdA.needed() ? &c_dABdA : 0, _dABdB.needed() ? &c_dABdB : 0); ^~~ /home/benben/code/SC-wLS/dsacstar/dsacstar_util_rgbd.h:223:53: note: suggested alternative: ‘_dABdA’ cvCalcMatMulDeriv(&matA, &matB, _dABdA.needed() ? &c_dABdA : 0, _dABdB.needed() ? &c_dABdB : 0); ^~~ _dABdA /home/benben/code/SC-wLS/dsacstar/dsacstar_util_rgbd.h:223:85: error: ‘c_dABdB’ was not declared in this scope cvCalcMatMulDeriv(&matA, &matB, _dABdA.needed() ? &c_dABdA : 0, _dABdB.needed() ? &c_dABdB : 0); ^~~ /home/benben/code/SC-wLS/dsacstar/dsacstar_util_rgbd.h:223:85: note: suggested alternative: ‘_dABdB’ cvCalcMatMulDeriv(&matA, &matB, _dABdA.needed() ? &c_dABdA : 0, _dABdB.needed() ? &c_dABdB : 0); ^~~ _dABdB In file included from /home/benben/code/SC-wLS/dsacstar/dsacstar.cpp:42:0: /home/benben/code/SC-wLS/dsacstar/dsacstarutil.h: In function ‘std::vector<cv::Point > dsacstar::createSampling_seq(dsacstar::p_ts&, int)’: /home/benben/code/SC-wLS/dsacstar/dsacstar_util.h:94:25: warning: comparison between signed and unsigned integer expressions [-Wsign-compare] for(unsigned x = 0; x < N; x++) ^ /home/benben/code/SC-wLS/dsacstar/dsacstar_util.h: In function ‘std::vector dsacstar::getHypScores_seq(const std::vector<std::vector<float, std::allocator > >&, float, float)’: /home/benben/code/SC-wLS/dsacstar/dsacstar_util.h:711:20: warning: comparison between signed and unsigned integer expressions [-Wsign-compare] for(int x = 0; x < reproErrs[h].size(); x++) ^~~~~~~ /home/benben/code/SC-wLS/dsacstar/dsacstar_util.h: In function ‘std::vector<float, std::allocator > dsacstar::getReproErrs_seq(dsacstar::coord_ts&, const poset&, const std::vector<cv::Point >&, const cv::Mat&, float, cv::Mat_&, bool)’: /home/benben/code/SC-wLS/dsacstar/dsacstar_util.h:862:20: warning: comparison between signed and unsigned integer expressions [-Wsign-compare] for(int x = 0; x < sampling.size(); x++) ^~~~~~~ /home/benben/code/SC-wLS/dsacstar/dsacstar_util.h: In function ‘void dsacstar::refineHyp_seq(dsacstar::coordts&, const std::vector<float, std::allocator >&, const std::vector<cv::Point >&, const cv::Mat_&, float, unsigned int, float, dsacstar::pose_t&, std::vector<int, std::allocator >&)’: /home/benben/code/SC-wLS/dsacstar/dsacstar_util.h:1125:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare] for(int x = 0; x < sampling.size(); x++) ^~~~~~~ /home/benben/code/SC-wLS/dsacstar/dsacstar.cpp: In function ‘void dsacstar_rgb_forward(at::Tensor, at::Tensor, int, float, float, float, float, float, float, int)’: /home/benben/code/SC-wLS/dsacstar/dsacstar.cpp:302:9: warning: unused variable ‘hypEntropy’ [-Wunused-variable] double hypEntropy = dsacstar::entropy(hypProbs); // measure distribution entropy ^~~~~~ /home/benben/code/SC-wLS/dsacstar/dsacstar.cpp: In function ‘double dsacstar_rgb_backward(at::Tensor, at::Tensor, at::Tensor, int, float, float, float, float, float, float, float, float, float, int, int)’: /home/benben/code/SC-wLS/dsacstar/dsacstar.cpp:456:9: warning: unused variable ‘hypEntropy’ [-Wunused-variable] double hypEntropy = dsacstar::entropy(hypProbs); // measure distribution entropy ^~~~~~ ninja: build stopped: subcommand failed. Traceback (most recent call last): File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1666, in _run_ninja_build subprocess.run( File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/subprocess.py", line 512, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "setup.py", line 36, in setup( File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/init.py", line 87, in setup return distutils.core.setup(**attrs) File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup return run_commands(dist) File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands dist.run_commands() File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands self.run_command(cmd) File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/dist.py", line 1208, in run_command super().run_command(command) File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/command/install.py", line 74, in run self.do_egg_install() File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/command/install.py", line 123, in do_egg_install self.run_command('bdist_egg') File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command self.distribution.run_command(command) File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/dist.py", line 1208, in run_command super().run_command(command) File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/command/bdist_egg.py", line 165, in run cmd = self.call_command('install_lib', warn_dir=0) File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/command/bdist_egg.py", line 151, in call_command self.run_command(cmdname) File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command self.distribution.run_command(command) File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/dist.py", line 1208, in run_command super().run_command(command) File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/command/install_lib.py", line 11, in run self.build() File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/_distutils/command/install_lib.py", line 112, in build self.run_command('build_ext') File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command self.distribution.run_command(command) File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/dist.py", line 1208, in run_command super().run_command(command) File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 84, in run _build_ext.run(self) File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 346, in run self.build_extensions() File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 709, in build_extensions build_ext.build_extensions(self) File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 468, in build_extensions self._build_extensions_serial() File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 494, in _build_extensions_serial self.build_extension(ext) File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 246, in build_extension _build_ext.build_extension(self, ext) File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 549, in build_extension objects = self.compiler.compile( File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 530, in unix_wrap_ninja_compile _write_ninja_file_and_compile_objects( File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1355, in _write_ninja_file_and_compile_objects _run_ninja_build( File "/home/benben/anaconda3/envs/SCwLS/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1682, in _run_ninja_build raise RuntimeError(message) from e RuntimeError: Error compiling objects for extension` 请问一下之前你在之前复现的过程中遇到这个问题了吗？ 请问一下是我的配置问题吗？ 打扰了，感谢！

MisEty commented 1 year ago

应该是opencv版本的问题，按照readme所说，您应该先安装opencv=3.4.15版本并且在setup.py中设置对应的路径，再进行编译

XinWu98 commented 1 year ago

I want to reproduce the results and test the released code with the following command: python test_e2e_DLT.py <dataset path> --dataset 7Scenes --scene chess --config_file configs/single_frame.ini --model DenseSC_GNN --pretrained-model checkpoints/7scenes_unsup_model/chess/model.pkl The code run successfuly, and the output is:

val loader: 2000 median t,r error for DLT: 0.021533024 0.6407075225990593 ptc5 for DLT: 0.886

I noticed that the result in the paper is "Ours (dlt) 0.029/0.78". The results are differen. Can you show the right command of the paper's results?

Update: I also test another command and it seems that the results of end-to-end model is OK: python test_e2e_DLT.py /home/pengzhexi/camera_relocalization/dataset/7scenes --dataset 7Scenes --scene pumpkin -b 1 --config_file configs/single_frame.ini --model DenseSC_GNN --pretrained-model checkpoints/7scenes_e2e_model/pumpkin/model.pkl The outputs are equal to paper

Update 4.18: I try to run SC-wLS on my dataset and I have processed my data in 7Scene format. I want to use the 7Scene dataloader but the code need coords_mean.txt file. Can you tell me the influence if I use zero instead, and how can I compute this file?

Update 4.19: I try to use 7Scene dataloader to run my data and I have done the following steps: first, resize my rgb images to 640*480; second, change the camera K to my resized camera ([[373.3, 0, 308.67], [0, 497.7, 249.2], [0, 0, 1]]); third, compute the mean of gt camera pose translation as scene_mean. I only use 50 reference images as a simple test but the result is not good. Here scene coordinate initialization network best output: t_median q_median lr 3.5635636 1.4805614300851522 0.0001 I also test 50 reference in 7scene chess dataset: t_median q_median lr 0.05904792 0.815205435638383 0.0001 Are there other important parameters that I should change?

Sincerely, thank you.

Hi, thanks for your attention! To be clear, I would answer the questions in order:

  1. The commands for test We provide two types of models for --pretrained-model here . One is *_e2e_model, which is the model for Ours (dlt+e2e) (The command you need). The other is *_unsup_model for Ours (dlt+e2e+self), which is finetuned using test set (after self-supervised test-time adaptation).

  2. coords_mean.txt We follow Dsac* to use the mean value of scene coordinates for training scene coordinate regression network. It's used here, and then the network could learn the residual of scene coordinate instead. The motivation of using the mean value is illustrated in this issue.

To calculate the mean coordinate value of your scenes, you could refer to these code.

  1. For your own datasets, make sure the camera K is right for your resized images. Besides, could you give more details of your datasets? Such as whether they are indoor scenes or outdoor, and the size of training set and test test. You could also debug following these steps:

Firstly, you could change the pose solver to RANSAC to see whether the problem happens on the training step of scene coordinate network initialization. If the views are too sparse, or the scene is too chanllenge, the training of scene coordinate network could fail.

Secondly, if the RANSAC results are good, you could turn to train our OANet-like weight network for each scene, and calculate dlt results finally.

MisEty commented 1 year ago

Thanks for your reply. The test demo of my dataset is an outdoor scene of 50 images for reference and 1 image for query. The images are captured from continue scan so the views are quite close and the pose is computed by BundleFusion. I will debug my code as you suggest and update the results!

Update 4.30 I have checked my camera K, it is right. But the results of Scene coordinate initialization are still wrong. I run the training process by command: python train_init.py ./datasets/OurScan --dataset OurScan --scene demo -b1 --epochs 150 --config_file configs/single_frame.ini --lr 1e-4 --cal_training_pose --model DenseSC The best outputs are: t_median q_median lr 5.743898 2.7037772337783856 0.0001 Then, I train the weight network with --useRANSAC, the best outputs are: t_median q_median t_med_ransac q_med_ransac lr 0.7095688 83.66632369089574 7.291292 1.9391857944064714 0.0001 I noticed that the avg loss of Scene coordinate initialization seems OK(my data: 19.232, chess: 16.431) but the output poses are wrong. The poses are computed by compute_pose_lm_pnp so can you tell me the possible reason

Update 5.1: I find that the DenseSC network use SCNet/DROID in 7 scene/Other scene. Previously I used DROID and the predicted 3d points got a lot of deviation. I have changed it to SCNet and the results are much better. Can you tell me the difference between the two network?

Update 5.3: SC-wLS works well when I only use 500 images for reference. However, although I have changed the model to SCNet, SC-wLS still cannot work on my full scene which is much bigger than 7scenes dataset (7000 reference images of indoor scene, about 100m2). I wonder that can SC-wLS handle such a big indoor scene?