search

ToniRV / Kimera-VIO-Evaluation

Code to evaluate and tune SPARK VIO pipeline.
MIT License
21 stars 2 forks source link

main_evaluation.py run error #41

Open zhangli3000 opened 4 years ago

zhangli3000 commented 4 years ago

Description: main_evaluation.py run error about vio_performance_template.html

Command:

./evaluation/main_evaluation.py -r -a --plot --verbose_sparkvio --save_plots --save_boxplots --save_results ../main_ev.yaml

Console output:


Traceback (most recent call last):                                                                    
  File "./evaluation/main_evaluation.py", line 72, in <module>                                        
    if run(args):                                                                                     
  File "./evaluation/main_evaluation.py", line 21, in run                                             
    dataset_evaluator = DatasetEvaluator(experiment_params, args, extra_flagfile_path)                
  File "/usr/local/lib/python2.7/dist-packages/evaluation/evaluation_lib.py", line 291, in __init__   
    self.website_builder = evt.WebsiteBuilder(self.results_dir)                                       
  File "/usr/local/lib/python2.7/dist-packages/evaluation/tools/website_utils.py", line 27, in __init_
_                                                                                                     
    self.boxplot_template = self.env.get_template('vio_performance_template.html')                    
  File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line 883, in get_template      
    return self._load_template(name, self.make_globals(globals))                                      
  File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line 857, in _load_template    
    template = self.loader.load(self, name, globals)                                                  
  File "/usr/local/lib/python2.7/dist-packages/jinja2/loaders.py", line 115, in load                  
    source, filename, uptodate = self.get_source(environment, name)                                   
  File "/usr/local/lib/python2.7/dist-packages/jinja2/loaders.py", line 249, in get_source            
    raise TemplateNotFound(template)                                                                  
jinja2.exceptions.TemplateNotFound: vio_performance_template.html

Additional files: Please attach all the files needed to reproduce the error.

Please give also the following information:

executable_path: '../Kimera-VIO/buil/stereoVIOEuroc'
vocabulary_path: '../Kimera-VIO/vocabulary/ORBvoc.yml'
results_dir: 'Kimera-VIO-Evaluation/reuslts'
params_dir: '../Kimera-VIO/params/Euroc'
dataset_dir: '../Kimera-VIO/script/euroc'

datasets_to_run:

rginjapan commented 3 years ago

same error

amihakko commented 3 years ago

Description: main_evaluation.py run error about vio_performance_template.html

Command:

./evaluation/main_evaluation.py -r -a --plot --verbose_sparkvio --save_plots --save_boxplots --save_results ../main_ev.yaml

Console output:



Traceback (most recent call last):                                                                    
  File "./evaluation/main_evaluation.py", line 72, in <module>                                        
    if run(args):                                                                                     
.....

I was able to resolve the issue. The fact is that in addition to the "website" package located in the: Kimera-VIO-Evaluations/website there is another website package located in the: $Home/.local/lib/pythonX.X/site-packages/website , where X.X - your version Python. If you copy the "templates" folder there, the problem will be solved. Alternatively, you can tell Python to use the package that is in the Kimera-VIO-Evaluations/website folder first.

geothan commented 3 years ago

I have encountered the same problem and i tried to move templates folder to python directory, but the evaluation tool doesn't work. Have you solved the problem?

My yaml file is:

executable_path: '/home/george/catkin_ws/build/kimera_vio/stereoVIOEuroc' vocabulary_path: '/home/george/catkin_ws/src/Kimera-VIO/vocabulary/ORBvoc.yml' params_dir: '/home/george/catkin_ws/src/Kimera-VIO/params' dataset_dir: '/home/george/datasets/EuRoC/ASL_Dataset_format' results_dir: '/home/george/spark_vio_evaluation/results' datasets_to_run: name: V1_01_easy use_lcd: true plot_vio_and_pgo: true segments: [1] pipelines: ['S'] discard_n_start_poses: 0 discard_n_end_poses: 0 initial_frame: 10 final_frame: 220

the output of code execution before movement of folder templates:

(venv) george@pc:~/spark_vio_evaluation$ ./evaluation/main_evaluation.py -r -a --save_plots --save_results --save_boxplots experiments/george_example_euroc.yaml Traceback (most recent call last): File "./evaluation/main_evaluation.py", line 76, in if run(args): File "./evaluation/main_evaluation.py", line 25, in run dataset_evaluator = DatasetEvaluator(experiment_params, args, extra_flagfile_path) File "/home/george/venv/local/lib/python2.7/site-packages/evaluation/evaluation_lib.py", line 298, in init self.website_builder = evt.WebsiteBuilder(self.results_dir) File "/home/george/venv/local/lib/python2.7/site-packages/evaluation/tools/website_utils.py", line 29, in init self.boxplot_template = self.env.get_template('vio_performance_template.html') File "/home/george/venv/local/lib/python2.7/site-packages/jinja2/environment.py", line 883, in get_template return self._load_template(name, self.make_globals(globals)) File "/home/george/venv/local/lib/python2.7/site-packages/jinja2/environment.py", line 857, in _load_template template = self.loader.load(self, name, globals) File "/home/george/venv/local/lib/python2.7/site-packages/jinja2/loaders.py", line 115, in load source, filename, uptodate = self.get_source(environment, name) File "/home/george/venv/local/lib/python2.7/site-packages/jinja2/loaders.py", line 249, in get_source raise TemplateNotFound(template) jinja2.exceptions.TemplateNotFound: vio_performance_template.html

and the output of code execution after movement of folder templates:

(venv) george@pc:~/spark_vio_evaluation$ ./evaluation/main_evaluation.py -r -a --save_plots --save_results --save_boxplots experiments/george_example_euroc.yaml 0%| | 0/1 [00:00<?, ?it/s]I1116 22:53:56.135045 5048 evaluation_lib.py:305] Run dataset: V1_01_easy Run pipeline: S ----------/home/george/catkin_ws/src/Kimera-VIO/params/S/flags/stereoVIOEuroc.flags: No such file or directory E1116 22:53:56.236728 5048 evaluation_lib.py:168] Failed pipeline run! Finished evaluation for dataset: V1_01_easy I1116 22:53:56.237183 5048 evaluation_lib.py:308] Dataset: V1_01_easy failed!! 0%| | 0/1 [00:00<?, ?it/s] Traceback (most recent call last): File "./evaluation/main_evaluation.py", line 76, in if run(args): File "./evaluation/main_evaluation.py", line 26, in run dataset_evaluator.evaluate() File "/home/george/venv/local/lib/python2.7/site-packages/evaluation/evaluation_lib.py", line 309, in evaluate raise Exception("Failed to run dataset %s." % dataset['name']) Exception: Failed to run dataset V1_01_easy.

amihakko commented 3 years ago

I have encountered the same problem and i tried to move templates folder to python directory, but the evaluation tool doesn't work. Have you solved the problem?

My yaml file is:

executable_path: '/home/george/catkin_ws/build/kimera_vio/stereoVIOEuroc' vocabulary_path: '/home/george/catkin_ws/src/Kimera-VIO/vocabulary/ORBvoc.yml' params_dir: '/home/george/catkin_ws/src/Kimera-VIO/params' dataset_dir: '/home/george/datasets/EuRoC/ASL_Dataset_format' results_dir: '/home/george/spark_vio_evaluation/results' datasets_to_run: name: V1_01_easy use_lcd: true plot_vio_and_pgo: true segments: [1] pipelines: ['S'] discard_n_start_poses: 0 discard_n_end_poses: 0 initial_frame: 10 final_frame: 220

the output of code execution before movement of folder templates:

(venv) george@pc:~/spark_vio_evaluation$ ./evaluation/main_evaluation.py -r -a --save_plots --save_results --save_boxplots experiments/george_example_euroc.yaml Traceback (most recent call last): File "./evaluation/main_evaluation.py", line 76, in if run(args): File "./evaluation/main_evaluation.py", line 25, in run dataset_evaluator = DatasetEvaluator(experiment_params, args, extra_flagfile_path) File "/home/george/venv/local/lib/python2.7/site-packages/evaluation/evaluation_lib.py", line 298, in init self.website_builder = evt.WebsiteBuilder(self.results_dir) File "/home/george/venv/local/lib/python2.7/site-packages/evaluation/tools/website_utils.py", line 29, in init self.boxplot_template = self.env.get_template('vio_performance_template.html') File "/home/george/venv/local/lib/python2.7/site-packages/jinja2/environment.py", line 883, in get_template return self._load_template(name, self.make_globals(globals)) File "/home/george/venv/local/lib/python2.7/site-packages/jinja2/environment.py", line 857, in _load_template template = self.loader.load(self, name, globals) File "/home/george/venv/local/lib/python2.7/site-packages/jinja2/loaders.py", line 115, in load source, filename, uptodate = self.get_source(environment, name) File "/home/george/venv/local/lib/python2.7/site-packages/jinja2/loaders.py", line 249, in get_source raise TemplateNotFound(template) jinja2.exceptions.TemplateNotFound: vio_performance_template.html

and the output of code execution after movement of folder templates:

(venv) george@pc:~/spark_vio_evaluation$ ./evaluation/main_evaluation.py -r -a --save_plots --save_results --save_boxplots experiments/george_example_euroc.yaml 0%| | 0/1 [00:00<?, ?it/s]I1116 22:53:56.135045 5048 evaluation_lib.py:305] Run dataset: V1_01_easy Run pipeline: S ----------/home/george/catkin_ws/src/Kimera-VIO/params/S/flags/stereoVIOEuroc.flags: No such file or directory E1116 22:53:56.236728 5048 evaluation_lib.py:168] Failed pipeline run! Finished evaluation for dataset: V1_01_easy I1116 22:53:56.237183 5048 evaluation_lib.py:308] Dataset: V1_01_easy failed!! 0%| | 0/1 [00:00<?, ?it/s] Traceback (most recent call last): File "./evaluation/main_evaluation.py", line 76, in if run(args): File "./evaluation/main_evaluation.py", line 26, in run dataset_evaluator.evaluate() File "/home/george/venv/local/lib/python2.7/site-packages/evaluation/evaluation_lib.py", line 309, in evaluate raise Exception("Failed to run dataset %s." % dataset['name']) Exception: Failed to run dataset V1_01_easy.

You have dealt with the previous error, it seems thanks to my solution. And now you have a completely different error. It is connected with the fact that Kimera-Evaluate cannot detect parameter files. You need to rename the parameter "pipelines" in Yaml-file from ['S'] to ['Euroc'].

geothan commented 3 years ago

Thanks @amihakko , it seems to work. However it should have worked with parameters such as S, SP and SRP. Now, my execution stuck in that step (line 580) where there is an exception regarding traj_ref:

(venv) george@pc:~/spark_vio_evaluation$ ./evaluation/main_evaluation.py -r -a --save_plots --save_results --save_boxplots experiments/george_example_euroc.yaml 0%| | 0/1 [00:00<?, ?it/s]I1117 23:13:02.545836 7292 evaluation_lib.py:305] Run dataset: V1_02_medium Run pipeline: Euroc Successful pipeline run. All pipeline runs were successful. Finished evaluation for dataset: V1_02_medium I1117 23:13:35.605365 7292 evaluation_lib.py:326] Evaluate dataset: V1_02_medium Starting analysis of pipeline: Euroc 0%| | 0/1 [00:33<?, ?it/s] Traceback (most recent call last): File "./evaluation/main_evaluation.py", line 76, in if run(args): File "./evaluation/main_evaluation.py", line 26, in run dataset_evaluator.evaluate() File "/home/george/venv/local/lib/python2.7/site-packages/evaluation/evaluation_lib.py", line 313, in evaluate self.evaluate_dataset(dataset) File "/home/george/venv/local/lib/python2.7/site-packages/evaluation/evaluation_lib.py", line 329, in evaluate_dataset if not self.__evaluate_run(pipeline_type, dataset): File "/home/george/venv/local/lib/python2.7/site-packages/evaluation/evaluation_lib.py", line 369, in __evaluate_run dataset_name, discard_n_start_poses, discard_n_end_poses) File "/home/george/venv/local/lib/python2.7/site-packages/evaluation/evaluation_lib.py", line 407, in run_analysis traj_ref, traj_est_vio, traj_est_pgo = self.read_traj_files(traj_ref_path, traj_vio_path, traj_pgo_path) File "/home/george/venv/local/lib/python2.7/site-packages/evaluation/evaluation_lib.py", line 580, in read_traj_files traj_ref = pandas_bridge.df_to_trajectory(pd.read_csv(traj_ref_path, sep=',', index_col=0)) AttributeError: 'module' object has no attribute 'df_to_trajectory'

Did you face that problem? What is the outcome of this execution? What does someone expect to see regarding VIO?

amihakko commented 3 years ago

@geothan , This is the problem of missing df_to_trajectory functions in the EVO library, namely the pandas_bridge.py file. To fix this, you need to remove your EVO library and install the EVO-1 fork, which is listed in the Prerequisites section, instead: evo-1 // Fork from evo

geothan commented 3 years ago

@amihakko thanks a lot, i didn't noticed that! Now, the result files were generated and i moved on with jupyter notebook frontend. In Frontend Mono RANSAC, i get the below error, have you met this problem?

i followed up the below installation steps, including your suggested move of website folder:

sudo apt-get install virtualenv virtualenv -p python2.7 ./venv source ./venv/bin/activate

git clone https://github.com/ToniRV/Kimera-VIO-Evaluation cd Kimera-VIO-Evaluation pip install . python setup.py develop

git clone https://github.com/ToniRV/evo-1.git cd evo-1 pip install . --upgrade --no-binary evo cd ..

Also, the path which i defined in Jupyter are: vio_output_dir = "/home/george/Kimera-VIO-Evaluation/results/V1_01_easy/Euroc" gt_data_file = "/home/george/datasets/EuRoC/ASL_Dataset_format/V1_01_easy/mav0/state_groundtruth_estimate0/data.csv" left_cam_calibration_file = "/home/george/datasets/EuRoC/ASL_Dataset_format/V1_01_easy/mav0/cam0/sensor.yaml"

KeyError Traceback (most recent call last)

in () 1 # Generate some trajectories for later plots 2 # Convert to evo trajectory objects ----> 3 traj_ref_unassociated = pandas_bridge.df_to_trajectory(gt_df) 4 5 # Use the mono ransac file as estimated trajectory. /home/george/venv/local/lib/python2.7/site-packages/evo/tools/pandas_bridge.pyc in df_to_trajectory(df) 50 if not isinstance(df, pd.DataFrame): 51 raise TypeError("pandas.DataFrame or derived required") ---> 52 positions_xyz = df.loc[:,['x','y','z']].to_numpy() 53 quaternions_wxyz = df.loc[:,['qw','qx','qy','qz']].to_numpy() 54 # NOTE: df must have timestamps as index /home/george/venv/local/lib/python2.7/site-packages/pandas/core/indexing.pyc in __getitem__(self, key) 1492 except (KeyError, IndexError, AttributeError): 1493 pass -> 1494 return self._getitem_tuple(key) 1495 else: 1496 # we by definition only have the 0th axis /home/george/venv/local/lib/python2.7/site-packages/pandas/core/indexing.pyc in _getitem_tuple(self, tup) 886 continue 887 --> 888 retval = getattr(retval, self.name)._getitem_axis(key, axis=i) 889 890 return retval /home/george/venv/local/lib/python2.7/site-packages/pandas/core/indexing.pyc in _getitem_axis(self, key, axis) 1900 raise ValueError('Cannot index with multidimensional key') 1901 -> 1902 return self._getitem_iterable(key, axis=axis) 1903 1904 # nested tuple slicing /home/george/venv/local/lib/python2.7/site-packages/pandas/core/indexing.pyc in _getitem_iterable(self, key, axis) 1203 # A collection of keys 1204 keyarr, indexer = self._get_listlike_indexer(key, axis, -> 1205 raise_missing=False) 1206 return self.obj._reindex_with_indexers({axis: [keyarr, indexer]}, 1207 copy=True, allow_dups=True) /home/george/venv/local/lib/python2.7/site-packages/pandas/core/indexing.pyc in _get_listlike_indexer(self, key, axis, raise_missing) 1159 self._validate_read_indexer(keyarr, indexer, 1160 o._get_axis_number(axis), -> 1161 raise_missing=raise_missing) 1162 return keyarr, indexer 1163 /home/george/venv/local/lib/python2.7/site-packages/pandas/core/indexing.pyc in _validate_read_indexer(self, key, indexer, axis, raise_missing) 1244 raise KeyError( 1245 u"None of [{key}] are in the [{axis}]".format( -> 1246 key=key, axis=self.obj._get_axis_name(axis))) 1247 1248 # We (temporarily) allow for some missing keys with .loc, except in KeyError: u"None of [Index([u'x', u'y', u'z'], dtype='object')] are in the [columns]"
amihakko commented 3 years ago

Glad I was able to help you with previous problems. But I haven't worked with Jupyter notebook, so I can't help here.