Closed chinnusai25 closed 1 year ago
What is your experiment that you are trying to evaluate? the evaluation instructions are not uploaded yet, but I can help you. You should point to a folder that has an experiment inside e.g. a .hydra dir and a checkpoint dir. First, you should run sample_seq
to get the samples and then you should use the evaluate to get the desired numbers
I want to reproduce the APE and AVE of past conditioned (TEACH) using the pretrained model provided
Then download the checkpoints and .hydra directory from the website put the them in a folder and use this folder as argument to the eval script. First, you should use python sample_seq.py folder=/path/to/folder align=full slerp_ws=8
. That is an example for generating the samples using full alignment and slerp=8 frames see the results tables for the other options (i.e. align=trans and slerp_ws=null (aka no slerp)). After you have generated those samples (you will see the folders appear in your experiment folder), you should be able to run eval for the above command the corresponding eval is python eval.py folder=/path/to/folder slerp=true align=true
.
Thanks for the reply!
I had executed sample_seq.py
which resulted a folder /path/to/folder/samples_slerp_aligned_pairs/checkpoint-last/val/
with all sampled npy files. ( On a side note, there is a typo in configs/sampler/all_conseq.yaml:1:target: teachg.data.sampling.FrameSampler, which can be fixed by replacing teach. inplace of teachg. ).
Executing python eval.py folder=/path/to/folder slerp=true align=true
after the sampling raised the below error,
Traceback (most recent call last):
File "/home/miniconda3/envs/teach/lib/python3.9/logging/config.py", line 385, in resolve
found = self.importer(used)
ModuleNotFoundError: No module named 'temos'
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/home/miniconda3/envs/teach/lib/python3.9/logging/config.py", line 552, in configure filters[name] = self.configure_filter(filters[name]) File "/home/miniconda3/envs/teach/lib/python3.9/logging/config.py", line 689, in configure_filter result = self.configure_custom(config) File "/home/miniconda3/envs/teach/lib/python3.9/logging/config.py", line 470, in configure_custom c = self.resolve(c) File "/home/miniconda3/envs/teach/lib/python3.9/logging/config.py", line 398, in resolve raise v File "/home/miniconda3/envs/teach/lib/python3.9/logging/config.py", line 385, in resolve found = self.importer(used) ValueError: Cannot resolve 'temos.tools.logging.LevelsFilter': No module named 'temos'
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/home/miniconda3/envs/teach/lib/python3.9/site-packages/hydra/_internal/utils.py", line 211, in run_and_report return func() File "/home/miniconda3/envs/teach/lib/python3.9/site-packages/hydra/_internal/utils.py", line 378, in lambda: hydra.run( File "/home/miniconda3/envs/teach/lib/python3.9/site-packages/hydra/_internal/hydra.py", line 88, in run cfg = self.compose_config( File "/home/miniconda3/envs/teach/lib/python3.9/site-packages/hydra/_internal/hydra.py", line 566, in compose_config configure_log(cfg.hydra.hydra_logging, cfg.hydra.verbose) File "/home/miniconda3/envs/teach/lib/python3.9/site-packages/hydra/core/utils.py", line 48, in configure_log logging.config.dictConfig(conf) File "/home/miniconda3/envs/teach/lib/python3.9/logging/config.py", line 809, in dictConfig dictConfigClass(config).configure() File "/home/miniconda3/envs/teach/lib/python3.9/logging/config.py", line 554, in configure raise ValueError('Unable to configure ' ValueError: Unable to configure filter 'onlyimportant'
Can you please try again I made a minor commit, which may be relevant and resolve your issue.
Thanks!! This issue is resolved
I downloaded the pretrained teach model provided and tried executing the "python eval.py folder=/path/to/experiment align=true slerp=true", but got this error as below, Traceback (most recent call last): File "/home/miniconda3/envs/teach/lib/python3.9/logging/config.py", line 385, in resolve found = self.importer(used) ModuleNotFoundError: No module named 'temos'
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/home/miniconda3/envs/teach/lib/python3.9/logging/config.py", line 552, in configure filters[name] = self.configure_filter(filters[name]) File "/home/miniconda3/envs/teach/lib/python3.9/logging/config.py", line 689, in configure_filter result = self.configure_custom(config) File "/home/miniconda3/envs/teach/lib/python3.9/logging/config.py", line 470, in configure_custom c = self.resolve(c) File "/home/miniconda3/envs/teach/lib/python3.9/logging/config.py", line 398, in resolve raise v File "/home/miniconda3/envs/teach/lib/python3.9/logging/config.py", line 385, in resolve found = self.importer(used) ValueError: Cannot resolve 'temos.tools.logging.LevelsFilter': No module named 'temos'
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/home/miniconda3/envs/teach/lib/python3.9/site-packages/hydra/_internal/utils.py", line 211, in run_and_report return func() File "/home/miniconda3/envs/teach/lib/python3.9/site-packages/hydra/_internal/utils.py", line 378, in
lambda: hydra.run(
File "/home/miniconda3/envs/teach/lib/python3.9/site-packages/hydra/_internal/hydra.py", line 88, in run
cfg = self.compose_config(
File "/home/miniconda3/envs/teach/lib/python3.9/site-packages/hydra/_internal/hydra.py", line 566, in compose_config
configure_log(cfg.hydra.hydra_logging, cfg.hydra.verbose)
File "/home/miniconda3/envs/teach/lib/python3.9/site-packages/hydra/core/utils.py", line 48, in configure_log
logging.config.dictConfig(conf)
File "/home/miniconda3/envs/teach/lib/python3.9/logging/config.py", line 809, in dictConfig
dictConfigClass(config).configure()
File "/home/miniconda3/envs/teach/lib/python3.9/logging/config.py", line 554, in configure
raise ValueError('Unable to configure '
ValueError: Unable to configure filter 'onlyimportant'