christsa / dgrasp

Official code release for CVPR 2022 paper D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactions
Other
80 stars 13 forks source link

Having trouble when 'python setup.py develop' #9

Closed xyaochen closed 1 year ago

xyaochen commented 1 year ago

I encountered something wrong when setup DGRASP environment. When I use "python setup.py develop" in raisimGymTorch " /home/xuanyao/anaconda3/envs/grasp/lib/python3.8/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools. warnings.warn( /home/xuanyao/anaconda3/envs/grasp/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. warnings.warn( -- pybind11 v2.9.0 -- raisim: -- Version: 1.1.6 -- Includes: /home/xuanyao/raisim/dgrasp/raisim/linux/include -- Libraries: raisim::raisimPng;raisim::raisimODE;raisim::raisimMine;raisim::raisim raisim os linux raisim arch raisim os linux raisim arch -- Configuring done -- Generating done -- Build files have been written to: /home/xuanyao/raisim/dgrasp/raisimGymTorch/build/temp.linux-x86_64-cpython-38 Consolidate compiler generated dependencies of target dgrasp Consolidate compiler generated dependencies of target dgrasp_test [ 33%] Building CXX object CMakeFiles/dgrasp.dir/raisimGymTorch/env/raisim_gym.cpp.o [ 33%] Building CXX object CMakeFiles/dgrasp_test.dir/raisimGymTorch/env/raisim_gym.cpp.o In file included from /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/LU:29, from /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/Dense:2, from /home/xuanyao/raisim/dgrasp/raisim/linux/include/raisim/math/Core.hpp:12, from /home/xuanyao/raisim/dgrasp/raisim/linux/include/raisim/math.hpp:17, from /home/xuanyao/raisim/dgrasp/raisim/linux/include/raisim/RaiSimTinyXmlWrapper.hpp:14, from /home/xuanyao/raisim/dgrasp/raisim/linux/include/raisim/Materials.hpp:13, from /home/xuanyao/raisim/dgrasp/raisim/linux/include/raisim/contact/BisectionContactSolver.hpp:15, from /home/xuanyao/raisim/dgrasp/raisim/linux/include/raisim/object/Object.hpp:12, from /home/xuanyao/raisim/dgrasp/raisim/linux/include/raisim/World.hpp:13, from /home/xuanyao/raisim/dgrasp/raisimGymTorch/raisimGymTorch/env/envs/dgrasp_test/../../RaisimGymEnv.hpp:13, from /home/xuanyao/raisim/dgrasp/raisimGymTorch/raisimGymTorch/env/envs/dgrasp_test/Environment.hpp:10, from /home/xuanyao/raisim/dgrasp/raisimGymTorch/raisimGymTorch/env/raisim_gym.cpp:9: /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/LU/PartialPivLU.h: In static member function ‘static Eigen::Index Eigen::internal::partial_lu_impl<Scalar, StorageOrder, PivIndex, SizeAtCompileTime>::unblocked_lu(MatrixTypeRef&, PivIndex, PivIndex&)’: /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/LU/PartialPivLU.h:390:24: error: ‘fix’ was not declared in this scope 390 | lu.col(k).tail(fix(rrows)) /= lu.coeff(k,k); | ^~~ /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/LU/PartialPivLU.h:400:30: error: ‘fix’ was not declared in this scope 400 | lu.bottomRightCorner(fix(rrows),fix(rcols)).noalias() -= lu.col(k).tail(fix(rrows)) lu.row(k).tail(fix(rcols)); | ^~~ In file included from /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/Householder:24, from /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/QR:15, from /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/Dense:4: /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/Householder/HouseholderSequence.h: In member function ‘typename Eigen::internal::matrix_type_times_scalar_type<typename Eigen::internal::traits<Eigen::HouseholderSequence<VectorsType, CoeffsType, Side> >::Scalar, OtherDerived>::Type Eigen::HouseholderSequence<VectorsType, CoeffsType, Side>::operator(const Eigen::MatrixBase&) const’: /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/Householder/HouseholderSequence.h:427:41: error: ‘is_identity’ is not a member of ‘Eigen::internal’ 427 | applyThisOnTheLeft(res, internal::is_identity::value && res.rows()==res.cols()); | ^~~ /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/Householder/HouseholderSequence.h:427:65: error: expected primary-expression before ‘>’ token 427 | applyThisOnTheLeft(res, internal::is_identity::value && res.rows()==res.cols()); | ^ /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/Householder/HouseholderSequence.h:427:68: error: ‘::value’ has not been declared 427 | applyThisOnTheLeft(res, internal::is_identity::value && res.rows()==res.cols()); | ^~~~~ In file included from /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/LU:29, from /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/Dense:2, from /home/xuanyao/raisim/dgrasp/raisim/linux/include/raisim/math/Core.hpp:12, from /home/xuanyao/raisim/dgrasp/raisim/linux/include/raisim/math.hpp:17, from /home/xuanyao/raisim/dgrasp/raisim/linux/include/raisim/RaiSimTinyXmlWrapper.hpp:14, from /home/xuanyao/raisim/dgrasp/raisim/linux/include/raisim/Materials.hpp:13, from /home/xuanyao/raisim/dgrasp/raisim/linux/include/raisim/contact/BisectionContactSolver.hpp:15, from /home/xuanyao/raisim/dgrasp/raisim/linux/include/raisim/object/Object.hpp:12, from /home/xuanyao/raisim/dgrasp/raisim/linux/include/raisim/World.hpp:13, from /home/xuanyao/raisim/dgrasp/raisimGymTorch/raisimGymTorch/env/envs/dgrasp/../../RaisimGymEnv.hpp:13, from /home/xuanyao/raisim/dgrasp/raisimGymTorch/raisimGymTorch/env/envs/dgrasp/Environment.hpp:10, from /home/xuanyao/raisim/dgrasp/raisimGymTorch/raisimGymTorch/env/raisim_gym.cpp:9: /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/LU/PartialPivLU.h: In static member function ‘static Eigen::Index Eigen::internal::partial_lu_impl<Scalar, StorageOrder, PivIndex, SizeAtCompileTime>::unblocked_lu(MatrixTypeRef&, PivIndex, PivIndex&)’: /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/LU/PartialPivLU.h:390:24: error: ‘fix’ was not declared in this scope 390 | lu.col(k).tail(fix(rrows)) /= lu.coeff(k,k); | ^~~ In file included from /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/QR:37: /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/QR/CompleteOrthogonalDecomposition.h: At global scope: /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/QR/CompleteOrthogonalDecomposition.h:58:27: error: ‘solve_assertion’ in namespace ‘Eigen::internal’ does not name a type 58 | friend struct internal::solve_assertion; | ^~~~~~~ /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/LU/PartialPivLU.h:400:30: error: ‘fix’ was not declared in this scope 400 | lu.bottomRightCorner(fix(rrows),fix(rcols)).noalias() -= lu.col(k).tail(fix(rrows)) lu.row(k).tail(fix(rcols)); | ^~~ In file included from /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/SVD:36, from /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/Dense:5: /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/SVD/SVDBase.h:22:42: error: ‘SVDBase’ was not declared in this scope 22 | template struct traits<SVDBase > | ^~~ /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/SVD/SVDBase.h:22:57: error: template argument 1 is invalid 22 | template struct traits<SVDBase > | ^ /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/SVD/SVDBase.h:22:59: error: expected unqualified-id before ‘>’ token 22 | template struct traits<SVDBase > | ^ /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/SVD/SVDBase.h:65:27: error: ‘solve_assertion’ in namespace ‘Eigen::internal’ does not name a type 65 | friend struct internal::solve_assertion; | ^~~~~~~ In file included from /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/Householder:24, from /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/QR:15, from /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/Dense:4: /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/Householder/HouseholderSequence.h: In member function ‘typename Eigen::internal::matrix_type_times_scalar_type<typename Eigen::internal::traits<Eigen::HouseholderSequence<VectorsType, CoeffsType, Side> >::Scalar, OtherDerived>::Type Eigen::HouseholderSequence<VectorsType, CoeffsType, Side>::operator(const Eigen::MatrixBase&) const’: /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/Householder/HouseholderSequence.h:427:41: error: ‘is_identity’ is not a member of ‘Eigen::internal’ 427 | applyThisOnTheLeft(res, internal::is_identity::value && res.rows()==res.cols()); | ^~~ /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/Householder/HouseholderSequence.h:427:65: error: expected primary-expression before ‘>’ token 427 | applyThisOnTheLeft(res, internal::is_identity::value && res.rows()==res.cols()); | ^ /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/Householder/HouseholderSequence.h:427:68: error: ‘::value’ has not been declared 427 | applyThisOnTheLeft(res, internal::is_identity::value && res.rows()==res.cols()); | ^~~~~ In file included from /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/QR:37: /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/QR/CompleteOrthogonalDecomposition.h: At global scope: /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/QR/CompleteOrthogonalDecomposition.h:58:27: error: ‘solve_assertion’ in namespace ‘Eigen::internal’ does not name a type 58 | friend struct internal::solve_assertion; | ^~~~~~~ In file included from /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/SVD:36, from /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/Dense:5: /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/SVD/SVDBase.h:22:42: error: ‘SVDBase’ was not declared in this scope 22 | template struct traits<SVDBase > | ^~~ /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/SVD/SVDBase.h:22:57: error: template argument 1 is invalid 22 | template struct traits<SVDBase > | ^ /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/SVD/SVDBase.h:22:59: error: expected unqualified-id before ‘>’ token 22 | template struct traits<SVDBase > | ^ /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/SVD/SVDBase.h:65:27: error: ‘solve_assertion’ in namespace ‘Eigen::internal’ does not name a type 65 | friend struct internal::solve_assertion; | ^~~~~~~ In file included from /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/Geometry:44, from /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/Dense:6: /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/Geometry/Transform.h: In member function ‘Eigen::Transform<Scalar, Dim, Mode, Options>& Eigen::Transform<Scalar, Dim, Mode, _Options>::pretranslate(const Eigen::MatrixBase&)’: /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/Geometry/Transform.h:920:6: error: there are no arguments to ‘EIGEN_CONST_CONDITIONAL’ that depend on a template parameter, so a declaration of ‘EIGEN_CONST_CONDITIONAL’ must be available [-fpermissive] 920 | if(EIGEN_CONST_CONDITIONAL(int(Mode)==int(Projective))) | ^~~~~~~ /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/Geometry/Transform.h:920:6: note: (if you use ‘-fpermissive’, G++ will accep your code, but allowing the use of an undeclared name is deprecated) In file included from /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/Geometry:44, from /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/Dense:6: /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/Geometry/Transform.h: In member function ‘Eigen::Transform<Scalar, Dim, Mode, Options>& Eigen::Transform<Scalar, Dim, Mode, _Options>::pretranslate(const Eigen::MatrixBase&)’: /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/Geometry/Transform.h:920:6: error: there are no arguments to ‘EIGEN_CONST_CONDITIONAL’ that depend on a template parameter, so a declaration of ‘EIGEN_CONST_CONDITIONAL’ must be available [-fpermissive] 920 | if(EIGEN_CONST_CONDITIONAL(int(Mode)==int(Projective))) | ^~~~~~~ /home/xuanyao/raisim/dgrasp/thirdParty/Eigen3/include/eigen3/Eigen/src/Geometry/Transform.h:920:6: note: (if you use ‘-fpermissive’, G++ will accept your code, but allowing the use of an undeclared name is deprecated) make[2]: [CMakeFiles/dgrasp_test.dir/build.make:76: CMakeFiles/dgrasp_test.dir/raisimGymTorch/env/raisim_gym.cpp.o] Error 1 make[1]: [CMakeFiles/Makefile2:102: CMakeFiles/dgrasp_test.dir/all] Error 2 make[1]: Waiting for unfinished jobs.... make[2]: [CMakeFiles/dgrasp.dir/build.make:76: CMakeFiles/dgrasp.dir/raisimGymTorch/env/raisim_gym.cpp.o] Error 1 make[1]: [CMakeFiles/Makefile2:128: CMakeFiles/dgrasp.dir/all] Error 2 make: [Makefile:91: all] Error 2 Traceback (most recent call last): File "setup.py", line 68, in setup( File "/home/xuanyao/anaconda3/envs/grasp/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup return run_commands(dist) File "/home/xuanyao/anaconda3/envs/grasp/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands dist.run_commands() File "/home/xuanyao/anaconda3/envs/grasp/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands self.run_command(cmd) File "/home/xuanyao/anaconda3/envs/grasp/lib/python3.8/site-packages/setuptools/dist.py", line 1208, in run_command super().run_command(command) File "/home/xuanyao/anaconda3/envs/grasp/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/home/xuanyao/anaconda3/envs/grasp/lib/python3.8/site-packages/setuptools/command/develop.py", line 34, in run self.install_for_development() File "/home/xuanyao/anaconda3/envs/grasp/lib/python3.8/site-packages/setuptools/command/develop.py", line 114, in install_for_development self.run_command('build_ext') File "/home/xuanyao/anaconda3/envs/grasp/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command self.distribution.run_command(command) File "/home/xuanyao/anaconda3/envs/grasp/lib/python3.8/site-packages/setuptools/dist.py", line 1208, in run_command super().run_command(command) File "/home/xuanyao/anaconda3/envs/grasp/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "setup.py", line 38, in run self.build_extension(ext) File "setup.py", line 66, in build_extension subprocess.check_call(['cmake', '--build', '.'] + build_args, cwd=self.build_temp) File "/home/xuanyao/anaconda3/envs/grasp/lib/python3.8/subprocess.py", line 364, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j2']' returned non-zero exit status 2. "

I've been struggling with this issue for a few days now and I would really appreciate it if someone could take a look and offer some guidance on how to resolve it. Thank you in advance!

handover-sim2real commented 1 year ago

Hi,

I have a few questions to better help you debug this:

  1. Can you let me know your system setup?
  2. Do you have the libeigen package installed? sudo apt install libeigen3-dev
  3. Did you previously have dgrasp installed before I switched everything to the new RaiSim version (1.1.6)?

Thanks, Sammy

xyaochen commented 1 year ago

It seems that I have successfully set up environments.

python setup.py develop

_/home/xuanyao/anaconda3/envs/grasp/lib/python3.8/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools. warnings.warn( /home/xuanyao/anaconda3/envs/grasp/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. warnings.warn( -- pybind11 v2.9.0 -- raisim: -- Version: 1.1.6 -- Includes: /home/xuanyao/raisim/dgrasp/raisim/linux/include -- Libraries: raisim::raisimPng;raisim::raisimODE;raisim::raisimMine;raisim::raisim raisim os linux raisim arch raisim os linux raisim arch -- Configuring done (0.3s) -- Generating done (0.1s) -- Build files have been written to: /home/xuanyao/raisim/dgrasp/raisimGymTorch/build/temp.linux-x86_64-cpython-38 [100%] Built target dgrasp [100%] Built target dgrasp_test

However, when I try to evaluate:

python raisimGymTorch/env/envs/dgrasp_test/runner.py -o 12 -e '021_bleach_dexycb' -sd 'pretrained_policies' -w 'full_3000.pt' 

It shows that: Traceback (most recent call last): File "raisimGymTorch/env/envs/dgrasp_test/runner.py", line 2, in from raisimGymTorch.env.bin import dgrasp_test as mano ImportError: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /home/xuanyao/raisim/dgrasp/raisimGymTorch/raisimGymTorch/env/bin/dgrasp_test.cpython-38-x86_64-linux-gnu.so)

Do I have something wrong about my set up or compliration?

xyaochen commented 1 year ago

the libeigen is installed in /usr/include/eigen3, so I change the #include<Eigen/Core> to #include<eigens/Eigen/Core> in some files. I tried install dgrasp in recent days, so I don't install the previous version.

the libeigen is installed in /usr/include/eigen3, so I change the #include<Eigen/Core> to #include<eigens/Eigen/Core> in some files. I tried install dgrasp in recent days, so I don't install the previous version.

christsa commented 1 year ago

Hm, I haven't seen this issue before. Seems like some issues with libstdc++6 Could you try the following solution (from https://github.com/lhelontra/tensorflow-on-arm/issues/13):

sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt update
sudo apt install gcc-4.9
sudo apt install --only-upgrade libstdc++6
xyaochen commented 1 year ago

Thanks! I will try it. Thanks again for your prompt response!

xyaochen commented 1 year ago

Thanks for your help! I have set up successfully. When I evaluate the example

python raisimGymTorch/env/envs/dgrasp_test/runner.py -o 12 -e '021_bleach_dexycb' -sd 'pretrained_policies' -w 'full_3000.pt'

I get the results:


object: 12 success: 0.944 disp mean: 0.422 disp std: 0.684

It seems a little degeneration compared to the results reported. Is it normal?

By the way, I tried to visualize the results. I think I have installed the prerequisite packages successfully. But, after I run the following:

python raisimGymTorch/env/envs/dgrasp_test/runner.py -o 12 -e '021_bleach_dexycb' -sd 'pretrained_policies' -w 'full_3000.pt' -ev

I get:


Traceback (most recent call last): File "raisimGymTorch/env/envs/dgrasp_test/runner.py", line 256, in env.reset_state(qpos_reset_seq, np.zeros((num_envs,51)), obj_pose_reset_seq) File "/home/xuanyao/raisim/dgrasp/raisimGymTorch/raisimGymTorch/env/RaisimGymVecEnv.py", line 84, in reset_state self.wrapper.reset_state(init_state, init_vel, obj_pose) TypeError: reset_state(): incompatible function arguments. The following argument types are supported: 1. (self: raisimGymTorch.env.bin.dgrasp_test.RaisimGymEnv, arg0: numpy.ndarray[numpy.float32[m, n], flags.writeable, flags.c_contiguous], arg1: numpy.ndarray[numpy.float32[m, n], flags.writeable, flags.c_contiguous], arg2: numpy.ndarray[numpy.float32[m, n], flags.writeable, flags.c_contiguous]) -> None

I will appreciate it if you could have a look. And what is the meaning of 'run raisimUnity executable in raisimUnity//RaiSimUnity', could you provide the corresponding command?

christsa commented 1 year ago

The slight discrepancy between the results seems normal. It can be due to slight changes in the new version of RaiSim or the different hardware used to run the evaluation.

Well, you need to go to the raisimUnity folder that corresponds to your OS and run the exectuable file for Unity, e.g., this file for Linux. To run an executable, you just double click on the file ;-) On Linux, you may need to first change the permissions via chmod to allow for execution.

Let me know if it works out.

xyaochen commented 1 year ago

Thanks for your guidance!

I think I have executed the Unity. But after I run the command

python raisimGymTorch/env/envs/dgrasp/runner_motion.py -o 12 -e '021_bleach_dexycb' -w 'full_3000.pt' -sd -'pretrained_policies' -sv

It is always pending and has endless output 'stored video' on the screen. But I didn't find no video stored. I notice that "this does not work on headless servers". However, I exactly use the remote servers to run the code. Is that possible to store the video on server?

christsa commented 1 year ago

Glad you got it to work.

Yes, this is expected. Last time I checked RaiSim did not offer headless rendering. For visualization, you need to run it on a local machine. This is however related to the physics simulation and not to the D-Grasp code. Please consult the RaiSim Github directly (this issue here seems related: https://github.com/raisimTech/raisimLib/issues/10).

xyaochen commented 1 year ago

Thanks! Your help is always nice and prompt. By the way, I want to explore the D-Grasp performance under the dynamic motion synthesis task. I have noticed that the final_obj_pos in one sample has small differences. Did these ground-truth labels sampled from a continuous sequence in DexYCB?

christsa commented 1 year ago

To try out more difficult labels, you can change the following line in runner_motion.py to: eval_dict = joblib.load('raisimGymTorch/data/motion_eval_dict_hard.pkl').

The way I created the labels was to add angular noise to the initial poses of the objects and sample the final position from a pre-defined grid.

xyaochen commented 1 year ago

Thanks!

I'm confused with 16 hand contact points. Are these the same part as 16 hand joints? What's the meaning of "link_palm_rz", "link_ff_pm_z", etc ?

christsa commented 1 year ago

Yes, they are the same parts as the 16 hand joints (without the fingertips).

The names correspond to the collision bodies defined in the mano_mean.urdf file.

The convention I used for naming the finger parts is "pm" primal, "md" medial, and "dd" distal. "link_palm_rz" is the palm collision body.

xyaochen commented 1 year ago

Thanks again! Your reply is always prompt and kind.

I am diving into your code details and I have some questions.

  1. where is the feature extraction layer $\psi(s, T_g, D)$ in motion synthesis? I didn't find it in the model.
  2. the gc variable is the current hand pose state. Is it in global coordinate or origin hand coordinate? Is gc[0:3] the root position and gc_[3:6] the root orientation?
  3. I am a little confused by how the PD controller works. In my understanding, pTarget = actionMean + predictedaction*scaling, where actionMean[6: 52] = gc_[6:52] is the current hand joint angles. In traning of motionsynthesis, actionMean[0:6] = (current relative root 6D pose compared to final pose) + gc[0:6]. Why is actionMean[0:6] just the current root 6D pose in testing as in test code? It seems to have a mismatch between training and testing in motion synthesis.
christsa commented 1 year ago

These are good questions.

  1. You should find it here.
  2. gc_ is the generalized coordinate in the root frame. In this case, it corresponds to the the origin hand coordinate at the beginning of a sequence, as set here and here.
  3. The referenced "testing" corresponds to the main experiment reported in the paper, which is to test the stability of grasps. To do this the global pose of the hand is fixed and it is evaluated whether the object is grasped without dropping. Therefore, the PD target is set as the current pose once it switches from approaching/grasping to testing.
xyaochen commented 1 year ago

I am extremely grateful!

So the motion synthesis module will not be used in testing as the pTarget_[0:6] has been overwritten in here?

christsa commented 1 year ago

Correct, in testing, the motion synthesis is not used.

Motion synthesis is run through the dgrasp folder with the runner_motion.py file.

christsa commented 1 year ago

Closing this issue here to make it easier to distinguish questions by topic. Feel free to open a new issues if anything is unclear :)