search

YNCris / SBeA_release

The release version of Social Behavior Atlas (SBeA)
GNU General Public License v3.0
35 stars 1 forks source link

calibration data: does it need to be generated independently? + installation #5

Closed delaroob closed 4 months ago

delaroob commented 4 months ago

Hi!

In the tracking manual (https://github.com/YNCris/SBeA_release/blob/main/README_SBeA_tracker.md#2-set-configurations), it mentions a calibration file ("caliParas.mat: the calibration file"). Is this generated automatically somehow throughout the process, or should I generate it separately (with some other software probably using a charuco board or something like this)?

YNCris commented 4 months ago

I recommend you to refer the issue#2. The format of the calibration file is based on the 3D-AI behavior atlas system. You can find some details on https://www.behavioratlas.cn/.

delaroob commented 4 months ago

Yes, I saw the behaviouratlas.cn link, but most of the page isnt’t in English, so it was hard to navigate, and the institution definitely does not have this device. What would you suggest that I do regarding the calibration?

HYN (Yaning Han) @.***> (időpont: 2024. ápr. 26., P, 15:48) ezt írta:

I recommend you to refer the issue#2. The format of the calibration file is based on the 3D-AI behavior atlas system. You can find some details on https://www.behavioratlas.cn/.

— Reply to this email directly, view it on GitHub https://github.com/YNCris/SBeA_release/issues/5#issuecomment-2079434272, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZZIMFQ4FVAXW3KTZRIFO2TY7JLLZAVCNFSM6AAAAABG2SHUMSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANZZGQZTIMRXGI . You are receiving this because you authored the thread.Message ID: @.***>

YNCris commented 4 months ago

Hello! I recommend you first check the demo data we provided. You can figure out the format of caliParas.mat. Then you can go to check the process of pose3d (https://github.com/SwathiSheshadri/pose3d). The format of the calibration file is similar to pose3d. For camera calibration, you can also refer to the tutorial of camera calibration of matlab (https://ww2.mathworks.cn/help/vision/ref/stereoparameters.html?s_tid=doc_ta).

delaroob commented 4 months ago

Okay, will do. Meanwhile I ran into some issues while installing. Here are the steps I followed:

  1. created a new environment in conda from the yml file conda env create -f environment.yml (and changed the env name to 'sbea')
  2. activated it conda activate sbea
  3. git cloned the repo git clone https://github.com/YNCris/SBeA_release.git
  4. cd to the setup.py file, and then tried installing: python setup.py install

first, I got an error about CUDA_HOME not being specified, so I installed cuda manually from nvidias website https://developer.nvidia.com/cuda-downloads?target_os=Windows&target_arch=x86_64&target_version=11&target_type=exe_local , and set the path manually by running set CUDA_HOME=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4

now, it seems like I get some kind of dependency issue with pytorch:

(sbea) C:\mentes\sbea\SBeA_release\SBeA_tracker\SBeA-Windows-main>python setup.py install
No CUDA runtime is found, using CUDA_HOME='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4'
running install
C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\command\easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
running bdist_egg
running egg_info
writing deform_conv.egg-info\PKG-INFO
writing dependency_links to deform_conv.egg-info\dependency_links.txt
writing top-level names to deform_conv.egg-info\top_level.txt
C:\mentes\conda\envs\sbea\lib\site-packages\torch\utils\cpp_extension.py:476: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
  warnings.warn(msg.format('we could not find ninja.'))
reading manifest file 'deform_conv.egg-info\SOURCES.txt'
writing manifest file 'deform_conv.egg-info\SOURCES.txt'
installing library code to build\bdist.win-amd64\egg
running install_lib
running build_ext
C:\mentes\conda\envs\sbea\lib\site-packages\torch\utils\cpp_extension.py:358: UserWarning: Error checking compiler version for cl: [WinError 2] A rendszer nem találja a megadott fájlt
  warnings.warn(f'Error checking compiler version for {compiler}: {error}')
Traceback (most recent call last):
  File "C:\mentes\sbea\SBeA_release\SBeA_tracker\SBeA-Windows-main\setup.py", line 3, in <module>
    setup(
  File "C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
    return run_commands(dist)
  File "C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
    dist.run_commands()
  File "C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\_distutils\dist.py", line 968, in run_commands
    self.run_command(cmd)
  File "C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\dist.py", line 1217, in run_command
    super().run_command(command)
  File "C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\_distutils\dist.py", line 987, in run_command
    cmd_obj.run()
  File "C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\command\install.py", line 74, in run
    self.do_egg_install()
  File "C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\command\install.py", line 123, in do_egg_install
    self.run_command('bdist_egg')
  File "C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\_distutils\cmd.py", line 319, in run_command
    self.distribution.run_command(command)
  File "C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\dist.py", line 1217, in run_command
    super().run_command(command)
  File "C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\_distutils\dist.py", line 987, in run_command
    cmd_obj.run()
  File "C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\command\bdist_egg.py", line 165, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\command\bdist_egg.py", line 151, in call_command
    self.run_command(cmdname)
  File "C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\_distutils\cmd.py", line 319, in run_command
    self.distribution.run_command(command)
  File "C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\dist.py", line 1217, in run_command
    super().run_command(command)
  File "C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\_distutils\dist.py", line 987, in run_command
    cmd_obj.run()
  File "C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\command\install_lib.py", line 11, in run
    self.build()
  File "C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\_distutils\command\install_lib.py", line 112, in build
    self.run_command('build_ext')
  File "C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\_distutils\cmd.py", line 319, in run_command
    self.distribution.run_command(command)
  File "C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\dist.py", line 1217, in run_command
    super().run_command(command)
  File "C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\_distutils\dist.py", line 987, in run_command
    cmd_obj.run()
  File "C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\command\build_ext.py", line 84, in run
    _build_ext.run(self)
  File "C:\mentes\conda\envs\sbea\lib\site-packages\Cython\Distutils\old_build_ext.py", line 186, in run
    _build_ext.build_ext.run(self)
  File "C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\_distutils\command\build_ext.py", line 346, in run
    self.build_extensions()
  File "C:\mentes\conda\envs\sbea\lib\site-packages\torch\utils\cpp_extension.py", line 499, in build_extensions
    _check_cuda_version(compiler_name, compiler_version)
  File "C:\mentes\conda\envs\sbea\lib\site-packages\torch\utils\cpp_extension.py", line 386, in _check_cuda_version
    raise RuntimeError(CUDA_MISMATCH_MESSAGE.format(cuda_str_version, torch.version.cuda))
RuntimeError:
The detected CUDA version (12.4) mismatches the version that was used to compile
PyTorch (11.7). Please make sure to use the same CUDA versions.

Do I download earlier version of CUDA? Or do I do something with pytorch? Or something with the cl compiler?

Thanks in advance for your help!

YNCris commented 4 months ago

You see the RuntimeError "The detected CUDA version (12.4) mismatches the version that was used to compile". Pytorch need 11.7 CUDA. I think you should use the earlier version of CUDA.

delaroob commented 4 months ago

did that, the runtime error is gone now, but its still not working

(sbea) C:\mentes\sbea\SBeA_release\SBeA_tracker\SBeA-Windows-main>python setup.py install

running install
C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
C:\mentes\conda\envs\sbea\lib\site-packages\setuptools\command\easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
running bdist_egg
running egg_info
writing deform_conv.egg-info\PKG-INFO
writing dependency_links to deform_conv.egg-info\dependency_links.txt
writing top-level names to deform_conv.egg-info\top_level.txt
C:\mentes\conda\envs\sbea\lib\site-packages\torch\utils\cpp_extension.py:476: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
  warnings.warn(msg.format('we could not find ninja.'))
reading manifest file 'deform_conv.egg-info\SOURCES.txt'
writing manifest file 'deform_conv.egg-info\SOURCES.txt'
installing library code to build\bdist.win-amd64\egg
running install_lib
running build_ext
C:\mentes\conda\envs\sbea\lib\site-packages\torch\utils\cpp_extension.py:358: UserWarning: Error checking compiler version for cl: [WinError 2] The system cannot find the file specified
  warnings.warn(f'Error checking compiler version for {compiler}: {error}')
creating build\bdist.win-amd64\egg
copying build\lib.win-amd64-cpython-39\_C.cp39-win_amd64.pyd -> build\bdist.win-amd64\egg
creating stub loader for _C.cp39-win_amd64.pyd
byte-compiling build\bdist.win-amd64\egg\_C.py to _C.cpython-39.pyc
creating build\bdist.win-amd64\egg\EGG-INFO
copying deform_conv.egg-info\PKG-INFO -> build\bdist.win-amd64\egg\EGG-INFO
copying deform_conv.egg-info\SOURCES.txt -> build\bdist.win-amd64\egg\EGG-INFO
copying deform_conv.egg-info\dependency_links.txt -> build\bdist.win-amd64\egg\EGG-INFO
copying deform_conv.egg-info\top_level.txt -> build\bdist.win-amd64\egg\EGG-INFO
writing build\bdist.win-amd64\egg\EGG-INFO\native_libs.txt
zip_safe flag not set; analyzing archive contents...
__pycache__._C.cpython-39: module references __file__
creating 'dist\deform_conv-0.0.0-py3.9-win-amd64.egg' and adding 'build\bdist.win-amd64\egg' to it
removing 'build\bdist.win-amd64\egg' (and everything under it)
Processing deform_conv-0.0.0-py3.9-win-amd64.egg
removing 'c:\mentes\conda\envs\sbea\lib\site-packages\deform_conv-0.0.0-py3.9-win-amd64.egg' (and everything under it)
creating c:\mentes\conda\envs\sbea\lib\site-packages\deform_conv-0.0.0-py3.9-win-amd64.egg
Extracting deform_conv-0.0.0-py3.9-win-amd64.egg to c:\mentes\conda\envs\sbea\lib\site-packages
deform-conv 0.0.0 is already the active version in easy-install.pth

Installed c:\mentes\conda\envs\sbea\lib\site-packages\deform_conv-0.0.0-py3.9-win-amd64.egg
Processing dependencies for deform-conv==0.0.0
Finished processing dependencies for deform-conv==0.0.0

I reinstalled VisualStudio2022 and the build tools too but this cl compiler thing still persists. Also, I wasn't sure if there was an actual error, because it mostly just says "warning", so I tried to continue:

(sbea) C:\mentes\sbea\SBeA_release\SBeA_tracker\SBeA-Windows-main>cd gui

(sbea) C:\mentes\sbea\SBeA_release\SBeA_tracker\SBeA-Windows-main\gui>python main.py

Traceback (most recent call last):
  File "C:\mentes\sbea\SBeA_release\SBeA_tracker\SBeA-Windows-main\gui\main.py", line 1, in <module>
    from tab import MainWindow
  File "gui\tab.py", line 1, in init tab
  File "gui\eval.py", line 7, in init eval
  File "C:\mentes\sbea\SBeA_release\SBeA_tracker\SBeA-Windows-main\SocialBehaviorAtlas\__init__.py", line 5, in <module>
    from SocialBehaviorAtlas.train_seg_models import train_seg_models
  File "SocialBehaviorAtlas\train_seg_models.py", line 15, in init train_seg_models
  File "C:\mentes\sbea\SBeA_release\SBeA_tracker\SBeA-Windows-main\SocialBehaviorAtlas\train_utils\__init__.py", line 2, in <module>
    from SocialBehaviorAtlas.train_utils.train_yolact import train_yolact
  File "SocialBehaviorAtlas\train_utils\train_yolact.py", line 9, in init train_yolact
  File "C:\mentes\sbea\SBeA_release\SBeA_tracker\SBeA-Windows-main\SocialBehaviorAtlas\train_utils\yolact\__init__.py", line 1, in <module>
    from SocialBehaviorAtlas.train_utils.yolact.sbea_train import sbea_train
ModuleNotFoundError: No module named 'SocialBehaviorAtlas.train_utils.yolact.sbea_train'

(sbea) C:\mentes\sbea\SBeA_release\SBeA_tracker\SBeA-Windows-main\gui>python main.py
Traceback (most recent call last):
  File "C:\mentes\sbea\SBeA_release\SBeA_tracker\SBeA-Windows-main\gui\main.py", line 1, in <module>
    from tab import MainWindow
  File "gui\tab.py", line 1, in init tab
  File "gui\eval.py", line 7, in init eval
  File "C:\mentes\sbea\SBeA_release\SBeA_tracker\SBeA-Windows-main\SocialBehaviorAtlas\__init__.py", line 5, in <module>
    from SocialBehaviorAtlas.train_seg_models import train_seg_models
  File "SocialBehaviorAtlas\train_seg_models.py", line 15, in init train_seg_models
  File "C:\mentes\sbea\SBeA_release\SBeA_tracker\SBeA-Windows-main\SocialBehaviorAtlas\train_utils\__init__.py", line 2, in <module>
    from SocialBehaviorAtlas.train_utils.train_yolact import train_yolact
  File "SocialBehaviorAtlas\train_utils\train_yolact.py", line 9, in init train_yolact
  File "C:\mentes\sbea\SBeA_release\SBeA_tracker\SBeA-Windows-main\SocialBehaviorAtlas\train_utils\yolact\__init__.py", line 1, in <module>
    from SocialBehaviorAtlas.train_utils.yolact.sbea_train import sbea_train
ModuleNotFoundError: No module named 'SocialBehaviorAtlas.train_utils.yolact.sbea_train'

any ideas?

YNCris commented 4 months ago

Yes, you met this bug. This is because the synchronization of code in GitHub has a bug. I will send you a zip. Use the zip, and you will not have this bug.

YNCris commented 4 months ago

And yes this "Finished processing dependencies for deform-conv==0.0.0" seems you are right.

delaroob commented 4 months ago

So, what worked was:

1)) to match the version of pytorch, downloaded earlier version (11.7) of CUDA from: https://developer.nvidia.com/cuda-11-7-0-download-archive?target_os=Windows&target_arch=x86_64&target_version=11&target_type=exe_local

2)) in cmd: set CUDA_HOME=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7

3)) followed the instructions in the user manual:

  1. downloded environment.yml file
  2. cd to the environment. yml file in anaconda prompt, then conda env create -f environment.yml
  3. conda activate sbaw
  4. git clone https://github.com/YNCris/SBeA_release.git --> here I downloaded the .zip file (SBeA-Windows-main.zip) and merged it with the original SBeA-Windows-main folder from the github repo, overwriting any conflicts
  5. cd to the setup.py file, then python setup.py install
  6. cd into ./gui, where the main.py file is, and then: python main.py

Aaand the GUI opens, as it should:) Also, I'm not sure if that was necessary but I downloaded visual studio build tools because there were some errors about the compiler. And thank you so much for your help!