thousandbrainsproject / tbp.monty

Monty is a sensorimotor learning framework based on the thousand brains theory of the neocortex.
https://thousandbrainsproject.readme.io
MIT License
102 stars 25 forks source link

Windows OS not supported? #52

Open internetscooter opened 21 hours ago

internetscooter commented 21 hours ago

Describe the bug

Hi TBP,

I think Windows OS is not supported due to https://github.com/facebookresearch/habitat-sim/issues/1570 - if this is the case there should be a note on this in the code/docs?

Below is the complete message. If the above is the case I will see what workaround I can come up with.

PS C:\code\tbp.monty> conda env create C:\Users\paulm\anaconda3\Lib\argparse.py:2006: FutureWarning:remote_definitionis deprecated and will be removed in 25.9. Useconda env create --file=URL` instead. action(self, namespace, argument_values, option_string) Channels:

PackagesNotFoundError: The following packages are not available from current channels:

Current channels:

To search for alternate channels that may provide the conda package you're looking for, navigate to

https://anaconda.org

and use the search bar at the top of the page.`

tristanls commented 12 hours ago

Hi @internetscooter, thank you for the report.

We did not intend to exclude Windows OS. While Habitat is not available on Windows, Monty should be available.

Does the following environment_win.yml file work for installation (Habitat dependencies omitted)?

# This file may be used to create an environment using:
#
# ## Miniconda or Anaconda
#     $ conda env create --file environment_win.yml
# If you are using the zsh shell, run:
#     $ conda init zsh
# Or, if you are using a different shell, run:
#     $ conda init
# After init, if you do not want conda to change your global shell when
# you open a new terminal, run:
#     $ conda config --set auto_activate_base false
# Finally, activate the environment with:
#     $ conda activate tbp.monty
#
# platform: win
name: tbp.monty
channels:
  - pytorch
  - pyg
  - defaults
  - conda-forge

dependencies:
  - python=3.8
  - cmake>=3.14.0
  - pyg::pyg
  - wget

  - pytorch::pytorch=1.11.0
  - conda-forge::quaternion=2023.0.3 # later versions missing np.long
  - pytorch::torchvision

  - pip
  - pip:
      - -e .[dev]

For example, the Monty Meets World benchmarks are intended to be independent of Habitat: https://thousandbrainsproject.readme.io/docs/benchmark-experiments#monty-meets-world. (A documentation update is pending in #57, which shares instructions for downloading the data used in the benchmarks.)

Unfortunately, once you pointed out the problem, it became apparent that Monty is accidentally coupled to Habitat in some of the motor policies: https://github.com/thousandbrainsproject/tbp.monty/blob/d1c8369aa98470afc253c879c929f605094cde6c/src/tbp/monty/frameworks/models/motor_policies.py#L18 I think this coupling can be removed as only a quat_rotate_vector and common.quat_to_magnum are used, and can likely be replaced by non-Habitat implementations.

internetscooter commented 8 hours ago

Would yo be happy including SciPy?

scipy.spatial.transform.Rotation

tristanls commented 8 hours ago

@vkakerbeck, @nielsleadholm any thoughts on habitat_sim.utils replacements and @internetscooter 's suggestion above?

vkakerbeck commented 16 minutes ago

I agree, scipy's Rotation function seems the most straight forward function to replace the use of quat_rotate_vector (especially since we are using this function in all the other places). It should already be included in the installed dependencies as well (https://github.com/thousandbrainsproject/tbp.monty/blob/70d610af588c71f2d239d8fb0f1146fac44649a1/pyproject.toml#L26) For the quaternion <-> magnum conversions (quat_to_magnum and quat_from_magnum) it seems like we could just call whatever habitat implements in the respective functions instead (https://github.com/facebookresearch/habitat-sim/blob/d7da04897f234727813aee3368d7e44e0f08c50e/src_python/habitat_sim/utils/common.py#L48) since its just a straight forward call to magnum functionality (mn.Quaternion(quat.imag, quat.real))