Add Linux and Mac arm platforms

AlbertDeFusco commented 2 years ago

This adds linux-aarch64 and osx-arm64 as known platforms.

Further, this PR also solves #274 and allows CONDA_SUBDIR env var to be used to override the current platform.

For example, on an Apple M1 you can now force it to create envs for osx-64. First you'll see that Conda determines my platform is osx-arm64. I can force anaconda-project to create an osx-64 env_spec by setting CONDA_SUBDIR=osx-64. Otherwise the prepare would fail because Python 3.7 is not available for osx-arm64. Note that you must keep CONDA_SUBDIR set for all anaconda-project actions, not just the prepare step.

> conda info --json | jq .platform
"osx-arm64"

> cat anaconda-project.yml
name: arm
packages:
- python=3.7

> anaconda-project prepare
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... failed

PackagesNotFoundError: The following packages are not available from current channels:

  - python=3.7

Current channels:

  - https://repo.anaconda.com/pkgs/main/osx-arm64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/osx-arm64
  - https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

missing requirement to run this project: The project needs a Conda environment containing all required packages.
  '/Users/adefusco/Desktop/arm/envs/default' doesn't look like it contains a Conda environment yet.

> CONDA_SUBDIR=osx-64 anaconda-project prepare
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: /Users/adefusco/Desktop/arm/envs/default

  added / updated specs:
    - python=3.7

The following NEW packages will be INSTALLED:

  ca-certificates    pkgs/main/osx-64::ca-certificates-2021.10.26-hecd8cb5_2
  certifi            pkgs/main/osx-64::certifi-2021.10.8-py37hecd8cb5_0
  libcxx             pkgs/main/osx-64::libcxx-12.0.0-h2f01273_0
  libffi             pkgs/main/osx-64::libffi-3.3-hb1e8313_2
  ncurses            pkgs/main/osx-64::ncurses-6.3-hca72f7f_2
  openssl            pkgs/main/osx-64::openssl-1.1.1l-h9ed2024_0
  pip                pkgs/main/osx-64::pip-21.2.2-py37hecd8cb5_0
  python             pkgs/main/osx-64::python-3.7.11-h88f2d9e_0
  readline           pkgs/main/osx-64::readline-8.1.2-hca72f7f_0
  setuptools         pkgs/main/osx-64::setuptools-58.0.4-py37hecd8cb5_0
  sqlite             pkgs/main/osx-64::sqlite-3.37.0-h707629a_0
  tk                 pkgs/main/osx-64::tk-8.6.11-h7bc2e8c_0
  wheel              pkgs/main/noarch::wheel-0.37.1-pyhd3eb1b0_0
  xz                 pkgs/main/osx-64::xz-5.2.5-h1de35cc_0
  zlib               pkgs/main/osx-64::zlib-1.2.11-h4dc903c_4

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
#
# To activate this environment, use
#
#     $ conda activate /Users/adefusco/Desktop/arm/envs/default
#
# To deactivate an active environment, use
#
#     $ conda deactivate

The project is ready to run commands.
Use `anaconda-project list-commands` to see what's available.

mcg1969 commented 2 years ago

By the way, I'd be open to dropping Python 2.7 support if necessary.

AlbertDeFusco commented 2 years ago

I'm going to keep working on the tests.

I may wait to drop 2.7 tests when I complete a PR to add py 3.9 testing. I think some tests fail when anaconda-project is installed in py 3.9 env and I'ld check if there are bugs.

AlbertDeFusco commented 2 years ago

For those here who are interested I've gotten tests passing for all but three scenarios added the following exclusions to the github action.

The Ubuntu Conda 4.8 + py 3[7-8] fail for one test that checks a timestamp on the live project environment, but it does not exist for older or newer versions of Conda.

I plan to drop testing support for Py 2.7 in the near future.

          # Odd timestamp behavior
          - pyver: 3.7
            os: ubuntu-latest
            cver: 4.8
          - pyver: 3.8
            os: ubuntu-latest
            cver: 4.8
          # Old versions will be trimmed from tests soon
          - os: windows-latest
            pyver: 2.7
            cver: 4.6
          - os: windows-latest
            pyver: 3.6
            cver: 4.6

AlbertDeFusco commented 2 years ago

A question for those of you who maybe interested (@jbednar , @jlstevens , @philippjfr ?).

Anaconda Project had previously decided that the default platforms are linux-64, osx-64, and win-64. You see this when initializing a project or locking a project

platforms:
  - linux-64
  - osx-64
  - win-64

At some point osx-arm64 is going to start gaining in popularity, but right now only Python >=3.8 is available and many packages are missing so I may feel a little hesitant about adding the platform

Should osx-arm64 be added to the default platforms? Having osx-armt64 in default platform may cause locking to fail more often for users, and especially those who do not use osx-arm64.
If not should I attempt to contextualize the default platforms such that osx-arm64 is considered default only if anaconda-project is being run on osx-arm64? This would allow osx-arm64 to enter into the platforms: key without the user having to explicitly write it, as they would now and with this PR

AlbertDeFusco commented 2 years ago

oh, nevermind above! Anaconda Project already does option 2 above. I just need to adjust tests to account for this. Some tests were not originally intended to run on non default platforms hardware.

AlbertDeFusco commented 2 years ago

Tests pass on all platforms. Any comments or concerns before I merge?

anaconda / anaconda-project

Add Linux and Mac arm platforms #352