search

opensearch-project / opensearch-build

🧰 OpenSearch / OpenSearch-Dashboards Build Systems
Apache License 2.0
132 stars 260 forks source link

Support aarch64 OpenSearch distribution for MacOS architecture #4670

Open reta opened 2 months ago

reta commented 2 months ago

Is your feature request related to a problem? Please describe

The OpenSeach core does support the MacOS aarch64 based distribution (Mx CPUs) but we never publish those distribution during the release.

Describe the solution you'd like

Publish aarch64 OpenSearch distributions for MacOS architecture.

Describe alternatives you've considered

N/A

Additional context

@peterzhuamazon fyi

Acceptance Criteria

peterzhuamazon commented 2 months ago

Previous issue:

peterzhuamazon commented 2 months ago

Will sync up with team on the next steps after 2.14.0:

Thanks.

peterzhuamazon commented 2 months ago

Will take a look next week after 2.14.0.

peterzhuamazon commented 1 month ago

Taking a look on adding ec2 instances with:

  1. Changing existing x64 mac1.metal instance from 6 executors to 4, so that each will have 3cpu/8GB ram.
  2. New arm64 will use the mac2-m2pro.metal instance with the same 4 executors, with 3cpu/8GB ram each container setup.
peterzhuamazon commented 1 month ago

Seems like mac1.metal is still using the host tenancy, trying to switch to on-demand default tenancy and have issues:

2024-05-28 22:02:22.403+0000 [id=115853]        WARNING h.i.i.InstallUncaughtExceptionHandler#handleException: Caught unhandled exception with ID 7ab1f0b4-a262-4186-bdd8-2ba5ecd55bd1
com.amazonaws.services.ec2.model.AmazonEC2Exception: The requested tenancy is not supported for this instance type. Please check the documentation for supported configurations. (Service: AmazonEC2; Status Code: 400; Error Code: Unsupported; Request ID: dfc7b12e-635d-4ed7-8c0c-8117eedafd4b; Proxy: null)
peterzhuamazon commented 1 month ago

Seems like mac1.metal is still using the host tenancy, trying to switch to on-demand default tenancy and have issues:

2024-05-28 22:02:22.403+0000 [id=115853]        WARNING h.i.i.InstallUncaughtExceptionHandler#handleException: Caught unhandled exception with ID 7ab1f0b4-a262-4186-bdd8-2ba5ecd55bd1
com.amazonaws.services.ec2.model.AmazonEC2Exception: The requested tenancy is not supported for this instance type. Please check the documentation for supported configurations. (Service: AmazonEC2; Status Code: 400; Error Code: Unsupported; Request ID: dfc7b12e-635d-4ed7-8c0c-8117eedafd4b; Proxy: null)

Per this docs: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-mac-instances.html

Mac instances are available only as bare metal instances on Dedicated Hosts, with a minimum allocation period of 24 hours before you can release the Dedicated Host. You can launch one Mac instance per Dedicated Host. You can share the Dedicated Host with the AWS accounts or organizational units within your AWS organization, or the entire AWS organization.

peterzhuamazon commented 1 month ago

Need to apply for a dedicated host for testing.

peterzhuamazon commented 1 month ago

Added acceptance criteria to the description.

peterzhuamazon commented 1 month ago

Get the dedicated hosts for m2 pro and able to setup 1 host, tho there are few issues to provision the instance onto the host through Jenkins:

  1. m2 pro on ec2 was announced last year on 2023/09/19: https://aws.amazon.com/blogs/aws/new-amazon-ec2-m2-pro-mac-instances-built-on-apple-silicon-m2-pro-mac-mini-computers/
  2. The corresponding sdk change of aws-java-sdk-ec2 was updated with this instance type on 1.12.556 (2023/09/22): https://github.com/aws/aws-sdk-java/blame/ee7313b7505156127a3dfe251acdf2a79133e7ad/aws-java-sdk-ec2/src/main/java/com/amazonaws/services/ec2/model/InstanceType.java#L763
  3. Our current Jenkins is using ec2 plugin 2.0.7 which in turn uses sdk 1.12.406 (https://github.com/jenkinsci/ec2-plugin/blob/ec2-2.0.7/pom.xml#L121). At the time of the creation of Jenkins main node docker we update the sdk to 1.12.481. We might need to wait for Jenkins upgrade in this issue to complete: https://github.com/opensearch-project/opensearch-ci/issues/389. The ideal case is to use the latest version of ec2 plugin and at least 1.12.556 version of aws-java-sdk-ec2 to support mac2-m2pro.metal instance type.

Thanks.

peterzhuamazon commented 1 month ago

Able to confirm the latest version of EC2 plugin is able to select mac2m2prometal: Screenshot 2024-06-06 at 2 34 04 PM

peterzhuamazon commented 3 weeks ago

The dedicated hosts seems have availability limitations. Previous mac1 are all launched on one zone, while this time it is on other zone with previous zone have limited availability.

Need more digging.

peterzhuamazon commented 4 days ago

I am eventually able to allocate enough dedicated hosts in the particular zone our jenkins located. Will start the next step of adding a new packer template.

Thanks.

peterzhuamazon commented 3 days ago

After scheduling the same version of macOS 12 on arm64 instance, it failed with check 0/2 and never able to launch.

Investigating why that is the case and potentially move to macos13.

Thanks.

peterzhuamazon commented 3 days ago

Apparently even tho macOS 12 support arm64, it only support M1 where as our instance of arm64 is M2. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-mac-instances.html#mac-instance-considerations


    macOS Mojave (version 10.14) (x86 Mac instances only)

    macOS Catalina (version 10.15) (x86 Mac instances only)

    macOS Big Sur (version 11) (x86 and M1 Mac instances)

    macOS Monterey (version 12) (x86 and M1 Mac instances)

    macOS Ventura (version 13) (all Mac instances, M2 and M2 Pro Mac instances support macOS Ventura version 13.2 or later)

    macOS Sonoma (version 14) (all Mac instances)

I will now switch arm64 to macOS 13 and potentially update the x64 one to macOS 13 as well.