search

WorksOnArm / equinix-metal-arm64-cluster

Arm and Equinix Metal have partnered to make powerful Neoverse based Armv8 bare metal infrastructure including latest generation Ampere systems — available for open source software developers to build, test and optimize for Arm64 architecture.
http://www.worksonarm.com
77 stars 12 forks source link

Request for conda-forge #193

Closed isuruf closed 4 years ago

isuruf commented 4 years ago

Name, email, company, job title

Isuru Fernando, isuruf@gmail.com, conda-forge, Core developer Matthew R. Becker, becker.mr@gmail.com, conda-forge, Core developer

Project Title and description

conda-forge

conda-forge is a community-led collection of recipes, build infrastructure and distributions for the conda package manager.

Which members of the community would benefit from your work?

Packages from conda-forge are used by hundreds of thousands of users and have a million downloads for some packages every month.

Is the code that you’re going to run 100% open source? If so, what is the URL or URLs where it is located?

Yes, everything is going to be in https://github.com/conda-forge

What infrastructure (computing resources and network access) do you need? (see: https://www.packet.net/bare-metal/)?

We are using CI for building aarch64 packages using Drone (and sometimes Travis). One issue is that there are packages that require more than the 1 hour time limit provided by Drone CI and Travis CI. For these packages, we'd like to have access to a server. We'll need ongoing support, but the usage of the server will be very minimal as the time limits are enough for most packages and only a few packages require more time or debugging (A few hours per week is typical). We also don't need a dedicated server. A shared one is more than enough.

Describe the continuous integration (CI) system in use or desired for this project.

We already have CI as mentioned above.

Please state your contributions to the open source community and any other relevant initiatives

I've been a core contributor of conda-forge, SymPy and SymEngine for a few years and also help out on quite a few open source projects on packaging.

vielmetti commented 4 years ago

Thanks @isuruf - we'll be reviewing this on a call on Friday 2020-02-07.

If you happen to have links to the particular packages that are failing due to time limits, that would be useful input.

isuruf commented 4 years ago

For eg: PyPy, LLVM, Clang, GCC. All of these have x86_64 (due to Azure's 6 hour limit), ppc64le (OSU OSL gave access to a server) packages but not aarch64.

vielmetti commented 4 years ago

@isuruf - got it - did not realize how deep all of the dependencies went.

Are your build scripts using more than one core? The Arm servers we have on each of these all are manycore systems (32 to 96 cores) and there are considerable speedups available in nearly every case with parallelism.

isuruf commented 4 years ago

@vielmetti, yes. For example https://github.com/conda-forge/clangdev-feedstock/blob/master/recipe/build.sh#L30 where CPU_COUNT is set to 96 on Drone. Some recipes change this default as they see fit.

vielmetti commented 4 years ago

This has been approved - I'll follow up as soon as possible with access details!

vielmetti commented 4 years ago

Accounts set up, 1x c2.large.arm reserved, and further details sent via email. Closing here for now.