search

conda-forge / conda-forge.github.io

The conda-forge website.
https://conda-forge.org
BSD 3-Clause "New" or "Revised" License
131 stars 274 forks source link

Building packages on s390x architecture #1123

Open vibhaKulka opened 4 years ago

vibhaKulka commented 4 years ago

Continuing from issue https://github.com/conda-forge/conda-smithy/issues/1359

Opening an issue to start building packages on s390x .

We have had various internal discussions on having a channel with s390x build packages and we would like to take it forward.

I am tagging my team members who will work on all the requirements that is needed to enable package builds on s390x.

Please let us know , what are the next steps and we would be very happy to contribute.

Thanks for all the guidance and the interest so far .

Thanks, Vibha Kulkarni

vibhaKulka commented 4 years ago

@chandrureddy @@pradghos @ravigumm

vibhaKulka commented 4 years ago

@isuruf @jjhelmus

isuruf commented 4 years ago

First steps in order would be,

  1. Ask for volunteers for conda-forge/help-s390x team to support s390x. (At least two people who'll respond to github issues quickly)
  2. Ask Anaconda, Inc to support uploading s390x packages to anaconda.org.
  3. Send a PR to repackage the centos sysroot for s390x in https://github.com/conda-forge/linux-sysroot-feedstock/
  4. Build binutils for s390x in https://github.com/conda-forge/binutils-feedstock/
  5. Build a cross compiler for s390x in https://github.com/conda-forge/ctng-compilers-feedstock
  6. Build a cross-native compiler for s390x in https://github.com/conda-forge/ctng-compilers-feedstock
CJ-Wright commented 4 years ago
  1. After you get together a group of people for 1, please tell core so we can make the appropriate help team
  2. Once you are ready to do a full scale build out contact the bot team (@conda-forge/bot) and we can work with you to setup the architecture migration.
vibhaKulka commented 4 years ago

@isuruf @CJ-Wright Thank you for your response . We will take a look at the all the items and get back to you .

potula-chandra commented 4 years ago

@isuruf and @CJ-Wright thank you very much for your support !

We are looking on the different details that are outlined in the comment https://github.com/conda-forge/conda-forge.github.io/issues/1123#issuecomment-668399522

Since there is no official support in CentOS for s390x platform at this point in time we are relying on clefos which has all equivalent CentOS binaries/packages.

https://www.sinenomine.net/products/linux/clefos/packages

pradghos commented 4 years ago

@isuruf Regarding ctng-compilers-feedstock, It seems crosstool-NG samples (i.e powerpc64le-unknown-linux-gnu ) along with ctng_vendor are taking care of the builds of cross compiler and cross-native compiler depending upon which build arch is being used.

Just for example One of scenario would be - 

[EXTRA]  Dumping internal crosstool-NG configuration
[EXTRA]    Building a toolchain for:
[EXTRA]      build  = powerpc64le-unknown-linux-gnu
[EXTRA]      host   = powerpc64le-unknown-linux-gnu
[EXTRA]      target = powerpc64le-conda_cos7-linux-gnu

Probably we may need to have similar approach for s390x in crosstool-NG itself.

Pls let me know if anything I have missed out here. Thank you !

pradghos commented 4 years ago

@isuruf and @CJ-Wright thank you very much for your support !

We are looking on the different details that are outlined in the comment #1123 (comment)

Since there is no official support in CentOS for s390x platform at this point in time we are relying on clefos which has all equivalent CentOS binaries/packages.

https://www.sinenomine.net/products/linux/clefos/packages

I see same version of all rpms used in linux-sysroot-feedstock are available in http://download.sinenomine.net/clefos/7/os/s390x .. expect the tzdata-2019c-1.el7.noarch.rpm which is coming from http://download.sinenomine.net/clefos/7/os/noarch.

isuruf commented 4 years ago

Using clefos is fine. You can already build cross compilers with the current ctng-compilers-feedstock. This is enough to cross compile other packages and to bootstrap a conda installer. cross-native are currently not supported in ctng-compilers-feedstock, but it shouldn't be too much work to add it.

pradghos commented 4 years ago

@isuruf : I have created PR to introduce s390x-unknown-linux-gnu samples like other platform is having today.
https://github.com/crosstool-ng/crosstool-ng/pull/1376

isuruf commented 4 years ago

Looks good. Please send the PR over to https://github.com/conda-forge/crosstool-ng as well which is a fork we maintain with some extra patches.

pradghos commented 4 years ago

@isuruf : Looks like I cant create a personal fork for conda-forge/crosstool-ng because same name repository is already exists. Once it will merge , I will create PR https://github.com/conda-forge/crosstool-ng . For time being, I have created PR for https://github.com/conda-forge/crosstool-ng-feedstock/pull/15 and added it as patch for now . I will clean it up from here later. Please let me know if you are fine.

jakirkham commented 4 years ago

Shouldn’t need another fork as they share the same lineage. Would just add a new remote locally, add a new branch with your changes on top of the remote’s master branch, and submit a PR to our fork with your changes.

pradghos commented 4 years ago

@jakirkham yes, I was able to create the PR and its merged :-) .. Thanks to @isuruf for guiding me there ...

pradghos commented 4 years ago

Using clefos is fine. You can already build cross compilers with the current ctng-compilers-feedstock. This is enough to cross compile other packages and to bootstrap a conda installer. cross-native are currently not supported in ctng-compilers-feedstock, but it shouldn't be too much work to add it.

@isuruf created PR for cdt similar to the way ppc64le,aarch64 treating today - https://github.com/conda/conda-build/pull/4030, considering the clefos7 is equivalent to CentOS7