Azure CI pending - Githubissues

apache / incubator-xtable

Apache XTable (incubating) is a cross-table converter for lakehouse table formats that facilitates interoperability across data processing systems and query engines.

https://xtable.apache.org/

Apache License 2.0

919 stars 147 forks source link

Azure CI pending #438

Closed daragu closed 5 months ago

daragu commented 6 months ago

Search before asking

[X] I had searched in the issues and found no similar issues.

Please describe the bug 🐞

https://dev.azure.com/apache-xtable-ci-org/apache-xtable-ci/_build/results?buildId=969&view=logs&j=22fea640-1099-5f32-ec5d-316ad83f359a

##[error]This request was automatically failed because there were no enabled agents online to process the request for more than 7 days.

Are you willing to submit PR?

[ ] I am willing to submit a PR!
[ ] I am willing to submit a PR but need help getting started!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

daragu commented 6 months ago

hi @the-other-tim-brown, how about using github action?

daragu commented 6 months ago

Hi @the-other-tim-brown Recently, I have found that there are always problems with CI, and many CIs are in a pending state. I am familiar with GitHub Action and we can give it a try. I would be happy to hear your advice.

vinishjail97 commented 6 months ago

@daragu There are some apache guidelines for using a azure CI pipeline, l will check more on this.

daragu commented 6 months ago

@daragu There are some apache guidelines for using a azure CI pipeline, l will check more on this.

👍 thanks @vinishjail97.

But there is a limitation that forkers cannot trigger ci in their own warehouse with using Azure CI, https://github.com/daragu/incubator-xtable/actions/workflows/azure_ci_check.yml

charlesy6 commented 6 months ago

+1, Github Action is better for developers.

the-other-tim-brown commented 5 months ago

@jcamachor what is your opinion on switching to GitHub actions?

jcamachor commented 5 months ago

@the-other-tim-brown , if I remember correctly, one of the reasons we were using Azure infrastructure before moving to the ASF was because the runners provided by the free GitHub plan were too slow or couldn't handle the tests at all. Do you recall? Also, do you know if the current ASF-provided runners will be sufficient for our needs? I agree the current testing pipeline is a bit of a mess since Azure Pipelines necessary permissions don't mesh well with ASF, as described in https://issues.apache.org/jira/browse/INFRA-17030. This has led us to a somewhat makeshift solution (#363). It would be great to simplify things, and if GitHub-hosted runners for actions are viable, I'm +1. If we ever decide to use Azure infrastructure in the future for other types of tests, we can see how to run them on Azure at that time.

daragu commented 5 months ago

hi @the-other-tim-brown @jcamachor let's give it a try?

the-other-tim-brown commented 5 months ago

@the-other-tim-brown , if I remember correctly, one of the reasons we were using Azure infrastructure before moving to the ASF was because the runners provided by the free GitHub plan were too slow or couldn't handle the tests at all. Do you recall? Also, do you know if the current ASF-provided runners will be sufficient for our needs? I agree the current testing pipeline is a bit of a mess since Azure Pipelines necessary permissions don't mesh well with ASF, as described in https://issues.apache.org/jira/browse/INFRA-17030. This has led us to a somewhat makeshift solution (#363). It would be great to simplify things, and if GitHub-hosted runners for actions are viable, I'm +1. If we ever decide to use Azure infrastructure in the future for other types of tests, we can see how to run them on Azure at that time.

@jcamachor it was due to the credits we were able to get on Azure originally. We use GitHub actions internally and were able to run the tests. Where can I find information about the runners we have access to as part of ASF?

jcamachor commented 5 months ago

@daragu , yes, we should try. I found this information: https://cwiki.apache.org/confluence/display/BUILDS/GitHub+Actions+status Though it does not seem to provide details about the runners.

I found this as well, maybe something to consider if github-hosted runners are not sufficient: https://cwiki.apache.org/confluence/pages/viewpage.action?spaceKey=INFRA&title=ASF+Infra+provided+self-hosted+runners

daragu commented 5 months ago

hi @the-other-tim-brown @jcamachor let's give it a try?

hi @the-other-tim-brown, I would like to hear your opinion.

daragu commented 5 months ago

@daragu , yes, we should try. I found this information: https://cwiki.apache.org/confluence/display/BUILDS/GitHub+Actions+status Though it does not seem to provide details about the runners.

I found this as well, maybe something to consider if github-hosted runners are not sufficient: https://cwiki.apache.org/confluence/pages/viewpage.action?spaceKey=INFRA&title=ASF+Infra+provided+self-hosted+runners

👍

the-other-tim-brown commented 5 months ago

@daragu I am in support of this, let's make sure to follow the instructions in that doc. I am also more familiar with GitHub actions and the default runner should work for us based on my experience.

daragu commented 5 months ago

@daragu I am in support of this, let's make sure to follow the instructions in that doc. I am also more familiar with GitHub actions and the default runner should work for us based on my experience.

Glad to hear this news 👍👍👍