search

sakura-editor / sakura

SAKURA Editor (Japanese text editor for MS Windows)
https://sakura-editor.github.io/
Other
1.24k stars 162 forks source link

Azure Pipelines を調査する #446

Closed m-tmatma closed 1 month ago

m-tmatma commented 5 years ago

Azure Pipelines を調査する

https://azure.microsoft.com/ja-jp/services/devops/pipelines/

オープン ソース用に 10 個の無料の並列ジョブ (時間制限なし) を実行可能な、Linux、macOS、 および Windows 用のクラウドホステッド パイプラインを入手

とある。

調べること

KENCHjp commented 5 years ago

オンラインでコンパイルできるってことですよねぇ。 コンパイラ（例えばDelphi)とか持ち込めるのかなぁ。。。

m-tmatma commented 5 years ago

https://github.com/marketplace/azure-pipelines で Sign-up する

m-tmatma commented 5 years ago

ちなみに 2018年9月13日に発表されたばかりみたいです。 https://www.publickey1.jp/blog/18/cicdazure_pipeline130.html

2018年9月10日に発表されたみたいです。 https://azure.microsoft.com/en-us/blog/announcing-azure-pipelines-with-unlimited-ci-cd-minutes-for-open-source/

m-tmatma commented 5 years ago

10 個の無料の並列ジョブ が魅力ですね。

m-tmatma commented 5 years ago

appveyor に Azure Pipelines と同程度あるいはそれ以上の同時タスクをサポートしてほしいと送ってみた。 https://github.com/appveyor/ci/issues/2641

m-tmatma commented 5 years ago

オンラインでコンパイルできるってことですよねぇ。

Windows に対応している Cloud の CI として appveyor を使っていましたが、 採用当時、windows に対応しているが、appveyor しか選択肢がありませんでしたが、 他に候補ができたということです。

説明欄に調査項目を追加しました。

m-tmatma commented 5 years ago

https://github.com/MicrosoftDocs?utf8=%E2%9C%93&q=pipelines&type=&language= で検索すると pipeline の YAML の書き方のサンプルプロジェクトを検索できる。

以下は ASP.NET Core project using Azure Pipelines のサンプル https://github.com/MicrosoftDocs/pipelines-dotnet-core/blob/master/azure-pipelines.yml

c++ は見当たらないけどサポートしてないのかな?

以下関連ドキュメント https://docs.microsoft.com/en-us/azure/devops/pipelines/get-started-yaml?view=vsts https://docs.microsoft.com/en-us/azure/devops/pipelines/build/ci-public?view=vsts&tabs=github

m-tmatma commented 5 years ago

sakura editor には関係ないけど iOS のビルドもできそう。 https://github.com/MicrosoftDocs/pipelines-xcode

m-tmatma commented 5 years ago

以下はブランチをビルドする方法 https://docs.microsoft.com/en-us/azure/devops/pipelines/build/ci-build-git?view=vsts&tabs=yaml

m-tmatma commented 5 years ago

c++ は見当たらないけどサポートしてないのかな?

https://docs.microsoft.com/en-us/azure/devops/pipelines/apps/windows/cpp?view=vsts に cpp に関する記述がある

https://docs.microsoft.com/en-us/azure/devops/pipelines/get-started-designer?view=vsts&tabs=new-nav にリンクが張ってあって Web から設定する説明がある。

m-tmatma commented 5 years ago

appveyor は使うの簡単だったけど、 Azure Pipelines は使うの難しそう。

m-tmatma commented 5 years ago

バッチファイルも実行可能みたい。 https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/batch-script?view=vsts

m-tmatma commented 5 years ago

https://docs.microsoft.com/en-us/azure/devops/pipelines/index?view=vsts にドキュメントのトップページがある。

各ページの下に Feedback という項目があり、直接 GitHub 経由で Feedback を送ることが できるみたい。

m-tmatma commented 5 years ago

成果物の保存期間の調査

https://docs.microsoft.com/en-us/azure/devops/pipelines/policies/retention?view=vsts に記載があって、どれぐらいの期間保存するのかを確認方法が説明されているが、 具体的にどれぐらいかは記載されていない。

vtbassmatt commented 5 years ago

Hello. I'm a product manager for Azure Pipelines. If there's anything I can do to help, let me know. (I'm sorry for chiming in using English. I don't know Japanese.)

k-takata commented 5 years ago

@vtbassmatt Thank you for your comment. Currently we use AppVeyor in this project, and now we are investigating if we can use Azure Pipelines instead (or parallelly). We think that 10 parallel jobs for OSS projects is attractive, but using Azure Pipelines looks more difficult than AppVeyor.

vtbassmatt commented 5 years ago

I would be grateful to know which part is more difficult. We can make product changes or documentation improvements.

Azure Pipelines has a LOT of options and flexibility, which makes it challenging/overwhelming to get started with. Did you see our templates repository? That might be a good place to start - we have templates for most languages and platforms.

beru commented 5 years ago

@vtbassmatt

Azure Pipelines has a LOT of options and flexibility, which makes it challenging/overwhelming to get started with. Did you see our templates repository? That might be a good place to start - we have templates for most languages and platforms.

I don't see a template for Windows classic desktop application which would be of use to us.

m-tmatma commented 5 years ago

@vtbassmatt

We have the following issues to solve.

Do you have any user support repository for Azure Pipeline on GitHub like https://github.com/appveyor/ci ?

appveyor's document is very easy to understand. It explains what tools are installed on the build images, what environment variables are available, and how we write appveyor.yml.

beru commented 5 years ago

@vtbassmatt

I'd like to see pipelines-cplusplus sample app repository which isn't available yet.

I see 10 repositories on this query result page. https://github.com/MicrosoftDocs/?utf8=%E2%9C%93&q=pipeline&type=&language=

But I see only 5 programming languages listed on the documentation. https://docs.microsoft.com/en-us/azure/devops/pipelines/get-started-yaml?view=vsts#get-the-sample-code The other 5 missing repositories must be weeping...

beru commented 5 years ago

@vtbassmatt

Can you please provide a link to https://azure.microsoft.com/en-us/services/devops/pipelines/ on this page? I had to manually type the address which was quite challenging for me. The screenshots seem really helpful but due to the lack of the link, an average person might not start the procedure.

beru commented 5 years ago

@vtbassmatt

I didn't know the template named .NET Desktop can also be used for Windows classic desktop solutions. Disguised name!

image

vtbassmatt commented 5 years ago

Thanks for all the feedback!

Unfortunately, the one thing I haven't found a workaround for is changing the system locale. I don't believe you can reboot our hosted agents. We support private agents (which you can configure with any software or settings you want), but then you lose the value of having everything cloud-hosted.

I'm still looking into this, though, and maybe someone on my team will have a suggestion.

vtbassmatt commented 5 years ago

Oh - I missed two things.

m-tmatma commented 5 years ago

@vtbassmatt

When I tried Azure Pipelines, I was asked to log in by Microsoft Account.

If it is possible to log in by GitHub Account like appveyor, let me how. If not, please support it (including GitHub organization account).

I think most of GitHub users want to log in by GitHub accounts instead of Microsoft Accounts.

vtbassmatt commented 5 years ago

For now you have to use a Microsoft Account. We want to support GitHub identities but don't have a timeframe yet.

beru commented 5 years ago

@vtbassmatt I'm wondering if importers and migration guides from AppVeyor or Travis or Jenkins are available to us.

vtbassmatt commented 5 years ago

@beru we're writing some of that up. The person responsible for it is at a conference all week but I think he plans to get started next week.

Unfortunately we don't have a good workaround for needing to reboot the agents. It's been added to our backlog. If you're interested in still trying Azure Pipelines, you could set up self-hosted Windows agents that are already configured in ja-JP, so no reboot would be required.

m-tmatma commented 5 years ago

@vtbassmatt

Unfortunately we don't have a good workaround for needing to reboot the agents.

@berryzplus removed the necessity of rebooting agents at #482 by setting /execution-charset:shift_jis. So we don't need a workaround any more.

vtbassmatt commented 5 years ago

Oh, great! When I get a few minutes, I'll see if I can port over your Appveyor config.

vtbassmatt commented 5 years ago

I took a run at this, but I don't know enough about building C++ apps or this code in particular to be successful. There are lots of hardcoded references to AppVeyor which I couldn't easily extract.

I'm still happy to help by answering questions and getting you information, though!

m-tmatma commented 5 years ago

@vtbassmatt

There are lots of hardcoded references to AppVeyor which I couldn't easily extract.

We use some appveyor environment variables in https://www.appveyor.com/docs/environment-variables/

Do you have alternative variables on Azure Pipelines?

Note: we support building this software locally. So you can build it without these appveyor-specific variables for an experiment.

m-tmatma commented 5 years ago

GitHub 連携始めたみたい。(It seems that the integration between GitHub and Azure Pipelines has alredy started) https://github.com/marketplace/azure-pipelines https://blog.github.com/2018-09-10-azure-pipelines-now-available-in-github-marketplace/

m-tmatma commented 5 years ago

@vtbassmatt

For now you have to use a Microsoft Account. We want to support GitHub identities but don't have a timeframe yet.

Do you have the timeframe now?

vtbassmatt commented 5 years ago

Nothing to announce right now.

berryzplus commented 5 years ago

さっき Microsoft Azure のアカウント登録してきました。 https://azure.microsoft.com/ja-jp/features/azure-portal/ 個人だとSMS認証とクレジットカードの登録がいるんですね。 他で使うアテがない場合、アカウント登録の敷居は高いように思いました。

GitHubアカウントを連携して使えるように・・・というか、 organizationで共用できるようになるまでは少しかかりそうかな？と思っています。

そろそろ少しずつ試験運用をしてみてもいいかな～と思い始めました。 個人アカウントを用意できる人で「色々やってみる」を試験運用と言ってます。 結果は できるだけここに書く というスタンスでまずはやってみようってことです。

@vtbassmatt

Nothing to announce right now.

thank you for your reply. we want to use the Azule-pipeline as a user-group named SAKURA editor organization on GitHub. we would like to start trial anyway, to learn about that on a personal basis.

KageShiron commented 5 years ago

Azure Pipelinesを個人的なプロジェクトで試して見ましたが、案外簡単そうです。azure-pipelines.ymlというファイルを適当な場所に置いて、環境設定やステップを順に並べてけばOKでした。

  1. Batch Script taskをいくつか並べる
  2. CopyFilesでArtifactsフォルダにコピー
  3. PublishBuildArtifactsでZIP作成

みたいに並べれば良さそう。GitHub Releaseタスクとか、Extract Filesタスク、サードパーティマーケットからChocolateyタスクなどもあるので、必要に応じて使えそうです。

↓個人的なプロジェクトのビルドの一部。build.shでsrc→distにビルドしてます。

trigger:
  - master
pool:
  vmImage: 'ubuntu-16.04'
steps:
- task: ShellScript@2
  inputs:
    scriptPath: build.sh
- task: CopyFiles@2
  inputs:
    contents: |
      dist/**
      src/**
    targetFolder: $(Build.ArtifactStagingDirectory)
- task: PublishBuildArtifacts@1
  inputs:
    pathtoPublish: $(Build.ArtifactStagingDirectory)
    artifactName: MyBuildOutputs
m-tmatma commented 5 years ago

Predefined build variables

* AppVeyorと同質の情報は得られそう

* https://docs.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=vsts
appveyor azure pipelines comment
APPVEYOR_ACCOUNT_NAME ?
APPVEYOR_BUILD_NUMBER BUILD_BUILDID
APPVEYOR_REPO_COMMIT BUILD_SOURCEVERSION can be obtained by git show -s --format=%H
APPVEYOR_REPO_TAG_NAME ?
APPVEYOR_URL SYSTEM_TASKDEFINITIONSURI
APPVEYOR_BUILD_VERSION BUILD_BUILDNUMBER https://docs.microsoft.com/ja-jp/azure/devops/pipelines/build/options?view=azure-devops&tabs=yaml
APPVEYOR_PROJECT_SLUG SYSTEM_TEAMPROJECT
APPVEYOR_PULL_REQUEST_HEAD_REPO_BRANCH SYSTEM_PULLREQUEST_SOURCEBRANCH
APPVEYOR_PULL_REQUEST_NUMBER SYSTEM_PULLREQUEST_PULLREQUESTNUMBER
APPVEYOR_PULL_REQUEST_HEAD_COMMIT SYSTEM_PULLREQUEST_SOURCECOMMITID
APPVEYOR_REPO_NAME BUILD_REPOSITORY_NAME
APPVEYOR_REPO_PROVIDER BUILD_REPOSITORY_PROVIDER
N/A BUILD_REPOSITORY_URI repository URL

https://www.appveyor.com/docs/environment-variables/ https://docs.microsoft.com/ja-jp/azure/devops/pipelines/build/variables?tabs=yaml&view=azure-devops#predefined-variables

m-tmatma commented 5 years ago

fixed BUILD_SOURCEVERSION and SYSTEM_PULLREQUEST_SOURCECOMMITID in the above table.

m-tmatma commented 5 years ago

@vtbassmatt

I enteroounterd the limition of free users. Is there any way to remove the restriction for open source organization?

https://docs.microsoft.com/ja-jp/azure/devops/organizations/billing/buy-basic-access-add-users?view=azure-devops&viewFallbackFrom=vsts

In this quickstart, you learn how to pay for more users in your organization. You only need to pay for users when your team size exceeds the free limits. It's free to add users who have a Visual Studio subscription. You also get five free additional users in your organization.

m-tmatma commented 5 years ago

azure pipelines project for this project is https://dev.azure.com/sakuraeditor/sakura/_build?definitionId=3

vtbassmatt commented 5 years ago

For open source, Pipelines-only usage, you don't need to pay for Basic users. You can use Stakeholder users, which are free and have full access to both Pipelines and administrative functions. Sorry for the confusion.

m-tmatma commented 5 years ago

@vtbassmatt

Thank you for the reply and the outstanding CI service for free.

m-tmatma commented 5 years ago

@vtbassmatt

When I tried Azure Pipelines, I was asked to log in by Microsoft Account.

If it is possible to log in by GitHub Account like appveyor, let me how.

I confirmed that log in by GitHub Account is supported now Thanks, I really appreciate it.

m-tmatma commented 5 years ago

@vtbassmatt

I found the way to rebuild a PR by commenting special messages. https://docs.microsoft.com/en-us/azure/devops/pipelines/repos/github?view=azure-devops&tabs=yaml#comment-triggers

But it is not easier than Appveyor. Users can trigger a PR by clicking just "RE-BUILD PR" link on Appveyor.

Is there similar way on Azure Pipelines?

re-build-PR
vtbassmatt commented 5 years ago

Once you're using the Azure Pipelines app, there is a "rebuild" button right on the Checks tab. I'm on mobile so I can't share a screenshot easily.

m-tmatma commented 5 years ago

@vtbassmatt

Thank you for the replay. You mean this?

re-build-PR-azure
vtbassmatt commented 5 years ago

That's it!

m-tmatma commented 5 years ago

@vtbassmatt

Unfortunately, the one thing I haven't found a workaround for is changing the system locale. I don't believe you can reboot our hosted agents. We support private agents (which you can configure with any software or settings you want), but then you lose the value of having everything cloud-hosted.

@berryzplus removed the necessity of rebooting agents at #482 by setting /execution-charset:shift_jis. So we don't need a workaround any more.

We have thought that we don't need to change system locale by using /execution-charset:shift_jis option. But it is not valid for compiling HTML Help.

Compiling HTML Help on system locale (en-us) has two problems.

  1. The first problem is that it sometimes fails. (#906)
  2. The second problem is that indexes of generated CHM files are garbled. (#922)

Is there any way to compile HTML Help on system locale (en-us) or to change system locate to ja-jp?

vtbassmatt commented 5 years ago

I don't think there's an HTML Help compiler that doesn't depend on system locale, and it's not possible to set system locale on the hosted agents.

Maybe something like Locale Emulator could help?