microsoft / AL-Go

The plug-and-play DevOps solution for Business Central app development on GitHub
MIT License
293 stars 125 forks source link

Multi AppSource Apps in one repo. #328

Closed jonaswre closed 1 year ago

jonaswre commented 1 year ago

Im trying to publish multiple apps in one repo to appsource.

You can only enter one productId per .AL-Go/settings.json. So I tried to make the repo multi project but now I get the following error.

Error: RunPipeline action failed. Error: For AppSource Apps with AppSourceCop enabled, you need to specify AppSourceCopMandatoryAffixes in .AL-Go\settings.json Stacktrace: at AnalyzeRepo, D:\a\_actions\microsoft\AL-Go-Actions\v2.2\AL-Go-Helper.ps1: line 539 at <ScriptBlock>, D:\a\_actions\microsoft\AL-Go-Actions\v2.2\RunPipeline\RunPipeline.ps1: line 79 at <ScriptBlock>, D:\a\_temp\d965a261-a9b2-4f1c-91d7-c20ad8eabeb2.ps1: line 1 at <ScriptBlock>, <No file>: line 1

https://github.com/microsoft/AL-Go-Actions/blob/53470cc1bf0524559a158d90b093170d7fb3ec76/AL-Go-Helper.ps1#L539

It seems like the analyse-repo part doesn't respect multi project AppSource Repos.

I'm doing something wrong here?

jonaswre commented 1 year ago

I found a solution

But this feels like cheating and this solution drasticly increases the build time.

Whats the reasoning to only allow one appsource app per project?

freddydk commented 1 year ago

One project is one shippable entity, you can have multiple apps in one project, but they would be under one appsource app.

Seems like a bug that you cannot have a multi-project repo with 2 appsource app projects - the above "solution" makes the root into a third project - and is likely not what you want.

jonaswre commented 1 year ago

If you set appFolders to an empty array there seems to be no harm

Is there anyway to eliminate wasted build time here? Maybe we can cache container images if all projects have the same requirements?

Because if you capsulate your code you will often create bridge apps, which are independently deployable, but are still tightly coupled.

jonaswre commented 1 year ago

There is no "good" way to get dependencies between projects within the same run.

The following option is problematic

  "appDependencyProbingPaths":  [
        {
            "repo":  "https://github.com/<org>/<repo>",
            "release_status":  "latestBuild",
            "projects":  "*"
        }
    ]

because this depends on the build to be successful before it can be used. So this will use the artifacts from the prevoius run and not the latest.

After release this might be okay. But during development you will often make changes in both projects. Specially with bridge apps, they are both shippable units but still tightly coupled.

So the only solution I see here is to create a repo for each app but this makes working with lots of apps very difficult. For us this would currently mean 30 additional repos.

freddydk commented 1 year ago

There are two solutions to this.

  1. Set "UseProjectDependencies" to true in .github/AL-Go-Settings.json and run update AL-Go System files.
  2. Use release_status = include in your probing paths.

The first one will turn your build pipelines into multi-stage pipelines. The second one will build the dependent apps together with the depending app.

The "right" solution is #1, but this one causes build time to shoot up. Over time (with up-coming performance improvements) this should go away again. My thinking is that I can build apps in all stages and run all tests in the latest stage, to avoid creating containers for all stages.

jonaswre commented 1 year ago

Thanks, I'll give it a try. I think both of these setting are missing in the documentation right? I looked at the following part. https://github.com/microsoft/AL-Go/blob/main/Scenarios/settings.md

freddydk commented 1 year ago

will add scenarios for these

jonaswre commented 1 year ago

The build worked correctly, but I now get this error when delivering to nuget.

D:\a\_actions\microsoft\AL-Go-Actions\v2.2\Deliver
Downloading BcContainerHelper latest version from CDN
BcContainerHelper version 4.0.7
BcContainerHelper emits usage statistics telemetry to Microsoft
Reading .github\AL-Go-Settings.json
Reading .AL-Go\settings.json
Artifacts D:\a\Project\Project\.artifacts
Projects:
Project1
Project2
.
Using GitHubPackagesContext
Project 'Project1'

    Directory: D:\a\Project\Project\.artifacts

Mode                 LastWriteTime         Length Name                                                                 
----                 -------------         ------ ----                                                                 
d-----          1/2/2023   5:58 PM                Project1-main-Apps-1.0.44.0                               
d-----          1/2/2023   5:58 PM                Project2-main-Apps-1.0.44.0                               

Error: Deliver action failed. Error: Internal error - multiple Apps folders located Stacktrace: at <ScriptBlock>, D:\a\_actions\microsoft\AL-Go-Actions\v2.2\Deliver\Deliver.ps1: line 197 at <ScriptBlock>, D:\a\_actions\microsoft\AL-Go-Actions\v2.2\Deliver\Deliver.ps1: line [193](https://github.com/byndit/BeyondAttributes/actions/runs/3824202039/jobs/6506328816#step:7:195) at <ScriptBlock>, D:\a\_actions\microsoft\AL-Go-Actions\v2.2\Deliver\Deliver.ps1: line 72 at <ScriptBlock>, D:\a\_temp\355272af-02c0-4b9e-8f05-ab384f03287e.ps1: line 1 at <ScriptBlock>, <No file>: line 1

I'm not sure if this is still a bug in the nuget delivery or if this is caused by having the third project in base.

freddydk commented 1 year ago

I will check this.

freddydk commented 1 year ago

Looking at the first error first - created a multi-project repo with 2 appsource apps - I cannot repro the first issue. When AnalyzeRepo is called - settings is repoSettings AND project setting for the project being built. If you can show me a repo, which reproes the first issue, I would be very interested to see when this happens - thanks.

I can repro the issue with deliver to NuGet failing and will look at that.

freddydk commented 1 year ago

There is a fix for this issue on https://github.com/freddydk/AL-Go-AppSource@issue328 - if you want to try it. I will run this through end 2 end testing and some more tests.

jonaswre commented 1 year ago

Thanks, the nuget issue is fixed.

I've created a repo that reproduces the "error".

But I now understand why this is happening. If you use multi project repos you need to run "Updated System files" if you add or remove projects. But this can't be done using a pr. Because the workflow will run on the old workflow which might have had to change.

https://github.com/byndit/AL-Go-Issue138/pull/2/checks?check_run_id=10518386337

But this is probably just something you need to know. Because I'm not sure whats the solution for this.

freddydk commented 1 year ago

Correct - if you use UseProjectDependencies - you need to run Update AL-Go System Files if you change the dependency depth (with the workflow will tell you). Just adding/removing projects should not require you to run the update workflow. The error you have now is actually a new error in my issue328 branch - any chance you can give me write access to your repo - then I will add a fix directly

freddydk commented 1 year ago

Wrong - the error is not new, the consequence is new. You should be able to re-run the jobs in the PR workflow now - I have fixed the bug in actions.

jonaswre commented 1 year ago

It works now. Feel free to close the issue once this is merged into al-go. I'll then delete the repo again.