Run app build after another app build was completed

[kostyay]

Use case:

App called base, it builds a base docker image. App called base-child, it depends on base docker image and extends it. Ideally I want base-child to build after base finished. Is there a way to do it?

As far as I can tell Baur doesnt build projects in parallel? So I suppose I could call base 0_base so it will be build before base-child due to alphabetical order.

WDYT?

[fho]

Cool idea, building a base-child docker image automatically when it's base image base changed would be very handy. We never considered to use baur for that.

A way to represent that might to declare an output of one task as the input of another task. This adds tons of complexity though, dependencies between tasks needs to resolved, order needs to be calculated, dependency must loops must be detected etc. Finding a more simple solution would be better. :-) I was also thinking about supporting to specify a remote docker image as input for tasks. The base-child build task could define the docker image myrepo/base:latest as input. When baur is run it could check if the digest of the remote docker image changed, if yes it reruns the base-child build task. Because the execution of the tasks is not defined, it might be that baur ran the base-child before base and would not detect a change in the same baur run, only in the following execution.

As far as I can tell Baur doesnt build projects in parallel?

You are right, it does not run tasks in parallel.

So I suppose I could call base 0_base so it will be build before base-child due to alphabetical order.

Yes that would work, all pending tasks are executed in their alphabetical order. You still would need to ensure that baur runs the base-child task when base changed somehow. It should work if you always pin the installed program versions and reference BASE docker images by their sha256 checksum in it's Dockerfile, to ensure that base only changes if it's Dockerfile changes.

[kostyay]

We solved the base-child issue by defining base-child's include = ['../base/**', '**']. I just wanted to make sure it was built first, but since baur builds in alphabetical order as long as child step doesnt start with letter < than base it should be fine.

Ideally, however, it would be great to have support of dependency on apps (the whole complex thing you describe) + parallel build :)

[gedw99]

The dependency issue of one app build needing to than cause other app builds to occur could be made event based .

I have a similar mono repo project with 1.000 of apps inside and use nats Jetstream to do the event based work. When 1 app buildings , it can cause many other apps to build.

the first app “ producers” a message of its name. The other apps are “consumers” of the build event and subscribe based on the namespace of the repo path .

[fho]

Nice idea! This would be a gigantic architectural change for baur. It's is not planned to go into that direction.

I have a similar mono repo project with 1.000 of apps inside and use nats Jetstream to do the event based work. When 1 app buildings , it can cause many other apps to build.

If the project is open-source, could you drop a link?

[gedw99]

It’s closed , because it’s government project . Sorry about that . They will open it in stages as it passes security audits . SOC 2 Type1.

it’s very very easy to use nats serve to do what bar does though . The bar config files can work the same way the are now . The system just listens to nats and then carries out whatever operations are needed based on the bar config .

https://github.com/nats-io/nats-server

https://github.com/nats-io/nats.go Is he client .

https://github.com/nats-io/natscli Is the cli

https://github.com/nats-io/nkeys Is the auth system for adding , removing and revoking users and groups. Revocation and all others ops are real time !

nats is SOC 2 type 1, and so security audited.

it’s why I suggested it to remove the dependency of Postgres. Nats is like a reactive db and mesh , with full auth etc.

for me it’s nice because you can embed nats with any golang program ; and so can be embedded with bar too. In ci it will then be able to run itself . It can also run using in process and so not need a network at all - does millions of transactions a second. If you run it with a network connection - Now it’s got a reactive mesh integrated between many mono repos. That simple . So you can do reactive pipelines just like what bar is doing .

It can be embedded with bar. Then all mono repos can be joined up via nats acting as a mesh .

If you want I can help with bar and nats , ut it seems that your still wedded to postreql.

[fho]

It’s closed , because it’s government project . Sorry about that . They will open it in stages as it passes security audits . SOC 2 Type1.

Np, I would have been interested to check it out. :-)

If you want I can help with bar and nats , ut it seems that your still wedded to postreql.

Thanks for the offer, we currently have no plan to step away from PostgreSQL. It works well for our use case.