Versions of other folders as environment variables to current build

[nirradi]

I find it necessary to know the name of the artifacts of dependencies when building a folder.

for example: It could be convenient to use MBT_MODULE_VERSION for tagging all the artifacts, but when module B is built after module A, the build script no longer has access to MBT_MODULE_VERSION of module A.

The best plan I had was using something like

mbt describe branch $(git rev-parse --abbrev-ref HEAD) --json | jq .$1.Version | sed 's/"//g'

which uses jq to parse the json response of the current versions in this git branch.

What ideas for workarounds do you have? It would be convenient having environment variables like MBT_{MODULE_NAME}_VERSION for all the projects.

[buddhike]

Sounds like a reasonable feature request. You are on the right track with describe, however That still does not give you an easy way to identify deps of a given module.

Are you able to share a bit more about your use case?

Thanks

[nirradi]

I'm using docker to build artifacts, the image name (artifact) can simply be image_name:${MBT_MODULE_VERSION}, that works fine.

for a simple example, module parrot depends on module birb. When I change something in birb then running mbt build will trigger a build command in birb and a docker image named birb:somelonghash is created

since I've correctly added birb as a dependency in the parraot/.mbt.yml - the docker builld command is run in the parrot directory. Ideally I want the docker to start my parrot build from the bird image so parrot/Dockerfile needs to look something like this

FROM _birb_:somelonghash
RUN ...
CP ....

but clearly I can't hardcode the long hash into the file since I'll have to change that file each time I commit a change to birb, so I need to do something like

FROM _birb_:{CURRENT_BIRB_VERSION}

and instead of simply running docker-compose build as my build script, I need to write a bash script:

CURRENT_BIRB_VERSION=$(mbt describe branch $(git rev-parse --abbrev-ref HEAD) --json | jq ._birb_.Version | sed 's/"//g'
)

docker-compose build

If I had access to MBT_BIRB_VERSION as an evironment variable always, I could have left

name: _parrot_
build: 
  default:
    cmd: 'docker-compose'
    args: ['build']

instead of adding a ./build.sh script to each folder that prepares the environment variable from the mbt describe command.

[buddhike]

Thanks for providing more context @nirradi.

You should be able to create the environment your build needs before you execute mbt build with templating capabilities. To do that, create a template to provision all module versions in the environment

{{- range $mod := .Modules -}}
export MBT_{{$mod.Name}}_VERSION="{{$mod.Version}}"
{{end}}

Then you run mbt apply command to produce a file that you can source before you run mbt build. This would save you from writing wrapper script in each module directory.

e.g.

mbt apply commit <sha> --to .env.tmpl --out build/.env
#ensure build directory is listed in .gitignore file
source build/.env
mbt build commit <sha>

You might also be able to use the same templating technique to generate docker-compose file on the fly. However, without intimate knowledge about what your pipeline, I cannot think of exact steps 😉.

[nirradi]

Thanks!

although this doesn't save me from calling source ... before calling mbt, it does alleviates the need to use jq for parsing the describe json so this is definitely an improvement.

I hope I might have some time to add it to /lib/process_manager.go in the near future, and PR you - if that's something you see going in permanently.

On Fri, Apr 26, 2019 at 4:22 PM Buddhike de Silva notifications@github.com wrote:

Thanks for providing more context @nirradi https://github.com/nirradi.

You should be able to create the environment your build needs before you execute mbt build with templating capabilities. To do that, create a template to provision all module versions in the environment

{{- range $mod := .Modules -}}

export MBT_{{$mod.Name}}_VERSION="{{$mod.Version}}"

{{end}}

Then you run mbt apply command to produce a file that you can source before you run mbt build. This would save you from writing wrapper script in each module directory.

e.g.

mbt apply commit --to .env.tmpl --out build/.env

ensure build directory is listed in .gitignore file

source build/.env

mbt build commit

You might also be able to use the same templating technique to generate docker-compose file on the fly. However, without intimate knowledge about what your pipeline, I cannot think of exact steps 😉.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mbtproject/mbt/issues/119#issuecomment-487054880, or mute the thread https://github.com/notifications/unsubscribe-auth/AHNV2KDOT3QWQW674UNU72DPSL625ANCNFSM4HIAWQTA .