search

fleetdm / fleet

Open-source platform for IT, security, and infrastructure teams. (Linux, macOS, Chrome, Windows, cloud, data center)
https://fleetdm.com
Other
2.7k stars 386 forks source link

Create and publish a Docker image with rcodesign + transporter #7052

Open roperzh opened 1 year ago

roperzh commented 1 year ago

During the 4.34 release there was a bug in rcodesign that prevented Fleet from releasing the version. This bug blocked the release until it was fixed by the maintainer of rcodesign.

Goal

User story
As a contributor,
I want to pin a Docker image to a specific version of rcodesign
so that the Fleet release works every time

To sign and notarize pkg files, we introduced a dependency that has to be compiled, this has been problematic because:

  1. Compilation fails on M1 macs, so the fleetdm/fleetctl image can't be build locally on them
  2. Compilation is slow, adding additional, unnecessary time to CI
  3. Code duplication between the fleetdm/fleetctl and the infra Dockerfiles

It would be good if we we could pre-compile and download the necessary dependencies.

How?

Create and publish a new Docker image that contains exactly this:

https://github.com/fleetdm/fleet/blob/7cb54728a3e9a059b0cda1fff6ae1ab799924d40/tools/fleetctl-docker/Dockerfile#L1-L7

Use COPY --from to copy from the prebuilt image in:

https://github.com/fleetdm/fleet/blob/7cb54728a3e9a059b0cda1fff6ae1ab799924d40/tools/fleetctl-docker/Dockerfile#L20-L21

And also in infrastructure/sandbox/PreProvisioner/lambda/Dockerfile

roperzh commented 1 year ago

cc: @lucasmrod @zwinnerman-fleetdm does this makes sense to you folks?

lucasmrod commented 1 year ago

The optimization does make sense.

I haven't been involved in the development/testing so I don't know how urgent this is. If devs using M1 cannot develop+build the image locally then it does make sense to solve this for that one reason.

Maybe next person using M1 to develop changes to fleetctl package can tackle this? :)

roperzh commented 1 year ago

I haven't been involved in the development/testing so I don't know how urgent this is. If devs using M1 cannot develop+build the image locally then it does make sense to solve this for that one reason.

indeed they can't, but unless you're tweaking some functionality for fleetctl package --native-tooling you don't need to deal with this (as you can use a pre-built image).

Maybe next person using M1 to develop changes to fleetctl package can tackle this? :)

sounds good!

roperzh commented 1 year ago

leaving a note for future selves: we had a release blocking problem that could have been prevented if we had a pre-built image in place.

One of the Rust dependencies for apple-codesign mistakenly published a broken version, which prevented the image from building (example run).

Fortunately the maintainer was very responsive and they fixed the problem right away, but this is a point of failure in our release process.

lukeheath commented 1 year ago

@roperzh Thanks for pointing this out. Because the issue this addresses blocked a release, it's worth taking another look at prioritizing this. Please work with @georgekarrv and @noahtalerman to move this through the engineering-initiated stories process?

georgekarrv commented 1 year ago

@noahtalerman if you see anything product want's to add to here to comment, otherwise you can unassign yourself and I will pull it through design review tomorrow

noahtalerman commented 1 year ago

Added to the feature fest doc here.

georgekarrv commented 1 year ago

Hey team! Please add your planning poker estimate with Zenhub @ghernandez345 @gillespi314 @marcosd4h @roperzh

noahtalerman commented 12 months ago

@roperzh this didn't make it in to the current sprint. Please bring this back to FF for prioritization if still desired.

noahtalerman commented 11 months ago

Hey @zhumo this didn't make it in to the current sprint. Please bring it back to FF for prioritization if still desired.

georgekarrv commented 2 weeks ago

@lukeheath @noahtalerman Looking at this and bringing it back to drafting to get pulled through the eng-initiated pipeline since I think it's still valid.

cc: @roperzh let me know if that doesn't seem to be the case

lukeheath commented 2 weeks ago

@georgekarrv This should get ~engineering-initiated but not :product label at first until it is prioritized to the drafting board. That said, I'm leaving it on because I think this is worth taking another look at when we have the capacity.

georgekarrv commented 1 week ago

Hey team! Please add your planning poker estimate with Zenhub @dantecatalfamo @gillespi314 @jahzielv @mna