Add FCOS to AWS Marketplace

[bgilbert]

@davdunc has proposed a mechanism for listing FCOS in AWS Marketplace, in which AWS directly manages the Marketplace listing on Fedora's behalf via automation driven from fedmsg. See that ticket for details. This ticket tracks FCOS-specific work; the details below are from a conversation with @davdunc.

AWS infrastructure would listen on fedmsg for new release AMIs, which would be automatically cloned to an AWS internal account and processed through the Marketplace pipeline. We'd need to implement https://github.com/coreos/fedora-coreos-tracker/issues/225, and decide whether Marketplace publication should happen at the release commit point or when we actually update stream metadata. Note that Marketplace publication latency is measured in hours, because there's an automated security scan.

Marketplace AMIs would be published to every region, including some we currently don't have access to (GovCloud). The Marketplace AMI for each region would have a distinct ID, as with the community AMIs, and could be launched directly by users without clicking through a registration page first (since there's no EULA or extra fee). Thus it'd make sense to incorporate Marketplace AMI IDs into stream metadata, in a separate section from community AMIs. We'd want AWS infrastructure to emit a fedmsg when new Marketplace AMIs are published; we'd listen for that message and automatically PR stream metadata. Because of the publication delay and the indirection through Amazon's pipeline, we'd still recommend that users stick with the community AMIs if possible. But including Marketplace AMIs in stream metadata would help users in additional regions, or with additional compliance requirements, to use FCOS on AWS.

One other issue: rollbacks. If we roll back our AWS images to a previous release via stream metadata, we'd want AWS to update our Marketplace listings accordingly. #225 already includes a "stream metadata synced" fedmsg which AWS could use to intercept this case. However, rollbacks should be an infrequent occurrence, and IMO it'd be okay to instead have a rollback SOP that includes a manual ticket filing.

[dustymabe]

We discussed this in last week's meeting.

There was no objection to listing our images in AWS marketplace. Mostly just some questions about logistics.

[bgilbert]

Lightly-edited IRC log:

<bgilbert> davdunc: let us know what help you need from us re FCOS in AWS Marketplace.  I believe we'll
           need to add some fedmsgs at least.
<davdunc> yea. I need to report back the detail in a consumable way.
<bgilbert> +1, and we'll need to report publication events
<bgilbert> for you to consume
<davdunc> I can publish it to a stream or an sns notification that  we can subscribe
<bgilbert> do you have write access to fedmsg?
<davdunc> bgilbert: I can only really publish stable in the MP because of their policies.
<davdunc> I don't have it now bgilbert, but I can provide the code to someone who does.
<bgilbert> re stable, not completely surprised, but thanks for checking
<davdunc> yea. I'll make sure the other streams are referenced in the product listings for bravehearts.
<davdunc> I can consume the stream though now, so that's going into the pipeline and, as soon as I figure
           out what I am doing wrong in pushing to my own fork on pagure, I'll set up the details in
           the cloud-sig directory that King_InuYasha made for me.
<bgilbert> davdunc: you're okay polling stream metadata?  or would you prefer a fedmsg?
<davdunc> fedmsg is better.
<davdunc> but the stream is good.
<bgilbert> cool cool, we'll see if we can get some eyes on this

[bgilbert]

In summary, I think there are three parts to this, in priority order:

1. fedmsg notification that AWS should use to ingest a build. This might be at the FCOS release commit point or when the stream metadata is updated; we'll need to decide.
2. Some mechanism to listen on e.g. an SNS topic for the resulting Marketplace AMI IDs, and feed them back into FCOS release metadata and/or (via a PR) stream metadata, after the fact. If we're feeling ambitious, we could rebroadcast the info as a fedmsg.
3. Some mechanism (a fedmsg for stream metadata updates?) for AWS to detect an image rollback and update the Marketplace listing accordingly.

1 is important (or else AWS will need to poll stream metadata), 2 is nice to have (or else we can just link the Marketplace listing from the download page), 3 is optional (or else we can ping davdunc if we ever actually have to roll back).

[jlebon]

1. fedmsg notification that AWS should use to ingest a build. This might be at the FCOS release commit point or when the stream metadata is updated; we'll need to decide.

This part is done in https://github.com/coreos/fedora-coreos-pipeline/pull/288. That patch actually adds msgs for both the release point, and the metadata update point. Though for the purposes here, the metadata update point seems more appropriate because that's also when the website gets updated, and it'd be good for the MP listing and the getfedora.org AMI to be for the same version.

And actually if the AWS side keys off of just that, I think that addresses 1 and 3, right? I.e. every time the stream metadata changes, it makes sure that the MP listing matches the metadata.

[bgilbert]

I could see it going either way because of pipeline latency on the AWS side, but the metadata update point is easier to reason about when creating a release. :+1:

For point 3, I believe the metadata update message should be sufficient from our side. The AWS side would need to carry logic to detect whether to process a new release image or roll back to an existing one.

[jlebon]

1. fedmsg notification that AWS should use to ingest a build. This might be at the FCOS release commit point or when the stream metadata is updated; we'll need to decide.

Just to note, this is done now. Here is an example fedmsg emitted when stream metadata is updated:

https://apps.fedoraproject.org/datagrepper/id?id=2020-b127ae0b-6833-46d4-ab9c-de31499752d1&is_raw=true&size=extra-large

Copying relevant snippet for posterity:

{
  "topic": "org.fedoraproject.prod.coreos.stream.metadata.update", 
  "headers": {}, 
  "msg": {
    "stream": "next"
  }
}

/cc @davdunc