Need to create comprehensive tests for zuul.ansible.action.*

[missaugustina]

See: https://storyboard.openstack.org/#!/story/2000878

All Zuul v3 Issues: https://storyboard.openstack.org/#!/board/41

Reviews: https://review.openstack.org/#/q/project:openstack-infra/zuul+branch:feature/zuulv3

Tasks:

• [ ] Use zuul's fake nodepool to create static nodes.
• [ ] Create job config suitable for multi-node testing.
• [ ] Run integration tests using static nodes.

[missaugustina]

Hey @eggshell when you have a review up in Gerrit for this, feel free to link it here and then move this issue to "Needs Review" so folks on the team can review it!

[eggshell]

I popped into #zuul today and asked some questions about how to get this done. The irc log is below:

<eggshell> anyone in here have opinions on https://storyboard.openstack.org/#!/story/2000878 ?
<mordred> eggshell: I'm sure we all have opinions - but they may or may not be _good_ opinions
<jeblair> eggshell: my first thoughts are "let's not go down the docker container path"
* mordred likes the multi-node integration test option
<eggshell> jeblair: thought that might be the case, and agree.
<mordred> since that gets us both testing of the action plugins _and_ testing of multi-node jobs in a real way
<jeblair> eggshell, mordred: yeah, i think multi-node sounds good.  we don't have a way of attaching a static node to zuul right now though...
<mordred> jeblair: do we need a static node?
<jeblair> well, we need a node
<jeblair> and i'm not super keen on having this depend on devstack
<mordred> golly no
<mordred> oh
<mordred> ok - I was asking the questions in more detail and as a result of typing I now understand the issue
<mordred> jeblair: maybe we should just made test-action-plugins blocked on support-static-node ?
<clarkb> could you preseed zookeeper with a record that says node (self on external IP) belongs to test box?
<jeblair> part of me thinks "maybe we can do this as a functional test and put 127.0.0.1 (maybe aliased) in the inventory" but i think that's sort of "begging the question" as far as testing goes.  since this is really designed to make sure we don't have any holes there, and that would greatly alter the very thing we're trying to test, yeah?
<jeblair> which is why the "static node" idea is there, right?
<mordred> yah
<jeblair> clarkb: i think that could work.  it'd be a small throwaway utility.  i guess the question is: is it worth it to write that now, or defer till we have nodepool static nodes as mordred suggests?
<eggshell> I was under the impression that testing the plugins locally was a no-go.
<mordred> yah - testing them locally is a no-go
<jeblair> eggshell: yeah, i think that's the case.  i'm mostly just refreshing my memory as to why.  :)
<eggshell> jeblair: cool, just wanna make sure I'm on the same page.
<jeblair> clarkb: similarly -- we could run these in a zuul functional test environment with a modified version of zuul's fake nodepool which returns the static node.
<jeblair> eggshell: so i think we've got 3 options for the multi-node job approach: 1) defer until zuul and/or nodepool support static nodes.  2) run with nodepool in fake mode but pre-seed the zookeeper data structure with the static node's info.  3) use zuul's fake nodepool and have it return the static node.
<eggshell> jeblair: 3) seems like the best approach today imo. Not sure what the time cost is on waiting for 1)
<jeblair> pabelanger: done
<pabelanger> jeblair: ty
<jeblair> eggshell: i think static node support is not a very high priority task right now, so i agree, 3 is probably simplest.

So I'll be researching how to implement this according to the direction Jim and Monty provided.

[eggshell]

I've broken this story up into three more digestible tasks. They are noted in the initial comment above.

[eggshell]

Initial review for first task: https://review.openstack.org/#/c/454826/