Rebase release-0.4 to pick up release fix

[imjasonh]

Fixes #566 with #567 and also picks up #564 which is just test-infra updates.

[knative-prow-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ImJasonH, nbarthwal

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/knative/build/blob/release-0.4/OWNERS)~~ [ImJasonH] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment

[nbarthwal]

/lgtm