TAS: add e2e tests for rank ordering for Job

[mimowo]

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

Which issue(s) this PR fixes:

Part of #3450 Part of #3533

Special notes for your reviewer:

• I was also thinking about an alternative assert that rack(pod0)==rack(pod1) and rack(pod2)==rack(pod3), but I think it will be harder to debug when the assert fails. With the exact assignment check we can easily see the full assignment.
• I have confirmed locally that when not using TAS the distributions of pods is completely random (I commented out setting the TopologyName and TAS PodSet annotation), then got:

  [FAILED] Expected object to be comparable, diff:   map[string]string{
  -     "0": "kind-worker",
  +     "0": "kind-worker5",
  -     "1": "kind-worker2",
  +     "1": "kind-worker7",
        "2": "kind-worker3",
        "3": "kind-worker4",
    }

I ran it a 4 times and it failed all repeats.

Does this PR introduce a user-facing change?

NONE

[netlify[bot]]

✅ Deploy Preview for kubernetes-sigs-kueue canceled.

Name	Link
🔨 Latest commit	985b955410c864d405428a25176037f85ec5547b
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-sigs-kueue/deploys/673b81cb9fcc4c000804500c

[mimowo]

/assign @PBundyra cc @tenzen-y

[mimowo]

/test pull-kueue-test-tas-e2e-main

[PBundyra]

LGTM, good job! I'll hold for @tenzen-y to take a look /hold /lgtm

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: d52ed29323337d92ac431c26372b19e1b6982709

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mimowo, tenzen-y

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[test/OWNERS](https://github.com/kubernetes-sigs/kueue/blob/main/test/OWNERS)~~ [mimowo,tenzen-y] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment

[tenzen-y]

/hold cancel

[mimowo]

Do we have integration testing for this situation?

no, but we have unit testing which is quite useful here, because it is relatively high-level (for the exported function), which makes it similar to TestSchedule which proved to be very useful over time.

So, personally I'm using unit tests for debugging, the e2e tests are useful to confirm the feature works.

I considered adding integration tests, but they are actually tricky, because TopologyUngater operates on pods, but we don't enable the k8s Job controller in the integaration tests (no kube-controller-manager, only API server). So, we would need to create the pods ourselves, just like in unit tests, so I don't see much gain.