Pytorch feedstock changes for s390x

[manogy]

Checklist before submitting

• [x] Did you read the contributor guide?
• [x] Did you update any affected documentation?
• [x] Did you write any tests to validate this change?

Description

As part of pytorch conda package build enablement through open-ce, we are proposing the change in dependencies for s390x. We are not building the torchvision through conda hence putting a conditional for torchvision-base

associated PR : https://github.com/open-ce/open-ce/pull/781. -->open-ce https://github.com/open-ce/numactl-feedstock/pull/22 --> numactl

Fixes # (issue).

Review process to land

1. All tests and other checks must succeed.
2. At least one maintainer must review and approve.
3. If any maintainer requests changes, they must be addressed.