make more cuda-only tests device-agnostic

[faaany]

What does this PR do?

Below is the test results on XPU:

====================================== test session starts ======================================
platform linux -- Python 3.9.0, pytest-8.0.0, pluggy-1.5.0
rootdir: /home/fanli/workspace/tmp/accelerate
plugins: subtests-0.12.1, xdist-3.6.1
collected 9 items                                                                               

tests/test_big_modeling.py .                                                              [ 12%]
tests/test_examples.py .                                                                                        [ 25%]
tests/test_modeling_utils.py ......                                                                             [100%]

================================================== warnings summary ===================================================
../../../miniforge3/envs/acc-upstream/lib/python3.9/site-packages/intel_extension_for_pytorch/nn/utils/_weight_prepack.py:5
  /home/fanli/miniforge3/envs/acc-upstream/lib/python3.9/site-packages/intel_extension_for_pytorch/nn/utils/_weight_prepack.py:5: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================================================= PASSES ========================================================
=============================================== short test summary info ===============================================
PASSED tests/test_big_modeling.py::BigModelingTester::test_dispatch_model_move_model_warning
PASSED tests/test_examples.py::FeatureExamplesTests::test_ddp_comm_hook
PASSED tests/test_modeling_utils.py::ModelingUtilsTester::test_load_checkpoint_in_model_disk_offload
PASSED tests/test_modeling_utils.py::ModelingUtilsTester::test_load_checkpoint_in_model_one_gpu
PASSED tests/test_modeling_utils.py::ModelingUtilsTester::test_load_checkpoint_in_model_two_gpu
PASSED tests/test_modeling_utils.py::ModelingUtilsTester::test_set_module_tensor_to_cpu_and_gpu
PASSED tests/test_modeling_utils.py::ModelingUtilsTester::test_set_module_tensor_between_gpus
PASSED tests/test_modeling_utils.py::ModelingUtilsTester::test_set_module_tensor_to_meta_and_gpu
============================================ 8 passed, 1 warning in 16.90s ============================================

PASSED tests/test_examples.py::FeatureExamplesTests::test_distributed_inference_examples_phi2
PASSED tests/test_examples.py::FeatureExamplesTests::test_distributed_inference_examples_stable_diffusion
PASSED tests/test_accelerator.py::AcceleratorTester::test_get_state_dict_from_offload_use_pytorch
PASSED tests/test_accelerator.py::AcceleratorTester::test_get_state_dict_from_offload_use_safetensors

[yao-matrix]

OK for me.

[faaany]

@SunMarc as well as this one. Thx!

[faaany]

@BenjaminBossan

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[faaany]

In general, this change LGTM, thanks. Do we know if the affected tests would actually pass on all these devices?

Yes, all of them passed on XPU.

[faaany]

Thanks! Can you adjust the merge conflict then I can land this? :)

sure, merge conflict is resolved.