Using xdist to run pytest in parallel

[unkcpz]

Running tests in parallel using pytest-xdist reduce the time of ci-codes (2 cores).

• ci-code / tests (3.9): ~28m -> ~15m

• ci-code / tests (3.12): ~20m -> 12m

• ci-code / presto: ~11m -> 6m

• [x] Fix all failed tests, see if it possible.

• [x] Bring the inconsistent test back to run.

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 77.91%. Comparing base (ef60b66) to head (9f591fe). Report is 146 commits behind head on main.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #6620 +/- ## ========================================== + Coverage 77.51% 77.91% +0.41% ========================================== Files 560 567 +7 Lines 41444 42178 +734 ========================================== + Hits 32120 32859 +739 + Misses 9324 9319 -5 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[unkcpz]

Run with my 16 cores, I found some issues that storage is accessed at the same time. The problem is caused in tests/tools/archive/test_simple.py::test_base_data_nodes didn't clean the storage before the test that cause the shared storage has unsealed nodes.

• [x] The restapi tests failed randomly because the test open restapi server using port 5000 at the same time. (fixed by https://github.com/aiidateam/aiida-core/pull/6620/commits/3bf2c7210b57b8e8fc9456edf15e04a30cfb656b)

• [ ] The following one seems be able to be solved by https://github.com/aiidateam/aiida-core/pull/6610

    def test_load_backend_if_not_loaded_load_once(manager, monkeypatch):
        """Test :meth:`aiida.cmdline.utils.decorators.load_backend_if_not_loaded` calls ``get_profile_storage`` once."""
        mocked = mock.Mock()

        # This test assumes the ``load_backend_if_not_loaded`` uses ``get_profile_storage`` to load the profile, so we need
        # to first check that this is the case. If this changes, this first test will fail alerting that it needs to be
        # adapted.
        with monkeypatch.context() as context:
            context.setattr(manager.__class__, 'get_profile_storage', mocked)
            load_backend_if_not_loaded()
>           assert mocked.call_count == 1
E           AssertionError: assert 0 == 1
E            +  where 0 = <Mock id='125376138318368'>.call_count

tests/cmdline/utils/test_decorators.py:82: AssertionError

• [ ] transport/test_all_plugins.py should use independent tmp path. It is addressed in https://github.com/aiidateam/aiida-core/pull/6627

The typical error of not using tmp_path fixture is:

FAILED tests/transports/test_all_plugins.py::test_put_get_empty_string_file[core.ssh] - OSError: Error during mkdir of 'tmp_try' from folder '/tmp', maybe you don't have the permissions to do it, or the dire
ctory already exists? (Failure)

More tests randomly failed, should be fixed

ERROR tests/tools/archive/orm/test_groups.py::test_nodes_in_group - sqlalchemy.orm.exc.DetachedInstanceError: Instance <DbUser at 0x73ad11c478f0> is not bound to a Session; attribute refresh operation cannot
 proceed (Background on this error at: https://sqlalche.me/e/20...
ERROR tests/cmdline/commands/test_status.py::test_status_no_profile - TimeoutError: disconnect after 10 seconds
FAILED tests/cmdline/commands/test_rabbitmq.py::test_revive - AssertionError: Traceback (most recent call last):
FAILED tests/orm/test_querybuilder.py::TestBasic::test_tuples - AssertionError: assert 11 == 1
FAILED tests/tools/archive/orm/test_authinfo.py::test_create_all_no_authinfo - aiida.tools.archive.exceptions.ExportValidationError: All ProcessNodes must be sealed before they can be exported. Node(s) with PK(s): 61 is/are not se
aled.
FAILED tests/tools/archive/orm/test_authinfo.py::test_create_all_with_authinfo - aiida.tools.archive.exceptions.ExportValidationError: All ProcessNodes must be sealed before they can be exported. Node(s) with PK(s): 61 is/are not 
sealed.