search

Azure / azure-cli

Azure Command-Line Interface
MIT License
3.92k stars 2.9k forks source link

CannotOverwriteExistingCassetteException #28283

Open mwesigwaguma opened 5 months ago

mwesigwaguma commented 5 months ago

Describe the bug

I am having this issue where running tests in live mode works well and the tests pass but in playback mode, it seems there are missing http requests in the recordings. Details below: Here is a link to an active Pull request's pipeline error: https://dev.azure.com/azclitools/public/_build/results?buildId=123946&view=logs&j=8b87d530-5472-5986-1c88-cfbbd7928469&t=11cc7245-08a2-5894-29fb-f5b6001226e6&l=1148

Related command

az sf cluster creat az sf cluster show sf cluster setting set az sf cluster node-type add ... Generally all servicefabric commands

Errors

except CannotOverwriteExistingCassetteException as ex: raise AssertionError(ex) AssertionError: Can't overwrite existing cassette ('/mnt/vss/_work/1/s/src/azure-cli/azure/cli/command_modules/servicefabric/tests/latest/recordings/test_update_settings_and_reliability.yaml') in your current record mode ('once'). No match for the request (<Request (GET) https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.KeyVault/vaults/clitestrg000001?api-version=2023-02-01>) was found. Found 3 similar requests with 1 different matcher(s) :

Issue script & Debug output

Test on modules: test_sf_cluster

The tests are set to run against current profile "latest" ================================================= test session starts ================================================= platform win32 -- Python 3.9.13, pytest-7.4.4, pluggy-1.4.0 -- E:\repos\azure-cli\env\Scripts\python.exe cachedir: .pytest_cache rootdir: e:\repos\azure-cli\src\azure-cli plugins: forked-1.6.0, xdist-3.5.0 8 workers [4 items] scheduling tests via LoadScheduling

src\azure-cli\azure\cli\command_modules\servicefabric\tests\latest\test_sf_cluster.py::ServiceFabricClusterTests::test_create_cluster_with_separate_kv src\azure-cli\azure\cli\command_modules\servicefabric\tests\latest\test_sf_cluster.py::ServiceFabricClusterTests::test_primary_nt_add_remove_node src\azure-cli\azure\cli\command_modules\servicefabric\tests\latest\test_sf_cluster.py::ServiceFabricClusterTests::test_update_settings_and_reliability src\azure-cli\azure\cli\command_modules\servicefabric\tests\latest\test_sf_cluster.py::ServiceFabricClusterTests::test_add_secondary_node_type_add_remove_node [gw2] [ 25%] SKIPPED src\azure-cli\azure\cli\command_modules\servicefabric\tests\latest\test_sf_cluster.py::ServiceFabricClusterTests::test_primary_nt_add_remove_node [gw0] [ 50%] SKIPPED src\azure-cli\azure\cli\command_modules\servicefabric\tests\latest\test_sf_cluster.py::ServiceFabricClusterTests::test_add_secondary_node_type_add_remove_node [gw3] [ 75%] FAILED src\azure-cli\azure\cli\command_modules\servicefabric\tests\latest\test_sf_cluster.py::ServiceFabricClusterTests::test_update_settings_and_reliability [gw1] [100%] PASSED src\azure-cli\azure\cli\command_modules\servicefabric\tests\latest\test_sf_cluster.py::ServiceFabricClusterTests::test_create_cluster_with_separate_kv

====================================================== FAILURES ======================================================= ___ ServiceFabricClusterTests.test_update_settings_and_reliability ____ [gw3] win32 -- Python 3.9.13 E:\repos\azure-cli\env\Scripts\python.exe

self = <azure.cli.testsdk.base.ExecutionResult object at 0x000001C9AB3B6490> cli_ctx = <azure.cli.core.mock.DummyCli object at 0x000001C9AB40E6D0> command = 'sf cluster create -g clitest.rg000001 -c sfrp-cli-000004 -l westus --certificate-subject-name sfrp-cli-000004 --vm-password "Pass123!@#" --cluster-size 5' expect_failure = False

def _in_process_execute(self, cli_ctx, command, expect_failure=False):
    from io import StringIO
    from vcr.errors import CannotOverwriteExistingCassetteException

    if command.startswith('az '):
        command = command[3:]

    stdout_buf = StringIO()
    logging_buf = StringIO()
    try:
        # issue: stderr cannot be redirect in this form, as a result some failure information
        # is lost when command fails.
      self.exit_code = cli_ctx.invoke(shlex.split(command), out_file=stdout_buf) or 0

src\azure-cli-testsdk\azure\cli\testsdk\base.py:301:

env\lib\site-packages\knack\cli.py:245: in invoke exit_code = self.exception_handler(ex) src\azure-cli-core\azure\cli\core__init__.py:127: in exception_handler return handle_exception(ex) src\azure-cli-testsdk\azure\cli\testsdk\patches.py:33: in _handle_main_exception raise ex env\lib\site-packages\knack\cli.py:233: in invoke cmd_result = self.invocation.execute(args) src\azure-cli-core\azure\cli\core\commands__init__.py:663: in execute raise ex src\azure-cli-core\azure\cli\core\commands__init__.py:726: in _run_jobs_serially results.append(self._run_job(expanded_arg, cmd_copy)) src\azure-cli-core\azure\cli\core\commands__init__.py:697: in _run_job result = cmd_copy(params) src\azure-cli-core\azure\cli\core\commands__init.py:333: in call__ return self.handler(args, kwargs) src\azure-cli-core\azure\cli\core\commands\command_operation.py:121: in handler return op(command_args) src\azure-cli\azure\cli\command_modules\servicefabric\custom.py:192: in new_cluster result = _create_certificate(cmd, src\azure-cli\azure\cli\command_modules\servicefabric\custom.py:1095: in _create_certificate vault = _safe_get_vault(cli_ctx, vault_resource_group_name, vault_name) src\azure-cli\azure\cli\command_modules\servicefabric\custom.py:1355: in _safe_get_vault vault = key_vault_client.get(resource_group_name, vault_name) env\lib\site-packages\azure\core\tracing\decorator.py:76: in wrapper_use_tracer return func(args, kwargs) env\lib\site-packages\azure\mgmt\keyvault\v2023_02_01\operations_vaults_operations.py:900: in get pipeline_response: PipelineResponse = self._client._pipeline.run( # pylint: disable=protected-access env\lib\site-packages\azure\core\pipeline_base.py:213: in run return first_node.send(pipeline_request) env\lib\site-packages\azure\core\pipeline_base.py:70: in send response = self.next.send(request) env\lib\site-packages\azure\core\pipeline_base.py:70: in send response = self.next.send(request) env\lib\site-packages\azure\core\pipeline_base.py:70: in send response = self.next.send(request) env\lib\site-packages\azure\core\pipeline_base.py:70: in send response = self.next.send(request) env\lib\site-packages\azure\core\pipeline_base.py:70: in send response = self.next.send(request) env\lib\site-packages\azure\mgmt\core\policies_base.py:47: in send response = self.next.send(request) env\lib\site-packages\azure\core\pipeline\policies_redirect.py:181: in send response = self.next.send(request) env\lib\site-packages\azure\core\pipeline\policies_retry.py:467: in send response = self.next.send(request) env\lib\site-packages\azure\core\pipeline\policies_authentication.py:115: in send response = self.next.send(request) env\lib\site-packages\azure\core\pipeline_base.py:70: in send response = self.next.send(request) env\lib\site-packages\azure\core\pipeline_base.py:70: in send response = self.next.send(request) env\lib\site-packages\azure\core\pipeline_base.py:70: in send response = self.next.send(request) env\lib\site-packages\azure\core\pipeline_base.py:70: in send response = self.next.send(request) env\lib\site-packages\azure\core\pipeline_base.py:70: in send response = self.next.send(request) env\lib\site-packages\azure\core\pipeline_base.py:108: in send self._sender.send(request.http_request, request.context.options), env\lib\site-packages\azure\core\pipeline\transport_requests_basic.py:338: in send response = self.session.request( # type: ignore env\lib\site-packages\requests\sessions.py:589: in request resp = self.send(prep, send_kwargs) env\lib\site-packages\requests\sessions.py:703: in send r = adapter.send(request, kwargs) env\lib\site-packages\requests\adapters.py:486: in send resp = conn.urlopen( env\lib\site-packages\urllib3\connectionpool.py:715: in urlopen httplib_response = self._make_request( env\lib\site-packages\urllib3\connectionpool.py:458: in _make_request httplib_response = conn.getresponse(buffering=True)

self = <vcr.patch.VCRRequestsHTTPSConnectione:\repos\azure-cli\src\azure-cli\azure\cli\command_modules\servicefabric\tests\latest\recordings\test_update_settings_andreliability.yaml object at 0x000001C9AD9A8CD0> = False, kwargs = {'buffering': True}

def getresponse(self, _=False, **kwargs):
    """Retrieve the response"""
    # Check to see if the cassette has a response for this request. If so,
    # then return it
    if self.cassette.can_play_response_for(self._vcr_request):
        log.info(f"Playing response for {self._vcr_request} from cassette")
        response = self.cassette.play_response(self._vcr_request)
        return VCRHTTPResponse(response)
    else:
        if self.cassette.write_protected and self.cassette.filter_request(self._vcr_request):
          raise CannotOverwriteExistingCassetteException(

cassette=self.cassette, failed_request=self._vcr_request, ) E vcr.errors.CannotOverwriteExistingCassetteException: Can't overwrite existing cassette ('e:\repos\azure-cli\src\azure-cli\azure\cli\command_modules\servicefabric\tests\latest\recordings\test_update_settings_and_reliability.yaml') in your current record mode ('once'). E No match for the request (<Request (GET) https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.KeyVault/vaults/clitestrg000001?api-version=2023-02-01>) was found. E Found 3 similar requests with 1 different matcher(s) : E E 1 - (<Request (GET) https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.KeyVault/vaults/clitestrg5b3ldlllqxmq?api-version=2023-02-01>). E Matchers succeeded : ['method', 'scheme', 'host', 'port', '_custom_request_query_matcher'] E Matchers failed : E path - assertion failure : E /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.KeyVault/vaults/clitestrg000001 != /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.KeyVault/vaults/clitestrg5b3ldlllqxmq E E 2 - (<Request (GET) https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.KeyVault/vaults/clitestrg5b3ldlllqxmq?api-version=2023-02-01>). E Matchers succeeded : ['method', 'scheme', 'host', 'port', '_custom_request_query_matcher'] E Matchers failed : E path - assertion failure : E /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.KeyVault/vaults/clitestrg000001 != /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.KeyVault/vaults/clitestrg5b3ldlllqxmq E E 3 - (<Request (GET) https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.KeyVault/vaults/clitestrg5b3ldlllqxmq?api-version=2023-02-01>). E Matchers succeeded : ['method', 'scheme', 'host', 'port', '_custom_request_query_matcher'] E Matchers failed : E path - assertion failure : E /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.KeyVault/vaults/clitestrg000001 != /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.KeyVault/vaults/clitestrg5b3ldlllqxmq

env\lib\site-packages\vcr\stubs__init__.py:263: CannotOverwriteExistingCassetteException

During handling of the above exception, another exception occurred:

self = 

@ResourceGroupPreparer()
def test_update_settings_and_reliability(self):
    self.kwargs.update({
        'kv_name': self.create_random_name('sfrp-cli-kv-', 24),
        'loc': 'westus',
        'cert_name': self.create_random_name('sfrp-cli-', 24),
        'cluster_name': self.create_random_name('sfrp-cli-', 24),
        'vm_password': "Pass123!@#",
        'primary_node_type': 'nt1vm',
        'new_node_type': 'nt2',
        'cluster_size': '5'
    })
  _create_cluster(self, self.kwargs)

src\azure-cli\azure\cli\command_modules\servicefabric\tests\latest\test_sf_cluster.py:44:

src\azure-cli\azure\cli\command_modules\servicefabric\tests\latest\test_util.py:50: in _create_cluster test.cmd('az sf cluster create -g {rg} -c {cluster_name} -l {loc} --certificate-subject-name {cluster_name} --vm-password "{vm_password}" --cluster-size {cluster_size}') src\azure-cli-testsdk\azure\cli\testsdk\base.py:175: in cmd return execute(self.cli_ctx, command, expect_failure=expect_failure).assert_with_checks(checks) src\azure-cli-testsdk\azure\cli\testsdk\base.py:250: in init self._in_process_execute(cli_ctx, command, expect_failure=expect_failure)

self = <azure.cli.testsdk.base.ExecutionResult object at 0x000001C9AB3B6490> cli_ctx = <azure.cli.core.mock.DummyCli object at 0x000001C9AB40E6D0> command = 'sf cluster create -g clitest.rg000001 -c sfrp-cli-000004 -l westus --certificate-subject-name sfrp-cli-000004 --vm-password "Pass123!@#" --cluster-size 5' expect_failure = False

def _in_process_execute(self, cli_ctx, command, expect_failure=False):
    from io import StringIO
    from vcr.errors import CannotOverwriteExistingCassetteException

    if command.startswith('az '):
        command = command[3:]

    stdout_buf = StringIO()
    logging_buf = StringIO()
    try:
        # issue: stderr cannot be redirect in this form, as a result some failure information
        # is lost when command fails.
        self.exit_code = cli_ctx.invoke(shlex.split(command), out_file=stdout_buf) or 0
        self.output = stdout_buf.getvalue()
        self.applog = logging_buf.getvalue()

    except CannotOverwriteExistingCassetteException as ex:
      raise AssertionError(ex)

E AssertionError: Can't overwrite existing cassette ('e:\repos\azure-cli\src\azure-cli\azure\cli\command_modules\servicefabric\tests\latest\recordings\test_update_settings_and_reliability.yaml') in your current record mode ('once'). E No match for the request (<Request (GET) https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.KeyVault/vaults/clitestrg000001?api-version=2023-02-01>) was found. E Found 3 similar requests with 1 different matcher(s) : E E 1 - (<Request (GET) https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.KeyVault/vaults/clitestrg5b3ldlllqxmq?api-version=2023-02-01>). E Matchers succeeded : ['method', 'scheme', 'host', 'port', '_custom_request_query_matcher'] E Matchers failed : E path - assertion failure : E /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.KeyVault/vaults/clitestrg000001 != /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.KeyVault/vaults/clitestrg5b3ldlllqxmq E E 2 - (<Request (GET) https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.KeyVault/vaults/clitestrg5b3ldlllqxmq?api-version=2023-02-01>). E Matchers succeeded : ['method', 'scheme', 'host', 'port', '_custom_request_query_matcher'] E Matchers failed : E path - assertion failure : E /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.KeyVault/vaults/clitestrg000001 != /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.KeyVault/vaults/clitestrg5b3ldlllqxmq E E 3 - (<Request (GET) https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.KeyVault/vaults/clitestrg5b3ldlllqxmq?api-version=2023-02-01>). E Matchers succeeded : ['method', 'scheme', 'host', 'port', '_custom_request_query_matcher'] E Matchers failed : E path - assertion failure : E /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.KeyVault/vaults/clitestrg000001 != /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/clitest.rg000001/providers/Microsoft.KeyVault/vaults/clitestrg5b3ldlllqxmq

src\azure-cli-testsdk\azure\cli\testsdk\base.py:306: AssertionError ---------- generated xml file: C:\Users\mwesigwaguma.azdev\env_config\repos\azure-cli\env\test_results.xml ----------- =============================================== short test summary info =============================================== FAILED src\azure-cli\azure\cli\command_modules\servicefabric\tests\latest\test_sf_cluster.py::ServiceFabricClusterTests::test_update_settings_and_reliability - AssertionError: Can't overwrite existing cassette ('e:\repos\azure-cli\src\azure-cli\azure\cli\command_modul... !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! xdist.dsession.Interrupted: stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ======================================= 1 failed, 1 passed, 2 skipped in 11.90s =======================================

Expected behavior

Tests pass in playback mode

Environment Summary

Windows 11

Additional context

No response

yonzhan commented 5 months ago

Thank you for opening this issue, we will look into it.

mwesigwaguma commented 5 months ago

Hey, @yonzhan is there any progress on this ?

jsntcy commented 5 months ago

Please provide the related PR link.

mwesigwaguma commented 5 months ago

Please provide the related PR link.

https://github.com/Azure/azure-cli/pull/28218

yonzhan commented 5 months ago

Please fix conflicting files.

mwesigwaguma commented 5 months ago

Please fix conflicting files.

Updated