search

Azure / azure-sdk-for-java

This repository is for active development of the Azure SDK for Java. For consumers of the SDK we recommend visiting our public developer docs at https://docs.microsoft.com/java/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-java.
MIT License
2.25k stars 1.93k forks source link

[resource manager] Refactor resourcemanager tests to work with latest TestProxy #40209

Open samvaity opened 1 month ago

samvaity commented 1 month ago

To address the timeout issues with ResourceManager tests when using the latest Test Proxy, the tests are being marked as LiveOnly The central sanitization process on the recordings may be causing the timeouts.

Need to figure out what central sanitizers should be disabled or refactoring the tests to make it compatible with the sanitization process, ensuring they can run without the need for live-only conditions. This will likely involve adjusting the test setup, and the data validation steps within the tests.

scbedd commented 1 month ago

Tag me in @weidongxu-microsoft with questions. I have reason to believe that something in the upstream sanitizers is messing with the batching of azure-core on java side, so requests aren't flowing as quickly as they should be.

We saw that with .NET, due to the sanitizer AZSDK3448

weidongxu-microsoft commented 1 month ago

resourcemanager sanitizer mostly here https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/resourcemanager/azure-resourcemanager-test/src/main/java/com/azure/resourcemanager/test/ResourceManagerTestProxyTestBase.java#L439-L480

a few module would add a few more, e.g. https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/resourcemanager/azure-resourcemanager-containerregistry/src/test/java/com/azure/resourcemanager/containerregistry/RegistryTest.java#L28-L33

weidongxu-microsoft commented 1 month ago

We are actually fine with @liveOnly.

If we upgrade api-version of a service, we generally require dev to re-run them as LIVE or RECORD, as there is no promise that backend does not have behavior change that breaks customer at runtime (which can only be figured out when run LIVE). If SDK dev can catch the behavior change, SDK customer may not get impacted, or at least get a warning in changelog.

weidongxu-microsoft commented 1 month ago

@v-hongli1 Search @LiveOnly in sdk/resourcemanager, and see if you can remove it, record, and pass playback.

Please do it one module at a time.

Make sure your branch is after PR https://github.com/Azure/azure-sdk-for-java/pull/40441