[Custom threshold] Improve api integration tests by using retry service and alertingApi

[maryam-saeidi]

📝 Summary

Currently, for the custom threshold API integration tests, we are using x-pack/test/alerting_api_integration/observability/helpers functions that we defined ourselves, but we already have an alertingApi service that we can use instead. The goal of this ticket is to remove all these helpers and use alertingApi service instead.

Also, retry common helper was added due to not having similar functionality in retry service but this PR solved this issue and we should be able to get rid of all the retry functionalities outside of retry service (such as pRetry, or retry)

🎯 Acceptance Criteria

• Remove alert-related helpers and use alertingApi instead
• Make sure we don't have any usage of pRetry in our tests and we use retryService instead
• [Optional] Consider using publicBaseUrl config here and similar places instead of hardcoding the value

[elasticmachine]

Pinging @elastic/obs-ux-management-team (Team:obs-ux-management)

[jasonrhodes]

@maryam-saeidi can you confirm that this work is all still relevant? I'd like to group together things we are looking to do to improve our tests and this would fit in nicely, if it still needs to be done.

[maryam-saeidi]

@jasonrhodes yes, it is still valid.

I was talking to @mgiota about her work on the agnostic testing framework, and since the issue described in this ticket is fixed in serverless tests (as you see below, we use alertingApi service in that case), we can use serverless custom threshold tests as a base when we move to the agnostic framework, so both works can be done in one go (still need to check pRetry usage and using publicBaseUrl)