improve logging and retry of trial add/remove TC test

fix an issue with failing test by adding logging, increasing retries, separating validation steps

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    def validate():
        trial_components = list(trial_obj.list_trial_components())
>       assert 1 == len(trial_components)
E       assert 1 == 0
E        +  where 0 = len([])

tests/integ/test_trial.py:115: AssertionError
----------------------------- Captured stdout call -----------------------------
Retrying assert 0 == 1
 +  where 1 = len([TrialComponentSummary(trial_component_name='smexperiments-integ-f3c230a1-0a64-45ae-80c2-c40e9b05fb68',trial_component...ted_by={},last_modified_time=datetime.datetime(2022, 4, 22, 14, 17, 35, 738000, tzinfo=tzlocal()),last_modified_by={})])
Retrying assert 1 == 0
 +  where 0 = len([])
Retrying assert 1 == 0
 +  where 0 = len([])

tox -e py37 -- tests/integ/test_trial.py
...
py37 run-test-pre: PYTHONHASHSEED='3330764980'
py37 run-test: commands[0] | coverage run --source smexperiments -m pytest tests/integ/test_trial.py
========================================================= test session starts =========================================================
platform darwin -- Python 3.7.11, pytest-4.4.1, py-1.10.0, pluggy-0.13.1
cachedir: .tox/py37/.pytest_cache
rootdir: /Users/danabens/workplace/github/aws/sagemaker-experiments, inifile: pytest.ini
plugins: cov-2.9.0, rerunfailures-8.0, xdist-1.34.0, forked-1.3.0
collected 8 items

tests/integ/test_trial.py ........                                                                                              [100%]

====================================================== 8 passed in 22.61 seconds ======================================================
py37 run-test: commands[1] | coverage report --fail-under=95

aws / sagemaker-experiments

improve logging and retry of trial add/remove TC test #159