Closed kloczek closed 8 months ago
Here is pytest output:
compare to https://github.com/aio-libs/aiobotocore/runs/8013297817?check_suite_focus=true where it's passing. Are you using an environment created by the Pipfile.lock?
No I'm using environment created inside LXC zone using rpm packages.
BTW in CI output I see that you are sunning python -m pytest
. This is incorrect way of using pytest because when is executed python -m foo
python automatically adds current directory to sys.path
which is causing many issues.
This is pytest provides pytest
as executable script which should be used instead.
So you have no idea what could be missing in my env?
that's fine, the issue is you need to use pipenv to ensure your python environment has the packages required.
The issue is that the test suite is not compatible with pytest-asyncio >= 0.19 yet. If I add the following line to pytest.ini, most tests can pass.
diff --git a/pytest.ini b/pytest.ini
index 319ae64..753266c 100644
--- a/pytest.ini
+++ b/pytest.ini
@@ -1,4 +1,5 @@
[pytest]
+asyncio_mode = auto
markers =
moto
config_kwargs
The issue is that the test suite is not compatible with pytest-asyncio >= 0.19 yet. If I add the following line to pytest.ini, most tests can pass.
Summary result with asyncio_mode = auto
============================================================================= warnings summary =============================================================================
tests/conftest.py:25
/home/tkloczko/rpmbuild/BUILD/aiobotocore-2.4.0/tests/conftest.py:25: PytestRemovedIn8Warning: The pytest_cmdline_preparse hook is deprecated and will be removed in a future release.
Please use pytest_load_initial_conftests hook instead.
def pytest_cmdline_preparse(args):
tests/test_basic_s3.py: 23 warnings
tests/test_config.py: 2 warnings
tests/test_eventstreams.py: 2 warnings
tests/test_monitor.py: 1 warning
tests/test_session.py: 1 warning
tests/python3.8/boto_tests/test_signers.py: 2 warnings
/usr/lib/python3.8/site-packages/botocore/utils.py:1719: FutureWarning: The S3RegionRedirector class has been deprecated for a new internal replacement. A future version of botocore may remove this class.
warnings.warn(
tests/test_basic_s3.py::test_can_make_request_no_verify[False]
/home/tkloczko/rpmbuild/BUILD/aiobotocore-2.4.0/aiobotocore/httpsession.py:111: DeprecationWarning: verify_ssl is deprecated, use ssl=False instead
self._create_connector = lambda: aiohttp.TCPConnector(
tests/test_config.py::test_connector_args
/home/tkloczko/rpmbuild/BUILD/aiobotocore-2.4.0/tests/test_config.py:52: DeprecationWarning: The object should be created within an async function
AioConfig({"resolver": aiohttp.resolver.DefaultResolver()})
tests/boto_tests/test_utils.py::test_idmsfetcher_timeout
/usr/lib/python3.8/site-packages/botocore/awsrequest.py:452: RuntimeWarning: coroutine 'fake_aiohttp_session.<locals>.FakeAioHttpSession.FakeResponse._content' was never awaited
params = {} if params is None else params
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/boto_tests/test_utils.py::test_idmsfetcher_timeout
/usr/lib/python3.8/site-packages/botocore/awsrequest.py:452: RuntimeWarning: coroutine 'fake_aiohttp_session.<locals>.FakeAioHttpSession.FakeResponse._text' was never awaited
params = {} if params is None else params
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/python3.8/boto_tests/test_utils.py::test_containermetadatafetcher_retrieve_url_bad_status
/usr/lib64/python3.8/inspect.py:2985: RuntimeWarning: coroutine 'fake_aiohttp_session.<locals>.FakeAioHttpSession.FakeResponse._text' was never awaited
for param in itertools.chain(parameters_ex, parameters):
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/python3.8/boto_tests/test_utils.py::test_containermetadatafetcher_retrieve_url_not_json
/usr/lib64/python3.8/unittest/mock.py:2030: RuntimeWarning: coroutine 'fake_aiohttp_session.<locals>.FakeAioHttpSession.FakeResponse._text' was never awaited
setattr(_type, entry, MagicProxy(entry, self))
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================================================================= short test summary info ==========================================================================
SKIPPED [1] tests/test_basic_s3.py:415: Not supported
ERROR tests/test_basic_s3.py::test_succeed_proxy_request[False] - botocore.exceptions.ProxyConnectionError: Failed to connect to proxy URL: "http://127.0.0.1:54321"
ERROR tests/test_basic_s3.py::test_can_get_bucket_location - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_delete_urlencoded_object - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_paginate - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_paginate_with_page_size - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_search_paginate - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_paginate_iterator - assert 200 == 204
ERROR tests/test_basic_s3.py::test_result_key_iters - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_get_and_put_object - assert 200 == 204
ERROR tests/test_basic_s3.py::test_adaptive_retry - assert 200 == 204
ERROR tests/test_basic_s3.py::test_get_object_stream_wrapper - assert 200 == 204
ERROR tests/test_basic_s3.py::test_get_object_stream_context - assert 200 == 204
ERROR tests/test_basic_s3.py::test_paginate_max_items - botocore.exceptions.ParamValidationError: Parameter validation failed:
ERROR tests/test_basic_s3.py::test_paginate_within_page_boundaries - assert 200 == 204
ERROR tests/test_basic_s3.py::test_unicode_key_put_list[False] - botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL: "http://169.254.169.254/latest...
ERROR tests/test_basic_s3.py::test_unicode_system_character[False] - botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL: "http://169.254.169.254/la...
ERROR tests/test_basic_s3.py::test_non_normalized_key_paths - assert 200 == 204
ERROR tests/test_basic_s3.py::test_copy_with_quoted_char - assert 200 == 204
ERROR tests/test_basic_s3.py::test_copy_with_query_string - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_copy_with_dict_form - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_copy_with_dict_form_with_version - assert 200 == 204
ERROR tests/test_basic_s3.py::test_copy_with_s3_metadata - assert 200 == 204
ERROR tests/test_basic_s3.py::test_presign_with_existing_query_string_values[False-s3-us-east-1] - botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint U...
ERROR tests/test_basic_s3.py::test_presign_sigv4[False-s3v4-us-east-1] - botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL: "http://169.254.169.25...
ERROR tests/test_basic_s3.py::test_can_follow_signed_url_redirect[False-s3v4] - botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL: "http://169.254...
ERROR tests/test_basic_s3.py::test_bucket_redirect[False-us-west-2-eu-west-1] - botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL: "http://169.254...
ERROR tests/test_basic_s3.py::test_head_object_keys[s3v4] - assert 200 == 204
ERROR tests/boto_tests/test_credentials.py::test_sso_credential_fetcher_can_fetch_credentials - botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL:...
ERROR tests/boto_tests/test_credentials.py::test_sso_cred_fetcher_raises_helpful_message_on_unauthorized_exception - botocore.exceptions.ConnectTimeoutError: Connect tim...
ERROR tests/boto_tests/test_credentials.py::test_load_sso_credentials_without_cache
ERROR tests/boto_tests/test_credentials.py::test_load_sso_credentials_with_cache - botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL: "http://169....
ERROR tests/boto_tests/test_credentials.py::test_load_sso_credentials_with_cache_expired - botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL: "htt...
ERROR tests/boto_tests/test_credentials.py::test_required_config_not_set - botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL: "http://169.254.169....
FAILED tests/test_basic_s3.py::test_can_delete_urlencoded_object - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObject operat...
FAILED tests/test_basic_s3.py::test_can_paginate - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached max retries: 4)...
FAILED tests/test_basic_s3.py::test_can_paginate_with_page_size - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached ...
FAILED tests/test_basic_s3.py::test_can_search_paginate - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached max retr...
FAILED tests/test_basic_s3.py::test_can_paginate_iterator - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached max re...
FAILED tests/test_basic_s3.py::test_result_key_iters - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObject operation: The spe...
FAILED tests/test_basic_s3.py::test_can_get_and_put_object - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached max r...
FAILED tests/test_basic_s3.py::test_adaptive_retry - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached max retries: ...
FAILED tests/test_basic_s3.py::test_get_object_stream_wrapper - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached ma...
FAILED tests/test_basic_s3.py::test_get_object_stream_context - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached ma...
FAILED tests/test_basic_s3.py::test_paginate_max_items - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the CreateMultipartUpload oper...
FAILED tests/test_basic_s3.py::test_paginate_within_page_boundaries - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reac...
FAILED tests/test_basic_s3.py::test_non_normalized_key_paths - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObject operation:...
FAILED tests/test_basic_s3.py::test_copy_with_quoted_char - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObject operation: Th...
FAILED tests/test_basic_s3.py::test_copy_with_query_string - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObject operation: T...
FAILED tests/test_basic_s3.py::test_can_copy_with_dict_form - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObject operation: ...
FAILED tests/test_basic_s3.py::test_can_copy_with_dict_form_with_version - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObjec...
FAILED tests/test_basic_s3.py::test_copy_with_s3_metadata - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached max re...
FAILED tests/test_basic_s3.py::test_head_object_keys[s3v4] - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached max r...
FAILED tests/test_lambda.py::test_run_lambda - botocore.exceptions.ClientError: An error occurred (500) when calling the CreateFunction operation (reached max retries: 4...
FAILED tests/test_mturk.py::test_mturk_stubber - botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL: "http://169.254.169.254/latest/api/token"
FAILED tests/test_patches.py::test_patches - assert False
FAILED tests/test_waiter.py::test_sqs - botocore.parsers.ResponseParserError: Unable to parse response (syntax error: line 1, column 0), invalid XML received. Further re...
FAILED tests/boto_tests/test_credentials.py::test_get_credentials - botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://[fd00:ec2...
FAILED tests/python3.8/test_eventstreams.py::test_kinesis_stream_json_parser - botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL: "http://169.254....
FAILED tests/python3.8/boto_tests/test_signers.py::test_signers_generate_presigned_urls - AssertionError: expected call not found.
============================================== 26 failed, 142 passed, 1 skipped, 38 warnings, 33 errors in 193.01s (0:03:13) ===============================================
Some of the units are failing because docker
module is missing. Is it really necessary to use that module amnd virtualenv
cannot be used? 🤔
docker is used by moto for lambda testing
if we don't have lambda then probably don't need
botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL: "http://169.254.169.254/latest...
FWIW, I have once got similar errors, and I got rid of them by adding some environment variables. See https://github.com/aio-libs/aiobotocore/issues/948#issuecomment-1190689924
Tested ant it does not change anything and more even units fails.
========================================================================= short test summary info ==========================================================================
SKIPPED [1] tests/test_basic_s3.py:415: Not supported
ERROR tests/test_basic_s3.py::test_can_get_bucket_location - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_delete_urlencoded_object - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_paginate - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_paginate_with_page_size - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_search_paginate - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_paginate_iterator - assert 200 == 204
ERROR tests/test_basic_s3.py::test_result_key_iters - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_get_and_put_object - assert 200 == 204
ERROR tests/test_basic_s3.py::test_adaptive_retry - assert 200 == 204
ERROR tests/test_basic_s3.py::test_get_object_stream_wrapper - assert 200 == 204
ERROR tests/test_basic_s3.py::test_get_object_stream_context - assert 200 == 204
ERROR tests/test_basic_s3.py::test_paginate_max_items - botocore.exceptions.ParamValidationError: Parameter validation failed:
ERROR tests/test_basic_s3.py::test_paginate_within_page_boundaries - assert 200 == 204
ERROR tests/test_basic_s3.py::test_unicode_key_put_list[False] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the CreateBucket op...
ERROR tests/test_basic_s3.py::test_unicode_key_put_list[False] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the ListObjectVersi...
ERROR tests/test_basic_s3.py::test_unicode_system_character[False] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the CreateBucke...
ERROR tests/test_basic_s3.py::test_unicode_system_character[False] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the ListObjectV...
ERROR tests/test_basic_s3.py::test_non_normalized_key_paths - assert 200 == 204
ERROR tests/test_basic_s3.py::test_copy_with_quoted_char - assert 200 == 204
ERROR tests/test_basic_s3.py::test_copy_with_query_string - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_copy_with_dict_form - assert 200 == 204
ERROR tests/test_basic_s3.py::test_can_copy_with_dict_form_with_version - assert 200 == 204
ERROR tests/test_basic_s3.py::test_copy_with_s3_metadata - assert 200 == 204
ERROR tests/test_basic_s3.py::test_presign_with_existing_query_string_values[False-s3-us-east-1] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId...
ERROR tests/test_basic_s3.py::test_presign_with_existing_query_string_values[False-s3-us-east-1] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId...
ERROR tests/test_basic_s3.py::test_presign_sigv4[False-s3v4-us-east-1] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the CreateB...
ERROR tests/test_basic_s3.py::test_presign_sigv4[False-s3v4-us-east-1] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the ListObj...
ERROR tests/test_basic_s3.py::test_can_follow_signed_url_redirect[False-s3v4] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the ...
ERROR tests/test_basic_s3.py::test_can_follow_signed_url_redirect[False-s3v4] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the ...
ERROR tests/test_basic_s3.py::test_bucket_redirect[False-us-west-2-eu-west-1] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the ...
ERROR tests/test_basic_s3.py::test_head_object_keys[s3v4] - assert 200 == 204
ERROR tests/boto_tests/test_credentials.py::test_load_sso_credentials_without_cache
FAILED tests/test_basic_s3.py::test_succeed_proxy_request[False] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the ListBuckets o...
FAILED tests/test_basic_s3.py::test_can_delete_urlencoded_object - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObject operat...
FAILED tests/test_basic_s3.py::test_can_paginate - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached max retries: 4)...
FAILED tests/test_basic_s3.py::test_can_paginate_with_page_size - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached ...
FAILED tests/test_basic_s3.py::test_can_search_paginate - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached max retr...
FAILED tests/test_basic_s3.py::test_can_paginate_iterator - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached max re...
FAILED tests/test_basic_s3.py::test_result_key_iters - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObject operation: The spe...
FAILED tests/test_basic_s3.py::test_can_get_and_put_object - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached max r...
FAILED tests/test_basic_s3.py::test_adaptive_retry - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached max retries: ...
FAILED tests/test_basic_s3.py::test_get_object_stream_wrapper - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached ma...
FAILED tests/test_basic_s3.py::test_get_object_stream_context - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached ma...
FAILED tests/test_basic_s3.py::test_paginate_max_items - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the CreateMultipartUpload oper...
FAILED tests/test_basic_s3.py::test_paginate_within_page_boundaries - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reac...
FAILED tests/test_basic_s3.py::test_non_normalized_key_paths - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObject operation:...
FAILED tests/test_basic_s3.py::test_copy_with_quoted_char - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObject operation: Th...
FAILED tests/test_basic_s3.py::test_copy_with_query_string - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObject operation: T...
FAILED tests/test_basic_s3.py::test_can_copy_with_dict_form - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObject operation: ...
FAILED tests/test_basic_s3.py::test_can_copy_with_dict_form_with_version - botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObjec...
FAILED tests/test_basic_s3.py::test_copy_with_s3_metadata - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached max re...
FAILED tests/test_basic_s3.py::test_bucket_redirect[False-us-west-2-eu-west-1] - botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the...
FAILED tests/test_basic_s3.py::test_head_object_keys[s3v4] - botocore.exceptions.ClientError: An error occurred (500) when calling the PutObject operation (reached max r...
FAILED tests/test_lambda.py::test_run_lambda - botocore.exceptions.ClientError: An error occurred (500) when calling the CreateFunction operation (reached max retries: 4...
FAILED tests/test_mturk.py::test_mturk_stubber - AttributeError: 'MTurk' object has no attribute 'list_hi_ts_for_qualification_type'
FAILED tests/test_patches.py::test_patches - assert False
FAILED tests/test_waiter.py::test_sqs - botocore.parsers.ResponseParserError: Unable to parse response (syntax error: line 1, column 0), invalid XML received. Further re...
FAILED tests/boto_tests/test_credentials.py::test_get_credentials - botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://[fd00:ec2...
FAILED tests/boto_tests/test_credentials.py::test_load_sso_credentials_with_cache_expired - botocore.exceptions.StubAssertionError: Error getting response stub for opera...
FAILED tests/python3.8/test_eventstreams.py::test_kinesis_stream_json_parser - botocore.exceptions.NoRegionError: You must specify a region.
FAILED tests/python3.8/boto_tests/test_signers.py::test_signers_generate_presigned_urls - AssertionError: expected call not found.
============================================== 29 failed, 146 passed, 1 skipped, 47 warnings, 32 errors in 209.63s (0:03:29) ===============================================
Looks like test suite needs access to AWS services
After add to my build procedure all those units which are falis or ends with error to --deselect list I was able to produce clean output with some warnings. It would be good at least take care of tjhose warnings.
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-aiobotocore-2.4.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-aiobotocore-2.4.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra --deselect tests/boto_tests/test_credentials.py::test_get_credentials --deselect tests/boto_tests/test_credentials.py::test_load_sso_credentials_with_cache --deselect tests/boto_tests/test_credentials.py::test_load_sso_credentials_with_cache_expired --deselect tests/boto_tests/test_credentials.py::test_load_sso_credentials_without_cache --deselect tests/boto_tests/test_credentials.py::test_required_config_not_set --deselect tests/boto_tests/test_credentials.py::test_sso_credential_fetcher_can_fetch_credentials --deselect tests/boto_tests/test_credentials.py::test_sso_cred_fetcher_raises_helpful_message_on_unauthorized_exception --deselect tests/python3.8/boto_tests/test_signers.py::test_signers_generate_presigned_urls --deselect tests/python3.8/test_eventstreams.py::test_kinesis_stream_json_parser --deselect tests/test_basic_s3.py::test_adaptive_retry --deselect 'tests/test_basic_s3.py::test_bucket_redirect[False-us-west-2-eu-west-1]' --deselect tests/test_basic_s3.py::test_can_copy_with_dict_form --deselect tests/test_basic_s3.py::test_can_copy_with_dict_form_with_version --deselect tests/test_basic_s3.py::test_can_delete_urlencoded_object --deselect 'tests/test_basic_s3.py::test_can_follow_signed_url_redirect[False-s3v4]' --deselect tests/test_basic_s3.py::test_can_get_and_put_object --deselect tests/test_basic_s3.py::test_can_get_bucket_location --deselect tests/test_basic_s3.py::test_can_paginate --deselect tests/test_basic_s3.py::test_can_paginate_iterator --deselect tests/test_basic_s3.py::test_can_paginate_with_page_size --deselect tests/test_basic_s3.py::test_can_search_paginate --deselect tests/test_basic_s3.py::test_copy_with_query_string --deselect tests/test_basic_s3.py::test_copy_with_quoted_char --deselect tests/test_basic_s3.py::test_copy_with_s3_metadata --deselect tests/test_basic_s3.py::test_get_object_stream_context --deselect tests/test_basic_s3.py::test_get_object_stream_wrapper --deselect 'tests/test_basic_s3.py::test_head_object_keys[s3v4]' --deselect tests/test_basic_s3.py::test_non_normalized_key_paths --deselect tests/test_basic_s3.py::test_paginate_max_items --deselect tests/test_basic_s3.py::test_paginate_within_page_boundaries --deselect 'tests/test_basic_s3.py::test_presign_sigv4[False-s3v4-us-east-1]' --deselect 'tests/test_basic_s3.py::test_presign_with_existing_query_string_values[False-s3-us-east-1]' --deselect tests/test_basic_s3.py::test_result_key_iters --deselect 'tests/test_basic_s3.py::test_succeed_proxy_request[False]' --deselect 'tests/test_basic_s3.py::test_unicode_key_put_list[False]' --deselect 'tests/test_basic_s3.py::test_unicode_system_character[False]' --deselect tests/test_lambda.py::test_run_lambda --deselect tests/test_mturk.py::test_mturk_stubber --deselect tests/test_patches.py::test_patches --deselect tests/test_waiter.py::test_sqs
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.15, pytest-7.1.3, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/aiobotocore-2.4.0, configfile: pytest.ini
plugins: asyncio-0.20.2
asyncio: mode=auto
collected 182 items / 40 deselected / 142 selected
tests/test_adaptive.py ........... [ 7%]
tests/test_basic_s3.py ...s [ 10%]
tests/test_batch.py . [ 11%]
tests/test_config.py ..... [ 14%]
tests/test_dynamodb.py ...... [ 19%]
tests/test_ec2.py . [ 19%]
tests/test_eventstreams.py .. [ 21%]
tests/test_monitor.py . [ 21%]
tests/test_patches.py . [ 22%]
tests/test_response.py ............... [ 33%]
tests/test_session.py .. [ 34%]
tests/test_sns.py ..... [ 38%]
tests/test_sqs.py .... [ 40%]
tests/test_version.py . [ 41%]
tests/boto_tests/test_credentials.py ............................ [ 61%]
tests/boto_tests/test_signers.py . [ 61%]
tests/boto_tests/test_utils.py ........ [ 67%]
tests/python3.8/boto_tests/test_credentials.py ............... [ 78%]
tests/python3.8/boto_tests/test_signers.py ....... [ 83%]
tests/python3.8/boto_tests/test_utils.py ........................ [100%]
============================================================================= warnings summary =============================================================================
tests/conftest.py:25
/home/tkloczko/rpmbuild/BUILD/aiobotocore-2.4.0/tests/conftest.py:25: PytestRemovedIn8Warning: The pytest_cmdline_preparse hook is deprecated and will be removed in a future release.
Please use pytest_load_initial_conftests hook instead.
def pytest_cmdline_preparse(args):
tests/test_basic_s3.py: 3 warnings
tests/test_config.py: 2 warnings
tests/test_eventstreams.py: 2 warnings
tests/test_monitor.py: 1 warning
tests/test_session.py: 1 warning
tests/python3.8/boto_tests/test_signers.py: 1 warning
/usr/lib/python3.8/site-packages/botocore/utils.py:1719: FutureWarning: The S3RegionRedirector class has been deprecated for a new internal replacement. A future version of botocore may remove this class.
warnings.warn(
tests/test_basic_s3.py::test_can_make_request_no_verify[False]
/home/tkloczko/rpmbuild/BUILD/aiobotocore-2.4.0/aiobotocore/httpsession.py:111: DeprecationWarning: verify_ssl is deprecated, use ssl=False instead
self._create_connector = lambda: aiohttp.TCPConnector(
tests/test_config.py::test_connector_args
/home/tkloczko/rpmbuild/BUILD/aiobotocore-2.4.0/tests/test_config.py:52: DeprecationWarning: The object should be created within an async function
AioConfig({"resolver": aiohttp.resolver.DefaultResolver()})
tests/boto_tests/test_utils.py::test_idmsfetcher_timeout
/home/tkloczko/rpmbuild/BUILD/aiobotocore-2.4.0/tests/boto_tests/test_utils.py:28: RuntimeWarning: coroutine 'fake_aiohttp_session.<locals>.FakeAioHttpSession.FakeResponse._content' was never awaited
class FakeAioHttpSession:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/boto_tests/test_utils.py::test_idmsfetcher_timeout
/home/tkloczko/rpmbuild/BUILD/aiobotocore-2.4.0/tests/boto_tests/test_utils.py:28: RuntimeWarning: coroutine 'fake_aiohttp_session.<locals>.FakeAioHttpSession.FakeResponse._text' was never awaited
class FakeAioHttpSession:
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/python3.8/boto_tests/test_credentials.py::test_session_credentials
/usr/lib64/python3.8/inspect.py:2821: RuntimeWarning: coroutine 'fake_aiohttp_session.<locals>.FakeAioHttpSession.FakeResponse._content' was never awaited
params = OrderedDict(((param.name, param)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/python3.8/boto_tests/test_credentials.py::test_session_credentials
/usr/lib64/python3.8/inspect.py:2821: RuntimeWarning: coroutine 'fake_aiohttp_session.<locals>.FakeAioHttpSession.FakeResponse._text' was never awaited
params = OrderedDict(((param.name, param)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/python3.8/boto_tests/test_utils.py::test_containermetadatafetcher_retrieve_url_bad_status
/usr/lib64/python3.8/enum.py:339: RuntimeWarning: coroutine 'fake_aiohttp_session.<locals>.FakeAioHttpSession.FakeResponse._text' was never awaited
return cls.__new__(cls, value)
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
tests/python3.8/boto_tests/test_utils.py::test_containermetadatafetcher_retrieve_url_not_json
/usr/lib64/python3.8/unittest/mock.py:2076: RuntimeWarning: coroutine 'fake_aiohttp_session.<locals>.FakeAioHttpSession.FakeResponse._text' was never awaited
self.name = name
Enable tracemalloc to get traceback where the object was allocated.
See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================================================================= short test summary info ==========================================================================
SKIPPED [1] tests/test_basic_s3.py:415: Not supported
======================================================= 141 passed, 1 skipped, 40 deselected, 19 warnings in 49.95s ========================================================
sys:1: RuntimeWarning: coroutine 'fake_aiohttp_session.<locals>.FakeAioHttpSession.FakeResponse._text' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
Tested ant it does not change anything and more even units fails.
I also got many failures from tests/test_basic_s3.py
with botocore 1.29.16, so I keep botocore 1.27.96 for now and only one test fails (test_load_sso_credentials_with_cache_expired
). As a side note, support for botocore >= 1.28 is tracked in https://github.com/aio-libs/aiobotocore/issues/976.
Just retested 2.4.2. Here is summary pytest output:
And full log python-aiobotocore-pytest.txt
Tested 2.5.1. Here is result:
The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.
aiobotocore/client.py:378: ClientError ------------------------------------------------------------------------------------ Captured log setup ------------------------------------------------------------------------------------- INFO werkzeug:_internal.py:187 127.0.0.1 - - [28/Jun/2023 06:55:27] "GET /static HTTP/1.1" 404 - INFO werkzeug:_internal.py:187 127.0.0.1 - - [28/Jun/2023 06:55:28] "GET /static HTTP/1.1" 500 - ERROR werkzeug:_internal.py:187 Error on request: Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/werkzeug/serving.py", line 364, in run_wsgi execute(self.server.app) File "/usr/lib/python3.8/site-packages/werkzeug/serving.py", line 325, in execute application_iter = app(environ, start_response) File "/usr/lib/python3.8/site-packages/moto/moto_server/werkzeug_app.py", line 238, in __call__ backend_app = self.get_application(environ) File "/usr/lib/python3.8/site-packages/moto/moto_server/werkzeug_app.py", line 180, in get_application app = self.create_app(backend) File "/usr/lib/python3.8/site-packages/moto/moto_server/werkzeug_app.py", line 260, in create_backend_app backend_dict = backends.get_backend(service) File "/usr/lib/python3.8/site-packages/moto/backends.py", line 48, in get_backend return _import_backend(module_name, backends_name) File "/usr/lib/python3.8/site-packages/moto/backends.py", line 18, in _import_backend module = importlib.import_module("moto." + module_name) File "/usr/lib64/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "The se...nd was unable to complete your request. Either the server is overloaded or there is an error in the application.
\n' def _parse_xml_string_to_dom(self, xml_string): try: parser = ETree.XMLParser( target=ETree.TreeBuilder(), encoding=self.DEFAULT_ENCODING ) > parser.feed(xml_string) E xml.etree.ElementTree.ParseError: syntax error: line 1, column 0 /usr/lib/python3.8/site-packages/botocore/parsers.py:503: ParseError During handling of the above exception, another exception occurred: cloudformation_client =The se...nd was unable to complete your request. Either the server is overloaded or there is an error in the application.
\n' def _parse_xml_string_to_dom(self, xml_string): try: parser = ETree.XMLParser( target=ETree.TreeBuilder(), encoding=self.DEFAULT_ENCODING ) parser.feed(xml_string) root = parser.close() except XMLParseError as e: > raise ResponseParserError( "Unable to parse response (%s), " "invalid XML received. Further retries may succeed:\n%s" % (e, xml_string) ) E botocore.parsers.ResponseParserError: Unable to parse response (syntax error: line 1, column 0), invalid XML received. Further retries may succeed: E b'\n\nThe server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.
\n' /usr/lib/python3.8/site-packages/botocore/parsers.py:506: ResponseParserError ------------------------------------------------------------------------------------ Captured log setup ------------------------------------------------------------------------------------- INFO werkzeug:_internal.py:187 127.0.0.1 - - [28/Jun/2023 06:55:55] "GET /static HTTP/1.1" 500 - ERROR werkzeug:_internal.py:187 Error on request: Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/werkzeug/serving.py", line 364, in run_wsgi execute(self.server.app) File "/usr/lib/python3.8/site-packages/werkzeug/serving.py", line 325, in execute application_iter = app(environ, start_response) File "/usr/lib/python3.8/site-packages/moto/moto_server/werkzeug_app.py", line 238, in __call__ backend_app = self.get_application(environ) File "/usr/lib/python3.8/site-packages/moto/moto_server/werkzeug_app.py", line 180, in get_application app = self.create_app(backend) File "/usr/lib/python3.8/site-packages/moto/moto_server/werkzeug_app.py", line 260, in create_backend_app backend_dict = backends.get_backend(service) File "/usr/lib/python3.8/site-packages/moto/backends.py", line 48, in get_backend return _import_backend(module_name, backends_name) File "/usr/lib/python3.8/site-packages/moto/backends.py", line 18, in _import_backend module = importlib.import_module("moto." + module_name) File "/usr/lib64/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "only the moto marked tests are supported, so you need to run with -m moto
. The others are just for local testing
only the moto marked tests are supported, so you need to run with
-m moto
. The others are just for local testing
For what this marker is? 🤔
it tells you which tests need moto to run
Just FTR 2.6.0 pytest is failing in seleniumbase (4.17.0) module:
Any hints? 🤔
Update: 2.7.0 is failing in 3 units and pytest shows some warnings
this isn't an issue with aiobotocore, this is an issue with your packaging (unless I'm missing something). Can you move this to discussions? You need to make sure you're running unittests the same way we are: https://github.com/aio-libs/aiobotocore/blob/master/.github/workflows/python-package.yml#L55 which calls: https://github.com/aio-libs/aiobotocore/blob/master/Makefile#L25
and here's how we install the requirements: https://github.com/aio-libs/aiobotocore/blob/master/.github/workflows/python-package.yml#L46
Just tested 2.8.0 and I have one unit failing. May I ask to have look? 🤔
so the lambda tests require moto, and further require docker. Do you have docker installed and running?
In this case moto
was installed.
Why docker? venv/virtualenv is not enough? 🤔
because moto uses the lambda base image to run the lambda function similar to how it runs in AWS. I actually wrote that part originally :)
On Wed, Nov 29, 2023, 12:47 AM Tomasz Kłoczko @.***> wrote:
In this case moto was installed. Why docker? venv/virtualenv is not enough? 🤔
— Reply to this email directly, view it on GitHub https://github.com/aio-libs/aiobotocore/issues/965#issuecomment-1831460123, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA6Q77N5J3ZKBAXVWNS6MYTYG3ZALAVCNFSM57TCB5G2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBTGE2DMMBRGIZQ . You are receiving this because you commented.Message ID: @.***>
After control rebuild in updated env I found that now two units are failing
BTW build
shows some warnings as well
ERROR moto.awslambda.models:models.py:989 error running docker: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
I'm going to close this issue, this is unrelated to an issue with aiobotocore. Feel free to continue in a discussion or stackoverflow or similar area. This area is for resolving issues with aiobotocore
discussion area for reference: https://github.com/aio-libs/aiobotocore/discussions
I'm going to close this issue, this is unrelated to an issue with aiobotocore. Feel free to continue in a discussion or stackoverflow or similar area. This area is for resolving issues with aiobotocore
OK. Can you provide your expertise what it cud be? 🤔
Describe the bug A clear and concise description of what the bug is.
Checklist
pip check
passes without errorspip freeze
resultspip freeze results
Environment:
Additional context I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.
python3 -sBm build -w --no-isolation
build
with--no-isolation
I'm using during all processes only locally installed modulesBecaus eone unit is failing on missing fixture I'm assuming that at least one module is missing in my build env. May I ask fo rdome hints what may be missing?