The root cause only occurs while multithreading. The one_or_many_urls in download_from_urls() is of type class 'pydantic.networks.AnyUrl' and when we try to cast it to a list in the below code, we tell it to only type-cast if it’s of type class 'pydantic.networks.HttpUrl'. Again it’s only an issue in multithreaded runs because single threaded already returns a list of type pydantic.networks.AnyUrl . Noting once again that we’re still using the wrong type for the check
That’s why we were getting the error AttributeError: 'str' object has no attribute 'host' because we listified ‘http://*’ so for the first iteration, url == ‘h’
@staticmethod
def _coerce_to_list(
possible_list: Union[Iterable[Any], Any], obj_type: Type[T]
) -> Iterable[T]:
"""Cast possible single object into list
Args:
possible_list (Union[Iterable[Any], Any]): Variable to be converted to a list
obj_type (Type): The type of the item within the returned list.
Even though `obj_type` isn't used in implementation, it's key in
determining return type
"""
if isinstance(possible_list, obj_type):
possible_list = [possible_list]
return possible_list
Type of change
Please delete options that are not relevant.
[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update
How Has This Been Tested?
I ran it in debug mode and asserted that we were getting just 'h' for the url array and that the cast wasn't actually converting to a list
Checklist:
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules
[x] I have checked my code and corrected any misspellings
Description
The root cause only occurs while multithreading. The one_or_many_urls in download_from_urls() is of type class 'pydantic.networks.AnyUrl' and when we try to cast it to a list in the below code, we tell it to only type-cast if it’s of type class 'pydantic.networks.HttpUrl'. Again it’s only an issue in multithreaded runs because single threaded already returns a list of type pydantic.networks.AnyUrl . Noting once again that we’re still using the wrong type for the check
That’s why we were getting the error AttributeError: 'str' object has no attribute 'host' because we listified ‘http://*’ so for the first iteration, url == ‘h’
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
I ran it in debug mode and asserted that we were getting just 'h' for the url array and that the cast wasn't actually converting to a list
Checklist:
Next Steps
[x] Assign a reviewer based on the code owner document.
[x] Once your review is approved, merge and delete the feature branch
On behalf of the Modis Tools Dev Team, thank you for your hard work! ✨