Nwis Client fails trying to iterate a datetime.datetime object

[hellkite500]

The following is a stack trace of a failing test in a project that is using the nwis client. I just created a new virtual env and installed all project dependencies from their latest versions (including hydrotools) and this suddenly popped up in my test suite:

venv/lib/python3.9/site-packages/hypy/hydrolocation/nwis_location.py:50: in get_data
    return self._nwis.get(self._station_id, startDT=start, endDT=end)
venv/lib/python3.9/site-packages/hydrotools/nwis_client/_utilities.py:93: in wrapper
    return fn(*args, **kwargs)
venv/lib/python3.9/site-packages/hydrotools/nwis_client/iv.py:236: in get
    raw_data = self.get_raw(
venv/lib/python3.9/site-packages/hydrotools/nwis_client/iv.py:446: in get_raw
    results = self._restclient.mget(parameters=query_params, headers=self._headers)
venv/lib/python3.9/site-packages/forge/_revision.py:328: in inner
    return callable(*mapped.args, **mapped.kwargs)
venv/lib/python3.9/site-packages/hydrotools/_restclient/_restclient.py:162: in mget
    return self._add_to_loop(
venv/lib/python3.9/site-packages/hydrotools/_restclient/async_helpers.py:23: in _add_to_loop
    return self._loop.run_until_complete(coro)
../../homebrew/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py:647: in run_until_complete
    return future.result()
venv/lib/python3.9/site-packages/forge/_revision.py:322: in inner
    return await callable(*mapped.args, **mapped.kwargs)
venv/lib/python3.9/site-packages/hydrotools/_restclient/_restclient.py:231: in _mget
    return await asyncio.gather(
venv/lib/python3.9/site-packages/forge/_revision.py:322: in inner
    return await callable(*mapped.args, **mapped.kwargs)
venv/lib/python3.9/site-packages/hydrotools/_restclient/_restclient.py:184: in _get
    resp = await self._session.get(url, headers=_headers, **kwargs)
venv/lib/python3.9/site-packages/forge/_revision.py:322: in inner
    return await callable(*mapped.args, **mapped.kwargs)
venv/lib/python3.9/site-packages/hydrotools/_restclient/async_client.py:49: in _request
    resp = await bound_resp()
venv/lib/python3.9/site-packages/forge/_revision.py:322: in inner
    return await callable(*mapped.args, **mapped.kwargs)
venv/lib/python3.9/site-packages/aiohttp_client_cache/session.py:47: in _request
    self.cookie_jar.update_cookies(response.cookies or {}, response.url)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <aiohttp.cookiejar.CookieJar object at 0x1293b3f40>
cookies = datetime.datetime(2022, 5, 2, 16, 17, 40, 466031)
response_url = URL('https://nwis.waterservices.usgs.gov/nwis/iv/?endDT=2015-12-30T23:00%2B0000&format=json&sites=02146211&parameterCd=00060&siteStatus=all&startDT=2015-12-01T00:00%2B0000')

    def update_cookies(self, cookies: LooseCookies, response_url: URL = URL()) -> None:
        """Update cookies."""
        hostname = response_url.raw_host

        if not self._unsafe and is_ip_address(hostname):
            # Don't accept cookies from IPs
            return

        if isinstance(cookies, Mapping):
            cookies = cookies.items()

>       for name, cookie in cookies:
E       TypeError: 'datetime.datetime' object is not iterable

venv/lib/python3.9/site-packages/aiohttp/cookiejar.py:146: TypeError

[jarq6c]

@aaraney Well I'm stumped.

[jarq6c]

So far, I can't duplicate it. I've tested it with pandas.Timestamp, datetime.datetime, and str

Python 3.9.7 (default, Sep 16 2021, 13:09:58) [GCC 7.5.0] :: Anaconda, Inc. on linux

Package Version

---

aiodns 3.0.0 aiohttp 3.7.4.post0 aiohttp-client-cache 0.6.1 aiosqlite 0.17.0 async-timeout 3.0.1 attrs 21.4.0 brotlipy 0.7.0 cchardet 2.1.7 certifi 2021.10.8 cffi 1.15.0 chardet 4.0.0 charset-normalizer 2.0.4 click 8.1.3 conda 4.12.0 conda-package-handling 1.8.1 cryptography 37.0.1 hydrotools.nwis-client 3.2.1 hydrotools.-restclient 3.0.5 idna 3.3 itsdangerous 2.1.2 multidict 6.0.2 numpy 1.22.3 pandas 1.4.2 pip 22.0.4 pycares 4.1.2 pycosat 0.6.3 pycparser 2.21 pyOpenSSL 22.0.0 PySocks 1.7.1 python-dateutil 2.8.2 python-forge 18.6.0 pytz 2022.1 requests 2.27.1 ruamel-yaml-conda 0.15.100 setuptools 61.2.0 six 1.16.0 tqdm 4.64.0 typing_extensions 4.2.0 url-normalize 1.4.3 urllib3 1.26.9 wheel 0.37.1 yarl 1.7.2

from hydrotools.nwis_client.iv import IVDataService
from datetime import datetime as dt

client = IVDataService(
)

df = client.get(
    sites='01646500',
    startDT=dt(2020, 8, 1),
    endDT=dt(2020, 8, 3)
    )

print(df)

[hellkite500]

As I pondered the stack trace and looked at the session cache, it occurred to me that maybe the old nwisiv_cache.sqlite was to blame, and sure enough, I renamed that file, and everything worked. (I had one hanging around from the previous version before the upgrade...)

[jarq6c]

As I pondered the stack trace and looked at the session cache, it occurred to me that maybe the old nwisiv_cache.sqlite was to blame, and sure enough, I renamed that file, and everything worked. (I had one hanging around from the previous version before the upgrade...)

Nice! Let's blame @aaraney anyway ;)

[aaraney]

Glad you all got it sorted out! Now to go plant my next bug 😝

[hellkite500]

This has come back to haunt me again...though at least this time I had some de-ja-vu to assist me and didn't spend too much time banging my head on the wall. I'm curious if there is a way to trap some of these conditions and at least warn the user to maybe delete old cache?