flickr.photos.search returns total="" instead of total="0"

oPromessa commented 6 years ago

Hey guys. Excellent job on flickrapi.

Issue

Since a few days ago I'm getting a weird result from flickrapi/Flickr on TravisCI testing.

I get an stat="ok" on flickr.photos.search but when I process the result, the field total comes up empty and not with an appropriate int value -- it should be 1 in my test case. (This code has been working for months).
Is this normal or did something change onflickrapi or Flickr itself andI now have to protect also against null values on total field?

This happens wiht python 2.6 and 3.7.

<rsp stat="ok">
<photos page="1" pages="0" perpage="100" total="" />
</rsp>

Source code

    @retry(attempts=3, waittime=20, randtime=True)
    def R_photos_search(kwargs):
        return nuflickr.photos.search(**kwargs)

    try:
        searchIsUploaded = None
        searchIsUploaded = R_photos_search(dict(user_id="me",
                                                tags='checksum:{}'
                                                     .format(xchecksum),
                                                extras='tags'))
    except flickrapi.exceptions.FlickrError as ex:
        reportError(Caught=True,
                    CaughtPrefix='+++',
                    CaughtCode='180',
                    CaughtMsg='Flickrapi exception on photos.search',
                    exceptUse=True,
                    exceptCode=ex.code,
                    exceptMsg=ex)
    except (IOError, httplib.HTTPException):
        reportError(Caught=True,
                    CaughtPrefix='+++',
                    CaughtCode='181',
                    CaughtMsg='Caught IO/HTTP Error in photos.search')
    except:
        reportError(Caught=True,
                    CaughtPrefix='+++',
                    CaughtCode='182',
                    CaughtMsg='Caught exception in photos.search',
                    exceptSysInfo=True)
    finally:
        if searchIsUploaded is None or not self.isGood(searchIsUploaded):
            logging.error('searchIsUploadedOK:[{!s}]'
                          .format('None'
                                  if searchIsUploaded is None
                                  else self.isGood(searchIsUploaded)))
            # CODING: how to indicate an error... different from False?
            # Possibly raising an exception?
            # raise Exception('photos_search: Max attempts exhausted.')
            niceprint('return: IS_PHOTO_UPLOADED: ERROR#1',
                      fname='is_photo_already_uploaded')
            logging.warning('return: IS_PHOTO_UPLOADED: ERROR#1')
            return returnIsPhotoUploaded, \
                   returnPhotoUploaded, \
                   returnPhotoID, \
                   returnUploadedNoSet

    logging.debug('searchIsUploaded:')
    logging.debug(xml.etree.ElementTree.tostring(searchIsUploaded,
                                                 encoding='utf-8',
                                                 method='xml'))

    # Number of pics with specified checksum
    returnPhotoUploaded = int(searchIsUploaded
                              .find('photos').attrib['total'])        # this is row 3581 reported on the Traceback below

Logging.

[1961][2018.02.21 19:10:01]:[MainProcess][WARNING ]:[root] ___Retry f():[R_photos_search] Max:[3] Delay:[20] Rnd[True]
[1961][2018.02.21 19:10:01]:[MainProcess][WARNING ]:[root] ___Retry f():[R_photos_search] arg[0]={'extras': 'tags', 'user_id': 'me', 'tags': 'checksum:b8d88bfc7feecc4fbe8e51b30e8e4680'}
[1961][2018.02.21 19:10:01]:[MainProcess][WARNING ]:[root] ___Retry f():[R_photos_search]: Attempt:[1] of [3]
[1961][2018.02.21 19:10:01]:[MainProcess][INFO    ]:[flickrapi.core] Calling {'nojsoncallback': 1, 'method': 'flickr.photos.search', 'format': 'etree'}
[1961][2018.02.21 19:10:01]:[MainProcess][DEBUG   ]:[flickrapi.core] Wrapping call <bound method FlickrAPI._flickr_call of [FlickrAPI for key "[secure]"]>(self, (), {'user_id': 'me', 'tags': 'checksum:b8d88bfc7feecc4fbe8e51b30e8e4680', 'format': 'rest', 'extras': 'tags', 'timeout': None, 'nojsoncallback': 1, 'method': 'flickr.photos.search'})
[1961][2018.02.21 19:10:01]:[MainProcess][DEBUG   ]:[flickrapi.core] Calling {'nojsoncallback': 1, 'format': 'rest', 'tags': 'checksum:b8d88bfc7feecc4fbe8e51b30e8e4680', 'extras': 'tags', 'user_id': 'me', 'method': 'flickr.photos.search'}
[1961][2018.02.21 19:10:01]:[MainProcess][DEBUG   ]:[requests_oauthlib.oauth1_auth] Signing request <PreparedRequest [POST]> using client <Client nonce=None, signature_method=HMAC-SHA1, realm=None, encoding=utf-8, timestamp=None, resource_owner_secret=****, decoding=utf-8, verifier=None, signature_type=AUTH_HEADER, rsa_key=None, resource_owner_key=[secure], client_secret=****, callback_uri=None, client_key=[secure]>
[1961][2018.02.21 19:10:01]:[MainProcess][DEBUG   ]:[requests_oauthlib.oauth1_auth] Including body in call to sign: True
[1961][2018.02.21 19:10:01]:[MainProcess][DEBUG   ]:[oauthlib.oauth1.rfc5849] Collected params: [(u'oauth_nonce', u'78127102800448378911519240201'), (u'oauth_timestamp', u'1519240201'), (u'oauth_consumer_key', u'[secure]'), (u'oauth_signature_method', u'HMAC-SHA1'), (u'oauth_version', u'1.0'), (u'oauth_token', u'[secure]'), (u'nojsoncallback', u'1'), (u'format', u'rest'), (u'tags', u'checksum:b8d88bfc7feecc4fbe8e51b30e8e4680'), (u'extras', u'tags'), (u'user_id', u'me'), (u'method', u'flickr.photos.search')]
[1961][2018.02.21 19:10:01]:[MainProcess][DEBUG   ]:[oauthlib.oauth1.rfc5849] Normalized params: extras=tags&format=rest&method=flickr.photos.search&nojsoncallback=1&oauth_consumer_key=[secure]&oauth_nonce=78127102800448378911519240201&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1519240201&oauth_token=[secure]&oauth_version=1.0&tags=checksum%3Ab8d88bfc7feecc4fbe8e51b30e8e4680&user_id=me
[1961][2018.02.21 19:10:01]:[MainProcess][DEBUG   ]:[oauthlib.oauth1.rfc5849] Normalized URI: https://api.flickr.com/services/rest/
[1961][2018.02.21 19:10:01]:[MainProcess][DEBUG   ]:[oauthlib.oauth1.rfc5849] Signing: signature base string: POST&https%3A%2F%2Fapi.flickr.com%2Fservices%2Frest%2F&extras%3Dtags%26format%3Drest%26method%3Dflickr.photos.search%26nojsoncallback%3D1%26oauth_consumer_key%3D[secure]%26oauth_nonce%3D78127102800448378911519240201%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1519240201%26oauth_token%3D[secure]%26oauth_version%3D1.0%26tags%3Dchecksum%253Ab8d88bfc7feecc4fbe8e51b30e8e4680%26user_id%3Dme
[1961][2018.02.21 19:10:01]:[MainProcess][DEBUG   ]:[oauthlib.oauth1.rfc5849] Signature: 8x5r8far8hw2BWQXLTxZxT4se8E=
[1961][2018.02.21 19:10:01]:[MainProcess][DEBUG   ]:[oauthlib.oauth1.rfc5849] Encoding URI, headers and body to utf-8.
[1961][2018.02.21 19:10:01]:[MainProcess][DEBUG   ]:[requests_oauthlib.oauth1_auth] Updated url: https://api.flickr.com/services/rest/
[1961][2018.02.21 19:10:01]:[MainProcess][DEBUG   ]:[requests_oauthlib.oauth1_auth] Updated headers: {'Content-Length': '128', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'User-Agent': 'python-requests/2.18.4', 'Connection': 'keep-alive', 'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'OAuth oauth_nonce="78127102800448378911519240201", oauth_timestamp="1519240201", oauth_version="1.0", oauth_signature_method="HMAC-SHA1", oauth_consumer_key="[secure]", oauth_token="[secure]", oauth_signature="8x5r8far8hw2BWQXLTxZxT4se8E%3D"'}
[1961][2018.02.21 19:10:01]:[MainProcess][DEBUG   ]:[requests_oauthlib.oauth1_auth] Updated body: 'nojsoncallback=1&format=rest&tags=checksum%3Ab8d88bfc7feecc4fbe8e51b30e8e4680&extras=tags&user_id=me&method=flickr.photos.search'
[1961][2018.02.21 19:10:01]:[MainProcess][DEBUG   ]:[urllib3.connectionpool] https://api.flickr.com:443 "POST /services/rest/ HTTP/1.1" 200 116
[1961][2018.02.21 19:10:01]:[MainProcess][INFO    ]:[flickrapi.core] REST Parser: using xml.etree.cElementTree
[1961][2018.02.21 19:10:01]:[MainProcess][DEBUG   ]:[root] searchIsUploaded:
[1961][2018.02.21 19:10:01]:[MainProcess][DEBUG   ]:[root] <rsp stat="ok">
<photos page="1" pages="0" perpage="100" total="" />
</rsp>
Traceback (most recent call last):
  File "uploadr.py", line 4623, in <module>
    flick.upload()
  File "uploadr.py", line 1342, in upload
    success = self.uploadFile(lock=None, file=file)
  File "uploadr.py", line 1828, in uploadFile
    setName)
  File "uploadr.py", line 3581, in is_photo_already_uploaded
    .find('photos').attrib['total'])
ValueError: invalid literal for int() with base 10: ''

sybrenstuvel commented 6 years ago

If you want to figure out which one is causing the issue (your local software, which includes the FlickrAPI library, or Flickr itself), visit the Flickr API service page and pass the same parameters there as you use in your test case. Then you'll be able to inspect the response body by yourself.

I haven't changed anything in the past days; such a change would also require you to download and install a new version of the FlickrAPI library, so you would have known.

oPromessa commented 6 years ago

Thanks for your feedback. Yes I had already checked with the Flickr App Garden and output would be with a zero (total="0").

Anyhow for some reason, the same way it started occurring it stopped, so I'm guessing it was related with some transient issue on Flickr side.

I'm closing this issue for now.

Thanks, oPromessa

sybrenstuvel / flickrapi