Query images yields bad request

[eor314]

Querying images with GeoImageConstraints returns a bad request to url ValueError: Bad request: http://fathomnet.org:8080/images/query.

from fathomnet.api import images
from fathomnet.models import GeoImageConstraints

constraints = GeoImageConstraints(
    concept='Actinernus',
    maxLatitude=37.0538,
    minLatitude=36.4458,
    maxLongitude=-121.7805,
    minLongitude=-122.5073,
    startTimestamp='2007-08-02',
    minDepth=1300
)

imgs = images.find(constraints)

It looks like the ROOT of the endpoint manger in fathomnet/api/init.py needs to be updated?

[hohonuuli]

Actinerus doesn't return any values, i.e. the HTTP response body is []. @eor314 If you sub in something like: Nanomia what happens?

@kevinsbarnard Maybe the python library just needs to be modified to handle empty return values?

[hohonuuli]

Sanity check with a simplified curl request:

> curl -X 'POST' \
  'http://fathomnet.org:8080/images/query' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "concept": "Actinerus",
  "offset": 0
}'

[]

[kevinsbarnard]

Looks like it's returning an error due to the startTimestamp format. This is pretty strict on the ISO 8601 formatting... should be something like:

startTimestamp='2007-08-02T00:00:00.000Z'

[kevinsbarnard]

Two lessons here:

1. The error message (from fathomnet-py) is not very helpful when the HTTP response's status code is in the 400's
2. Timestamp formatting is not forgiving

I'll open an issue to improve each of these.

[eor314]

Right on, it was the timestamp format in my case. Thanks for helping with troubleshooting.

Kevin, how did you increase the verbosity on the errors? I think I could have sorted it out with a bit more information.

[kevinsbarnard]

There isn't a great way to increase verbosity like you describe, but there's a function in fathomnet-py called fathomnet.util.debug_format_response.

In this case I just put a debug print statement where the error was raised:

elif res.status_code < 500:  # User error
    print(debug_format_response(res))
    raise ValueError('Bad request: {} {}'.format(method, url))

which gave me

REQUEST:
    Method: POST
    URL: http://fathomnet.org:8080/images/query
    Headers:
        User-Agent: python-requests/2.28.2
        Accept-Encoding: gzip, deflate
        Accept: */*
        Connection: keep-alive
        Content-Length: 406
        Content-Type: application/json

    Body:
    b'{"concept": "Actinernus", "taxaProviderName": null, "contributorsEmail": null, "startTimestamp": "2007-08-02", "endTimestamp": null, "imagingTypes": null, "includeUnverified": null, "includeVerified": null, "minLongitude": -122.5073, "maxLongitude": -121.7805, "minLatitude": 36.4458, "maxLatitude": 37.0538, "minDepth": 1300, "maxDepth": null, "ownerInstitutionCodes": null, "limit": null, "offset": null}'

RESPONSE:
    Status: 400
    Headers:
        Content-Type: application/json
        date: Tue, 24 Jan 2023 01:00:25 GMT
        content-length: 567
        connection: keep-alive
    Content:
    b'{"message":"Bad Request","_links":{"self":{"href":"/images/query","templated":false}},"_embedded":{"errors":[{"message":"Failed to convert argument [constraints] for value [null] due to: Cannot deserialize value of type `java.time.Instant` from String \\"2007-08-02\\": Failed to deserialize java.time.Instant: (java.time.format.DateTimeParseException) Text \'2007-08-02\' could not be parsed at index 10\\n at [Source: UNKNOWN; byte offset: #UNKNOWN] (through reference chain: org.fathomnet.repositories.GeoImageConstraints[\\"startTimestamp\\"])","path":"/constraints"}]}}'

[hohonuuli]

@eor314 Just FYI, the one timestamp to rule them all ... https://en.wikipedia.org/wiki/ISO_8601