IGSDB returns only 100 optical data sets

[christoph-maurer]

@danielmcquillen When I query https://igsdb-v2.herokuapp.com/graphql/ with

{
  allOpticalData {
    edges {
      node {
        id
        uuid
        opticalDataId
        componentId
      }
    }
  }
}

I receive only 100 optical data sets. I have expected around 6800 optical data sets. Can IGSDB return all of them?

We need to implement planning software which uses IGSDB. I hope that it is easy to fix for you.

[christoph-maurer]

When I query for data sets with a minimum visible transmittance of 0.8, I also receive the metadata of only 100 data sets although there are more in IGSDB.

{
  allOpticalData(
    where: {nearnormalHemisphericalVisibleTransmittances: {some: {greaterThanOrEqualTo: 0.8}}}
  ) {
    edges {
      node {
        componentId
        nearnormalHemisphericalVisibleTransmittances
      }
    }
  }
}

[danielmcquillen]

@christoph-maurer The GraphQL endpoint is set up to use the Relay convention for paging. You need to include the pageInfo object...(more in the simple docs page I wrote: https://igsdb-v2.herokuapp.com/docs/help/using-api/graphql)

{
  allOpticalData(
    where: {nearnormalHemisphericalVisibleTransmittances: {some: {greaterThanOrEqualTo: 0.8}}}
  ) {
    edges {
      node {
        componentId
        nearnormalHemisphericalVisibleTransmittances
      }
    }
    pageInfo {
        hasNextPage
        startCursor
        endCursor
    }
  }
}

Which gives a pagination object in the results (at the bottom):

      "pageInfo": {
        "hasNextPage": true,
        "startCursor": "YXJyYXljb25uZWN0aW9uOjA=",
        "endCursor": "YXJyYXljb25uZWN0aW9uOjk5"
      }
    }

The problem we have is we're hosting on Heroku, which has a 30 second request timeout. If we try to return all results I think we sometimes exceed that limit and the response cuts out. However, I haven't tried recently so I guess I can set up the default query to try to return all results and we'll see what happens!

I'll look into this today or early next week, but in the meantime can you use the paging mechanism?

[christoph-maurer]

@danielmcquillen Thanks for trying! The API specification allows pagination. When pagination is not requested, the metadata should be returned of all data sets which fulfil the where argument. Hopefully, Heroku can respond all data sets. Sure, in the meantime, I'll use the paging mechanism for manual examples with the other project partners.