Embedding and billing

[Martin-F]

Hi,

We don't entirely follow how "embedding" interacts with your API call billing structure, and were hoping you could clear things up.

It'd be easiest to explain with an example: let's say, for the sake of argument, that there's a customer with 40 properties, each of which has 25 images, and we want to fetch the image data at the same time as the properties.

If we made an API call like "/properties?embed=images&pageSize=100", we'd get back a response with all of the properties, and all 1000 of their images. But how many API calls would that count as for billing purposes? It'd be one call for the properties, but would the additional calls for the embedded images be billed as, say...

• one call, because each property has only one page of images at pageSize=100;
• ten calls, because 1000 images is ten pages at that pageSize;
• or 40 calls, one for each property?

Or, for comparison, if we made a call like "/properties?embed=images&pageSize=10", we'd get back a response with only 10 properties, and 250 images between them. In that case, would the additional calls for the embedded images be billed as, say...

• three calls, because each property has three pages of images at pageSize=10;
• 25 calls, because 250 images is 25 pages at that pageSize;
• or 30 calls, three for each property in the response?

Or maybe we've got this all wrong, and the actual numbers would be something else entirely?

Your clarification would be greatly appreciated.

[HollyJoyPhillips]

Hi @Martin-F, can you confirm you've reviewed our documentation with regards to embedding? https://foundations-documentation.reapit.cloud/faqs#embedding

[github-actions[bot]]

We have recently requested additional information relating to the issue you have raised. Please can you take the time to review this ticket and where applicable, provide the information requested. For more information on our processes, please click here

[Martin-F]

Hi @HollyJoyPhillips - yes, I've seen that. It's ambiguous, that's why I'm here seeking clarification.

[plittlewood-rpt]

Hi @Martin-F

The embed mechanism is simply for convenience and makes no difference to how you are billed versus if you made the requests yourself. That said, it can indeed be a bit hard to get your head around as it's impossible to really know before you make the request how many subresources you'll end up fetching, and therefore you don't know how many pages will be fetched in the background.

Taking your lighter weight example above (/properties?embed=images&pageSize=10) where each property has 25 images, that's a total of 250 images.

You'll be billed for: 1 call to the properties API 3 calls to the propertyImages API since we always use the maximum page size of 100 for any embed operations

The internal calls for embedded resources use the full collection of parent resource ids from the initial result set (in this case 10 property ids) which are all passed to the respective service that's providing the embedded resources (in this case the propertyImages API). It makes a call in the format GET /embeddedResourceEndpoint?id={firstResourceId}&id={secondResourceId} and so on.

It looks like the documentation is slightly out of date as we did used to pass the pageSize parameter down to the services handling the embed operations, however to give developers a bit of leeway with the billing aspect, we took the decision to always use the maximum page size when processing the embedded resources as it also reduced latency due to the lower number of pages that need to be iterated. I'll arrange for this to be updated

I hope this helps clear this up, but please let me know if anything is still unclear.

Thanks Pete

[Martin-F]

Hi @plittlewood-rpt,

Yes, I think that does clarify things nicely - that change since the docs were last revised is particularly good to know about, thanks for that.

Just to double-check I've understood you 100% correctly, I now understand that:

• In my first example, we'd be billed for one properties call and ten propertyImages calls;
• but if we called "/properties?embed=images&pageSize=50", that would also be one properties call and ten propertyImages calls, because 50 is still enough for all the properties, and the embedded call is always pageSize=100.

Have I got that right?

[plittlewood-rpt]

Hi @Martin-F

Yes that's correct :) if there were 51 properties in your results but the total number of images overall was still 1000 you'd have an additional call to make to get the 51st property which would be on a separate page.

Thanks

[Martin-F]

Hi @plittlewood-rpt,

Great, thank you - I think that's my original question answered to perfection!

...but I do have another one now - this bit you wrote got me thinking about how we might yet squeeze a little more juice out of our API calls:

The internal calls for embedded resources use the full collection of parent resource ids from the initial result set (in this case 10 property ids) which are all passed to the respective service that's providing the embedded resources (in this case the propertyImages API). It makes a call in the format GET /embeddedResourceEndpoint?id={firstResourceId}&id={secondResourceId} and so on.

Am I right in thinking, as an alternative to embedding, we could do that "many IDs in the query-string" trick ourselves, and it would still only count as one API call per page of results, no matter how many IDs we passed in?

The scenario I'm thinking of is when we need to filter down the list of properties at our end, by some criterion that can't be expressed in the parameters of a /properties API call - we wouldn't need the images for any properties we're discarding, obviously, so it'd be a waste of API calls to embed them.

Let's take another worked example: say a client has 300 properties with 20 images each, but we only actually need 200 of those properties.

If we pulled all the data at once, by paging through "/properties?pageSize=100&embed=images", that'd be...

• three API calls for the properties;
• then, for each page of properties, 20 embedded API calls to get that page's 2000 images;
• so (1 + 20) × 3 = 63 API calls to get the lot.

But instead we could...

• page through "/properties?pageSize=100" to get all 300 properties;
• select the 200 properties we actually want, and collate their IDs;
• build an alarmingly long URL, of the form "/propertyImages?pageSize=100&propertyId=ABC000001&propertyId=ABC000002&...&propertyId=ABC000200" with 200 propertyId parameters, one for each the properties we're interested in;
• then page through that (200 × 20 = 4000 images, so 40 pages) to get just the images we're after;
• and we'd have the data we needed, while only running up a bill for 43 API calls.

Does that work?

[plittlewood-rpt]

Sort of. You're always bound to the maximum page size of 100 resources, but yes you can pass multiple ids in the query string as per the Swagger docs exactly as your example call to the propertyImages service is doing above.

Ultimately the embed mechanism is just for convenience to reduce round trips and make things a little easier for developers but it does come with it's problems, particularly on larger datasets. Each embed operation adds latency. Some developers choose not to use it so they retain as much control as possible, which for more advanced applications is probably sensible, particularly for more seasoned developers.

[Martin-F]

Got it - can I just confirm that the number of propertyId parameters doesn't have any effect on the number of API calls counted for billing purposes (except insofar as more property IDs may result in more pages of results, as each page would require a separate API call)?

Also, I've done a few experiments since my last reply, and have found that passing 200 propertyId parameters works as I'd expect, as does 300, even 350... but passing 400 property IDs fails with HTTP/1.1 414 Request-URI Too Long. What's the limit there exactly? Is it based on the number of parameters in the query-string, the length of the URL in bytes, something else?

(Here's the full response to my "400 property IDs" request, in case it's of any use to you.)

HTTP/1.1 414 Request-URI Too Long Date: Fri, 26 Apr 2024 08:08:03 GMT Content-Type: application/json Content-Length: 171 Connection: keep-alive x-amzn-RequestId: 8e9f6c85-206d-4f0a-9404-d2bf6f27a1e4 x-amzn-Remapped-Connection: close x-amz-apigw-id: W0wToGMpLPEEZ3Q= x-amzn-Remapped-Server: Server X-Amzn-Trace-Id: Root=1-662b60e3-3a2b0ac92343b5bb1486c62c;Parent=00dc4a73c96c8b0c;Sampled=0;lineage=a4a37df8:0|62bc53eb:0 x-amzn-Remapped-Date: Fri, 26 Apr 2024 08:08:03 GMT

414 Request-URI Too Large

---

Server

[plittlewood-rpt]

The number of parameters has no bearing on billing no. It's how many physical calls you make. You'd never want to pass more than 100 ids as you can't physically retrieve more than that so you'd need to be a bit sensible about how you structure your calls.

I believe the maximum length of the full URI is 8kb but I'd need to run some tests to confirm that.

[Martin-F]

Maybe images make for a bad example in this case, as the vast majority of properties will have at least one, usually several - but if we were trying to fetch something that a lot of properties might not have any of (say, appointments or tenancies), surely it could make sense to pass more than 100 property IDs?

Eg, if we had a list of 200 properties, which happened to have 100 tenancies scattered between them, we could pass all 200 property IDs to "/tenancies?pageSize=100" to find them all in one go.

[plittlewood-rpt]

I've just checked and the limit should be considered as 8192 characters (8Kb). It's likely that the APIs will currently accept up to 10,240 characters however this may change in the future due to infrastructure constraints