search

onepf / OpenAEP

Open Application Exchange Protocol (for stores)
3 stars 1 forks source link

Purchases: specify package, data range #2

Open oorlov opened 10 years ago

oorlov commented 10 years ago

Current Purchases Suggest following:

Mandatory:

Optional:

jesper-sjovall commented 10 years ago

For select data-range I suggest to use same format as discussed here. https://github.com/onepf/OpenAEP/issues/3#issuecomment-26695745

I think the 'country-code' also is a good selector for limit the result which also can be use for the Review request also.

oorlov commented 10 years ago

Updated version:

https://www.distributorappstore.com/openaep/purchases
?package=com.softspb.flashcards.sv - single parameter
[&datefrom=YYYY-MM-DD] - optional
[&dateto=YYYY-MM-DD] - optional
[&country=US] - optional
[&limit=100] - optional
[&offset=7j8ad9go] - optional

Do paginate output there should be offset param used together with limit. And we need to return it in output like this

<?xml version="1.0" encoding="UTF-8"?>
<purchases version="1" offset="7j8ad9go"> <!-- encoded value of request params, db state and offset -->
  <purchase>
...

Offset is critical here, nobody wants to lose financial data

jesper-sjovall commented 10 years ago

Yes offset is critical as you say, but I can see a problem here. How to known what next offset is to be use or misunderstand I you? For example: We has get the purchase 1 to 10 from www.distributorappstore.com, how to get purchase 11 to 20,

And in this case would it not be good to also include total number of purchase if "limit" or "offset" not was set to known how many pages this exists.

  <?xml version="1.0" encoding="UTF-8"?>
  <purchases version="1" offset="7j8ad9go" totalCount="123"> <!-- encoded value of request params, db state and offset -->
     <purchase>
  ...
oorlov commented 10 years ago

First time you specify only 'limit' param. Server returns non-empty offset if there is more purchases remained. If there is nothing to request next time - empty offset returns.

We have 27 purchases, Requests 10, gets purchases 0..9 and non-empty offset Requests 10 more, gets 10..19 and non-empty offset Requests 10 more, gets 20..26 and empty offset

If you decided to request data by 10 items bulks will you change it's size if get totalCount value? Another thought is: if client specify complex request and Purchases table is huge, doesn't it hurt performance to calculate totalCount for every request?

jesper-sjovall commented 10 years ago

Ok, 'offset' will point on the next page to fetch, it works but I think this is not the correct way to work, the main problem here is this only allow us to fetch data in one order, you can't from the current response fetch previous page of data.

The only reason I can see to do as your propose is if the data not has a natural indexing, but I think all data will has a natural indexing. (as my proposal to always sort the result in chronological order base on date and time for the Purchases).

oorlov commented 10 years ago

Could you help me to understand use case for fetching data in opposite order? I assumed batches is needed to retrieve big amount of data like purchases for a day or a week. Probably it doesn't work well for pulling data multiple times in a day

jesper-sjovall commented 10 years ago

True, this is probably a very uncommon case to read in opposite order. If we use as your suggests I think this is possible to pre create data before it is need if this is wanted on backend as a optimization. In this case I think we want to use limit xor offset:

Example:

https://www.distributorappstore.com/openaep/purchases?...&limit=10

Give result 0 to 9 and offset "7j8ad9go"

https://www.distributorappstore.com/openaep/purchases?...&offset=7j8ad9go

Give result 10 to 19 and offset "8iwadhj0"

A ide I have is how to optimize the result and for who, See the new issues https://github.com/onepf/OpenAEP/issues/5

morozstepan commented 10 years ago

Hi! Actually why are we using this "offset" in some magically-encoded form? I see problems in this approach. For example, I have 15 purchases of some product, all in one second. Client comes and asks about first 10 purchases. What should I encode in offset to give other portion of data for next request? I would have to save some information like "take all purchases from time

morozstepan commented 10 years ago

So, with all that stuff server system for providing feed will be either complicated, or non-consistent. What if we define page size (e.g. 10 items) and prepare these static pages on server? All we have to tell client is url to next page. Yes, it limits the control, but it's much easier to implement and very stable. What do you think?

jesper-sjovall commented 10 years ago

Morozstepan, have you read my issues nr https://github.com/onepf/OpenAEP/issues/5 I have in this issues a idea to precreate result-set base of day to day for minimize the load of the server, but the main problem with this is this will limit the customize of the fetch result.

I also see the problem with how to use "offset" but I think this will be no big problem, In my case I probably will use the database index or time-stamp. A other thing we can do here is to limit the result can't be fetch today that will prevent the system to fetch data that is change under the day.