eventespresso / eea-rest-api

Event Espresso 4 Rest API
GNU General Public License v2.0
6 stars 2 forks source link

Reading Equivalent of Registration CSV Report #44

Open Cowthulu opened 1 year ago

Cowthulu commented 1 year ago

Hello -

I am trying to use the EE REST API to essentially read all of the data that is present in the report that you get when you run the Registration CSV export. It seems like I can assemble the data, but it requires a lot of sub-queries. A few questions:

  1. Is there a more straightforward method to read that data?
  2. If not, is there a recommended approach (start with registrations? start with transactions? etc.)
  3. I notice that putting in a call for attendees (https:///wp-json/ee/v4.8.36/attendees gives a lot of old data and doesn't include up-to-date data. Is this a bug or am I misunderstanding what this call does? Is there a different call to get a list of attendees?
tn3rb commented 1 year ago
  1. Is there a more straightforward method to read that data?

LOLZ, no sorry, uh... welcome to REST APIs?

Unfortunately with any REST API there is a 1:1 relationship between an endpoint and a specific resource. Any kind of relational data will require multiple queries to retreive that data. There really is no way around that other than to use something other than REST, such as GraphQL which can return a full set of relational data in a single query. But unfortunately, our GraphQL implementation is not completed fully and therefore can most likely not be used to retrieve the data you require.

  1. If not, is there a recommended approach

That kinda depends on what you are doing with the data once you've received it. If you are intending to generate objects out of the data and need to maintain relationships between the objects, then I would probably start with top level objects like Transactions which are on the "1" side of 1:many relationships. But even this is totally arbitrary depending on what you are going to do with the data and how you write code. So ultimately it's kinda up to you to do whatever you want. Sorry I don't have a better answer.

  1. I notice that putting in a call for attendees gives a lot of old data and doesn't include up-to-date data.

In Event Espresso, an attendee is our data model for a person that has attended an event. Their record of having attended an event is saved in the Registraion model. An attendee can have multiple registrations associated with them (such as a returning customer). New attendee records are only created whenever a new uniqiue person has been identified. This is done by comparing the person's first name, last name, and email address. If an existing attendee record is found with those three items matching, then no new record is created and the existing one is used. Therefore, attendee records with lower IDs represent attendees that have been in the system longer, but do not necessarily mean they do not have any current related registrations. What you are observing is likely fairly typical, but is also a result of how people are attending your events (ie: repeat customers will use an existing attendee record if the name and email fields match, but you are seeing more unique one time attendees).

Now, all of that said, if you are finding that too many attendee records associated with old registrations are being returned and you are having to perform too many "paging" queries to retreive the rest of the data, then you have (at least) a couple of options.

You could try:

Is this a bug or am I misunderstanding what this call does? Is there a different call to get a list of attendees?

It sounds to me like your definition of an attendee is more like our definition of a registration. It's confusing because the terminology can easily have a lot of overlap. A good way to understand our definitions is to imagine a use case where you have a multiday conference with multiple workshops occurring during the conference. Let's assume you are setting things up in the following manener:

John Doe decides to go to the conference and sign up for three workshops. This will result in the following records being created (not inculding records for the above entities):

So even though you have a total of 4 registration records created for that single transaction, there is only one attendee record created for John Doe, because he was the same attendee for the conference and each workshop.

I hope that helps you out and good luck with whatever awesome thing you are building.

Cowthulu commented 1 year ago

Brent –

Thank you for your quick response – it is appreciated.

Basically, I just want to denormalize the data and get something like Name of Person, Item purchased, Amount Paid. I can do it the hard way, though.

One issue – I don’t see any real support for pagination – just the ability to increase the page limit. I have a small enough dataset that I can probably get away with that (and testing that I am now seeing all the attendees), but since you mentioned pagination, I’m wondering if there is a way to do that?

Again, thanks!

Arlen

From: Brent Christensen @.> Sent: Friday, November 25, 2022 6:21 PM To: eventespresso/eea-rest-api @.> Cc: Arlen Feldman @.>; Author @.> Subject: Re: [eventespresso/eea-rest-api] Reading Equivalent of Registration CSV Report (Issue #44)

  1. Is there a more straightforward method to read that data?

LOLZ, no sorry, uh... welcome to REST APIs?

Unfortunately with any REST API there is a 1:1 relationship between an endpoint and a specific resource. Any kind of relational data will require multiple queries to retreive that data. There really is no way around that other than to use something other than REST, such as GraphQL which can return a full set of relational data in a single query. But unfortunately, our GraphQL implementation is not completed fully and therefore can most likely not be used to retrieve the data you require.

  1. If not, is there a recommended approach

That kinda depends on what you are doing with the data once you've received it. If you are intending to generate objects out of the data and need to maintain relationships between the objects, then I would probably start with top level objects like Transactions which are on the "1" side of 1:many relationships. But even this is totally arbitrary depending on what you are going to do with the data and how you write code. So ultimately it's kinda up to you to do whatever you want. Sorry I don't have a better answer.

  1. I notice that putting in a call for attendees gives a lot of old data and doesn't include up-to-date data.

In Event Espresso, an attendee is our data model for a person that has attended an event. Their record of having attended an event is saved in the Registraion model. An attendee can have multiple registrations associated with them (such as a returning customer). New attendee records are only created whenever a new uniqiue person has been identified. This is done by comparing the person's first name, last name, and email address. If an existing attendee record is found with those three items matching, then no new record is created and the existing one is used. Therefore, attendee records with lower IDs represent attendees that have been in the system longer, but does not necessarily mean they do not have any current related registrations. What you are observing is likely fairly typical, but is also a result of how people are attending your events (ie: repeat customers will use an existing attendee record if the name and email fields match). Now, all of that said, if you are finding that too many attendee records associated with old registrations are being returned and you are having to perfomr too many "paging" queries to retreive the rest of the data, then you have a couple of options (at least). You could try:

Is this a bug or am I misunderstanding what this call does? Is there a different call to get a list of attendees?

It sounds to me like your definition of an attendee is our definition of a registration. It's confusing because the terminology can easily have a lot of overlap. A good way to understand our definitions is to imagine a use case where you have a multiday conference with multiple workshops occurring during the conference. Let's assume you are setting things up in the following manener:

John Doe decides to go to the conference and sign up for three workshops. This will result in the following records being created (not inculding records for the above entities):

So even though you have a total of 4 registration records created for that single transaction, there is only one attendee record created for John Doe, because he was the same attendee forthe conference and each workshop.

I hope that helps you out and good luck with whatever awesome thing you are building.

— Reply to this email directly, view it on GitHubhttps://github.com/eventespresso/eea-rest-api/issues/44#issuecomment-1327951389, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AKV6WSY3IB2K5VMUFQBH55LWKFQWPANCNFSM6AAAAAASLVX2H4. You are receiving this because you authored the thread.Message ID: @.**@.>>

tn3rb commented 1 year ago

I just want to denormalize the data and get something like Name of Person, Item purchased, Amount Paid. I can do it the hard way, though.

unfortunately with REST there is only the hard way when you have relational data

I don’t see any real support for pagination... I’m wondering if there is a way to do that?

AFAIK there is no built in way to do that and you just have to manually set your page size and offset.

Cowthulu commented 1 year ago

When you say “manually set your page size and offset” what do you mean? That would imply that there was a way to specify a page size and an offset and I don’t see anything like that…

Again, I appreciate the help. One thing about REST – there is no requirement that it has to model your data structures. In fact, it is generally not considered appropriate to expose your underlying data structures via an API unless it is extremely low-level. Not a criticism—just an alternate way of looking at the topic.

Thanks!

Arlen

From: Brent Christensen @.> Sent: Friday, November 25, 2022 7:45 PM To: eventespresso/eea-rest-api @.> Cc: Arlen Feldman @.>; Author @.> Subject: Re: [eventespresso/eea-rest-api] Reading Equivalent of Registration CSV Report (Issue #44)

I just want to denormalize the data and get something like Name of Person, Item purchased, Amount Paid. I can do it the hard way, though.

unfortunately with REST there is only the hard way when you have relational data

I don’t see any real support for pagination... I’m wondering if there is a way to do that?

AFAIK there is no built in way to do that and you just have to manually set your page size and offset.

— Reply to this email directly, view it on GitHubhttps://github.com/eventespresso/eea-rest-api/issues/44#issuecomment-1327965990, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AKV6WS6JJEQOXWQSTUE7A3DWKF2T5ANCNFSM6AAAAAASLVX2H4. You are receiving this because you authored the thread.Message ID: @.**@.>>

tn3rb commented 1 year ago

When you say “manually set your page size and offset” what do you mean? That would imply that there was a way to specify a page size and an offset and I don’t see anything like that…

our REST API implementation allows you to set the limit and offset. See here: https://github.com/eventespresso/event-espresso-core/blob/master/docs/C--REST-API/ee4-rest-api-GET-filtering-results.md#limit

One thing about REST – there is no requirement that it has to model your data structures. In fact, it is generally not considered appropriate to expose your underlying data structures via an API unless it is extremely low-level.

you are 100% correct, but that's not how the developer that built our REST API did things

Cowthulu commented 1 year ago

Awesome, thanks!

And, speaking as a developer, you can’t trust developers—they do weird things :-).

Arlen

From: Brent Christensen @.> Sent: Saturday, November 26, 2022 2:40 PM To: eventespresso/eea-rest-api @.> Cc: Arlen Feldman @.>; Author @.> Subject: Re: [eventespresso/eea-rest-api] Reading Equivalent of Registration CSV Report (Issue #44)

When you say “manually set your page size and offset” what do you mean? That would imply that there was a way to specify a page size and an offset and I don’t see anything like that…

our REST API implementation allows you to set the limit and offset. See here: https://github.com/eventespresso/event-espresso-core/blob/master/docs/C--REST-API/ee4-rest-api-GET-filtering-results.md#limit

One thing about REST – there is no requirement that it has to model your data structures. In fact, it is generally not considered appropriate to expose your underlying data structures via an API unless it is extremely low-level.

you are 100% correct, but that's not how the developer that built our REST API did things

— Reply to this email directly, view it on GitHubhttps://github.com/eventespresso/eea-rest-api/issues/44#issuecomment-1328118665, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AKV6WS5FX2E7VDAC3IXNSBDWKJ7UHANCNFSM6AAAAAASLVX2H4. You are receiving this because you authored the thread.Message ID: @.**@.>>