search

ThreeSixtyGiving / registry-classic

Registry for 360Giving files that were published
Other
0 stars 0 forks source link

What is the order of records when a publisher has more than one? #40

Closed stevieflow closed 5 years ago

stevieflow commented 5 years ago

Take a look at the big lottery:

data registry 360giving 1

It isn't clear how this order is generated. @BibianaC please can you clarify? We can then discuss if we want to alter this

Observed by @KDuerden

robredpath commented 5 years ago

Right now, they're basically unordered.

@BibianaC has proposed an ordering in #41 , which would sort by the date of the first grant in the file. How does that sound @stevieflow @KDuerden?

KDuerden commented 5 years ago

@robredpath I don't understand - 'sort by the date of the first grant in the file' First means what? Most recently awarded grant or the first to be awarded (eg oldest)? Top in the file? And this will apply across several files so - 'first' based on all the grants in all the files for a particular publisher?

BibianaC commented 5 years ago

@KDuerden the table will be sorted by publisher name (as it is now) and if the publisher has more than one grant, these ones will be sorted by year (based on the min_award_date field).

screen shot 2018-11-29 at 15 48 09
KDuerden commented 5 years ago

@BibianaC thanks for clarifying. We need it to be the opposite - ordered by max_award_date field This will be most helpful for the cases of name change - which is what prompted this ticket - as outlined GN behaviour here

BibianaC commented 5 years ago

@KDuerden I can change it to max_award_date. Does this mean that you also want it in reverse order? The newest first? In the case of the Big Lottery the first to be 'Big Lottery Fund - grants data 2018-19 year-to-date'?

KDuerden commented 5 years ago

Yes the most recent first please.

BibianaC commented 5 years ago

@KDuerden I have done the change. Thanks for the feedback.

KDuerden commented 5 years ago

The order of files on the Registry doesn't seem to be sorted by year based on the max_award_date field. Has this change not been push to live? I'm sure I had seen this happen.

robredpath commented 5 years ago

@KDuerden So, the code to do this is definitely live, and it looks like it's ordered by max_award_date descending apart from Lottery and SCVO. Which is odd.

@BibianaC can you take a look at this when you get a chance? I've checked that the latest code is deployed on the server and I can't see why the code would behave differently in either of those cases. SCVO has one file that's invalid that I can understand might mess with ordering, but all the Lottery data seems fine.

https://console.cloud.google.com/storage/browser/datagetter-360giving-output/branch/master/?project=datagetter-360giving&organizationId=980843748997 is the bucket with the raw data if, like me, you don't have it bookmarked.

KDuerden commented 5 years ago

Heart of England, John Moores and Seafarers UK are also out of order if based on max award date.

BibianaC commented 5 years ago

@robredpath @KDuerden The problem is that the sorting is done by year without taking in account the month. We will have to fix that.

BibianaC commented 5 years ago

@KDuerden we have fixed a few publishers but SCVO and the Heart of England are still not correct.

We will investigate further in the next non-sprint slot. I have one after the Cove 360 sprint.

KDuerden commented 5 years ago

Thank you, and that's fine. Heart of England looks in order to me now. With SCVO potentially a json thing?

eta - sorry missed that HoE is still out of order, because the latest file is at the top.

BibianaC commented 5 years ago

@KDuerden I think we need to check properly why these two are not correct.

BibianaC commented 5 years ago

The order is now correct.