REST API for SESAR Statistics

lulin-song commented 5 years ago

On 6/4/19 7:30 AM, Kerstin Lehnert wrote:

How have sample types changes over the last 3-4 years (= number of samples registered by quarter grouped by object type)

How many samples are registered by institution over the past 3 years?

How many active users do we have by year (last 8 years if possible)?

Number of batches processed by month over past 8 years

lulin-song commented 4 years ago

How many samples are registered by institution over the past 3 years?

The following REST API is implemented.
- /v1/igsns/count/institution?start_date=2011-01-01&end_date=2012-01-01

lulin-song commented 4 years ago

How many active users do we have by year (last 8 years if possible)?

Implemented the new user count by year/month REST APIs

/v1/newusers/count/month
/v1/newusers/count/year

Implemented the active user count by year/month REST APIs

/v1/activeusers/count/month
/v1/activeusers/count/year

lulin-song commented 4 years ago

number of samples registered by quarter grouped by object type

Implemented the following rest api. Get the sample count by sample type

/v1/igsns/count/sampletype

lulin-song commented 4 years ago

Number of batches processed by month over past 8 years

There is not enough uploaded batch information in the database. I will leave this for future implementation. See newly created #17

ramdeensarah commented 4 years ago

How many samples are registered by institution over the past 3 years?

If provide a start date and no end date, search fails. http://grg-doc-dev.ldeo.columbia.edu:8082/v1/igsns/count/institution?start_date=2019-12-31

If provide an end date in the future, search is successful http://grg-doc-dev.ldeo.columbia.edu:8082/v1/igsns/count/institution?end_date=2020-06-12&start_date=2019-12-31
- Search results for this year seem incorrect. Only lists Ronin and CNRS. http://grg-doc-dev.ldeo.columbia.edu:8082/v1/igsns/count/institution?end_date=2020-06-12&start_date=2019-12-31
- Search for samples registered within 2019 tested for hideprivate (set to 1 and null) successfully shows a change in number http://grg-doc-dev.ldeo.columbia.edu:8082/v1/igsns/count/institution?end_date=2019-12-12&start_date=2019-01-01

How many active/new users do we have by year (last 8 years if possible)?
- Implemented the new user count by year REST APIs Question about results This shows 90 new users for 2019, but the results for active users in 2019 was only 83 (see below).

Implemented the new user count by month REST APIs Test appears successful. See note above.
Implemented the active user count by year REST APIs Question about results I assume these are unique visitors, why are the numbers so high in some past years if this is on sesar-test? Results "2005": "7", "2006": "2", "2007": "10", "2008": "4", "2009": "7", "2010": "17", "2011": "27", "2012": "35", "2013": "45", "2014": "55", "2015": "87", "2016": "114", "2017": "112", "2018": "87", "2019": "83", "2020": "2"
Implemented the active user count by month REST APIs Question about results -- This looks at the users on test-sesar. This seems low and there are months with gaps (was there no work on test-sesar during March and April?). Has Lulin logged into test-sesar this month? Should the count be at least 2?

Results "2020-06": "1", "2020-05": "1", "2020-02": "1", "2020-01": "2",

Implemented the following rest api. Get the sample count by sample type
- Execute with no end date and include unpublished metadata Test successful
- Execute with no end date and exclude unpublished metadata Test successful, some sample types show decrease in count.
- Execute with end date and include unpublished metadata Test successful. If put end date as a future date, test still successful.
- Execute with end date and exclude unpublished metadata Test successful, some sample types show decrease in count. If put end date as a future date, test still successful.

lulin-song commented 4 years ago

I updated the REST API and let the API connect to SESAR production database.

If provide a start date and no end date, search fails.

The end date is supposed to be required. I fixed the bug.

Implemented the new user count by year REST APIs Question about results This shows 90 new users for 2019, but the results for active users in 2019 was only 83 (see below).

New user count is from sesar_user table. If someone registered a sesar account, it will be counted.
Active user count is from sample table. If a user registered a sample, the user will be counted as 'active user'. If a new user registered an account but the user did not register any sample. This user is not counted in 'active user'.
This is what it is coded now but we can discuss how 'active user' to be counted.

Implemented the active user count by month REST APIs Question about results -- This looks at the users on test-sesar. This seems low and there are months with gaps (was there no work on test-sesar during March and April?). Has Lulin logged into test-sesar this month? Should the count be at least 2?

The test database has not been updated for a long time. I don't think I registered any sample in test database in March, April, 2020. I have switched the REST API to connect to production database, so the number reflects the real number now.

ramdeensarah commented 4 years ago

Comments on 'Implemented the new user count by year REST APIs' Recommend we clarify the phrasing of 'new user' to be (perhaps) 'new registered accounts' or a description for the 'new user' to indicate what you said - that they are newly registered accounts and may include accounts which have not registered any samples. And then we should probably define 'Active user' as well. Suggested text - 'Active user' reflects 'a count of unique user accounts which have registered samples during the range selected'.

Retested 'Implemented the active user count by month REST APIs' Test successful and look accurate (and look to be useful long term, clearly seeing trends etc that might be used in analysis/reports or perhaps metrics on the website - this particular set and others listed).

GeoSamples / sesar-rest-api