Generate number of agencies from providers.yml

18F / identity-analytics-etl

ETL and SQL scripts for Login.gov data warehouse and business intelligence

10 stars 6 forks source link

Generate number of agencies from providers.yml #176

Closed lauraGgit closed 5 years ago

lauraGgit commented 5 years ago

User story

As a prospective agency user, I would like to know how many agencies use LG, so that I can see if this is a commonly used service.

Notes

What is the value to the user in this story?* As a user, I want to know how to who is using this.
What are things we should consider when making this story *
this wouldn't be an sql query, but would use: https://github.com/18F/identity-idp/blob/master/config/service_providers.yml

Acceptance Criteria

[ ] As a biz person, I should can generate the number of agencies using this service

Tasks to complete the story

[ ] write script that can run this against the github
[ ] document
[ ] consider how this integrates with other SQL queries

Definition of Done

[ ] Pull requests meet technical definition of done
[ ] Usability tested

MacHu-GWU commented 5 years ago

@lauraGgit

Here's a small app demo: https://repl.it/@sanhe/ServiceProviderCount. And there's a problem:

service_providers.yml is not a pure data file, it is actually using template engine and dynamically rendering the content from ruby code.

A straight forward way could be reading counts from idp.service_providers table. And can be easily solved in admin console.

lauraGgit commented 5 years ago

@MacHu-GWU I'm not sure I see the demo?

And can you explain a bit more as to why it's not "pure data"?

MacHu-GWU commented 5 years ago

@lauraGgit it is not a parsable YML file. Because the idp ruby app dynamically load the content of the template engine, and dynamically render the real YML file, see https://github.com/18F/identity-idp/blob/master/config/service_providers.yml#L195, take a look at the <% if .... %> ... <% end %> part.

Which means we need idp app runtime config to generate the real content of service_providers.yml file. It is not doable by simply dealing with the file only.

MacHu-GWU commented 5 years ago

This issue will be solved in https://github.com/18F/identity-idp repo, @lauraGgit I think this issue can be closed when the PR / test are done in https://github.com/18F/identity-idp.

lauraGgit commented 5 years ago

Migrated to Jira