raft-tech / TANF-app

Repo for development of a new TANF Data Reporting System
Other
16 stars 3 forks source link

Implement storage of parsed data in Elasticsearch #1354

Closed jtwillis92 closed 1 year ago

jtwillis92 commented 2 years ago

Description: Using the queuing system introduced in #1352 we will need to provide Django with the ability to index documents in Elasticsearch as they are parsed from uploaded Data Files. Once the file has been successfully parsed into a dictionary / JSON format the backend should queue a job which will ingest the data into Elasticsearch.

Acceptance Criteria: Create a list of functional outcomes that must be achieved to complete this issue

Tasks: Create a list of granular, specific work items that must be completed to deliver the desired outcomes of this issue

Notes:

Supporting Documentation:

Open Questions:

stevenino commented 2 years ago

Data type agnostic

andrew-jameson commented 1 year ago

How to get fancy with our data storage: https://django-elasticsearch-dsl-drf.readthedocs.io/en/latest/advanced_usage_examples.html#sample-document

Credit to @jtimpe

lfrohlich commented 1 year ago

@Smithh-Co if a PR isn't merged, we should note why this was closed. Per @ADPennington it was covered in #1351 and #132 instead and we decided to close last week in backlog refinement

Smithh-Co commented 1 year ago

@lfrohlich Hey Lauren, great call out - apologies, missed adding the notation here.