infovote-io / dashboard

https://infovote.io/
1 stars 0 forks source link

Save ETL dataframe in a `csv` file #5

Closed gabrielbdornas closed 1 year ago

gabrielbdornas commented 1 year ago

@danybonfil as we discussed, I need that the dataframe used to generate the graphs saved in a csv file.

danybonfil commented 1 year ago

Data from the FRED API retrieves Real Disposable Income (RDI) in $US levels I use simple log transform to calculate yearly growth: value = log(value / lag(value)) * 100)

A separate json list with presidents name is used for filtering data. For example, Joe Biden (2024) means his term would include years 2021, 2022, 2023, and 2024 when elections take place

Dataframe is saved as "df_data" in the main repository folder. NOTE: odata, presidents_o, and df_o are based on a static file from the original experiment saved in Dropbox.

image
danybonfil commented 1 year ago

Using Vega Lite, I was able to plot the usvalue and growth graphs but could not include the cumulative graph. This was caused because the csv had NA in the year 1960 which is the base year to calculate growth and for the year 2023 that has no values yet.

To solve this, I reopen this issue to include a line of code that removes NAs. I will also modify the contents of the csv that currently are: date (years), usvalue (RDI in dollars), and value (RDI %growth) to also include the cumulative growth (cumsum of RDI %growth) and the names of the presidents.

Image

gabrielbdornas commented 1 year ago

@danybonfil I'll close this issue. From now on, remember to use See #issue_number and Fix #issue_number to link issues and commits!