search

michelleg06 / Educational_Outcomes_Ecuador

Educational and Labour Networks
2 stars 1 forks source link

translate/SaveAndMergePanel #4

Closed sipheleledanisa closed 3 years ago

AFbenHassine commented 3 years ago

I decided on using feather to save the merged dataframe (install as feather-format).

AFbenHassine commented 3 years ago

Ok, I think it may be better to leave the saving and loading. I am unable to store the merged panel as pickle or as ftr, getting memory errors. It currently takes 59 seconds to concatenate the dataframes and load them in the console. I think that will do.

The annual data is also in the drive. If anyone has trouble working with the data without storing it as pickle or ftr or something else, we can see whether we can perhaps export it in chunks.

AFbenHassine commented 3 years ago

Should be completed with commit https://github.com/michelleg06/Educational_Outcomes_Ecuador/commit/d836bdbc05f5c82cccdd88ed9bfb01873eabc5c7.