mountetna / monoetna

mono-repository version of etna projects
GNU General Public License v2.0
1 stars 0 forks source link

Add data_frame type to polyphemus metis loaders #1268

Closed dtm2451 closed 11 months ago

dtm2451 commented 1 year ago

Adds ability to fill in attributes for multiple records from csv/tsv files + a map of column name <-> attribute pairs.

ToDo:

graft commented 1 year ago

Testing by trying to put data into ipi demographics model, which is a table (somewhat of a strange case), I get: Job failed with error: Exception('MetisLoaderError("demographics-1.csv is missing column(s) targetted by demographic data_frame loader \'column_map\': id.")')

I'm curious to see how the successful result arrives; I suspect it will be append-only (i.e., it will add items to the demographics table, not overwrite them). Might need an option to allow replacement-of or append-to existing table rows, although I suspect there will remain some further edge cases about this.

dtm2451 commented 12 months ago

Oh big oops... As you may have already have figured out or intuited from that error message, this isn't designed with table models in mind at the moment. I had only 'standard' models in mind. I've not worked with table models enough, so definitely worth a pair to ensure I understand the workings and edge-cases to update for!