SEED-platform / seed

Standard Energy Efficiency Data (SEED) Platform™ is a web-based application that helps organizations easily manage data on the energy performance of large groups of buildings.
Other
106 stars 55 forks source link

Data Type Integration during Data Import #4701

Open kflemin opened 1 week ago

kflemin commented 1 week ago

Problem 1- Save Data Types during Data Import On data import, the data mapping screen looks like the following: import1

We need to provide the user with the ability to save the data types of the incoming data. If the file is a CSV, the data type is unknown, if it is Excel then it is typically correct. Regardless, the user needs to have the ability to select String, Integer, Number. This will improve importing and error checking. The possible data types can be found in the columns settings page. For example, it looks like this for extra_data columns: import2

Problem 2 - Number/Float Redundancy We need to remove the redundant Number/Float. Should probably just be Number. This will likely require a database migration to map existing Floats -> Number. Problem 3 If the field is a known unit type, e.g., EUI, GHG, then we should probably add a separator to denote those, because they are special. Problem 4 The columns mapping page looks like this: import3

Is this a problem? Do we need to add any of the datatype functionality here? Just check/doublecheck. Problem 5 Last, the SEED API allows for a file to be posted to create a "column mapping profile". The file format needs to be updated. https://github.com/SEED-platform/py-seed/blob/develop/tests/data/test-seed-data-mappings.csv Add column for "omit" mapping. Verify that if "omitted", that the column data won't be stored to the database and that it will be recalled correctly in the column mapping page represented in Problem 1 above. Problem 6 Add column for data type. Need to catch cases where the column data type changes because it may require that the data get an updated type in the database. If this is the case, then I think we want to error and not allow the user to change the datatype.