Closed fgalan closed 10 years ago
Comment done in behalf of @frbattid , created during issue migration:
In more detail --> Currently, stored data is based on 6 fields separated by '|', being the 6th field the "value" of a measure. If this value is created by concatenating M strings with the '|' character separator then the final stored table will be based on 6 -1 + M fields.
If the Hive external table is created for this 6 -1 + M fields there is no problem.
The problem arises when, having a Hive external table defined for 6 fields, the value field is injected with those M extra fields. Queries on that 6-length table will crash when reaching the first 6 - 1 + M fields line.
I think this issue has no sense anymore since the CSV-like serialization has been discarded at all.
Moved from https://github.com/telefonicaid/fiware-livedemoapp/issues/7:
ngsi2cosmos.py should parse the contextValue before writing it to Cosmos, escaping the delimiter (usually "|"). Otherwise, the user could "inject several columns in a single field" potentially breaking the schema defined by tools such as Hive.
This escaping should be a optional feature (typically, a flag in the CLI or configuration file), given that some cases it could be useful to have this injection to simplify NGSI model definition.
@frbattid is also having a look to this.