IKANOW / Aleph2

The IKANOW v2 meta-database and analytics platform
Apache License 2.0
2 stars 1 forks source link

Storage Schema doesn't use raw.properties #102

Open cburch opened 8 years ago

cburch commented 8 years ago

https://github.com/IKANOW/Aleph2-contrib/blob/master/aleph2_analytic_services_hadoop/src/main/java/com/ikanow/aleph2/analytics/hadoop/assets/BeFileInputReader.java#L291

The fields here are using storage_schema.processed().X instead of storage_schema.raw().X so if you create a schema that looks like { raw:{exist_age_max:"1 week"}} it does nothing, you currently have to handle it via { raw:{}, processed:{exist_age_max:"1 week"}}