terascope / teraslice

Scalable data processing pipelines in JavaScript
https://terascope.github.io/teraslice/
Apache License 2.0
50 stars 13 forks source link

ts-transform - issue transforming a field to an array without an extraction #1777

Closed ciorg closed 4 years ago

ciorg commented 4 years ago

trying to transform a single field in an incoming object without creating a new object. I need to retain all the properties of the incoming object and transform the one field on the record.

ldjson example

{ "selector": "_exists_:field1", "tag": "field1-to-array"}
{ "follow": "field1-to-array", "post_process": "array", "target":"field1"}

this returns the error TypeError: Cannot read property '__id' of undefined

using an extraction:

{ "selector": "_exists_:field1",  "source":"field1",  "target":"field1-array","tag": "field-to-array"}
{ "follow": "field-to-array", "post_process": "array", "target":"field1"}

creates a new object and the other properties are not carried over.

tried using "mutate": false and the result was the same.

jsnoble commented 4 years ago

Fix in the works,

Rules to what you need:

{ "selector": "_exists_:field", "tag": "testme" }
{ "follow": "testme", "post_process": "extraction", "mutate": true, "source": "field", "target": "field", "exp": "[field]"}

OR

{ "selector": "_exists_:field", "tag": "field1-to-array"}
{ "follow": "field1-to-array", "post_process": "array", "fields": ["field"], "target":"field"}