search

Azure / Azure-Sentinel

Cloud-native SIEM for intelligent security analytics for your entire enterprise.
https://azure.microsoft.com/en-us/services/azure-sentinel/
MIT License
4.65k stars 3.04k forks source link

DCR: Transformation Editor struggles while handling "_" in column names #10964

Closed kllnbrnjn closed 4 months ago

kllnbrnjn commented 4 months ago

Describe the bug We receive some logs using a CRIBL connection. The column names of the received logs do not match our needs, thus we are trying to create a Transformation within the DCR for the specific connection (and table). The timestamp of the log entry comes in the column "_timestamp", but here the Transformation Editor struggles (see Screenshots). The field will just be empty, no matter if we try to rename the column, or extend it. When removing the underline directly in the JSON file (which is not applicable in the real incoming data), it works (also see Screenshots).

To Reproduce Steps to reproduce the behavior:

  1. Go to any Data Collection Rule
  2. Open the Transformation Editor on any table
  3. Upload the following sample JSON: {"_timestamp":"2024-07-23T12:00:29.000+0200","testdata":"Some Test Data"}
  4. Try to extend the "TimeGenerated" field using the "_timestamp"

Expected behavior When renaming the "_timestamp" to "TimeGenerated", we expect the field to be still populated with the original data, as the Transformation obviously reads the correct value from the JSON.

Screenshots Uploading the sample JSON File (with underscore in front of timestamp) results in the tool correctly reading the time, but expecting us to rename it to "TimeGenerated": Transformation View With Underscore When trying to rename the column, the field will just be empty: Underscore Transformation With Project Also trying to extend a new column with the value does not work and the field remains empty: Underscore Transformation With Extend When directly removing the underscore in the JSON File (which we cannot do for real incoming data), renaming works just fine: Normal Transformation With Project

v-sudkharat commented 4 months ago

Hi @kllnbrnjn, Thank you for flagging issue. This DCR transformation issue needs to be investigated by concern (DCR) team, so we kindly request you to raise a support ticket case in azure portal, so our support team can check into it and connect with you if required. closing this issue from GitHub. If you still need support for this issue, feel free to re-open it any time. Thank you for your co-operation.