search

oracle-samples / oracle-functions-samples

Examples demonstrating how to use Oracle Functions
https://www.oracle.com/cloud/cloud-native/functions/
Universal Permissive License v1.0
124 stars 122 forks source link

oci-serviceconnector-streaming-json-to-csv-python not working #50

Open MarcGueury opened 2 years ago

MarcGueury commented 2 years ago

The sample is not working.

https://github.com/oracle-samples/oracle-functions-samples)/[samples](https://github.com/oracle-samples/oracle-functions-samples/tree/master/samples)/oci-serviceconnector-streaming-json-to-csv-python/

The input of the service connector looks like this:

"01GBK286021BT0Q4RZJ002PCNH - root - INFO - {'stream': 'mg-input-sc', 'partition': '0', 'key': None, 'value': 'W3sic3RyZWFtIjogIm15bmV3c3RyZWFtIiwicGFydGl0aW9uIjogIjAiLCJ2YWx1ZSI6ICJleUp1WVcxbElqb2lhV2R2Y2lJc0lDSnNiMk5oZEdsdmJpSTZJQ0pFZFdKc2FXNGlmUT09Iiwib2Zmc2V0IjogMCwidGltZXN0YW1wIjogIjIwMTgtMDQtMjZUMDE6MDM6MDYuMDUxWiJ9LHsic3RyZWFtIjogIm15bmV3c3RyZWFtIiwicGFydGl0aW9uIjogIjAiLCJ2YWx1ZSI6ICJleUp1WVcxbElqb2lZbkpwWVc0aUxDSm1hV1ZzWkRFaU9pSjBaWE4wSW4wPSIsIm9mZnNldCI6IDAsInRpbWVzdGFtcCI6ICIyMDE4LTA0LTI2VDAxOjAzOjA2LjA1MVoifV0=', 'offset': 10, 'timestamp': 1661718357219}"

Notice that key is None.

I am not sure why it is written this part of the code but this is causing issue:

        if 'key' in item:
            item['key'] = base64_decode(item['key'])

The same problem happens also with the test.json file that is in the sample directory. If you remove the lines, it works. I am not sure of the intention of these 2 lines.

rinkooy commented 1 year ago

I am also facing same issue, if there is only one tupple there in file ({"stream": "mynewstream"}), then code is working. But if more entries then not working.