bryanyang0528 / ksql-python

A python wrapper for the KSQL REST API.
MIT License
159 stars 67 forks source link

Querying KSQL table with ksql-python returning only header but no records #89

Closed Psykepro closed 3 years ago

Psykepro commented 3 years ago

Hello please help me with the following issue:

As background, I want to use Debezium to generate messages from Oracle Table from which to generate Events and I created KSQL table from the topic using AVRO format and Schema Registry. But the problem with the Table is that it returns me only the column types as the header but not the record itself. And when I try to use the same query with the ksql-cli it returns me that record.

I'm executing this:

client = KSQLAPI(url)
res = client.query("SELECT * FROM TRANSACTION_TYPES WHERE TRT_SERIAL = 4 EMIT CHANGES")
for x in res:
    print(x)

And what I got as a result is:

{"header":{"queryId":"none","schema":"`ROWTIME` BIGINT, `ROWKEY` STRING, `TRT_ARREARS_EFFECTIVE` STRING, `TRT_CODE` STRING, `TRT_DESCRIPTION` STRING, `TRT_SERIAL` BIGINT, `TRT_TRANSACTION_OR_ACCRUAL` STRING, `TRT_CASH_TRANSACTION` STRING, `TRT_CREDIT_ALLOWED` STRING, `TRT_DEBIT_ALLOWED` STRING, `TRT_DEBTOR_TRANSACTIONS` STRING, `TRT_INPUT_OR_AUTOMATIC` STRING, `TRT_PRODUCE_CHEQUE` STRING, `TRT_QUERY_ALLOWED` STRING, `TRT_SUSPENSE_TRANSACTION` STRING, `TRT_FEE_TRANSACTION` STRING, `TRT_ALIAS` BIGINT, `TRT_THIRD_PARTY_TRANSACTION` STRING, `TRT_INVOICE_TYPE` BIGINT, `TRT_SETTLEMENT_TRANSACTION` STRING, `TRT_INTEREST_ACCRUAL_EFFECTIVE` STRING, `TRT_EXPENSE_TRANSACTION` STRING, `TRT_RECOVERY_TRANSACTION` STRING, `TRT_CAPITAL_REPAYMENT` STRING, `TRT_SPLITTER_TYPE_VLC` BIGINT, `TRT_INCLUDE_ON_STATEMENT` STRING, `TRT_IGNORE_PAYOUT_DELAY` STRING, `SERIAL` BIGINT, `__OP` STRING, `__TABLE` STRING, `__SOURCE_TS_MS` BIGINT, `__DELETED` STRING"}}
Psykepro commented 3 years ago

Omg, the problem was that I wasn't passing stream_properties to reset the offset:

{
  "ksql.streams.auto.offset.reset": "earliest"
}