Closed kuldeeppoonia closed 6 years ago
print 'topic_vbrkheader' from beginning;
?CREATE STREAM vbrk_stream
specifies the wrong format (column types, data format, ...) for the topic topic_vbrkheader
? This could be the reason why a SELECT * FROM vbrk_stream
doesn't show any results, because the messages in the topic can't be parsed properly according to the STREAM definition.Hi,
Here are some records when I run print command as per suggested by you.
{"ROWTIME":1531808956749,"ROWKEY":"null","MANDT":"800","VBELN":"1118109411","FKART":"ZDR","FKTYP":"A","VBTYP":"P","WAERK":"INR","VKORG":"1000","VTWEG":"10","KALSM":"ZZDR09","KNUMV":"0026071530","VSBED":"01","FKDAT":"20180331","BELNR":" ","GJAHR":"0000","POPER":"000","KONDA":" ","KDGRP":" ","BZIRK":"000001","PLTYP":" ","INCO1":"CFR","INCO2":"CFR","EXPKZ":" ","RFBSK":"C","MRNKZ":" ","KURRF":"AYag","CPKUR":"X","VALTG":"00","VALDT":"00000000","ZTERM":"Z014","ZLSCH":" ","KTGRD":" ","LAND1":"IN","REGIO":" 0","COUNC":" ","CITYC":" ","BUKRS":"1000","TAXK1":"0","TAXK2":"0","TAXK3":" ","TAXK4":" ","TAXK5":" ","TAXK6":" ","TAXK7":" ","TAXK8":" ","TAXK9":" ","NETWR":"EMjg","ZUKRI":"00110091117940079","ERNAM":"SAVITA.VYAS","ERZET":"110053","ERDAT":"20180331","STAFO":" ","KUNRG":"0010000177","KUNAG":"0010000177","MABER":" ","STWAE":"INR","EXNUM":" ","STCEG":" ","AEDAT":"00000000","SFAKN":" ","KNUMA":" ","FKART_RL":"ZLR","FKDAT_RL":"00000000","KURST":" ","MSCHL":" ","MANSP":" ","SPART":"09","KKBER":"9000","KNKLI":"0010000177","CMWAE":"INR","CMKUF":"AYag","HITYP_PR":"A","BSTNK_VF":" ","VBUND":" ","FKART_AB":" ","KAPPL":" ","LANDTX":"IN","STCEG_H":"C","STCEG_L":"IN","XBLNR":"DD3108000619","ZUONR":"1117940079","MWSBK":"AA==","LOGSYS":" ","FKSTO":" ","XEGDR":" ","RPLNR":" ","LCNUM":" ","J_1AFITP":" ","KURRF_DAT":"20180316","AKWAE":" ","AKKUR":"AA==","KIDNO":"1118109411","BVTYP":" ","NUMPG":"000","BUPLA":"08RJ","VKONT":" ","FKK_DOCSTAT":" ","NRZAS":" ","ZZP01":"D","ZZCND":"00000000","ZZSDAT":"00000000","ZZPDAT":"00000000","ZZPLO":"GTN","ZSCODE":"A","ZZINVM":" ","ZZINVY":"2017","ZZREFN":"UDP31CCAD2066IN","ZZBBCH":"0000002050","ZZPD01":"00","ZZMUD":"X","ZZNUMR":"51","ZPRNTDT":"20180331","ZTCODE":" ","ZZCOBR":"0000002050","ZZORBB":" ","ZZCOOB":"0000002020","ZZTSBD":"20180407","J_1TPBUPL":" ","MNDID":" ","PAY_TYPE":" ","SEPON":" ","MNDVG":" ","ZZCCB":" ","ZZBLTP":"0010000177"} {"ROWTIME":1531808960669,"ROWKEY":"null","MANDT":"800","VBELN":"0090653056","FKART":"S2","FKTYP":"A","VBTYP":"S","WAERK":"INR","VKORG":"1000","VTWEG":"10","KALSM":"Z00016","KNUMV":"0003308703","VSBED":"01","FKDAT":"20120626","BELNR":" ","GJAHR":"0000","POPER":"000","KONDA":" ","KDGRP":" ","BZIRK":"000001","PLTYP":" ","INCO1":"CFR","INCO2":"Costs and Freight","EXPKZ":" ","RFBSK":"C","MRNKZ":" ","KURRF":"AYag","CPKUR":"X","VALTG":"00","VALDT":"00000000","ZTERM":"Z014","ZLSCH":" ","KTGRD":" ","LAND1":"IN","REGIO":"20","COUNC":" ","CITYC":" ","BUKRS":"1000","TAXK1":"1","TAXK2":" ","TAXK3":" ","TAXK4":" ","TAXK5":" ","TAXK6":" ","TAXK7":" ","TAXK8":" ","TAXK9":" ","NETWR":"D0JA","ZUKRI":"0011005","ERNAM":"2050-OPR01","ERZET":"232204","ERDAT":"20120626","STAFO":" ","KUNRG":"0010000177","KUNAG":"0010000177","MABER":" ","STWAE":"INR","EXNUM":" ","STCEG":" ","AEDAT":"00000000","SFAKN":"1118527849","KNUMA":" ","FKART_RL":" ","FKDAT_RL":"00000000","KURST":" ","MSCHL":" ","MANSP":" ","SPART":"05","KKBER":"9000","KNKLI":"0010000177","CMWAE":"INR","CMKUF":"AYag","HITYP_PR":"A","BSTNK_VF":" ","VBUND":" ","FKART_AB":" ","KAPPL":" ","LANDTX":"IN","STCEG_H":"C","STCEG_L":"IN","XBLNR":"14","ZUONR":"1116354399","MWSBK":"AA==","LOGSYS":" ","FKSTO":" ","XEGDR":" ","RPLNR":" ","LCNUM":" ","J_1AFITP":" ","KURRF_DAT":"20120514","AKWAE":" ","AKKUR":"AA==","KIDNO":"0090653056","BVTYP":" ","NUMPG":"000","BUPLA":"2050","VKONT":" ","FKK_DOCSTAT":" ","NRZAS":" ","ZZP01":"D","ZZCND":"00000000","ZZSDAT":"00000000","ZZPDAT":"00000000","ZZPLO":" ","ZSCODE":" ","ZZINVM":" ","ZZINVY":"0000","ZZREFN":"14","ZZBBCH":"0000002050","ZZPD01":"00","ZZMUD":" ","ZZNUMR":" ","ZPRNTDT":"20160419","ZTCODE":null,"ZZCOBR":"0000002050","ZZORBB":"0000002050","ZZCOOB":null,"ZZTSBD":null,"J_1TPBUPL":null,"MNDID":null,"PAY_TYPE":null,"SEPON":null,"MNDVG":null,"ZZCCB":null,"ZZBLTP":"0010000177"} {"ROWTIME":1531808960670,"ROWKEY":"null","MANDT":"800","VBELN":"1117798270","FKART":"ZF2","FKTYP":"L","VBTYP":"M","WAERK":"INR","VKORG":"1000","VTWEG":"10","KALSM":"ZDIVRA","KNUMV":"0023962096","VSBED":"01","FKDAT":"20171011","BELNR":" ","GJAHR":"0000","POPER":"000","KONDA":" ","KDGRP":" ","BZIRK":"000001","PLTYP":" ","INCO1":"CFR","INCO2":"CFR","EXPKZ":" ","RFBSK":"C","MRNKZ":" ","KURRF":"AYag","CPKUR":" ","VALTG":"00","VALDT":"00000000","ZTERM":"Z014","ZLSCH":" ","KTGRD":" ","LAND1":"IN","REGIO":" 0","COUNC":" ","CITYC":" ","BUKRS":"1000","TAXK1":"0","TAXK2":"0","TAXK3":" ","TAXK4":" ","TAXK5":" ","TAXK6":" ","TAXK7":" ","TAXK8":" ","TAXK9":" ","NETWR":"NQK0","ZUKRI":"001090010034470","ERNAM":"SAVITA.VYAS","ERZET":"114505","ERDAT":"20171011","STAFO":" ","KUNRG":"0010000177","KUNAG":"0010000177","MABER":" ","STWAE":"INR","EXNUM":" ","STCEG":" ","AEDAT":"00000000","SFAKN":" ","KNUMA":" ","FKART_RL":"ZLR","FKDAT_RL":"00000000","KURST":" ","MSCHL":" ","MANSP":" ","SPART":"09","KKBER":"9000","KNKLI":"0010000177","CMWAE":"INR","CMKUF":"AYag","HITYP_PR":"A","BSTNK_VF":" ","VBUND":" ","FKART_AB":" ","KAPPL":" ","LANDTX":"IN","STCEG_H":"C","STCEG_L":"IN","XBLNR":"DI3108002420","ZUONR":"1117798270","MWSBK":"AA==","LOGSYS":" ","FKSTO":" ","XEGDR":" ","RPLNR":" ","LCNUM":" ","J_1AFITP":" ","KURRF_DAT":"20171011","AKWAE":" ","AKKUR":"AA==","KIDNO":"1117798270","BVTYP":" ","NUMPG":"000","BUPLA":"08RJ","VKONT":" ","FKK_DOCSTAT":" ","NRZAS":" ","ZZP01":"D","ZZCND":"20170929","ZZSDAT":"20170929","ZZPDAT":"20171001","ZZPLO":"GTN","ZSCODE":"A","ZZINVM":" ","ZZINVY":"2017","ZZREFN":"UDP31AS1258","ZZBBCH":"0000002050","ZZPD01":"00","ZZMUD":"X","ZZNUMR":"51","ZPRNTDT":"20171011","ZTCODE":" ","ZZCOBR":"0000002050","ZZORBB":"0000002050","ZZCOOB":"0000002020","ZZTSBD":"20171013","J_1TPBUPL":" ","MNDID":" ","PAY_TYPE":" ","SEPON":" ","MNDVG":" ","ZZCCB":null,"ZZBLTP":"0010000177"} {"ROWTIME":1531808960670,"ROWKEY":"null","MANDT":"800","VBELN":"1117851571","FKART":"ZF2","FKTYP":"L","VBTYP":"M","WAERK":"INR","VKORG":"1000","VTWEG":"10","KALSM":"ZDIVRA","KNUMV":"0024694131","VSBED":"01","FKDAT":"20171207","BELNR":" ","GJAHR":"0000","POPER":"000","KONDA":" ","KDGRP":" ","BZIRK":"000001","PLTYP":" ","INCO1":"CFR","INCO2":"CFR","EXPKZ":" ","RFBSK":"C","MRNKZ":" ","KURRF":"AYag","CPKUR":" ","VALTG":"00","VALDT":"00000000","ZTERM":"Z016","ZLSCH":" ","KTGRD":" ","LAND1":"IN","REGIO":" 0","COUNC":" ","CITYC":" ","BUKRS":"1000","TAXK1":"0","TAXK2":"0","TAXK3":" ","TAXK4":" ","TAXK5":" ","TAXK6":" ","TAXK7":" ","TAXK8":" ","TAXK9":" ","NETWR":"elZc","ZUKRI":"001110010034828","ERNAM":"SAVITA.VYAS","ERZET":"152303","ERDAT":"20171207","STAFO":" ","KUNRG":"0010000177","KUNAG":"0010000177","MABER":" ","STWAE":"INR","EXNUM":" ","STCEG":" ","AEDAT":"00000000","SFAKN":" ","KNUMA":" ","FKART_RL":"ZLR","FKDAT_RL":"00000000","KURST":" ","MSCHL":" ","MANSP":" ","SPART":"09","KKBER":"9000","KNKLI":"0010000177","CMWAE":"INR","CMKUF":"AYag","HITYP_PR":"A","BSTNK_VF":" ","VBUND":" ","FKART_AB":" ","KAPPL":" ","LANDTX":"IN","STCEG_H":"C","STCEG_L":"IN","XBLNR":"DI3127012449","ZUONR":"1117851571","MWSBK":"AA==","LOGSYS":" ","FKSTO":" ","XEGDR":" ","RPLNR":" ","LCNUM":" ","J_1AFITP":" ","KURRF_DAT":"20171207","AKWAE":" ","AKKUR":"AA==","KIDNO":"1117851571","BVTYP":" ","NUMPG":"000","BUPLA":"27MH","VKONT":" ","FKK_DOCSTAT":" ","NRZAS":" ","ZZP01":"D","ZZCND":"20171102","ZZSDAT":"20171102","ZZPDAT":"20171108","ZZPLO":"GTN","ZSCODE":"A","ZZINVM":" ","ZZINVY":"2017","ZZREFN":"UDP31P1460","ZZBBCH":"0000002050","ZZPD01":"00","ZZMUD":"X","ZZNUMR":"51","ZPRNTDT":"20171207","ZTCODE":" ","ZZCOBR":"0000002050","ZZORBB":"0000003410","ZZCOOB":"0000002145","ZZTSBD":"20171208","J_1TPBUPL":" ","MNDID":" ","PAY_TYPE":" ","SEPON":" ","MNDVG":" ","ZZCCB":null,"ZZBLTP":"0010000177"} {"ROWTIME":1531808960670,"ROWKEY":"null","MANDT":"800","VBELN":"1117851574","FKART":"ZF2","FKTYP":"L","VBTYP":"M","WAERK":"INR","VKORG":"1000","VTWEG":"10","KALSM":"ZDIVRA","KNUMV":"0024694142","VSBED":"01","FKDAT":"20171207","BELNR":" ","GJAHR":"0000","POPER":"000","KONDA":" ","KDGRP":" ","BZIRK":"000001","PLTYP":" ","INCO1":"CFR","INCO2":"CFR","EXPKZ":" ","RFBSK":"C","MRNKZ":" ","KURRF":"AYag","CPKUR":" ","VALTG":"00","VALDT":"00000000","ZTERM":"Z016","ZLSCH":" ","KTGRD":" ","LAND1":"IN","REGIO":" 0","COUNC":" ","CITYC":" ","BUKRS":"1000","TAXK1":"0","TAXK2":"0","TAXK3":" ","TAXK4":" ","TAXK5":" ","TAXK6":" ","TAXK7":" ","TAXK8":" ","TAXK9":" ","NETWR":"AIgQ0A==","ZUKRI":"001110010034828","ERNAM":"SAVITA.VYAS","ERZET":"152516","ERDAT":"20171207","STAFO":" ","KUNRG":"0010000177","KUNAG":"0010000177","MABER":" ","STWAE":"INR","EXNUM":" ","STCEG":" ","AEDAT":"00000000","SFAKN":" ","KNUMA":" ","FKART_RL":"ZLR","FKDAT_RL":"00000000","KURST":" ","MSCHL":" ","MANSP":" ","SPART":"09","KKBER":"9000","KNKLI":"0010000177","CMWAE":"INR","CMKUF":"AYag","HITYP_PR":"A","BSTNK_VF":" ","VBUND":" ","FKART_AB":" ","KAPPL":" ","LANDTX":"IN","STCEG_H":"C","STCEG_L":"IN","XBLNR":"DI3127012450","ZUONR":"1117851574","MWSBK":"AA==","LOGSYS":" ","FKSTO":" ","XEGDR":" ","RPLNR":" ","LCNUM":" ","J_1AFITP":" ","KURRF_DAT":"20171207","AKWAE":" ","AKKUR":"AA==","KIDNO":"1117851574","BVTYP":" ","NUMPG":"000","BUPLA":"27MH","VKONT":" ","FKK_DOCSTAT":" ","NRZAS":" ","ZZP01":"D","ZZCND":"20171103","ZZSDAT":"20171103","ZZPDAT":"20171114","ZZPLO":"GTN","ZSCODE":"A","ZZINVM":" ","ZZINVY":"2017","ZZREFN":"UDP31P1461","ZZBBCH":"0000002050","ZZPD01":"00","ZZMUD":"X","ZZNUMR":"51","ZPRNTDT":"20171207","ZTCODE":" ","ZZCOBR":"0000002050","ZZORBB":"0000003410","ZZCOOB":"0000002145","ZZTSBD":"20171208","J_1TPBUPL":" ","MNDID":" ","PAY_TYPE":" ","SEPON":" ","MNDVG":" ","ZZCCB":null,"ZZBLTP":"0010000177"}
and here is the stream structure ksql> describe extended vbrk_stream;
Name : VBRK_STREAM
Type : STREAM
Key field : VBELN
Key format : STRING
Timestamp field : Not set - using
(Statistics of the local KSQL server interaction with the Kafka topic topic_vbrkheader)
Can you share the ksql server log when in the period when you run the select *
?
Hi Apurvam, here is my ksql logs:
[2018-07-19 10:23:49,667] WARN Writing to metrics Kafka topic will be disabled (io.confluent.support.metrics.PhoneHomeConfig:61)
[2018-07-19 10:23:49,676] ERROR No customer ID configured -- falling back to id 'anonymous' (io.confluent.support.metrics.BaseSupportConfig:342)
[2018-07-19 10:23:49,677] WARN Enforcing customer ID 'anonymous' (io.confluent.support.metrics.PhoneHomeConfig:66)
[2018-07-19 10:23:49,720] WARN Please note that the version check feature of KSQL is enabled. With this enabled, this instance is configured to collect and report anonymously the version information to Confluent, Inc. ("Confluent") or its parent, subsidiaries, affiliates or service providers every 24hours. This Metadata may be transferred to any country in which Confluent maintains facilities. For a more in depth discussion of how Confluent processes such information, please read our Privacy Policy located at http://www.confluent.io/privacy. By proceeding with confluent.support.metrics.enable=true
, you agree to all such collection, transfer and use of Version information by Confluent. You can turn the version check feature off by setting confluent.support.metrics.enable=false
in the KSQL configuration and restarting the KSQL. See the Confluent Platform documentation for further information. (io.confluent.ksql.version.metrics.KsqlVersionCheckerAgent:68)
Sorry, but that isn't the full log. IT would be good if you can upload it somewhere and share the link.
Hi Apurvam,
I got it from path /home/
Hi All, After spending long time on this issue , I found the cause of this behavior. Actual I have a amount field in my topic and when I create a stream then I take it as double. If I take it as varchar then stream is giving me proper data. Now question is that what's the problem with double datatype.?Now I am using confluent-5.0.0
Have you been able to solve your problem, @kuldeeppoonia ?
Hi Michael,
Yes, I have solved the problem
On Tue, Oct 2, 2018 at 7:59 PM Michael G. Noll notifications@github.com wrote:
Have you been able to solve your problem, @kuldeeppoonia https://github.com/kuldeeppoonia ?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/confluentinc/ksql/issues/1573#issuecomment-426295519, or mute the thread https://github.com/notifications/unsubscribe-auth/AStlTbzK0oFVuIuyQDUpwtl1M_ByF5RTks5ug3hYgaJpZM4VSpHp .
-- Thanks & Regards,
Kuldeep Poonia Sr. Software Eng. CJ Darcl Logistic Limited 19/3 Tilak Bazar, Hisar - 125001
Hi Team,
Today I have download beta version confluent-5.0.0-beta180702222458 for checking my issues which I am facing in previous version. But I am facing a strange problem in KSQL stream. It's not showing me records even if I have applied command set 'auto.offset.reset' = 'earliest';
here are the steps for regenerating the issue.
Create a topic like 'topic_vbrkheader' through jdbc source connector file.
create KQL stream using command
What works
When I use command
it showing me all records.
What does not work
When I run command
set 'auto.offset.reset' = 'earliest';
and thenselect * from vbrk_stream;
it shows nothing.Am I doing something wrong?