search

Informatica-EIC / Custom-Scanners

Custom Scanner Samples.
11 stars 12 forks source link

View Truncation statement in EDC log #19

Closed tylerinsd closed 1 year ago

tylerinsd commented 5 years ago

What does this mean? Do we risk getting incorrect metadata?

2019-08-05 16:23:11,944 [ExecuteThread-{Task_Denodo_Custom_Scanner_Sample_CustomMetadataScanner_CORE}] INFO com.infa.products.ldm.ingestion.client.impl.PublishRequest- Truncating fact value for attribute [com.infa.ldm.relational.ViewStatement] of length [10,372] to 10k characters in data exchange document with docId [Denodo_Custom_Scanner_Sample##Denodo_DEV#GISCCSAT#pspstatus_ops_history_Project_Summary_p1] 2019-08-05 16:23:11,962 [ExecuteThread-{Task_Denodo_Custom_Scanner_Sample_CustomMetadataScanner_CORE}] INFO com.infa.products.ldm.ingestion.client.impl.PublishRequest- Truncating fact value for attribute [com.infa.ldm.relational.ViewStatement] of length [13,305] to 10k characters in data exchange document with docId [Denodo_Custom_Scanner_Sample##Denodo_DEV#RO-AnalyticsHayward#ContainerDetails_training] 2

darren-wrigley commented 5 years ago

all attributes in the catalog are truncated at 10k characters. when you get the definition of an object in denodo - it includes a lot more than just the vql for the view. it can include dependent views and the datasource + wrapper. in the version i added today 0.9.8 - the scanner looks only for the create syntax for the actual object - so the occurrences of the 10k character truncation should be much less, or zero