Informatica-EIC / Custom-Scanners

Custom Scanner Samples.
11 stars 12 forks source link

Denodo scanner is not able to pull all the parent names from the EDC #51

Closed zfWazid closed 2 months ago

zfWazid commented 1 year ago

We have created custom scanner as per the suggestion provided below

https://www.informatica.com/content/dam/informatica-marketplace/documents/DENODO_custom_scanner.pdf

Parentnamemissing

And also We tried connection Denodo via JDBC also but this is failing when we are enabling "profiling" Data discovery options

"Please let us know if can acheive profiling via Denodo JDBC drivers" ?

darren-wrigley commented 1 year ago

there is a bug/error in Axon where it was not properly handling the core.dataSetUuid property for table/view columns. recent versions of the denodo scanner (1.1.000+) do not use this property (value is empty) to workaround that issue.
this may filter up to marketplace with the parent link.

I have been trying to get the download updated on marketplace - however that process is taking much longer than expected you can try the version here:- https://github.com/Informatica-EIC/Custom-Scanners/blob/master/Denodo_Scanner/packaged/DenodoScanner.zip

for the 2nd question - the denodo custom scanner does not currently work with profiling - for that to happen, a .csv file for each denodo table/view would need to be created in a tsftp folder & then the custom scanner framework can be configured to run some (not all) profiling.

if you are using a JDBC scan for denodo - profiling should work, but you will not have any lineage for the denodo objects, either within denodo, or to the upstream datasource objects.

zfWazid commented 1 year ago

Hi Darren,

Please help us on this

Thanks Wazid

darren-wrigley commented 1 year ago

i can only reproduce that issue if i set the JDBC Driver class name to com.denodo.vdb.jdbcdriver.VDBJDBCDriver in the connection from admin console. it gives me this message:_

The requested operation could not be performed due to the following error : The JDBC driver class name com.denodo.vdb.jdbcdriver.VDBJDBCDriver in the connection does not match with the JDBC driver class name com.denodo.vdp.jdbc.Driver used in the connection string jdbc:vdb://dw-sndbox01-w.infaaws.com:9999/tutorial?chunkTimeout=2000. Specify the correct connection string and the JDBC driver class name for the connection object.

since that test connect appears to be using com.denodo.vdb.jdbcdriver.VDBJDBCDriver - you can always try it.
there isn't much more i can do - since i can't reproduce it & it is not related to the denodo scanner.

some things you can look at:-

zfWazid commented 1 year ago

ok Thank now i'm able to test JDBC connection successfully.

But the profiling is still failing with

Function [INFASQLConnectW] failed in adapter [/infaedc/infa/plugins/dynamic/infajdbcruntime/libinfajdbc.so

Profile1 Profile

7c6934bf-6baf-47f8-88f7-d772f32b09db.zip

darren-wrigley commented 1 year ago

have you raised a support case with the platform team? i tried to reproduce, for jdbc connection i had to set ExecutionContextOptions.JVMMaxMemory custom property in the DIS (from this KB article: https://knowledge.informatica.com/s/article/578923?language=en_US )

i also set the sql identifier character to double-quotes and support mixed case identifiers to true.

is it working for other tables/views, or only failing for char[0] columns?

darren-wrigley commented 2 months ago

closing, no-activity/comments - a new ticket could be created for this if needed. does not appear to be an issue with the scanner