search

postgresql-interfaces / psqlodbc

Other
16 stars 15 forks source link

ODBC driver issue while working with Informatica Cloud #22

Open Maniyur1Git opened 6 months ago

Maniyur1Git commented 6 months ago

Informatica Engine is failing with fatal error, attached traces for your reference.

1 Thread 0x7fa5092c0700 (LWP 1016655) 0x00007fa50dfe9acf in raise () from /lib64/libc.so.6 2 Thread 0x7fa51bc23740 (LWP 1016646) 0x00007fa51b81045c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 3 Thread 0x7fa50c7ac700 (LWP 1016648) 0x00007fa51b8107aa in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 4 Thread 0x7fa50a2c2700 (LWP 1016652) 0x00007fa50e0bef41 in poll () from /lib64/libc.so.6 5 Thread 0x7fa509ac1700 (LWP 1016653) 0x00007fa51b81045c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 6 Thread 0x7fa50aac3700 (LWP 1016651) 0x00007fa51b8107aa in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 (gdb) #0 0x00007fa50dfe9acf in raise () from /lib64/libc.so.6

1 0x00007fa50dfbcea5 in abort () from /lib64/libc.so.6

2 0x00007fa5196e47a9 in PmAbortProcess() () from ./libpmuti.so

3 0x00000000005e88cf in fatalSignalHandler(int, siginfo, void) ()

4

5 0x00007fa50e06232e in __strcmp_avx2 () from /lib64/libc.so.6

6 0x00007fa519bd5019 in lh_insert () from ./libpmcrypto.so.1.0.0

7 0x00007fa519b32639 in OBJ_NAME_add () from ./libpmcrypto.so.1.0.0

8 0x00007fa4d7eb518c in ossl_init_ssl_baseossl () from /lib64/libssl.so.1.1

9 0x00007fa51b811e67 in __pthread_once_slow () from /lib64/libpthread.so.0

10 0x00007fa4d7b84f5d in CRYPTO_THREAD_run_once ()

from /lib64/libcrypto.so.1.1

11 0x00007fa4d7eb53eb in OPENSSL_init_ssl () from /lib64/libssl.so.1.1

12 0x00007fa4d813777c in pgtls_init () from /lib64/libpq.so.5

13 0x00007fa4d8122660 in PQconnectPoll () from /lib64/libpq.so.5

14 0x00007fa4d8123805 in connectDBComplete () from /lib64/libpq.so.5

15 0x00007fa4d8126748 in PQconnectdbParams () from /lib64/libpq.so.5

16 0x00007fa4d837a3ce in LIBPQ_connect () from /usr/lib64/psqlodbc.so

17 0x00007fa4d837f380 in CC_connect () from /usr/lib64/psqlodbc.so

18 0x00007fa4d837fcab in PGAPI_Connect () from /usr/lib64/psqlodbc.so

19 0x00007fa4d83b88f2 in SQLConnect () from /usr/lib64/psqlodbc.so

20 0x00007fa5081d473e in SQLConnect ()

from /opt/infaagent/apps/Data_Integration_Server/../../drivers/misc/base/bin/libodbc.so.2

21 0x00007fa5084485ba in TODBCDb::Connect(PmUString const&, PmUString const&, PmUString const&, short) () from ./libpmodbc2.so

22 0x00007fa515fa3867 in TDatabase::ConnectImpl(PmUString const&, PmUString const&, PmUString const&, short) () from ./libpmodl.so

23 0x00007fa515fa403a in TDatabase::Connect(PmUString const&, PmUString const&, PmUString const&, short, int) () from ./libpmodl.so

24 0x00007fa50867da98 in relReader::connect(int) () from ./libpmrelrdr.so

25 0x00007fa508682893 in relReader::Connect() () from ./libpmrelrdr.so

26 0x00000000007bffd2 in frame_blkReader::init() ()

27 0x00000000007c032d in SReaderRunnable::handleRequest(SManagerRequest const&) ()

28 0x0000000000633c62 in SWorker::run() ()

29 0x000000000060f3b1 in SDTMRunnable::MainEntryPoint() ()

30 0x00007fa51b5fcfea in runit(void*) () from ./libpmcef.so

31 0x00007fa51b80a1ca in start_thread () from /lib64/libpthread.so.0

32 0x00007fa50dfd4e73 in clone () from /lib64/libc.so.6

davecramer commented 6 months ago

What version of psqlodbc are you using version of postgresql Any other details ?

Maniyur1Git commented 6 months ago

OS: Linux RedHat server Driver Version: unixODBC 2.3.4

Connection test successful using iSQL utility & Informatica UI. Able to browse objects.

Let me know what additional information you need, I can provide. TY.

Maniyur1Git commented 6 months ago

Additional Stack trace form tomcat log.

UTC INFO [com.informatica.saas.dtm.executor.impl.ProcessStreamReader] dtm-executor-0-01CRID0E0000000001PT-wf_mtt_01CRID0Z000000000007-4 - [01CRID0Z000000000007 01CRID0E0000000001PT]

FATAL ERROR : Caught a fatal signal or exception.

FATAL ERROR : Aborting the DTM process due to fatal signal or exception.

***FATAL ERROR : Signal Received: SIGSEGV (11)

Stack trace produced by process [pmdtm(:wf_mtt_01CRID0Z000000000007.s_mtt_01CRID0Z000000000007)] with pid [1929074]. ./libpmasrt.so(PmDumpStackTrace+0x79)[0x7f7c949b3d99] /opt/infaagent/apps/Data_Integration_Server/68.0.5.1/ICS/main/bin/rdtm/pmdtm[0x5e8c7a] /lib64/libpthread.so.0(+0x12cf0)[0x7f7c96e6acf0]/lib64/libc.so.6(+0xc731e)[0x7f7c8969531e]. /libpmcrypto.so.1.0.0(lh_insert+0xa9)[0x7f7c95224019]./libpmcrypto.so.1.0.0(OBJ_NAME_add+0x79)[0x7f7c95181639] /lib64/libssl.so.1.1(+0x3718c)[0x7f7c6ddc418c]/lib64/libpthread.so.0(+0xfe67)[0x7f7c96e67e67] /lib64/libcrypto.so.1.1(CRYPTO_THREAD_run_once+0xd)[0x7f7c6da93a5d] /lib64/libssl.so.1.1(OPENSSL_init_ssl+0x5b)[0x7f7c6ddc43eb] /lib64/libpq.so.5(+0x2577c)[0x7f7c6e25977c] /lib64/libpq.so.5(PQconnectPoll+0xc70)[0x7f7c6e244660] /lib64/libpq.so.5(+0x11805)[0x7f7c6e245805] /lib64/libpq.so.5(PQconnectdbParams+0x28)[0x7f7c6e248748] /usr/lib64/psqlodbcw.so(+0x163ce)[0x7f7c6e49c3ce] /usr/lib64/psqlodbcw.so(+0x1b380)[0x7f7c6e4a1380] /usr/lib64/psqlodbcw.so(+0x1bcab)[0x7f7c6e4a1cab] /usr/lib64/psqlodbcw.so(SQLConnectW+0xff)[0x7f7c6e4e64df] /opt/infaagent/apps/Data_Integration_Server/../../drivers/misc/base/bin/libodbc.so(+0xb109e)[0x7f7c6f93609e] /opt/infaagent/apps/Data_Integration_Server/../../drivers/misc/base/bin/libodbc.so(+0xb091b)[0x7f7c6f93591b] /opt/infaagent/apps/Data_Integration_Server/../../drivers/misc/base/bin/libodbc.so(+0xb0d56)[0x7f7c6f935d56] /opt/infaagent/apps/Data_Integration_Server/../../drivers/misc/base/bin/libodbc.so(SQLConnectA+0x137)[0x7f7c6f91b723] ./libpmodbcdd.so(_ZN7TODBCDb7ConnectERK9PmUStringS2_S2_s+0x52a)[0x7f7c6fbb561a] ./libpmodl.so(_ZN9TDatabase11ConnectImplERK9PmUStringS2_S2_s+0x67)[0x7f7c915e1887] ./libpmodl.so(_ZN9TDatabase7ConnectERK9PmUStringS2_S2_si+0x12a)[0x7f7c915e205a] ./libpmwrtrel.so(_ZN22SRelTruncateConnection7connectEv+0x132)[0x7f7c6fdea462] ./libpmwrtrel.so(_ZN14SPreSessWrtRel21getTruncateConnectionEP20TRepSessionExtensionPK8SSessionRP22SRelTruncateConnection+0xd9)[0x7f7c6fde8209] ./libpmwrtrel.so(_ZN14SPreSessWrtRel14truncateTargetEP20TRepSessionExtensionPK8SSessioni+0x3d)[0x7f7c6fde833d] /opt/infaagent/apps/Data_Integration_Server/68.0.5.1/ICS/main/bin/rdtm/pmdtm(_ZN14prepostSessTbl15doTruncateTableEv+0x312)[0x8d6f22] /opt/infaagent/apps/Data_Integration_Server/68.0.5.1/ICS/main/bin/rdtm/pmdtm(_ZN16SPreSessRunnable16truncateTgtTableEv+0x2f)[0x8d440f] /opt/infaagent/apps/Data_Integration_Server/68.0.5.1/ICS/main/bin/rdtm/pmdtm(_ZN16SPreSessRunnable13handleRequestERK15SManagerRequest+0xc3)[0x8d48c3] /opt/infaagent/apps/Data_Integration_Server/68.0.5.1/ICS/main/bin/rdtm/pmdtm(_ZN7SWorker3runEv+0x372)[0x6341d2] /opt/infaagent/apps/Data_Integration_Server/68.0.5.1/ICS/main/bin/rdtm/pmdtm(_ZN12SDTMRunnable14MainEntryPointEv+0xf1)[0x60f901] ./libpmcef.so(_Z5runitPv+0xa)[0x7f7c96c52ffa]/lib64/libpthread.so.0(+0x81ca)[0x7f7c96e601ca]/lib64/libc.so.6(clone+0x43)[0x7f7c89607e73]

davecramer commented 6 months ago

This looks like this is happening right in the connect

What version of pgsql odbc ?

Maniyur1Git commented 6 months ago

Hi Dave,

Here's the version details.

postgresql-odbc.x86_64 10.03.0000-3.el8_6 @rhel-8-appstream-rhui-rpms

davecramer commented 6 months ago

That's pretty old. Any chance you can try with the latest version ?

Maniyur1Git commented 6 months ago

Thank you for checking.

Sure, please help me with documentation link, so that I can check the supported version and use? We tried to get the latest version but somehow we picked the old version, didn't realize till you mention about it.

davecramer commented 6 months ago

https://www.postgresql.org/ftp/odbc/versions.old/dll/

Maniyur1Git commented 6 months ago

I tried version 16.0.0 still same error.

https://www.postgresql.org/ftp/odbc/versions.old/src/

davecramer commented 6 months ago

OK, as I said this looks like a connection error. Can you share the details of the connection ?

Maniyur1Git commented 6 months ago

Using this driver, connection in informatica is successful and also I can browse the objects in the database. But when I run the mapping Informatica DTM (Data Transformation Manager) fails with fatal error.

Regarding, I have to check with Network security team and get back. May I know what exact details you need?

davecramer commented 6 months ago

How you created the ODBC datasource what is in the odbc.ini file

Maniyur1Git commented 6 months ago

Here it is.

odbc.ini

[ODBC_pgconnection] Driver=/usr/local/lib/psqlodbcw.so Description=PostgreSQL DSN Servername=<> PortNumber=5432 Database=<> SSLMode=require DriverUnicodeType=1 AllowedOpenSSLVersions=1.1.1

odbcinst.ini

[PostgreSQL] Description = ODBC for PostgreSQL Driver = /usr/local/lib/psqlodbcw.so Setup = /usr/local/lib/libodbcpsqlS.so Driver64 = /usr/local/lib/psqlodbcw.so Setup64 = /usr/local/lib/libodbcpsqlS.so FileUsage = 1 Threading = 2

davecramer commented 6 months ago

Why is there no servername or database name ?

Maniyur1Git commented 6 months ago

It's in the file, I have just removed it here for security reasons.

Maniyur1Git commented 6 months ago

Is it possible to have a call to review the details? If yes, what's the best time.

davecramer commented 6 months ago

This is not possible. I wonder if you can write a test to see if you can connect without using informatica ?

Maniyur1Git commented 6 months ago

Tested the connection from Linux server using isql utility, able to connect and browse objects.

davecramer commented 6 months ago

so it would seem something the informatica driver is doing.

Maniyur1Git commented 6 months ago

Yeah that’s what I thought initially but Informatica support told it’s ODBC driver libraries which are causing this issue.

davecramer commented 6 months ago

Ha! Of course they did. If you can connect and browse with isql, it seems that the driver is working. Did you mention that to them?

Maniyur1Git commented 6 months ago

Yes, Informatica support knows about it.

davecramer commented 6 months ago

Can I get the stack trace from the version 16 of the driver please

Maniyur1Git commented 6 months ago

Sure, I will upload it tomorrow once I login. TY

Maniyur1Git commented 6 months ago

Dave,

Here's the latest stack trace.

2024-05-28 19:50:51,951 UTC INFO [com.informatica.saas.dtm.executor.impl.ProcessStreamReader] dtm-executor-0-01CRID0E000000000209-wf_mtt_01CRID0Z000000000005-38 - [01CRID0Z000000000005 01CRID0E000000000209]*** FATAL ERROR : Caught a fatal signal or exception.

FATAL ERROR : Aborting the DTM process due to fatal signal or exception.

FATAL ERROR : Signal Received: SIGSEGV (11) ***\ nStack trace produced by process [pmdtm(:wf_mtt_01CRID0Z000000000005.s_mtt_01CRID0Z000000000005)] with pid [11203]. ./libpmasrt.so(PmDumpStackTrace+0x79)[0x7fe650f2bd99] /opt/infaagent/apps/Data_Integration_Server/68.0.12.1/ICS/main/bin/rdtm/pmdtm[0x5e8c7a] /lib64/libpthread.so.0(+0x12cf0)[0x7fe6533e2cf0]/lib64/libc.so.6(+0xc731e)[0x7fe645c0d31e] ./libpmcrypto.so.1.0.0(lh_insert+0xa9)[0x7fe65179c019] ./libpmcrypto.so.1.0.0(OBJ_NAME_add+0x79)[0x7fe6516f9639] /lib64/libssl.so.1.1(+0x3718c)[0x7fe60fa0f18c]/lib64/libpthread.so.0(+0xfe67)[0x7fe6533dfe67] /lib64/libcrypto.so.1.1(CRYPTO_THREAD_run_once+0xd)[0x7fe60f6dea5d] /lib64/libssl.so.1.1(OPENSSL_init_ssl+0x5b)[0x7fe60fa0f3eb] /lib64/libpq.so.5(+0x2577c)[0x7fe60fc9177c] /lib64/libpq.so.5(PQconnectPoll+0xc70)[0x7fe60fc7c660] /lib64/libpq.so.5(+0x11805)[0x7fe60fc7d805] /lib64/libpq.so.5(PQconnectdbParams+0x28)[0x7fe60fc80748] /usr/local/lib/psqlodbcw.so(+0x16a5e)[0x7fe60fed4a5e] /usr/local/lib/psqlodbcw.so(+0x1bad8)[0x7fe60fed9ad8] /usr/local/lib/psqlodbcw.so(+0x1c3d7)[0x7fe60feda3d7] /usr/local/lib/psqlodbcw.so(SQLConnect+0x9e)[0x7fe60ff1341e] /opt/infaagent/apps/Data_Integration_Server/../../drivers/misc/base/bin/libodbc.so.2(SQLConnect+0x1ee)[0x7fe61036973e] ./libpmodbc2.so(_ZN7TODBCDb7ConnectERK9PmUStringS2_S2_s+0x52a)[0x7fe6105dd5da] ./libpmodl.so(_ZN9TDatabase11ConnectImplERK9PmUStringS2_S2_s+0x67)[0x7fe64db59887] ./libpmodl.so(_ZN9TDatabase7ConnectERK9PmUStringS2_S2_si+0x12a)[0x7fe64db5a05a] ./libpmrelrdr.so(_ZN9relReader7connectEi+0xd8)[0x7fe640225a98] ./libpmrelrdr.so(_ZN9relReader7ConnectEv+0x93)[0x7fe64022a893] /opt/infaagent/apps/Data_Integration_Server/68.0.12.1/ICS/main/bin/rdtm/pmdtm(_ZN15frame_blkReader4initEv+0xe2)[0x7c06f2] /opt/infaagent/apps/Data_Integration_Server/68.0.12.1/ICS/main/bin/rdtm/pmdtm(_ZN15SReaderRunnable13handleRequestERK15SManagerRequest+0x4d)[0x7c0a4d] /opt/infaagent/apps/Data_Integration_Server/68.0.12.1/ICS/main/bin/rdtm/pmdtm(_ZN7SWorker3runEv+0x372)[0x6341d2] /opt/infaagent/apps/Data_Integration_Server/68.0.12.1/ICS/main/bin/rdtm/pmdtm(_ZN12SDTMRunnable14MainEntryPointEv+0xf1)[0x60f901] ./libpmcef.so(_Z5runitPv+0xa)[0x7fe6531caffa] /lib64/libpthread.so.0(+0x81ca)[0x7fe6533d81ca] /lib64/libc.so.6(clone+0x43)[0x7fe645b7fe73]

Maniyur1Git commented 6 months ago

Dave, did you get chance to look into the latest stack trace. Please let me know if you notice any discrepancy. TY

davecramer commented 6 months ago

Sorry, no. I'll try to find some time today. I'm at a conference for the week

davecramer commented 6 months ago

FYI, this looks very similar https://knowledge.informatica.com/s/article/000218542?language=en_US

Maniyur1Git commented 6 months ago

Yes, it is similar but no where can I find the version of driver which is working for PostgreSQL. Even Informatica support is not helping much.

davecramer commented 6 months ago

This appears to be their problem. Since this requires an informatica connector I don’t have any way to debug it

Dave Cramer

On Thu, May 30, 2024 at 6:37 AM Maniyur1Git @.***> wrote:

Yes, it is similar but no where can I find the version of driver which is working for PostgreSQL. Even Informatica support is not helping much.

— Reply to this email directly, view it on GitHub https://github.com/postgresql-interfaces/psqlodbc/issues/22#issuecomment-2139577860, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADDH5RLGTBCNHWAR6KTVTLZE4TS7AVCNFSM6AAAAABH2LSCIKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZZGU3TOOBWGA . You are receiving this because you commented.Message ID: @.***>

Maniyur1Git commented 6 months ago

Thank you Dave, let me know if there is a way to connect so that I can walkthrough the issue for debugging, TY.

Maniyur1Git commented 6 months ago

Dave, Informatica recommded to use driver version - 14.x or 15.x. Here is the article link - [(https://knowledge.informatica.com/s/article/FAQ-What-are-the-supported-application-versions-for-relational-NoSQL-and-real-time-connectors-and-supported-distribution-versions-for-big-data-connectors-in-IICS-Cloud-Data-Integration?language=en_US)]

But job failed with same error, sharing the stack trace for your reference.

2024-05-31 20:16:46,656 UTC INFO [com.informatica.saas.dtm.executor.impl.ProcessStreamReader] dtm-executor-1-01CRID0E00000000025T-wf_mtt_01CRID0Z000000000005-39 - [01CRID0Z000000000005 01CRID0E00000000025T]

***FATAL ERROR : Caught a fatal signal or exception.

FATAL ERROR : Aborting the DTM process due to fatal signal or exception.

FATAL ERROR : Signal Received: SIGSEGV (11)

Stack trace produced by process [pmdtm(:wf_mtt_01CRID0Z000000000005.s_mtt_01CRID0Z000000000005)] with pid [2394934]. ./libpmasrt.so(PmDumpStackTrace+0x79)[0x7f6128debd99] /opt/infaagent/apps/Data_Integration_Server/68.0.12.1/ICS/main/bin/rdtm/pmdtm[0x5e8c7a] /lib64/libpthread.so.0(+0x12d20)[0x7f612b2a2d20] /lib64/libc.so.6(+0xaa80b)[0x7f611da9f80b]. /libpmcrypto.so.1.0.0(lh_insert+0xa9)[0x7f612965c019] ./libpmcrypto.so.1.0.0(OBJ_NAME_add+0x79)[0x7f61295b9639] /lib64/libssl.so.1.1(+0x3718c)[0x7f60e7a0f18c]/lib64/libpthread.so.0(+0xfe97)[0x7f612b29fe97] /lib64/libcrypto.so.1.1(CRYPTO_THREAD_run_once+0xd)[0x7f60e76dea5d] /lib64/libssl.so.1.1(OPENSSL_init_ssl+0x5b)[0x7f60e7a0f3eb] /lib64/libpq.so.5(+0x2577c)[0x7f60e7c9177c] /lib64/libpq.so.5(PQconnectPoll+0xc70)[0x7f60e7c7c660] /lib64/libpq.so.5(+0x11805)[0x7f60e7c7d805] /lib64/libpq.so.5(PQconnectdbParams+0x28)[0x7f60e7c80748] /usr/local/lib/psqlodbcw.so(+0x16a5e)[0x7f60e7ed4a5e] /usr/local/lib/psqlodbcw.so(+0x1bad8)[0x7f60e7ed9ad8] /usr/local/lib/psqlodbcw.so(+0x1c3d7)[0x7f60e7eda3d7] /usr/local/lib/psqlodbcw.so(SQLConnect+0x9e)[0x7f60e7f1341e] /opt/infaagent/apps/Data_Integration_Server/../../drivers/misc/base/bin/libodbc.so.2(SQLConnect+0x1ee)[0x7f60e836973e] ./libpmodbc2.so(_ZN7TODBCDb7ConnectERK9PmUStringS2_S2_s+0x52a)[0x7f60e85dd5da] ./libpmodl.so(_ZN9TDatabase11ConnectImplERK9PmUStringS2_S2_s+0x67)[0x7f6125a19887] ./libpmodl.so(_ZN9TDatabase7ConnectERK9PmUStringS2_S2_si+0x12a)[0x7f6125a1a05a] ./libpmrelrdr.so(_ZN9relReader7connectEi+0xd8)[0x7f61180d4a98] ./libpmrelrdr.so(_ZN9relReader7ConnectEv+0x93)[0x7f61180d9893] /opt/infaagent/apps/Data_Integration_Server/68.0.12.1/ICS/main/bin/rdtm/pmdtm(_ZN15frame_blkReader4initEv+0xe2)[0x7c06f2] /opt/infaagent/apps/Data_Integration_Server/68.0.12.1/ICS/main/bin/rdtm/pmdtm(_ZN15SReaderRunnable13handleRequestERK15SManagerRequest+0x4d)[0x7c0a4d] /opt/infaagent/apps/Data_Integration_Server/68.0.12.1/ICS/main/bin/rdtm/pmdtm(_ZN7SWorker3runEv+0x372)[0x6341d2] /opt/infaagent/apps/Data_Integration_Server/68.0.12.1/ICS/main/bin/rdtm/pmdtm(_ZN12SDTMRunnable14MainEntryPointEv+0xf1)[0x60f901] ./libpmcef.so(_Z5runitPv+0xa)[0x7f612b08affa]/lib64/libpthread.so.0(+0x81ca)[0x7f612b2981ca]/lib64/libc.so.6(clone+0x43)[0x7f611da2e8d3]

davecramer commented 6 months ago

seems like exactly the same issue. This isn't particularly helpful

jraiford1 commented 4 months ago

Something that may be helpful is to capture the problem with ODBC tracing enabled. This will log every api call and return. It slows everything down quite a bit and the log grows quickly. I imagine only the tail end of the log would be useful to see what calls lead up to the crash. Note that I'm not sure if it will include personal information so you will probably want to review it before sending it.