CASA crash due to casacore import ?!?!

Until December 2018, from inside CASA, I used to: import casacore.tables as pt Everything worked fine. Today I updated: conda update --all And then, that import statement makes CASA to crash! (segmentation fault)

Some more information is needed. Can it be that the CASA and python-casacore shared libraries are conflicting?

On Fri, Feb 22, 2019 at 12:06 AM satya-yuga notifications@github.com wrote:

Reopened #164 https://github.com/casacore/python-casacore/issues/164.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/casacore/python-casacore/issues/164#event-2156451281, or mute the thread https://github.com/notifications/unsubscribe-auth/AHfIPA1gaF1SJiqFAL5cJR0xut82BToOks5vPyZjgaJpZM4bIgiO .

This could indeed well be a conflicting library issue. Note that python-casacore and casa are not designed to work together; in particular the way casa is shipped with its own python interpreter makes it hard to get the libraries right.

From casa, much of the python-casacore functionality is already accessible through tb. E.g. t = tb.open("bla.MS"), data = t.getcol("DATA"). See https://casa.nrao.edu/docs/CasaRef/table-Module.html

Some more information is needed. Can it be that the CASA and python-casacore shared libraries are conflicting? … On Fri, Feb 22, 2019 at 12:06 AM satya-yuga @.***> wrote: Reopened #164 <#164>. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#164 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/AHfIPA1gaF1SJiqFAL5cJR0xut82BToOks5vPyZjgaJpZM4bIgiO .

Conflicting shared libraries? ... hm ... maybe? ... but I employed python-casacore libraries from inside CASA during whole 2018, and it worked fine. After I updated python-casacore, it made CASA to crash. It has been absolutely annoying to split my scripts between the ones that employ python-casacore and the ones that doesn't.

This could indeed well be a conflicting library issue. Note that python-casacore and casa are not designed to work together; in particular the way casa is shipped with its own python interpreter makes it hard to get the libraries right.

From casa, much of the python-casacore functionality is already accessible through tb. E.g. t = tb.open("bla.MS"), data = t.getcol("DATA"). See https://casa.nrao.edu/docs/CasaRef/table-Module.html

I need to select some data from MS, alter it and update it in the MS. Hence, I did employ ms.getdata and ms.putdata. However, they have issues that I reported on September 7th 2018. The CASA Helpdesk recognized the 3 issues. They are still open under ticket "NRAO 12628".

After I reported those issues, I realized that TaQL works perfect for my needs. Sadly, I can only access it by casacore.tables which, after update, it makes CASA to crash. But then, I realized that CASA's python works fine with casacore.tables, as well as CONDA's python. Isn't it weird?

I insist: python-casacore worked fine inside CASA until the python-casacore update.

I'm sorry we cannot debug this without further information regarding your installation. Could you have a look at which libraries are loaded during import? E.g. within casa do an import casacore.tables as pt; print(pt.__file__); A really big hammer to hit this problem with is to start casa with LD_DEBUG=libs casapy and filter the output for casa and boost_python. My suspicion is that there will be two libraries with the same name in different locations.

A quicker solution for now would be to use the taql binding from within casa. You can use it from the tb module. E.g. t = tb.open("bla.MS"); tb.taql("update bla.MS set FLAG = mscal.azel("CygA")[1]>20deg") to flag all data where Cygnus A is higher than 20 degrees.

I'm sorry we cannot debug this without further information regarding your installation. Could you have a look at which libraries are loaded during import? E.g. within casa do an import casacore.tables as pt; print(pt.__file__); A really big hammer to hit this problem with is to start casa with LD_DEBUG=libs casapy and filter the output for casa and boost_python. My suspicion is that there will be two libraries with the same name in different locations.

A quicker solution for now would be to use the taql binding from within casa. You can use it from the tb module. E.g. t = tb.open("bla.MS"); tb.taql("update bla.MS set FLAG = mscal.azel("CygA")[1]>20deg") to flag all data where Cygnus A is higher than 20 degrees.

I don't have any casapy but only casa which I downloaded from casa.nrao.edu I followed your instructions and I got 4780 output lines. When I filtered by "casa" I got 2594 lines. When I filtered by "boost_python" I just got 2 lines that seemed fine.

However, I realized there were two FATAL output lines: 1) CrashReportPoster: error: symbol lookup error: undefined symbol: curl_easy_init (fatal) 2) /lib/x86_64-linux-gnu/libnss_mdns4_minimal.so.2: error: symbol lookup error: undefined symbol: _nss_mdns4_minimal_gethostbyname4_r (fatal)

Any thoughts?

I'm not doing Casa these days, but I did get this message.

I think the two lines are probably a secondary error. The program crashed (seg faulted or did an uncaught exception) and then when it tried to send a report out, it ran into the CrashReportPoster problem. This is definitely not a /casacore/ problem. Probably the installation resulted in Casa being paired with an unexpected version of the library; this problem would be latent unless Casa actually crashes.

So, there are two issues:

1) Whatever caused Casa to crash originally and generate a crash report.

2) A deployment issue that prevents Casa from sending the crash report back to here.

Solving issue 1 is what needs to be done to help the user. I would suggest looking in the log just a little bit above where the CrashReportPoster error appeared to see if there are any clues as to the cause of the original crash.

On 2/24/19 11:45 PM, satya-yuga wrote:

I'm sorry we cannot debug this without further information
regarding your installation. Could you have a look at which
libraries are loaded during import? E.g. within casa do an |import
casacore.tables as pt; print(pt.__file__);| A really big hammer to
hit this problem with is to start casa with |LD_DEBUG=libs casapy|
and filter the output for |casa| and |boost_python|. My suspicion
is that there will be two libraries with the same name in
different locations.

A quicker solution for now would be to use the taql binding from
within casa. You can use it from the |tb| module. E.g. |t =
tb.open("bla.MS"); tb.taql("update bla.MS set FLAG =
mscal.azel("CygA")[1]>20deg")| to flag all data where Cygnus A is
higher than 20 degrees.
I don't have any casapy but only casa which I downloaded from casa.nrao.edu I followed your instructions and I got 4780 output lines. When I filtered by "casa" I got 2594 lines. When I filtered by "boost_python" I just got 2 lines that seemed fine.

However, I realized there were two FATAL output lines:

CrashReportPoster: error: symbol lookup error: undefined symbol: curl_easy_init (fatal)

/lib/x86_64-linux-gnu/libnss_mdns4_minimal.so.2: error: symbol lookup error: undefined symbol: _nss_mdns4_minimal_gethostbyname4_r (fatal)

Any thoughts?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/casacore/python-casacore/issues/164#issuecomment-466891008, or mute the thread https://github.com/notifications/unsubscribe-auth/AGPGtN4nobBmRhD_AXsW3DpMnvznLH4iks5vQ4aIgaJpZM4bIgiO.Web Bug from https://github.com/notifications/beacon/AGPGtHcV_367p_hayEti-upsbTdoYxkkks5vQ4aIgaJpZM4bIgiO.gif

{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/casacore/python-casacore","title":"casacore/python-casacore","subtitle":"GitHub repository","main_image_url":"https://github.githubassets.com/images/email/message_cards/header.png","avatar_image_url":"https://github.githubassets.com/images/email/message_cards/avatar.png","action":{"name":"Open in GitHub","url":"https://github.com/casacore/python-casacore"}},"updates":{"snippets":[{"icon":"PERSON","message":"@satya-yuga in #164: \u003e I'm sorry we cannot debug this without further information regarding your installation. Could you have a look at which libraries are loaded during import? E.g. within casa do an import casacore.tables as pt; print(pt.__file__); A really big hammer to hit this problem with is to start casa with LD_DEBUG=libs casapy and filter the output for casa and boost_python. My suspicion is that there will be two libraries with the same name in different locations.\r\n\u003e \r\n\u003e A quicker solution for now would be to use the taql binding from within casa. You can use it from the tb module. E.g. t = tb.open(\"bla.MS\"); tb.taql(\"update bla.MS set FLAG = mscal.azel(\"CygA\")[1]\u003e20deg\") to flag all data where Cygnus A is higher than 20 degrees.\r\n\r\nI don't have any casapy but only casa which I downloaded from casa.nrao.edu\r\nI followed your instructions and I got 4780 output lines. When I filtered by \"casa\" I got 2594 lines. When I filtered by \"boost_python\" I just got 2 lines that seemed fine.\r\n\r\nHowever, I realized there were two FATAL output lines:\r\n1) CrashReportPoster: error: symbol lookup error: undefined symbol: curl_easy_init (fatal)\r\n2) /lib/x86_64-linux-gnu/libnss_mdns4_minimal.so.2: error: symbol lookup error: undefined symbol: _nss_mdns4_minimal_gethostbyname4_r (fatal)\r\n\r\nAny thoughts?\r\n"}],"action":{"name":"View Issue","url":"https://github.com/casacore/python-casacore/issues/164#issuecomment-466891008"}}}[ { "@context": "http://schema.org", "@type": "EmailMessage", "potentialAction": { "@type": "ViewAction", "target": "https://github.com/casacore/python-casacore/issues/164#issuecomment-466891008","url": "https://github.com/casacore/python-casacore/issues/164#issuecomment-466891008", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { "@type": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

-- Jim Jacobs

phone: +1 575.835.7235 office: 301

I'm not doing Casa these days, but I did get this message. I think the two lines are probably a secondary error. The program crashed (seg faulted or did an uncaught exception) and then when it tried to send a report out, it ran into the CrashReportPoster problem. This is definitely not a /casacore/ problem. Probably the installation resulted in Casa being paired with an unexpected version of the library; this problem would be latent unless Casa actually crashes. So, there are two issues: 1) Whatever caused Casa to crash originally and generate a crash report. 2) A deployment issue that prevents Casa from sending the crash report back to here. Solving issue 1 is what needs to be done to help the user. I would suggest looking in the log just a little bit above where the CrashReportPoster error appeared to see if there are any clues as to the cause of the original crash. … On 2/24/19 11:45 PM, satya-yuga wrote: I'm sorry we cannot debug this without further information regarding your installation. Could you have a look at which libraries are loaded during import? E.g. within casa do an |import casacore.tables as pt; print(pt.file);| A really big hammer to hit this problem with is to start casa with |LD_DEBUG=libs casapy| and filter the output for |casa| and |boost_python|. My suspicion is that there will be two libraries with the same name in different locations. A quicker solution for now would be to use the taql binding from within casa. You can use it from the |tb| module. E.g. |t = tb.open("bla.MS"); tb.taql("update bla.MS set FLAG = mscal.azel("CygA")[1]>20deg")| to flag all data where Cygnus A is higher than 20 degrees. I don't have any casapy but only casa which I downloaded from casa.nrao.edu I followed your instructions and I got 4780 output lines. When I filtered by "casa" I got 2594 lines. When I filtered by "boost_python" I just got 2 lines that seemed fine. However, I realized there were two FATAL output lines: 1. CrashReportPoster: error: symbol lookup error: undefined symbol: curl_easy_init (fatal) 2. /lib/x86_64-linux-gnu/libnss_mdns4_minimal.so.2: error: symbol lookup error: undefined symbol: _nss_mdns4_minimal_gethostbyname4_r (fatal) Any thoughts? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#164 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/AGPGtN4nobBmRhD_AXsW3DpMnvznLH4iks5vQ4aIgaJpZM4bIgiO.Web Bug from https://github.com/notifications/beacon/AGPGtHcV_367p_hayEti-upsbTdoYxkkks5vQ4aIgaJpZM4bIgiO.gif {"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/casacore/python-casacore","title":"casacore/python-casacore","subtitle":"GitHub repository","main_image_url":"https://github.githubassets.com/images/email/message_cards/header.png","avatar_image_url":"https://github.githubassets.com/images/email/message_cards/avatar.png","action":{"name":"Open in @. in #164: \u003e I'm sorry we cannot debug this without further information regarding your installation. Could you have a look at which libraries are loaded during import? E.g. within casa do an import casacore.tables as pt; print(pt.__file__); A really big hammer to hit this problem with is to start casa with LD_DEBUG=libs casapy and filter the output for casa and boost_python. My suspicion is that there will be two libraries with the same name in different locations.\r\n\u003e \r\n\u003e A quicker solution for now would be to use the taql binding from within casa. You can use it from the tb module. E.g. t = tb.open(\"bla.MS\"); tb.taql(\"update bla.MS set FLAG = mscal.azel(\"CygA\")[1]\u003e20deg\") to flag all data where Cygnus A is higher than 20 degrees.\r\n\r\nI don't have any casapy but only casa which I downloaded from casa.nrao.edu\r\nI followed your instructions and I got 4780 output lines. When I filtered by \"casa\" I got 2594 lines. When I filtered by \"boost_python\" I just got 2 lines that seemed fine.\r\n\r\nHowever, I realized there were two FATAL output lines:\r\n1) CrashReportPoster: error: symbol lookup error: undefined symbol: curl_easy_init (fatal)\r\n2) /lib/x86_64-linux-gnu/libnss_mdns4_minimal.so.2: error: symbol lookup error: undefined symbol: _nss_mdns4_minimal_gethostbyname4_r (fatal)\r\n\r\nAny thoughts?\r\n"}],"action":{"name":"View Issue","url":"#164 (comment)"}}}[ { @.": "http://schema.org", @.": "EmailMessage", "potentialAction": { @.": "ViewAction", "target": "#164 (comment)","url": "#164 (comment)", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { @.***": "Organization", "name": "GitHub", "url": "https://github.com" } } ] -- Jim Jacobs phone: +1 575.835.7235 office: 301

I tried to dig deeper into the problem, but I didn't get any light. However, I found something weird: all my installation and libraries are related to python2.7, but among the huge debug info there is one call to one library for python3. Mysterious. I will leave this issue here since I cannot waste additional time. Thanks for all your time. Greetings!

casacore / python-casacore

CASA crash due to casacore import ?!?! #164