ds9 not being registered

[KathleenLabrie]

I start ds9, I start Python. Then import imexam, and list_active_ds9(), and nothing, no active sessions registered.

(geminiconda)  klabrie$ ds9&
[1] 2881
(geminiconda)  klabrie$ python
Python 2.7.15 |Anaconda, Inc.| (default, May  1 2018, 18:37:05) 
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import imexam
>>> imexam.list_active_ds9()
No active sessions registered
{}
>>> 
>>> imexam.__version__
u'0.8.0'

I've had it work too. It seems to be random whether it works or not. I haven't been able to find a pattern.

The conda environment is created as follow: conda create -n geminiconda python=2.7 iraf-all pyraf-all stsci gemini

The ds9 version is v7.5.

[sosey]

arg...lemme see if I can replicate...

[sosey]

I can't reproduce you're error, but I wonder if it has something to do with the xpa.so that's getting installed in site-packages. I think the tar on pypi is messed up, running a local package install puts its in the imexam/ directory so it doesn't conflict with other packages. When I pull and build 0.8.0 from the repo the shared library goes to the proper place...give that a go in your environment and see if it fixes things for you?

[KathleenLabrie]

Thanks Megan. I can't build it. It complains that it cannot find xpa.h. (I'm doing a simple python setup.py install on a checkout of "stable".)

Also, I don't think that the location of the xpa.so library is the issue here, not the primary one anyway. I have the same environment on another machine, with xpa.so in site-packages, not in imexam/ and it works fine. (... most of the time... that's the thing, sometimes it stops working.)

The XPA_METHOD might be the issue, though I have about zero knowledge of that stuff. When I do in ds9 File->XPA->Information, on the system that does NOT work I get this:

XPA_VERSION: 2.1.14

XPA_CLASS: DS9

XPA_NAME: ds9

XPA_METHOD: /var/folders/qt/_25ftvnx3yb_7772q001mlkc00035n/T//DS9_ds9.3008

On the system that does work, I get this:

XPA_VERSION: 2.1.14 XPA_CLASS: DS9 XPA_NAME: ds9 XPA_METHOD: a014dac:63099

Does that tell you something?

Also, on the system that does not work, I get the annoying/dreaded "XPA unable to verify hostname" warning which I have tried to get rid of using all the tricks and tips I could find on the web, without success.

On Fri, May 4, 2018 at 8:27 AM, Megan Sosey notifications@github.com wrote:

I can't reproduce you're error, but I wonder if it has something to do with the xpa.so that's getting installed in site-packages. I think the tar on pypi is messed up, running a local package install puts its in the imexam/ directory so it doesn't conflict with other packages. When I pull and build 0.8.0 from the repo the shared library goes to the proper place...give that a go in your environment and see if it fixes things for you?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/spacetelescope/imexam/issues/137#issuecomment-386692827, or mute the thread https://github.com/notifications/unsubscribe-auth/ACDZfQGxK3smsev5UalYWQGHnNAq-VG5ks5tvJ2ZgaJpZM4Tx8lT .

[sosey]

you're getting the xpa.h file probably because the cextern directory needs to be initialized after the repo pull. git submodule update --init -- cextern/xpa should do it. Then you can python setup.py install

That's a really old xpa, you should consider updating to be at 2.1.18, as shown here: https://github.com/spacetelescope/imexam/tree/master/cextern

There are multiple methods that the XPA can use to talk to the dS9 windows, the one that is working looks like it's using an inet socket, the other one looks likes it's using a unix local host (though I don't know why it's going into a qt directory) that filename that you see is the communication socket. You could try setenv XPA_METHOD=inet on the failing system, that may force it. ds9 should default to inet, local sockets are sometimes used when there is no internet connection. The xpa server prefers that all connections for active ds9 windows are of the same socket type. imexam uses inet by default, but I imagine, if another xpaserver was started by a different program running a different type of connection, and one connected to the other, then it may have issues like you are seeing.

where is the xpa pulling from for you? which xpainfo, that should be the environment you're expecting it from, and not a system folder higher on your path.

[sosey]

oo, also if stable is not working, try using the the master branch... but there shouldn't be any xpa updates between the those two versions

[KathleenLabrie]

Wait a second here. People cannot use imexam with ds9 on a plane without internet, for example? Seems a bit ill advised. I'm just trying to display data that is local to my machine, why would I require internet for that?

which xpainfo does return the xpainfo from the correct environment. The same issues arise with v2.1.17 (which is what gets install in a fresh environment.)

Compiling 0.8.0 worked but didn't help.

I did a test on the machine where it works. Internet on, everything works. Internet off, boom.

ds9 = imexam.connect() ds9.view(data)

Traceback (most recent call last):

File "", line 1, in

File "/Users/klabrie/anaconda/envs/geminidev/lib/python2.7/site-packages/imexam/connect.py", line 489, in view

self.window.view(*args, **kwargs)

File "/Users/klabrie/anaconda/envs/geminidev/lib/python2.7/site-packages/imexam/ds9_viewer.py", line 1698, in view

frame = self.frame()

File "/Users/klabrie/anaconda/envs/geminidev/lib/python2.7/site-packages/imexam/ds9_viewer.py", line 965, in frame

frame = self.get("frame").strip()  # xpa returns '\n' for no frame

File "/Users/klabrie/anaconda/envs/geminidev/lib/python2.7/site-packages/imexam/ds9_viewer.py", line 683, in get

return self.xpa.get(param)

File "/Users/klabrie/anaconda/envs/geminidev/lib/python2.7/site-packages/imexam/xpa_wrap.py", line 15, in get

return super(XPA, self).get(param.encode('utf-8', 'strict')).decode()

File "wrappers/xpa.pyx", line 172, in xpa.xpa.get (wrappers/xpa.c:2378)

File "wrappers/xpa.pyx", line 112, in xpa._get (wrappers/xpa.c:1503)

xpa.XpaException: Unknown XPA Error : XPAGet returned 0!

BTW, I tried to force "inet", ds9 will have none of it. It resets it to local. (It does that on the "good" machine too when the internet is off.)

I don't know what to say. I do work offline sometimes in airport gate area, or in hotels with crappy internet. Even in conference when the internet gets overloaded and blows up. Requiring internet to display local data seems quite limiting to me.

At least, I think that we have figured out a likely source for the problem(s) I'm seeing. That's progress. (Now if I could just figure out why the other machine which is connected to internet still forces the XPA_METHOD to be local, then I would be half happy.)

On Wed, May 9, 2018 at 9:54 AM, Megan Sosey notifications@github.com wrote:

oo, also, don't checkout stable, checkout the v0.8.0 tag or use the master branch

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/spacetelescope/imexam/issues/137#issuecomment-387856162, or mute the thread https://github.com/notifications/unsubscribe-auth/ACDZfQiT-glWLYk58kwfyrqEMmGb718yks5tw0mNgaJpZM4Tx8lT .

[KathleenLabrie]

I found a solution! I don't know if it's THE solution but it sure works as a workaround. I can display offline and my XPA_METHOD local issue is gone too. The /etc/hosts needs to be edited. The first line needs to be: "127.0.0.1 machinename localhost". Not just localhost and not machinename.local (as recommended by some on the internet). (Tested on a Mac.) There might be consequences but that's easy to revert. No need for reboot.

Now I'm not just half happy, I'm full happy.

On Wed, May 9, 2018 at 3:48 PM, Kathleen Labrie < kathleen.labrie.phd@gmail.com> wrote:

Wait a second here. People cannot use imexam with ds9 on a plane without internet, for example? Seems a bit ill advised. I'm just trying to display data that is local to my machine, why would I require internet for that?

which xpainfo does return the xpainfo from the correct environment. The same issues arise with v2.1.17 (which is what gets install in a fresh environment.)

Compiling 0.8.0 worked but didn't help.

I did a test on the machine where it works. Internet on, everything works. Internet off, boom.

ds9 = imexam.connect() ds9.view(data)

Traceback (most recent call last):

File "", line 1, in

File "/Users/klabrie/anaconda/envs/geminidev/lib/python2.7/site-packages/imexam/connect.py", line 489, in view

self.window.view(*args, **kwargs)

File "/Users/klabrie/anaconda/envs/geminidev/lib/python2.7/site- packages/imexam/ds9_viewer.py", line 1698, in view

frame = self.frame()

File "/Users/klabrie/anaconda/envs/geminidev/lib/python2.7/site- packages/imexam/ds9_viewer.py", line 965, in frame

frame = self.get("frame").strip()  # xpa returns '\n' for no frame

File "/Users/klabrie/anaconda/envs/geminidev/lib/python2.7/site- packages/imexam/ds9_viewer.py", line 683, in get

return self.xpa.get(param)

File "/Users/klabrie/anaconda/envs/geminidev/lib/python2.7/site-packages/imexam/xpa_wrap.py", line 15, in get

return super(XPA, self).get(param.encode('utf-8', 'strict')).decode()

File "wrappers/xpa.pyx", line 172, in xpa.xpa.get (wrappers/xpa.c:2378)

File "wrappers/xpa.pyx", line 112, in xpa._get (wrappers/xpa.c:1503)

xpa.XpaException: Unknown XPA Error : XPAGet returned 0!

BTW, I tried to force "inet", ds9 will have none of it. It resets it to local. (It does that on the "good" machine too when the internet is off.)

I don't know what to say. I do work offline sometimes in airport gate area, or in hotels with crappy internet. Even in conference when the internet gets overloaded and blows up. Requiring internet to display local data seems quite limiting to me.

At least, I think that we have figured out a likely source for the problem(s) I'm seeing. That's progress. (Now if I could just figure out why the other machine which is connected to internet still forces the XPA_METHOD to be local, then I would be half happy.)

On Wed, May 9, 2018 at 9:54 AM, Megan Sosey notifications@github.com wrote:

oo, also, don't checkout stable, checkout the v0.8.0 tag or use the master branch

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/spacetelescope/imexam/issues/137#issuecomment-387856162, or mute the thread https://github.com/notifications/unsubscribe-auth/ACDZfQiT-glWLYk58kwfyrqEMmGb718yks5tw0mNgaJpZM4Tx8lT .

[sosey]

Wait a second here. People cannot use imexam with ds9 on a plane without internet, for example? Seems a bit ill advised. I'm just trying to display data that is local to my machine, why would I require internet for that?

no, imexam/pyds9 anything connecting with ds9 should work without internet, but not using inet, only with local or localhost. yah, if your hosts file was setting localhost to something other than 127.0.0.1 there were probably other programs having issues when there was no DNS server available. Glad you found the solution!

[jehturner]

If I have understood you both correctly, it's not that localhost was defined incorrectly on @KathleenLabrie's machine, it's that she had to add something like hostname (eg. "kathleen-laptop") to the usual 127.0.0.1 entry, which is not normal.

I'd be a bit wary about that as a general workaround, in terms of unexpected consequences and also some users not having root access to change it. I tend to find the fewest problems using display tools with a unix socket, if we could that working... I'll try testing imexam a bit better at some point too, though I don't have routine console access to an Apple machine like Kathleen's.

[sosey]

correct. I don't know why she had to change the localhost to point to her specific machine name. I don't have to do the same on my machine, whether or not I'm connected to the internet. I have the standard redirects:

127.0.0.1 localhost 255.255.255.255 broadcasthost

It would be great to have more testing...let me know if you run into the same problems so we can diagnose the best solution and add it to the solutions in the docs :) google-fu suggests also checking if there's a dns redirect in /etc/resolv.conf

[KathleenLabrie]

A follow up. I'm now testing a new laptop. OS X Mojave. Fresh computer. (like the other one) I am having the exact same problem that I solved the exact same way as before: adding the hostname before localhost on the 127.0.0.1 line in /etc/hosts. [imexam v0.8.0]

[mdhoney]

Was this ever resolved, @KathleenLabrie and @sosey? I am having the same problem with a fresh install of astroconda on Ubuntu 18.04. I get the dreaded No active sessions registered every time I try to imexam.list_active_ds9(). :frowning: Under DS9's XPA information, it says XPA_METHOD: 7f000101:34005 which seems to indicate it is using the proper inet sockets, correct? I swear this used to work on my previous machine... is there a way I can test my setup, maybe using something like xpainfo? Thanks.

[mdhoney]

I am having the same problem with a fresh install of astroconda on Ubuntu 18.04. I get the dreaded No active sessions registered every time I try to imexam.list_active_ds9().

I got this working now, the problem was apparently being caused by an incompatible DS9 version - anything > 7.5 will not work with Ubuntu 18.04 and astroconda!

[sosey]

good to hear. Are you using a conda installed binary or installing the binary yourself? Can I ask what the highest version you tried was? I had to move to 8.1b1 for the quartz app on mac myself.

[mdhoney]

good to hear. Are you using a conda installed binary or installing the binary yourself?

Using the conda binary in anaconda3/envs/astroconda/bin/ (getting DS9 via apt install saods9 is also possible, it installs v7.5).

Can I ask what the highest version you tried was? I had to move to 8.1b1 for the quartz app on mac myself.

astroconda installed v8.0.1. That didn't work, and neither did 7.6. 7.5-1 is the highest build that worked for me.

[sosey]

you might try installing v8.1b2 yourself just to see if it's working on ubuntu, if not it's worth giving Bill a heads up over at https://github.com/SAOImageDS9/SAOImageDS9 with a description of your issue.

[mdhoney]

I just compiled v8.1b2 on Ubuntu 18.04 (a few extra libraries needed installing) and am pleased to report that it registers with imexam!

[KathleenLabrie]

FYI, the saga continue. Different machine, still Mac. Same thing, won't work. I did the /etc/hosts trick, XPA_METHOD is local, I am running ds9 v8.1. imexam 0.9.1. No joy.

>>> ds9 = imexam.connect()

  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "/Users/klabrie/condaenvs/susd3_dev_20200710/lib/python3.7/site-packages/imexam/connect.py", line 96, in __init__
      quit_ds9_on_del=quit_window)
    File "/Users/klabrie/condaenvs/susd3_dev_20200710/lib/python3.7/site-packages/imexam/ds9_viewer.py", line 255, in __init__
      self._set_frameinfo()  # initial load
    File "/Users/klabrie/condaenvs/susd3_dev_20200710/lib/python3.7/site-packages/imexam/ds9_viewer.py", line 295, in _set_frameinfo
      frame = self.frame()
    File "/Users/klabrie/condaenvs/susd3_dev_20200710/lib/python3.7/site-packages/imexam/ds9_viewer.py", line 966, in frame
      frame = self.get("frame").strip()  # xpa returns '\n' for no frame
    File "/Users/klabrie/condaenvs/susd3_dev_20200710/lib/python3.7/site-packages/imexam/ds9_viewer.py", line 684, in get
      return self.xpa.get(param)
    File "/Users/klabrie/condaenvs/susd3_dev_20200710/lib/python3.7/site-packages/imexam/xpa_wrap.py", line 14, in get
      return super(XPA, self).get(param.encode('utf-8', 'strict')).decode()
    File "wrappers/xpa.pyx", line 173, in imexam.imexamxpa.xpa.get
    File "wrappers/xpa.pyx", line 113, in imexam.imexamxpa._get
  imexam.imexamxpa.XpaException: b'XPA$ERROR: no response from server during handshake (?:?)\n'

[sosey]

ok - let me see if I can find some time to work on this package in the next couple weeks and try to replicate the issue you are having on my local setup

[DBerke]

I'm having a possibly-related issue where imexam.list_active_ds9() shows an active window (after setting export XPA_METHOD="local" in my .bash_profile), but trying to connect to it using imexam.connect(imexam.list_active_ds9()) fails. I tried modifying /etc/hosts as Kathleen mentioned above, but it didn't change anything that I could tell. I created a simple conda environment with conda create -n test python=3.7 stsci to get the output below (this is using Bash, I've just got a fancy prompt).

~ via 🅒 test 
❯ ds9 &
[1] 78039

~ via 🅒 test 
✦ ❯ python
Python 3.7.11 (default, Jul 27 2021, 07:03:16) 
[Clang 10.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import imexam
>>> imexam.list_active_ds9()
DS9 ds9 gs /var/folders/01/3whmpm4d5jsgwc7nb1rtj62h000g0c/T//DS9_ds9.78039 daniel.berke
{'/var/folders/01/3whmpm4d5jsgwc7nb1rtj62h000g0c/T//DS9_ds9.78039': ('ds9', 'daniel.berke', 'DS9', 'gs')}
>>> ds9 = imexam.connect(imexam.list_active_ds9())
DS9 ds9 gs /var/folders/01/3whmpm4d5jsgwc7nb1rtj62h000g0c/T//DS9_ds9.78039 daniel.berke

DS9 target: {'/var/folders/01/3whmpm4d5jsgwc7nb1rtj62h000g0c/T//DS9_ds9.78039': ('ds9', 'daniel.berke', 'DS9', 'gs')} doesn't exist.
['/var/folders/01/3whmpm4d5jsgwc7nb1rtj62h000g0c/T//DS9_ds9.78039']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/daniel.berke/opt/anaconda3/envs/test/lib/python3.7/site-packages/imexam/connect.py", line 96, in __init__
    quit_ds9_on_del=quit_window)
  File "/Users/daniel.berke/opt/anaconda3/envs/test/lib/python3.7/site-packages/imexam/ds9_viewer.py", line 224, in __init__
    "Please choose an existing target."
ValueError: Please choose an existing target.
>>>

This is on macOS 11.6 Big Sur, imexam version 0.9.1, DS9 version 8.1.

EDIT: Nevermind, ignore this, sorry. I found that using ds9 = imexam.connect(list(imexam.list_active_ds9())[0]) works fine.