search

sdss / lvmcam

An actor implementation of sdss-araviscam for lvmagp
https://sdss-lvmcam.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
2 stars 0 forks source link

Add support for virtual cameras #49

Closed wasndas closed 2 years ago

wasndas commented 2 years ago

For testing purposes a virtual or simulated camera whould be good to have see lvmnps, put a global type key in the configuration file, eg:

type: araviscam
# type: skymakercam
# type: my_own_dummy_maybe_using fits_files

add in pyproject.toml:

 [tool.poetry.dependencies]
   ....
 sdss-skymakercam ">0.0.5"

Put in the actor some kind of a factory.

from araviscam import BlackflyCameraSystem, BlackflyCam
from skymakercam import SkymakerCameraSystem, SkymakerCamera

camera_types = {"araviscam" : lambda: BlackflyCameraSystem(BlackflyCam),
                "skymakercam" : lambda: SkymakerCameraSystem(SkymakerCamera)}

name = config["camtype"]
super().__init__(camera_types[name](), *args, **kwargs)

a better way whould be using import dynamically, instead of bloating it with imports.

camera_types = {"araviscam" : lambda: CameraType.BlackflyCameraSystem(CameraType.BlackflyCam),
                "skymakercam" : lambda: CameraType.SkymakerCameraSystem(CameraType.SkymakerCamera)}

name = config["camtype"]
CameraType = __import__(name)

super().__init__(camera_types[name](), *args, **kwargs)
mgjeon commented 2 years ago

Thank you for your code review and feedback. I will update lvmcam as soon as possible.

I have some questions about errors that happened in testing skymakercam

Here's what I did:

  1. git clone https://github.com/sdss/skymakercam && cd skymakercam
  2. Run AMQP actor named as lvm.sci.pwi
  3. Run command $ poetry run python utils/plot_skymakercam.py -v -c python/skymakercam/etc/cameras.yaml lvm.sci.agw.cam
  4. Then I got a Exception: Command '__commands' was aborted. error.
$ poetry run python utils/plot_skymakercam.py -v -c python/skymakercam/etc/cameras.yaml lvm.sci.agw.cam
[DEBUG]: [SKYMAKERCAMERASYSTEM]: read configuration file from python/skymakercam/etc/cameras.yaml
[DEBUG]: [SKYMAKERCAMERASYSTEM]: adding camera 'lvm.sci.agw.cam' with parameters {'uid': 'lvm.sci.agw.cam', 'descr': 'Guider Camera Science', 'default': {'gain': 5.0, 'binning': [4, 4]}, 'instpar': 'lvm_sci_agw_cam', 'focus_stage': 'lvm.sci.foc', 'kmirror': 'lvm.sci.km', 'tcs': 'lvm.sci.pwi', 'catalog_path': '$HOME/data/catalog/gaia'}
[DEBUG]: [LVM.SCI.AGW.CAM]: [1600, 1100]
[DEBUG]: [LVM.SCI.AGW.CAM]: connecting ...
[ERROR]: Traceback (most recent call last):
  File "/home/mgjeon/skymakercam/utils/plot_skymakercam.py", line 107, in <module>
    main()
  File "/home/mgjeon/skymakercam/utils/plot_skymakercam.py", line 102, in main
    asyncio.run(plot_skymakercam(args.exptime, args.binning, args.guiderect, args.camname, verb=args.verbose, config=args.cfg))
  File "/home/mgjeon/.pyenv/versions/3.9.8/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/home/mgjeon/.pyenv/versions/3.9.8/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/home/mgjeon/skymakercam/utils/plot_skymakercam.py", line 35, in plot_skymakercam
    cam = await cs.add_camera(name=camname, uid=cs._config[camname]["uid"])
  File "/home/mgjeon/.pyenv/versions/3.9.8/envs/ss20/lib/python3.9/site-packages/basecam/camera.py", line 386, in add_camera
    await camera.connect()
  File "/home/mgjeon/.pyenv/versions/3.9.8/envs/ss20/lib/python3.9/site-packages/basecam/camera.py", line 634, in connect
    await self._connect_internal(**conn_params)
  File "/home/mgjeon/skymakercam/python/skymakercam/camera.py", line 182, in _connect_internal
    await self._tcs.start()
  File "/home/mgjeon/.pyenv/versions/3.9.8/envs/ss20/lib/python3.9/site-packages/cluplus/proxy.py", line 131, in start_async
    commands = (await self.call_command(Proxy.__commands))[Proxy.__comkey]
  File "/home/mgjeon/.pyenv/versions/3.9.8/envs/ss20/lib/python3.9/site-packages/cluplus/proxy.py", line 214, in call_command
    raise self._errorMapToException(ret.replies[-1].body['error'])
Exception: Command '__commands' was aborted.

I'm not sure but I think the error occurred at tcs = Proxy(). But I don't know the solution to this error. So please let me know how to solve this problem.

wasndas commented 2 years ago

... did you start lvmtan & lvmpwi ? Are they the latest ? Check with clu lvm.sci.km help, if the command __command is available.

wasndas commented 2 years ago

... if you want to write your own virtual camera have a look here. You could use a bunch of fits files for example and return them when expose() is called.

mgjeon commented 2 years ago

There are some issues for running lvmtan and lvmpwi. This comment is written just for notifying these problems.

For lvmtan,

  1. poetry install error because of version for black and python in pyproject.toml I temporarily fixed it by changing python = ">=3.6, <4" -> python = ">=3.7, <4"
  2. syntax error because there is no expect: in python try: ... expect" ...
  3. Error when starting lvmtan 
    $ poetry run container_start --name lvm.all
False
podman run --rm -t --name lvm.all --network=host --runtime /usr/bin/crun -v /home/mgjeon/lvmtan:/root/lvmt:Z -e BASDARD_CONFIG=lvm/lvm.all-sim.conf localhost/ubuntu_lvmt_tan
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/mgjeon/.pyenv/versions/3.9.8/envs/ss20/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/home/mgjeon/.pyenv/versions/3.9.8/envs/ss20/lib/python3.9/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/home/mgjeon/.pyenv/versions/3.9.8/envs/ss20/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/mgjeon/.pyenv/versions/3.9.8/envs/ss20/lib/python3.9/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/home/mgjeon/lvmtan/container/__init__.py", line 95, in start
child.expect("Connected to")
File "/home/mgjeon/.pyenv/versions/3.9.8/envs/ss20/lib/python3.9/site-packages/pexpect/spawnbase.py", line 343, in expect
return self.expect_list(compiled_pattern_list,
File "/home/mgjeon/.pyenv/versions/3.9.8/envs/ss20/lib/python3.9/site-packages/pexpect/spawnbase.py", line 372, in expect_list
return exp.expect_loop(timeout)
File "/home/mgjeon/.pyenv/versions/3.9.8/envs/ss20/lib/python3.9/site-packages/pexpect/expect.py", line 179, in expect_loop
return self.eof(e)
File "/home/mgjeon/.pyenv/versions/3.9.8/envs/ss20/lib/python3.9/site-packages/pexpect/expect.py", line 122, in eof
raise exc
pexpect.exceptions.EOF: End Of File (EOF). Exception style platform.
<pexpect.pty_spawn.spawn object at 0x7f8f06745c70>
command: /usr/bin/podman
args: ['/usr/bin/podman', 'run', '--rm', '-t', '--name', 'lvm.all', '--network=host', '--runtime', '/usr/bin/crun', '-v', '/home/mgjeon/lvmtan:/root/lvmt:Z', '-e', 'BASDARD_CONFIG=lvm/lvm.all-sim.conf', 'localhost/ubuntu_lvmt_tan']
buffer (last 100 chars): b''
before (last 100 chars): b'2e". You have to remove that container to be able to reuse that name.: that name is already in use\r\n'
after: <class 'pexpect.exceptions.EOF'>
match: None
match_index: None
exitstatus: None
flag_eof: True
pid: 1987361
child_fd: 5
closed: False
timeout: 30
delimiter: <class 'pexpect.exceptions.EOF'>
logfile: None
logfile_read: None
logfile_send: None
maxread: 2000
ignorecase: False
searchwindowsize: None
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1
searcher: searcher_re:
0: re.compile(b'Connected to')

For lvmpwi,

  1. poetry install error because of version for sphinx-bootstrap-theme in pyproject.toml I temporarily fixed it just by deleting the line.
  2. There is 504 Gateway Timeout issue of mono on Ubuntu when building lvmpwi. It seems that this issue starts to occur recently and the same problem happens even in Japan.
$ poetry run container_build
...
E: Failed to fetch https://download.mono-project.com/repo/ubuntu/pool/main/m/mono/mono-runtime_6.12.0.122-0xamarin1+ubuntu2004b1_amd64.deb  504  Gateway Timeout [IP: 117.18.232.200 443]
E: Failed to fetch https://download.mono-project.com/repo/ubuntu/pool/main/m/mono/libmono-system-xml4.0-cil_6.12.0.122-0xamarin1+ubuntu2004b1_all.deb  504  Gateway Timeout [IP: 117.18.232.200 443]
E: Failed to fetch https://download.mono-project.com/repo/ubuntu/pool/main/g/gtk-sharp2/libglib2.0-cil-dev_2.12.45-0xamarin19+ubuntu2004b1_amd64.deb  504  Gateway Timeout [IP: 117.18.232.200 443]
E: Failed to fetch https://download.mono-project.com/repo/ubuntu/pool/main/m/mono/libmono-system-drawing4.0-cil_6.12.0.122-0xamarin1+ubuntu2004b1_all.deb  504  Gateway Timeout [IP: 117.18.232.200 443]
E: Failed to fetch https://download.mono-project.com/repo/ubuntu/pool/main/m/mono/libmono-codecontracts4.0-cil_6.12.0.122-0xamarin1+ubuntu2004b1_all.deb  504  Gateway Timeout [IP: 117.18.232.200 443]Fetched 153 MB in 3min 46s (676 kB/s)

E: Failed to fetch https://download.mono-project.com/repo/ubuntu/pool/main/m/mono/libmono-system-transactions4.0-cil_6.12.0.122-0xamarin1+ubuntu2004b1_all.deb  504  Gateway Timeout [IP: 117.18.232.200 443]
E: Failed to fetch https://download.mono-project.com/repo/ubuntu/pool/main/m/mono/libmono-webbrowser4.0-cil_6.12.0.122-0xamarin1+ubuntu2004b1_all.deb  504  Gateway Timeout [IP: 117.18.232.200 443]
E: Failed to fetch https://download.mono-project.com/repo/ubuntu/pool/main/m/mono/libmono-microsoft-csharp4.0-cil_6.12.0.122-0xamarin1+ubuntu2004b1_all.deb  504  Gateway Timeout [IP: 117.18.232.200 443]
E: Failed to fetch https://download.mono-project.com/repo/ubuntu/pool/main/m/mono/libmono-parallel4.0-cil_6.12.0.122-0xamarin1+ubuntu2004b1_all.deb  504  Gateway Timeout [IP: 117.18.232.200 443]
E: Failed to fetch https://download.mono-project.com/repo/ubuntu/pool/main/m/mono/libmono-system-reactive-windows-threading2.2-cil_6.12.0.122-0xamarin1+ubuntu2004b1_all.deb  504  Gateway Timeout [IP: 117.18.232.200 443]
E: Failed to fetch https://download.mono-project.com/repo/ubuntu/pool/main/m/mono/libmono-system-reflection-context4.0-cil_6.12.0.122-0xamarin1+ubuntu2004b1_all.deb  504  Gateway Timeout [IP: 117.18.232.200 443]
E: Failed to fetch https://download.mono-project.com/repo/ubuntu/pool/main/m/mono/libmono-system-runtime-durableinstancing4.0-cil_6.12.0.122-0xamarin1+ubuntu2004b1_all.deb  504  Gateway Timeout [IP: 117.18.232.200 443]
E: Failed to fetch https://download.mono-project.com/repo/ubuntu/pool/main/m/mono/libmono-system-web-extensions-design4.0-cil_6.12.0.122-0xamarin1+ubuntu2004b1_all.deb  504  Gateway Timeout [IP: 117.18.232.200 443]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
Error: error building at STEP "RUN apt-get update     && DEBIAN_FRONTEND="noninteractive" apt-get install -y --no-install-recommends         wget         wmctrl         xfonts-base xfonts-75dpi xfonts-100dpi         tightvncserver         eterm         fluxbox         xrdp         libgtk2.0         mono-devel         mono-basic-dbg": error while running runtime: exit status 100
wasndas commented 2 years ago
  1. lvmtan:
    • is there a stale lvmtan container ? podman ps -a or _poetry run containerstart --kill --name lvm.all
    • removed python 3.6 support
  2. lvmpwi:
    • maybe its comming back, it did work from here.
    • deleted the sphinx-bootstrap-theme
mgjeon commented 2 years ago

Thanks for replying.

  1. lvmtan

There was running lvmtan container. After stopping it, it seems that lvmtan works well. I didn't think of checking this. Thank you for letting me know.

$ podman ps -a
CONTAINER ID  IMAGE                             COMMAND               CREATED       STATUS           PORTS       NAMES
16f6b310c3b6  localhost/ubuntu_lvmt_tan:latest  /root/run-basdard...  39 hours ago  Up 39 hours ago              lvm.all
f1843f23b07d  localhost/ubuntu_lvmt_tan:latest  /root/run-basdard...  38 hours ago  Up 38 hours ago              test.first.focus_stage

$ poetry run container_start --kill --name lvm.all
lvm.all
False
podman run --rm -t --name lvm.all --network=host --runtime /usr/bin/crun -v /home/mgjeon/lvmtan:/root/lvmt:Z -e BASDARD_CONFIG=lvm/lvm.all-sim.conf localhost/ubuntu_lvmt_tan

$ podman ps -a                                    
CONTAINER ID  IMAGE                             COMMAND               CREATED        STATUS            PORTS       NAMES
f1843f23b07d  localhost/ubuntu_lvmt_tan:latest  /root/run-basdard...  38 hours ago   Up 38 hours ago               test.first.focus_stage
ca408392db28  localhost/ubuntu_lvmt_tan:latest  /root/run-basdard...  3 seconds ago  Up 3 seconds ago              lvm.all

$ poetry run container_stop --name lvm.all
lvm.all

$ poetry run container_stop               
test.first.focus_stage

$ podman ps -a             
CONTAINER ID  IMAGE       COMMAND     CREATED     STATUS      PORTS       NAMES

$ poetry run container_start --kill --name lvm.all
Error: no container with name or ID "lvm.all" found: no such container
False
podman run --rm -t --name lvm.all --network=host --runtime /usr/bin/crun -v /home/mgjeon/lvmtan:/root/lvmt:Z -e BASDARD_CONFIG=lvm/lvm.all-sim.conf localhost/ubuntu_lvmt_tan

$ podman ps -a                                    
CONTAINER ID  IMAGE                             COMMAND               CREATED        STATUS            PORTS       NAMES
0c8d1f0279f5  localhost/ubuntu_lvmt_tan:latest  /root/run-basdard...  4 seconds ago  Up 4 seconds ago              lvm.all
  1. lvmpwi

The 504 Gateway Timeout issue of mono still occurs for me in Korea. I think it's not resolved for some countries. I wrote an issue on mono repository.

wasndas commented 2 years ago

Thats annoying. Are you sure its not a problem at KHU ? At MPIA we have had a problem with github, after our official IT blocked one of the ip address of one github server in there firewall.

Maybe as a workaround, if you still can start the old pwi container and have checked out the latest lvmpwi you can run it with the --debug option.

 # run container using the local lvmpwi instead of the git version included into the container
 poetry run container_start --debug --simulator --name=lvm.sci.pwi

I should have a look at github docker packages thing.

mgjeon commented 2 years ago

I'm sure it's not a problem at KHU because I tried to install mono on my personal Ubuntu PC in my home and faced the same issue (504 Gateway Timeout). So it would be a problem in Korea or some countries.

By the way, I found a KHU team PC on which lvmpwi & lvmtan are already installed. In the pc, lvmpwi & lvmtan work well. And I checked skymakercam also works well while running lvmpwi & lvmtan.

So as a temporary solution, I'll use the PC for testing a virtual camera for lvmcam.

$ poetry run container_start --debug --simulator --name=lvm.sci.pwi
podman run --rm -d --name lvm.sci.pwi -e LVMT_RMQ=None -e DISPLAY -v /run/user/1000/gdm/Xauthority:/root/.Xauthority:Z --ipc=host  --network=host --device /dev/dri -p 8220 -e PWI_DEBUG=true -e PWI_SIMULATOR=true -v /dev:/dev:rslave -v /home/sumin/lvmpwi:/root/lvmt:Z -e PWI_NAME=lvm.sci.pwi localhost/ubuntu_lvmt_pwi
Port mappings have been discarded as one of the Host, Container, Pod, and None network modes are in use
90456b11949a4132848aa2ea463e80eb7c7fb5cbec4a37504a22bc3adf79b9dd
Could not set X locale modifiers
Gtk-Message: 10:01:50.630: Failed to load module "canberra-gtk-module"
Starting __main__.py ... BSC loaded in 23ms
BSC loaded in 18ms
OK
PointXPCore.OnModelChanged
PointXPCore.Calibrate() 1
Final thetaEncOffset=0, phiEncOffset=0
Calibrated in 1 ms
PointXPCore.CalPointErrors()
Calculated residual errors in 0 ms
PointXPCore.OnModelChanged
PointXPCore.Calibrate() 2
Final thetaEncOffset=0, phiEncOffset=0
Calibrated in 0 ms
PointXPCore.CalPointErrors()
Calculated residual errors in 0 ms
wasndas commented 2 years ago

... I think there is an export/import for container images, you could copy the image to your computer. Cheers, florianhttps://docs.podman.io/en/latest/markdown/podman-export.1.htmlVon meinem Huawei-Telefon gesendet-------- Ursprüngliche Nachricht --------Von: Mingyu Jeon @.>Datum: So., 12. Dez. 2021, 11:15An: sdss/lvmcam @.>Cc: "Florian B." @.>, Author @.>Betreff: Re: [sdss/lvmcam] Add support for virtual cameras (Issue #49) I'm sure it's not a problem at KHU because I tried to install mono on my personal Ubuntu PC in my home and faced the same issue (504 Gateway Timeout). So it would be a problem in Korea or some countries. By the way, I found a KHU team PC on which lvmpwi & lvmtan are already installed. In the pc, lvmpwi & lvmtan work well. And I've checked skymakercam also works well while running lvmpwi & lvmtan. So as a temporary solution, I'll use the PC for testing a virtual camera for lvmcam. $ poetry run container_start --debug --simulator --name=lvm.sci.pwi podman run --rm -d --name lvm.sci.pwi -e LVMT_RMQ=None -e DISPLAY -v /run/user/1000/gdm/Xauthority:/root/.Xauthority:Z --ipc=host --network=host --device /dev/dri -p 8220 -e PWI_DEBUG=true -e PWI_SIMULATOR=true -v /dev:/dev:rslave -v /home/sumin/lvmpwi:/root/lvmt:Z -e PWI_NAME=lvm.sci.pwi localhost/ubuntu_lvmt_pwi Port mappings have been discarded as one of the Host, Container, Pod, and None network modes are in use 90456b11949a4132848aa2ea463e80eb7c7fb5cbec4a37504a22bc3adf79b9dd Could not set X locale modifiers Gtk-Message: 10:01:50.630: Failed to load module "canberra-gtk-module" Starting main.py ... BSC loaded in 23ms BSC loaded in 18ms OK PointXPCore.OnModelChanged PointXPCore.Calibrate() 1 Final thetaEncOffset=0, phiEncOffset=0 Calibrated in 1 ms PointXPCore.CalPointErrors() Calculated residual errors in 0 ms PointXPCore.OnModelChanged PointXPCore.Calibrate() 2 Final thetaEncOffset=0, phiEncOffset=0 Calibrated in 0 ms PointXPCore.CalPointErrors() Calculated residual errors in 0 ms

—You are receiving this because you authored the thread.Reply to this email directly, view it on GitHub, or unsubscribe.Triage notifications on the go with GitHub Mobile for iOS or Android.

wasndas commented 2 years ago

Now there is docker/podman image on github, have to rename some stuff in the poetry startup, but you can download it with:

 podman pull ghcr.io/sdss/lvmpwi:0.0.9

or start it directly, fix every path with briegel in it.

 podman run --rm -d --name lvm.sci.pwi -e LVMT_RMQ=localhost -e DISPLAY -v /home/briegel/.Xauthority:/root/.Xauthority:Z --ipc=host  --network=host --device /dev/dri -e PWI_SIMULATOR=true -v /dev:/dev:rslave -v /home/briegel/workspace/lvmt/lvmpwi:/root/lvmt:Z -e PWI_NAME=lvm.sci.pwi ghcr.io/sdss/lvmpwi:0.0.9
wasndas commented 2 years ago

Containers for lvmtan & lvmpwi are available on github

Note: the containers have been renamed, so you should delete the old ones first.

podman rmi -f ubuntu_lvmt_pwi
podman rmi -f ubuntu_lvmt_tan
mgjeon commented 2 years ago

The new lvmtan & lvmpwi works well but the skymakercam could not detect whether lvmpwi is running. (Failed routing message to consumer 'lvm.sci.pwi'.)

So skymakercam with new lvmtan & lvmpwi does not run while skymakercam with old lvmtan & lvmpwi does run.

I think it's because the below error of new lvmpwi. Please check it also happens to you. 

FAILED
ERROR: Child exited immediately with exit code 1

The following is a log for lvmtan, lvmpwi, skymakercam.

new lvmtan (GUI opened)

$ podman ps -a
CONTAINER ID  IMAGE       COMMAND     CREATED     STATUS      PORTS       NAMES

$ lvmtan git:(main) ✗ poetry run container_start --kill --name lvm.all
Error: no container with name or ID "lvm.all" found: no such container
False
podman run --rm -t --name lvm.all --network=host --runtime /usr/bin/crun -e DISPLAY -v /run/user/1000/gdm/Xauthority:/root/.Xauthority:Z --ipc=host --device /dev/dri -e BASDARD_UI=lvm/lvm.all.ui -v /home/sumin/dev_lvmcam/lvmtan:/root/lvmt:Z -e BASDARD_CONFIG=lvm/lvm.all-sim.conf ghcr.io/sdss/lvmtan

new lvmpwi (GUI opened)

$ podman ps -a
CONTAINER ID  IMAGE                       COMMAND               CREATED         STATUS             PORTS       NAMES
20f559d494c0  ghcr.io/sdss/lvmtan:latest  /root/run-basdard...  12 seconds ago  Up 12 seconds ago              lvm.all

$ lvmpwi git:(main) ✗ poetry run container_start --name=lvm.sci.pwi --simulator 
podman run --rm -d --name lvm.sci.pwi -e LVMT_RMQ=None -e DISPLAY -v /run/user/1000/gdm/Xauthority:/root/.Xauthority:Z --ipc=host  --network=host --device /dev/dri -e PWI_SIMULATOR=true -v /dev:/dev:rslave -v /home/sumin/dev_lvmcam/lvmpwi:/root/lvmt:Z -e PWI_NAME=lvm.sci.pwi ghcr.io/sdss/lvmpwi
4d1a0855d845b7c6744ba1fcf7e1a3f21bf3287c25763c9b5db8bab02dc6a569
Gtk-Message: 02:28:40.419: Failed to load module "canberra-gtk-module"
Starting __main__.py ... BSC loaded in 26ms
BSC loaded in 19ms
FAILED
ERROR: Child exited immediately with exit code 1
PointXPCore.OnModelChanged
PointXPCore.Calibrate() 1
Final thetaEncOffset=0, phiEncOffset=0
Calibrated in 1 ms
PointXPCore.CalPointErrors()
Calculated residual errors in 0 ms
PointXPCore.OnModelChanged
PointXPCore.Calibrate() 2
Final thetaEncOffset=0, phiEncOffset=0
Calibrated in 0 ms
PointXPCore.CalPointErrors()
Calculated residual errors in 0 ms

skymakercam with new lvmtan & lvmpwi (GUI did not open)

$ podman ps -a
CONTAINER ID  IMAGE                       COMMAND               CREATED         STATUS             PORTS       NAMES
20f559d494c0  ghcr.io/sdss/lvmtan:latest  /root/run-basdard...  37 seconds ago  Up 38 seconds ago              lvm.all
4d1a0855d845  ghcr.io/sdss/lvmpwi:latest  /root/lvmt/contai...  16 seconds ago  Up 17 seconds ago              lvm.sci.pwi

$ poetry run python utils/plot_skymakercam.py -v -c python/skymakercam/etc/cameras.yaml lvm.sci.agw.cam
[DEBUG]: [SKYMAKERCAMERASYSTEM]: read configuration file from python/skymakercam/etc/cameras.yaml
[DEBUG]: [SKYMAKERCAMERASYSTEM]: adding camera 'lvm.sci.agw.cam' with parameters {'uid': 'lvm.sci.agw.cam', 'descr': 'Guider Camera Science', 'default': {'gain': 5.0, 'binning': [4, 4]}, 'instpar': 'lvm_sci_agw_cam', 'focus_stage': 'lvm.sci.foc', 'kmirror': 'lvm.sci.km', 'tcs': 'lvm.sci.pwi', 'catalog_path': '$HOME/data/catalog/gaia'}
[DEBUG]: [LVM.SCI.AGW.CAM]: [1600, 1100]
[DEBUG]: [LVM.SCI.AGW.CAM]: connecting ...
[ERROR]: Traceback (most recent call last):
  File "utils/plot_skymakercam.py", line 107, in <module>
    main()
  File "utils/plot_skymakercam.py", line 102, in main
    asyncio.run(plot_skymakercam(args.exptime, args.binning, args.guiderect, args.camname, verb=args.verbose, config=args.cfg))
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "utils/plot_skymakercam.py", line 35, in plot_skymakercam
    cam = await cs.add_camera(name=camname, uid=cs._config[camname]["uid"])
  File "/home/sumin/.cache/pypoetry/virtualenvs/sdss-skymakercam-POjD1PBZ-py3.8/lib/python3.8/site-packages/basecam/camera.py", line 386, in add_camera
    await camera.connect()
  File "/home/sumin/.cache/pypoetry/virtualenvs/sdss-skymakercam-POjD1PBZ-py3.8/lib/python3.8/site-packages/basecam/camera.py", line 634, in connect
    await self._connect_internal(**conn_params)
  File "/home/sumin/skymakercam/python/skymakercam/camera.py", line 182, in _connect_internal
    await self._tcs.start()
  File "/home/sumin/.cache/pypoetry/virtualenvs/sdss-skymakercam-POjD1PBZ-py3.8/lib/python3.8/site-packages/cluplus/proxy.py", line 131, in start_async
    commands = (await self.call_command(Proxy.__commands))[Proxy.__comkey]
  File "/home/sumin/.cache/pypoetry/virtualenvs/sdss-skymakercam-POjD1PBZ-py3.8/lib/python3.8/site-packages/cluplus/proxy.py", line 214, in call_command
    raise self._errorMapToException(ret.replies[-1].body['error'])
Exception: Failed routing message to consumer 'lvm.sci.pwi'.

old lvmtan (GUI open)

$ podman ps -a
CONTAINER ID  IMAGE       COMMAND     CREATED     STATUS      PORTS       NAMES

$ poetry run container_start --kill --name lvm.all
Error: no container with name or ID "lvm.all" found: no such container
False
podman run --rm -t --name lvm.all --network=host --runtime /usr/bin/crun -e DISPLAY -v /run/user/1000/gdm/Xauthority:/root/.Xauthority:Z --ipc=host --device /dev/dri -e BASDARD_UI=lvm/lvm.all.ui -v /home/sumin/lvmtan:/root/lvmt:Z -e BASDARD_CONFIG=lvm/lvm.all-sim.conf localhost/ubuntu_lvmt_tan

old lvmpwi (GUI open)

$ podman ps -a
CONTAINER ID  IMAGE                             COMMAND               CREATED        STATUS            PORTS       NAMES
f9680b7f31ef  localhost/ubuntu_lvmt_tan:latest  /root/run-basdard...  6 seconds ago  Up 7 seconds ago              lvm.all

$ poetry run container_start --name=lvm.sci.pwi --simulator 
podman run --rm -d --name lvm.sci.pwi -e LVMT_RMQ=None -e DISPLAY -v /run/user/1000/gdm/Xauthority:/root/.Xauthority:Z --ipc=host  --network=host --device /dev/dri -e PWI_SIMULATOR=true -v /dev:/dev:rslave -v /home/sumin/lvmpwi:/root/lvmt:Z -e PWI_NAME=lvm.sci.pwi localhost/ubuntu_lvmt_pwi
e9d71fec27457a4f165782c0cb51322f8c4086c0c4ba79f3581cfa366926c40b
Gtk-Message: 02:13:42.279: Failed to load module "canberra-gtk-module"
Starting __main__.py ... BSC loaded in 23ms
BSC loaded in 20ms
OK
PointXPCore.OnModelChanged
PointXPCore.Calibrate() 1
Final thetaEncOffset=0, phiEncOffset=0
Calibrated in 1 ms
PointXPCore.CalPointErrors()
Calculated residual errors in 0 ms
PointXPCore.OnModelChanged
PointXPCore.Calibrate() 2
Final thetaEncOffset=0, phiEncOffset=0
Calibrated in 0 ms
PointXPCore.CalPointErrors()
Calculated residual errors in 0 ms
PointXPCore.OnModelChanged
PointXPCore.Calibrate() 3
Final thetaEncOffset=0, phiEncOffset=0
Calibrated in 0 ms
PointXPCore.CalPointErrors()
Calculated residual errors in 0 ms
PointXPCore.OnModelChanged
PointXPCore.Calibrate() 4
Final thetaEncOffset=0, phiEncOffset=0
Calibrated in 0 ms
PointXPCore.CalPointErrors()
Calculated residual errors in 0 ms
PointXPCore.OnModelChanged
PointXPCore.Calibrate() 5
Final thetaEncOffset=0, phiEncOffset=0
Calibrated in 0 ms
PointXPCore.CalPointErrors()
Calculated residual errors in 0 ms

skymakercam with old lvmtan & lvmpwi (GUI open)

$ podman ps -a
CONTAINER ID  IMAGE                             COMMAND               CREATED         STATUS             PORTS       NAMES
f9680b7f31ef  localhost/ubuntu_lvmt_tan:latest  /root/run-basdard...  24 seconds ago  Up 25 seconds ago              lvm.all
e9d71fec2745  localhost/ubuntu_lvmt_pwi:latest  /root/lvmt/contai...  8 seconds ago   Up 8 seconds ago               lvm.sci.pwi

$ poetry run python utils/plot_skymakercam.py -v -c python/skymakercam/etc/cameras.yaml lvm.sci.agw.cam
[DEBUG]: [SKYMAKERCAMERASYSTEM]: read configuration file from python/skymakercam/etc/cameras.yaml
[DEBUG]: [SKYMAKERCAMERASYSTEM]: adding camera 'lvm.sci.agw.cam' with parameters {'uid': 'lvm.sci.agw.cam', 'descr': 'Guider Camera Science', 'default': {'gain': 5.0, 'binning': [4, 4]}, 'instpar': 'lvm_sci_agw_cam', 'focus_stage': 'lvm.sci.foc', 'kmirror': 'lvm.sci.km', 'tcs': 'lvm.sci.pwi', 'catalog_path': '$HOME/data/catalog/gaia'}
[DEBUG]: [LVM.SCI.AGW.CAM]: [1600, 1100]
[DEBUG]: [LVM.SCI.AGW.CAM]: connecting ...
[DEBUG]: [LVM.SCI.AGW.CAM]: camera connected.
[DEBUG]: cameras ['lvm.sci.agw.cam']
[DEBUG]: [LVM.SCI.AGW.CAM]: defocus 0.0
[DEBUG]: [LVM.SCI.AGW.CAM]: kmirror angle (deg): 0.0
[DEBUG]: [LVM.SCI.AGW.CAM]: separation 7007.580633945394
Gaia query:  SELECT source_id, ra,dec,phot_g_mean_mag FROM gaiaedr3.gaia_source WHERE phot_g_mean_mag <= 17 AND 1=CONTAINS(POINT('ICRS',ra,dec), CIRCLE('ICRS',242.88222345372895,-8.53871565560227, 0.692887394120578))
INFO: Query finished. [astroquery.utils.tap.core]
2884 stars found within 0.692887394120578 deg
[DEBUG]: [LVM.SCI.AGW.CAM]: defocus 0.0
[DEBUG]: [LVM.SCI.AGW.CAM]: kmirror angle (deg): 0.0
[DEBUG]: [LVM.SCI.AGW.CAM]: separation 3293.2928937841975
Gaia query:  SELECT source_id, ra,dec,phot_g_mean_mag FROM gaiaedr3.gaia_source WHERE phot_g_mean_mag <= 17 AND 1=CONTAINS(POINT('ICRS',ra,dec), CIRCLE('ICRS',298.5081593675085,-20.2949725906958, 0.692887394120578))
INFO: Query finished. [astroquery.utils.tap.core]
5518 stars found within 0.692887394120578 deg
[DEBUG]: [LVM.SCI.AGW.CAM]: defocus 0.0
[DEBUG]: [LVM.SCI.AGW.CAM]: kmirror angle (deg): 0.0
[DEBUG]: [LVM.SCI.AGW.CAM]: separation 3176.04865585497
Gaia query:  SELECT source_id, ra,dec,phot_g_mean_mag FROM gaiaedr3.gaia_source WHERE phot_g_mean_mag <= 17 AND 1=CONTAINS(POINT('ICRS',ra,dec), CIRCLE('ICRS',332.94076739169145,20.5380116838513, 0.692887394120578))
INFO: Query finished. [astroquery.utils.tap.core]
2760 stars found within 0.692887394120578 deg
[DEBUG]: [LVM.SCI.AGW.CAM]: defocus 0.0
[DEBUG]: [LVM.SCI.AGW.CAM]: kmirror angle (deg): 0.0
[DEBUG]: [LVM.SCI.AGW.CAM]: separation 22.3248895953375
Gaia query:  SELECT source_id, ra,dec,phot_g_mean_mag FROM gaiaedr3.gaia_source WHERE phot_g_mean_mag <= 17 AND 1=CONTAINS(POINT('ICRS',ra,dec), CIRCLE('ICRS',333.1841510630805,20.832290822152, 0.692887394120578))
INFO: Query finished. [astroquery.utils.tap.core]
2867 stars found within 0.692887394120578 deg
[DEBUG]: [LVM.SCI.AGW.CAM]: defocus 0.0
[DEBUG]: [LVM.SCI.AGW.CAM]: kmirror angle (deg): 0.0
[DEBUG]: [LVM.SCI.AGW.CAM]: separation 2008.561045657376
Gaia query:  SELECT source_id, ra,dec,phot_g_mean_mag FROM gaiaedr3.gaia_source WHERE phot_g_mean_mag <= 17 AND 1=CONTAINS(POINT('ICRS',ra,dec), CIRCLE('ICRS',0.0,0.0, 0.692887394120578))
INFO: Query finished. [astroquery.utils.tap.core]
817 stars found within 0.692887394120578 deg
[DEBUG]: [LVM.SCI.AGW.CAM]: defocus 0.0
[DEBUG]: [LVM.SCI.AGW.CAM]: kmirror angle (deg): 0.0
wasndas commented 2 years ago

LVMT_RMQ=None

 $ poetry run container_start --name=lvm.sci.pwi --simulator 
 podman run --rm -d --name lvm.sci.pwi -e LVMT_RMQ=None -e DISPLAY -v /run/user/1000/gdm/Xauthority:/root/.Xauthority:Z --ipc=host  --network=host --device /dev/dri -e PWI_SIMULATOR=true -v /dev:/dev:rslave -v /home/sumin/lvmpwi:/root/lvmt:Z -e PWI_NAME=lvm.sci.pwi localhost/ubuntu_lvmt_pwi

...it seems the enviroment variable HOSTNAME doesnt exist.

import OS
os.getenv('HOSTNAME')

... is there a variable HOST ?

env | grep HOST

... change it in container/init.py

mgjeon commented 2 years ago

It seems that in this case no HOSTNAME is not a cause of FAILED error of lvmpwi. Because old lvmpwi works well when LVMT_RMQ=None

I checked there is no HOSTNAME variable and python cannot get HOSTNAME.

$ env | grep NAME
LC_NAME=ko_KR.UTF-8
LOGNAME=sumin
USERNAME=sumin

$ echo $HOSTNAME
IRLABSML

$ cat getHOSTNAME.py 
import os
os.getenv('HOSTNAME')

$ python getHOSTNAME.py 
$

But lvmpwi shows a same error even if I hard-coded HOSTNAME in container/__init__.py.

$ sed -n 98p ./container/__init__.py 
run_base += f" -e LVMT_RMQ=IRLABSML"

$ poetry run container_start --name=lvm.sci.pwi --simulator
podman run --rm -d --name lvm.sci.pwi -e LVMT_RMQ=IRLABSML -e DISPLAY -v /run/user/1000/gdm/Xauthority:/root/.Xauthority:Z --ipc=host  --network=host --device /dev/dri -e PWI_SIMULATOR=true -v /dev:/dev:rslave -v /home/sumin/dev_lvmcam/lvmpwi:/root/lvmt:Z -e PWI_NAME=lvm.sci.pwi ghcr.io/sdss/lvmpwi
6818d7895e9a5d36b283a739f5b1807de235ac5d6780fda5ecfebba310d56858
Gtk-Message: 08:16:44.872: Failed to load module "canberra-gtk-module"
Starting __main__.py ... BSC loaded in 23ms
BSC loaded in 18ms
FAILED
ERROR: Child exited immediately with exit code 1
...

And also, the old lvmpwi shows LVMT_RMQ=None but it works well without FAILED.

$ poetry run container_start --name=lvm.sci.pwi --simulator 
podman run --rm -d --name lvm.sci.pwi -e LVMT_RMQ=None -e DISPLAY -v /run/user/1000/gdm/Xauthority:/root/.Xauthority:Z --ipc=host  --network=host --device /dev/dri -e PWI_SIMULATOR=true -v /dev:/dev:rslave -v /home/sumin/lvmpwi:/root/lvmt:Z -e PWI_NAME=lvm.sci.pwi localhost/ubuntu_lvmt_pwi
e9d71fec27457a4f165782c0cb51322f8c4086c0c4ba79f3581cfa366926c40b
Gtk-Message: 02:13:42.279: Failed to load module "canberra-gtk-module"
Starting __main__.py ... BSC loaded in 23ms
BSC loaded in 20ms
OK
mgjeon commented 2 years ago

Changing os.getenv('HOSTNAME') -> socket.gethostname() in container/__init__.py works for me.

I finally found what is wrong. LVMT_RMQ=None was the problem.

If I run lvmpwi LVMT_RMQ=localhost by hard-coding run_base += " -e LVMT_RMQ=localhost" or run_base += " -e LVMT_RMQ=lRLABSML" in container/__init__.py, then the problem is fixed and everything works well.

In the first time, there is no lvm.sci.pwi.yml in python/lvmpwi/etc. If I run $ podman run --rm -d --name lvm.sci.pwi -e LVMT_RMQ=None ..., the command makes lvm.sci.pwi.yml like the following.

actor:
    name: lvm.sci.pwi
    host: None
    port: 5672
    log_dir: '~/tmp/log'

This of course makes error. Once this file is made, then it is not overwritten anymore by using $ podman run --rm -d --name lvm.sci.pwi -e LVMT_RMQ=localhost ....

So this error occurs everytime once the command with wrong LVMT_RMQ.

I first fixed the problem by changing lvm.sci.pwi.yml manually.

FAILED
ERROR: Child exited immediately with exit code 1

The above error occurs when the actor cannot start:

$ python python/lvmpwi/__main__.py start
Starting __main__.py ... FAILED
ERROR: Child exited immediately with exit code 1

$ python python/lvmpwi/__main__.py start --debug
...
AssertionError: configuration path does not exists.

This error can be fixed by changing /etc/lvmpwi.yml -> /etc/lvm.pwi.yml

$ python python/lvmpwi/__main__.py start
Starting __main__.py ... OK

$ python python/lvmpwi/__main__.py stop
Stopping __main__.py ... OK

$ python python/lvmpwi/__main__.py start --debug
[ERROR]: Unexpected exception <class 'urllib.error.URLError'>: <urlopen error [Errno 111] Connection refused>

Thank you very much for your help to this problem. I really appreciate it.

wasndas commented 2 years ago

Yes, socket.gethostname() is a way better solution for retrieving the hostname, thanks - changed it.