Closed taldcroft closed 5 years ago
I just checked the rsync log to explicitly verify that files are getting transferred as expected.
2019:214:06:00:03.log:data/cmd_states/cmd_states.db3
2019:214:06:00:03.log:data/cmd_states/cmd_states.h5
2019:214:06:00:03.log:data/kadi/cmds.h5
2019:214:06:00:03.log:data/kadi/cmds.pkl
2019:216:06:00:02.log:data/kadi/events.db3
What are your thoughts about how this would be installed? Just rsync as aca user to update the ska2 task schedules?
Installed. For some reason the change in #121 was never installed, but now it is.
kadi$ diff task_schedule_cmds.cfg /proj/sot/ska/data/kadi/task_schedule_cmds.cfg
43c43,52
< exec update_cmds --data-root=$ENV{SKA_DATA}/kadi
---
> exec update_cmds --ftp --data-root=$ENV{SKA_DATA}/kadi
> <check>
> <error>
> # File Expression
> # ---------- ---------------------------
> kadi.log error
> kadi.log warning
> kadi.log fatal
> </error>
> </check>
kadi$ diff task_schedule_events.cfg /proj/sot/ska/data/kadi/task_schedule_events.cfg
46c46
< exec update_events --data-root=$ENV{SKA_DATA}/kadi/update_events
---
> exec update_events --ftp --data-root=$ENV{SKA_DATA}/kadi/update_events
aca-kadi% rsync -v task_schedule_events.cfg /proj/sot/ska/data/kadi/task_schedule_events.cfg
task_schedule_events.cfg
sent 2934 bytes received 31 bytes 5930.00 bytes/sec
total size is 2846 speedup is 0.96
aca-kadi% rsync -v task_schedule_cmds.cfg /proj/sot/ska/data/kadi/task_schedule_cmds.cfg
sent 2353 bytes received 31 bytes 4768.00 bytes/sec
total size is 2267 speedup is 0.95
And of course the issue with #121 just highlights the unreliable nature of the Ska2 architecture for task_schedule and out-of-environment files.
I think it just means that if there's still a Makefile, we should probably use it or remove it. That's still an issue in Ska3 like with this package. We can also use conda to install non-Python files with the Makefile, but I get the impression you like setup.py better for that in general.
Sorry, I meant "out-of-environment" files (e.g. /proj/sot/ska/data/kadi/task_schedule_cmds.cfg
). About conda and Makefile, are you talking about a post-install hook to run a Makefile? Anyway, any mechanism that installs out-of-environment files in conda is the wrong direction and unreliable.
I realize that Ska3 does not strictly enforce the package-based architecture just yet, but that is the plan with time and effort. One package at a time, e.g. cheta will be next.
I thought the issue here was that "make install" was not run in ska2 for kadi 3.15.4. I thought that was in-environment.
Regarding "About conda and Makefile, are you talking about a post-install hook to run a Makefile?" I don't know what you mean. For packages with a makefile, having the build.sh just run "make install" seems fine, so that's not a post-install step, that's just a package build step.
I thought the issue here was that "make install" was not run in ska2 for kadi 3.15.4.
I don't have a clue, so the plan is to fix process so memory is not required.
About conda and Makefile, I guess that works in the sheltered world of a production environment (assuming you set SKA to the install root before running make
in the build), but fails for a standalone where SKA != SKA_ARCH_OS
. If you think about how it was (is) in Ska2, $SKA/share is outside the installed environment which lives in $SKA_ARCH_OS. In Ska3, share is defined to live in $SKA_ARCH_OS, but legacy task schedule or other legacy scripts assume $SKA/share, which is (in general) outside the environment.
Takeaway: for Ska3 we should not use our custom Makefile that installs files based on $SKA. As packages are migrated to Ska3, the Makefile should be removed.
I think our old include logic wrt to SKA / SKA_ARCH_OS doesn't make sense for Makefile installs, but PREFIX should be reliable on the conda side.
But I'm not arguing to keep Makefiles, I'm just saying that if we have to jump through any other hoops to get setup.py to do something reasonable, we can examine this as needed.
Using
lucky
as a gateway to push commands and events to the GRETA network has been overtaken by pulling from GRETA via rsync. This is more reliable and simpler.Note that
task_schedule_occ_{cmds,events}.cfg
are no longer running on GRETA since moving to Wayside.