search

OSC / ood_core

Open OnDemand core library
https://osc.github.io/ood_core/
MIT License
10 stars 28 forks source link

LHA doesn't work with apptainer updates #784

Closed johrstrom closed 1 year ago

johrstrom commented 1 year ago

Well, this grepping through ps output has finally caught up with us. Looking at the grep expression we need to add appinit for the apptainer updates as an immediate patch.

https://github.com/OSC/ood_core/blob/69bb3430923a210b0ae4af1f89d4b137083293dc/lib/ood_core/job/adapters/linux_host/launcher.rb#L76

[johrstrom ~()]  pstree -p -l "$pane_id" 
tmp.4xgfywBRH3_(32660)─┬─tee(32662)
                       └─tmp.4xgfywBRH3_(32661)─┬─tee(32664)
                                                └─tmp.4xgfywBRH3_(32663)───timeout(32665)───starter(32666)─┬─appinit(32694)─┬─bash(32716)───script.sh(33366)───node(33774)─┬─node(33813)─┬─{node}(33814)
                                                                                                           │                │                                              │             ├─{node}(33815)
                                                                                                           │                │                                              │             ├─{node}(33816)
                                                                                                           │                │                                              │             ├─{node}(33817)
                                                                                                           │                │                                              │             ├─{node}(33818)
                                                                                                           │                │                                              │             ├─{node}(33819)
                                                                                                           │                │                                              │             ├─{node}(33827)
                                                                                                           │                │                                              │             ├─{node}(33828)
                                                                                                           │                │                                              │             ├─{node}(33829)
                                                                                                           │                │                                              │             └─{node}(33830)
                                                                                                           │                │                                              ├─{node}(33786)
                                                                                                           │                │                                              ├─{node}(33787)
                                                                                                           │                │                                              ├─{node}(33788)
                                                                                                           │                │                                              ├─{node}(33789)
                                                                                                           │                │                                              ├─{node}(33790)
                                                                                                           │                │                                              ├─{node}(33791)
                                                                                                           │                │                                              ├─{node}(33809)
                                                                                                           │                │                                              ├─{node}(33810)
                                                                                                           │                │                                              ├─{node}(33811)
                                                                                                           │                │                                              └─{node}(33812)
                                                                                                           │                ├─{appinit}(32711)
                                                                                                           │                ├─{appinit}(32712)
                                                                                                           │                ├─{appinit}(32713)
                                                                                                           │                ├─{appinit}(32714)
                                                                                                           │                ├─{appinit}(32715)
                                                                                                           │                ├─{appinit}(32721)
                                                                                                           │                └─{appinit}(32929)
                                                                                                           ├─squashfuse_ll(32707)
                                                                                                           ├─{starter}(32696)
                                                                                                           ├─{starter}(32697)
                                                                                                           ├─{starter}(32698)
                                                                                                           ├─{starter}(32699)
                                                                                                           ├─{starter}(32700)
                                                                                                           ├─{starter}(32709)
                                                                                                           └─{starter}(32710)

Longer term I'm sure actually using tmux commands to control the sessions (as opposed to grepping to find the right PID) is the better approach.

┆Issue is synchronized with this Asana task by Unito