scrapli / scrapligo

scrapli, but in go!
MIT License
252 stars 36 forks source link

defunct issue #94

Closed lulianbing58 closed 2 years ago

lulianbing58 commented 2 years ago

here is my sample code :

package main import ( "fmt" "github.com/scrapli/scrapligo/response" "github.com/scrapli/scrapligo/driver/options" "github.com/scrapli/scrapligo/platform" "time" )

func main() { for { _, err := ExecCmd([]string{"sho run","show ver"}, "x.x.x.x", "cml20") if err != nil { panic("wrong") } time.Sleep(10 * time.Duration(time.Second)) }

}

func ExecCmd(cmdlist []string, deviceip string, devicename string) (response.MultiResponse, error) { p, err := platform.NewPlatform( // cisco_iosxe refers to the included cisco iosxe platform definition "cisco_iosxr", deviceip, options.WithPort(22), options.WithAuthNoStrictKey(), options.WithTimeoutOps(120time.Second), options.WithAuthUsername("admin"), options.WithAuthPassword("admin"), options.WithTermWidth(256), options.WithTermHeight(0), ) if err != nil { fmt.Printf("failed to create platform for %s(%s); error: %+v\n", devicename, deviceip, err) return nil, err } d, err := p.GetNetworkDriver() if err != nil { fmt.Printf("failed to fetch network driver for %s(%s) ; error: %+v\n", devicename, deviceip, err) return nil, err } err = d.Open() if err != nil { fmt.Printf("failed to open driver %s(%s); error: %+v\n", devicename, deviceip, err) return nil, err } defer d.Close() mr, err := d.SendCommands(cmdlist, options.WithTimeoutOps(120*time.Second)) return mr, err }


I found lots of defunct processes, Every 1.0s: ps -ef | grep ssh | grep -v grep | grep -v sshd Mon Aug 8 04:40:00 2022 root 344 32616 0 04:39 ? 00:00:00 [ssh] root 428 32616 0 04:39 ? 00:00:00 [ssh] root 516 32616 0 04:39 ? 00:00:00 [ssh] root 601 32616 0 04:39 ? 00:00:00 [ssh] root 32622 32616 0 04:38 ? 00:00:00 [ssh] root 32692 32616 0 04:38 ? 00:00:00 [ssh] root 32740 32616 0 04:39 ? 00:00:00 [ssh]

How can we avoid it?

thanks.

carlmontanari commented 2 years ago

will take a bit closer look later, but my initial guess/thought is that because we use ptyprocess those child processes are staying alive until the parent process (the main program) exits. we of course call close the file descriptor we get back from ptyprocess but maybe there is an additional kill method or something we can use as well. let us know if you have a solution for this as well!

lulianbing58 commented 2 years ago

right now, I have no solution yet, I found the ssh defunct will consume all memory finally in my lab.

lulianbing58 commented 2 years ago

Whether it is caused by the tty colsed before pty, func StartWithAttrs(c exec.Cmd, sz Winsize, attrs syscall.SysProcAttr) (os.File, error) {

pty, tty, err := Open()
if err != nil {
    return nil, err
}
defer func() { _ = tty.Close() }() // Best effort.
if sz != nil {
    if err := Setsize(pty, sz); err != nil {
        _ = pty.Close() // Best effort.
        return nil, err
    }
}
if c.Stdout == nil {
    c.Stdout = tty
}
if c.Stderr == nil {
    c.Stderr = tty
}
if c.Stdin == nil {
    c.Stdin = tty
}
c.SysProcAttr = attrs
if err := c.Start(); err != nil {
    _ = pty.Close() // Best effort.
    return nil, err
}
return pty, err

}

carlmontanari commented 2 years ago

Whether it is caused by the tty colsed before pty not sure I am 100% following here, but I think you're maybe saying similar things to this thread?

I dont think there is anything else we can really be doing in scrapligo at the moment for this for this. The processes that are left, however, are just zombie processes that will get reaped when the main program exits. I think that the scenario demonstrated in the reproduction example here is very unlikely to be a normal thing (you could for example just keep a single session alive, or you could stop the program between sleeps and re-run it via cron or w/e) that I'm inclined to close this just due to lack of anything we can likely do and the kind of non-standard-ness of it.

One more thin -- you could also just use the "standard" transport that does not do any of this pty bits and is instead entirely in go -- no zombie processes that way! So perhaps that is a viable option for your situation?

carlmontanari commented 2 years ago

Hey there, going to close this out for now. If standard transport doesn't work for you and you can come up with a fix in either pty or here, feel free to reopen!