go-python does not execute a python module anymore if that module once raises an exception

[shtripat]

Having an issue where, we invoke a python code using go-python. If the python code raises some exception once, then onwards any call to that python module returns immediately from CallObject() with blank return value.

Sample code to invoke python looks as below

var pyinit = false
func Init() (err error) {
    if !pyinit {
        if err = python.Initialize(); err == nil {
            pyinit = true
        }
    }
    return
}

type PyFunction struct {
    *python.PyObject
}

var mutex sync.Mutex
func Import(module string, functions ...string) (funcs map[string]*PyFunction, err error) {
    if err = Init(); err != nil {
        return
    }

    if pymod := python.PyImport_ImportModuleNoBlock(module); pymod == nil {
        err = errors.New(fmt.Sprintf("gopy:%s: module import failed", module))
    } else {
        funcs = make(map[string]*PyFunction)
        for _, name := range functions {
            if pyfunc := pymod.GetAttrString(name); pyfunc == nil {
                err = errors.New(fmt.Sprintf("gopy:%s:%s: function not found", module, name))
                return
            } else {
                funcs[name] = &PyFunction{pyfunc}
            }
        }
    }
    return
}

func (f *PyFunction) Call(args ...interface{}) {
    var pyargs *python.PyObject

    if pyargs, err = ToPyObject(reflect.ValueOf(args)); err != nil {
        return
    }
    mutex.Lock()
    defer mutex.Unlock()
    name := python.PyString_AsString(f.GetAttrString("__name__"))
    if r = f.CallObject(pyargs); r == nil {
        err = errors.New(fmt.Sprintf("%s(): function failed at python side", name))
    }
 }

var funcNames = [...]string{
    "MyFunc",
}

var pyFuncs map[string]*PyFunction
func main() {
        pyFuncs, err := gopy.Import("mypackage", funcNames[:]...)
        if err != nil {
        panic(err)
    }
       for {
           if pyFuncs["MyFunc"].Call(); err !=nil {
               // handle the error
           }
      }
}

In above scenario say the python function mypackage.MyFunc raises an exception in some scenario. Then onwards, it never enters the python code at all and keeps showing the error "function failed at python side".

Not sure if we should re-load the python interpreter from go code once exception or something else. Kindly suggest.

Regards, Shubhendu

[sbinet]

(took the liberty to reformat slightly your submission)

[sbinet]

I haven't checked my hypothesis yet but I surmise this is because you haven't checked whether an exception was raised (possibly with PyErr_Occurred [1]) and then cleared it (with PyErr_Clear [2])

[1] https://godoc.org/github.com/sbinet/go-python#PyErr_Occurred [2] https://godoc.org/github.com/sbinet/go-python#PyErr_Clear

[shtripat]

Yes. Its not checked for exception in our code. Will try this and update. Thanks for the quick response.

Regards Shubhendu On 13 Jan 2016 02:03, "Sebastien Binet" notifications@github.com wrote:

I haven't checked my hypothesis yet but I surmise this is because you haven't checked whether an exception was raised (possibly with PyErr_Occurred [1]) and then cleared it (with PyErr_Clear [2])

[1] https://godoc.org/github.com/sbinet/go-python#PyErr_Occurred [2] https://godoc.org/github.com/sbinet/go-python#PyErr_Clear

— Reply to this email directly or view it on GitHub https://github.com/sbinet/go-python/issues/33#issuecomment-171045803.

[shtripat]

Thanks the solution works fine. ... On 13 Jan 2016 10:36, "Shubhendu Tripathi" shubhendu.tripathi@gmail.com wrote:

Yes. Its not checked for exception in our code. Will try this and update. Thanks for the quick response.

Regards Shubhendu On 13 Jan 2016 02:03, "Sebastien Binet" notifications@github.com wrote:

I haven't checked my hypothesis yet but I surmise this is because you haven't checked whether an exception was raised (possibly with PyErr_Occurred [1]) and then cleared it (with PyErr_Clear [2])

[1] https://godoc.org/github.com/sbinet/go-python#PyErr_Occurred [2] https://godoc.org/github.com/sbinet/go-python#PyErr_Clear

— Reply to this email directly or view it on GitHub https://github.com/sbinet/go-python/issues/33#issuecomment-171045803.

[shtripat]

Hi Sebastien,

Thanks for the support last time. We are facing another issue in our project while using go-python package. It seems something related to python version and compatibility with go-python package.

Issue Description

To simulate the issue I have python written as below

#!/bin/python

import sys

def PrintSysArgv():
    print (sys.argv)

if __name__ == "__main__":
    PrintSysArgv()

This python if I run from python CLI works fine as shown below

>python test.py
['test.py']
>

Now I have go program to invoke the same function PrintSysArgv as below

package main

import (
    "<package>/gopy"
    "fmt"
)

var funcNames = [...]string{
    "PrintSysArgv",
}

var pyFuncs map[string]*gopy.PyFunction

func main() {
    pyFuncs, err := gopy.Import("mymodule.test", funcNames[:]...)
    if err != nil {
        panic("Error importing python module")
    }

    _, err = pyFuncs["PrintSysArgv"].Call()
    if err != nil {
        panic(fmt.Sprintf("Error executing python function. error: %v",
err))
    }
    fmt.Println("Done")
}

Below is the snippet of gopy wrapper I am using in above go source

package gopy

import (
        "errors"
        "fmt"
        "github.com/sbinet/go-python"
        "reflect"
)

type PyFunction struct {
        *python.PyObject
}

func (f *PyFunction) Call(args ...interface{}) (r *python.PyObject, err
error) {
        var pyargs *python.PyObject

        if pyargs, err = ToPyObject(reflect.ValueOf(args)); err != nil {
                return
        }

        name := python.PyString_AsString(f.GetAttrString("__name__"))
        if r = f.CallObject(pyargs); r == nil {
                err = errors.New(fmt.Sprintf("%s(): function failed at
python side", name))
        }
        if pyobj := python.PyErr_Occurred(); pyobj != nil {
                err = errors.New(fmt.Sprintf("%s(): exception happened in
python side", name))
                python.PyErr_Clear()
        }
        return
}

var pyinit = false

func Init() (err error) {
        if !pyinit {
                if err = python.Initialize(); err == nil {
                        pyinit = true
                }
        }
        return
}

func Import(module string, functions ...string) (funcs
map[string]*PyFunction, err error) {
        if err = Init(); err != nil {
                return
        }

        if pymod := python.PyImport_ImportModuleNoBlock(module); pymod ==
nil {
                err = errors.New(fmt.Sprintf("gopy:%s: module import
failed", module))
        } else {
                funcs = make(map[string]*PyFunction)
                for _, name := range functions {
                        if pyfunc := pymod.GetAttrString(name); pyfunc ==
nil {
                                err = errors.New(fmt.Sprintf("gopy:%s:%s:
function not found", module, name))
                                return
                        } else {
                                funcs[name] = &PyFunction{pyfunc}
                        }
                }
        }
        return
}

The out of the execution of go program fails python side as shown below

go run test.go
panic: Error executing python function. error: PrintSysArgv(): exception
happened in python side

goroutine 1 [running]:
main.main()
    /root/go/bin/test.go:22 +0x291

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:2232 +0x1
exit status 2

Just a wierd thought that go-python uses a trimmed version of python which is not able to take care of "sys.argv" as such.

Can you please look into the same and suggest us accordingly.

If needed / suggested I can raise the same in wiki...

Thanks and Regards, Shubhendu

On Thu, Jan 14, 2016 at 1:48 AM, Sebastien Binet notifications@github.com wrote:

Closed #33 https://github.com/sbinet/go-python/issues/33.

— Reply to this email directly or view it on GitHub https://github.com/sbinet/go-python/issues/33#event-514168337.

[sbinet]

hi, please open a new issue, thanks.

[shtripat]

Thanks Sebastien. Raised one at https://github.com/sbinet/go-python/issues/36

Regards, Shubhendu

On Tue, Feb 9, 2016 at 1:48 PM, Sebastien Binet notifications@github.com wrote:

hi, please open a new issue, thanks.

— Reply to this email directly or view it on GitHub https://github.com/sbinet/go-python/issues/33#issuecomment-181755240.