Issue while invoking a python code which has reference to sys.argv

[shtripat]

We are facing an issue in our project while using go-python package. It seems something related to python version and compatibility with go-python package. (python version is 2.7.10)

Issue Description

To simulate the issue I have python written as below

#!/bin/python

import sys

def PrintSysArgv():
    print (sys.argv)

if __name__ == "__main__":
    PrintSysArgv()

This python if I run from python CLI works fine as shown below

>python test.py
['test.py']
>

Now I have go program to invoke the same function PrintSysArgv as below

package main

import (
    "<package>/gopy"
    "fmt"
)

var funcNames = [...]string{
    "PrintSysArgv",
}

var pyFuncs map[string]*gopy.PyFunction

func main() {
    pyFuncs, err := gopy.Import("mymodule.test", funcNames[:]...)
    if err != nil {
        panic("Error importing python module")
    }

    _, err = pyFuncs["PrintSysArgv"].Call()
    if err != nil {
        panic(fmt.Sprintf("Error executing python function. error: %v", err))
    }
    fmt.Println("Done")
}

Below is the snippet of gopy wrapper I am using in above go source

package gopy

import (
        "errors"
        "fmt"
        "github.com/sbinet/go-python"
        "reflect"
)

type PyFunction struct {
        *python.PyObject
}

func (f *PyFunction) Call(args ...interface{}) (r *python.PyObject, err error) {
        var pyargs *python.PyObject

        if pyargs, err = ToPyObject(reflect.ValueOf(args)); err != nil {
                return
        }

        name := python.PyString_AsString(f.GetAttrString("__name__"))
        if r = f.CallObject(pyargs); r == nil {
                err = errors.New(fmt.Sprintf("%s(): function failed at python side", name))
        }
        if pyobj := python.PyErr_Occurred(); pyobj != nil {
                err = errors.New(fmt.Sprintf("%s(): exception happened in python side", name))
                python.PyErr_Clear()
        }
        return
}

var pyinit = false

func Init() (err error) {
        if !pyinit {
                if err = python.Initialize(); err == nil {
                        pyinit = true
                }
        }
        return
}

func Import(module string, functions ...string) (funcs map[string]*PyFunction, err error) {
        if err = Init(); err != nil {
                return
        }

        if pymod := python.PyImport_ImportModuleNoBlock(module); pymod == nil {
                err = errors.New(fmt.Sprintf("gopy:%s: module import failed", module))
        } else {
                funcs = make(map[string]*PyFunction)
                for _, name := range functions {
                        if pyfunc := pymod.GetAttrString(name); pyfunc == nil {
                                err = errors.New(fmt.Sprintf("gopy:%s:%s: function not found", module, name))
                                return
                        } else {
                                funcs[name] = &PyFunction{pyfunc}
                        }
                }
        }
        return
}

The out of the execution of go program fails python side as shown below

go run test.go
panic: Error executing python function. error: PrintSysArgv(): exception happened in python side

goroutine 1 [running]:
main.main()
    /root/go/bin/test.go:22 +0x291

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:2232 +0x1
exit status 2

Just a wierd thought that go-python uses a trimmed version of python which is not able to take care of "sys.argv" as such.

Can you please look into the same and suggest us accordingly.

If needed / suggested I can raise the same in wiki...

Thanks and Regards, Shubhendu

[sbinet]

that's the way the C-API works: https://docs.python.org/2/c-api/init.html#c.Py_Initialize

void Py_Initialize()

Initialize the Python interpreter. In an application embedding Python, this should be
called before using any other Python/C API functions; with the exception of
Py_SetProgramName(), Py_SetPythonHome(), PyEval_InitThreads(), PyEval_ReleaseLock(),
and PyEval_AcquireLock().
This initializes the table of loaded modules (sys.modules), and creates the fundamental
modules __builtin__, __main__ and sys. It also initializes the module search path (sys.path).
It does not set sys.argv; use PySys_SetArgvEx() for that.
This is a no-op when called for a second time (without calling Py_Finalize() first).
There is no return value; it is a fatal error if the initialization fails.

[shtripat]

Thanks Sebastien for detailed explanation. Now how should we go around this? Should we invoke PySys_SetArgvEx() explicitly while initialize? Do we need to do any cleanup after each pyhon call or once calling PySys_SetArgvEx() while initialize is sufficient?

Regards Shubhendu On 10 Feb 2016 16:52, "Sebastien Binet" notifications@github.com wrote:

that's the way the C-API works: https://docs.python.org/2/c-api/init.html#c.Py_Initialize

void Py_Initialize()

Initialize the Python interpreter. In an application embedding Python, this should be called before using any other Python/C API functions; with the exception of Py_SetProgramName(), Py_SetPythonHome(), PyEval_InitThreads(), PyEval_ReleaseLock(), and PyEval_AcquireLock(). This initializes the table of loaded modules (sys.modules), and creates the fundamental modules builtin, main and sys. It also initializes the module search path (sys.path). It does not set sys.argv; use PySys_SetArgvEx() for that. This is a no-op when called for a second time (without calling Py_Finalize() first). There is no return value; it is a fatal error if the initialization fails.

— Reply to this email directly or view it on GitHub https://github.com/sbinet/go-python/issues/36#issuecomment-182323087.