DataDog / go-python3

Go bindings to the CPython-3 API
MIT License
376 stars 140 forks source link

how can i use tensorflow with library #53

Closed LuciferMS closed 2 years ago

LuciferMS commented 2 years ago

i write this code,but it did not work here is my code: def update_model(): print("1.hello") import tensorflow as ts print("2.hello")

it can print frist hello,but can not print the second.

note: i had use pip3 to install tensorflow library in my marchine, and this code can work in python3.7,but can not work when i use this lib to call this function.

christian-korneck commented 2 years ago

When import tensorflow fails, it crashes the Python host (no matter if it's the python executable or a custom Go executable that embeds Python). When the executable crashes, you don't see the second print(). Usually Tensorflow prints an error message to stderr (and Go should also show a stacktrace). I could imagine that for some reason you don't see that output, maybe because your IDE or shell redirects it in some way.

import tensorflow can fail for many reasons, for example if your CPU doesn't support avx2. You are saying that your code works in the python shell, so it's something else.

It noticed that on my machine import tensorflow works in a Python shell, but crashes an embedded Python:

GPU_TEST = re.search(r"(test_2?gpu|test_xla_2?gpu)$", sys.argv[0])
IndexError: list index out of range

This seems to be because sys.argv[0] isn't set by default in an embedded Python. (Not sure why, different topic). When you do something in your code to ensure it is set, the import works:

import sys
sys.argv.append('') #ensure sys.argv[0] isn't out of index
import tensorflow

So this code works for me (on a machine that meets the tensorflow CPU hardware requirements):

package main

import "github.com/DataDog/go-python3"

func main() {
  pycode := `
import os
import sys
sys.argv.append('') #ensure sys.argv[0] isn't out of index
os.environ["CUDA_VISIBLE_DEVICES"] = "-1" # also disable GPUs (avoid cuda issues) - probably not needed
print("hello1")
import tensorflow
print("hello2")
print(tensorflow.__version__)
`
  defer python3.Py_Finalize()
  python3.Py_Initialize()
  python3.PyRun_SimpleString(pycode)

}

and output:

go run .
hello1
<timestamp>: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
<timestamp>: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
hello2
2.7.0

Hope this helps.

LuciferMS commented 2 years ago

it wrok for me,thanks.