google / subpar

Subpar is a utility for creating self-contained python executables. It is designed to work well with Bazel.
Apache License 2.0
567 stars 69 forks source link

cython modules import fails when running par file #119

Open paulspixels opened 4 years ago

paulspixels commented 4 years ago

I suppose this is may be covered by the stated limitation that C extensions don't work...it would be great if someone could explain if that precludes using cython with subpar? What limits it from importing cython ".so" modules?

The issue I'm having is that when I try to package up a working version of a cython example, python is unable to import the compiled .so file when the .par file runs, even though it is included in the .par file and the non-.par'd version works fine. As an example, pyx_library can come from gRPC (https://github.com/grpc/grpc/blob/master/bazel/cython_library.bzl) and the BUILD file would look like this:

cc_library(
  name = "rectangle_lib",
  srcs = ["Rectangle.cpp"],
  hdrs = ["Rectangle.h"],
  deps = [
      "@com_google_absl//absl/strings",
  ],
)

pyx_library(
  name = "pyrectangle_lib",
  srcs = [
    "pyrectangle.pxd",
    "pyrectangle.pyx",
  ],
  deps = [
    ":rectangle_lib",
  ],
)

par_binary(
  name = "pyrectangle_bin",
  srcs = ["pyrectangle_bin.py"],
  deps = [
    ":pyrectangle_lib",
  ],
)
stevebairos commented 3 years ago

Not sure if this helps for you or not but may be useful to others who stumble upon this issue. I found that I was unable to run a par_binary that depended on grpcio because it couldn't locate GRPC's cython files. I tried adding the zip_safe = False, argument to my par_binary rule and suddenly, Python was able to locate the cython files and everything worked. Not super sure why par_binary doesn't play nice with cython files when they're zipped but maybe if you par_binary were unzipped, the imports would work?