search

digikar99 / py4cl2

Call python from Common Lisp
https://digikar99.github.io/py4cl2/
Other
41 stars 9 forks source link

Package TRANSFORMERS does not exist. #31

Open qubit55 opened 6 months ago

qubit55 commented 6 months ago

Hello, I'm trying to do the following:

(ql:quickload :py4cl2)

(py4cl2:defpymodule "transformers" t :lisp-package "TRANSFORMERS")

(defparameter *model-id* "meta-llama/Meta-Llama-3-8B-Instruct")

(defparameter *pipeline*
  (transformers:pipeline
   :task "text-generation"
   :model *model-id*))

but getting "Package TRANSFORMERS does not exist."

However, the following works fine:


(py4cl2:defpyfun "pipeline" "transformers" :lisp-fun-name "PIPELINE")

(defparameter *pipeline*
  (pipeline
   :task "text-generation"
   :model *model-id*))

I have a feeling I'm missing something obvious.

digikar99 commented 6 months ago

Hi, try passing :continue-ignoring-errors nil to py4cl2:defpymodule.

(py4cl2:defpymodule "transformers" t :lisp-package "TRANSFORMERS" :continue-ignoring-errors nil)

Also check if (py4cl2:config-var 'py4cl2:pycmd) is pointing to the correct python binary. And that the value of sys.path is as expected:

(py4cl2:raw-pyexec "import sys")
(py4cl2:raw-pyeval "sys.path")
digikar99 commented 6 months ago

I just tried this, and I run into an error pertaining to the name translation of transformers.models.megatron_gpt2. I will try to fix it. In the meanwhile, you can restrict to calling pycall directly.

digikar99 commented 6 months ago

eff3f0edb302fb2cef8b83de5cd876c15f9affcd should have fixed this.