vmatare
commented
4 months ago As a user/developer, you will have to use libpython-clj2.codegen/write-namespace! to generate a Clojure namespace with stubs for your Python facade. I'm currently playing around with this, and it works well in plain lein repl. The autogenerated namespace loads and the completions are there.
However somehow, the autogenerated namespace breaks Cursive. In an IntelliJ REPL I get this:
Starting nREPL server...
/usr/lib/jvm/java-11/bin/java -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=127.0.0.1:49999,suspend=y,server=n -Dfile.encoding=UTF-8 -Dfile.encoding=UTF-8 -Dclojure.compile.path=/home/vmatare/ces/distribution/incubator/spacy-adapter/target/classes -Dspacy-adapter.version=0.1.0-SNAPSHOT -Dclojure.debug=false -Dclojure.compiler.disable-locals-clearing=true -javaagent:/home/vmatare/.local/opt/idea-IC-203.7148.57/plugins/java/lib/rt/debugger-agent.jar=file:/tmp/capture.props -classpath [...] clojure.main -i /tmp/form-init9531518625439003328.clj
Connected to the target VM, address: '127.0.0.1:49999', transport: 'socket'
15:06:51.707 [main] INFO libpython-clj2.python.info - Detecting startup info for Python executable venv/bin/python
15:06:51.730 [main] INFO libpython-clj2.python - Startup info {:lib-version "3.11", :java-library-path-addendum "/usr/lib", :exec-prefix "/home/vmatare/ces/distribution/incubator/spacy-adapter/venv", :executable "/home/vmatare/ces/distribution/incubator/spacy-adapter/venv/bin/python", :libnames ("python3.11m" "python3.11"), :prefix "/home/vmatare/ces/distribution/incubator/spacy-adapter/venv", :base-prefix "/usr", :libname "python3.11m", :base-exec-prefix "/usr", :python-home "/usr", :version [3 11 8], :platform "linux"}
15:06:51.731 [main] INFO libpython-clj2.python - Prefixing java library path: /usr/lib
15:06:51.824 [main] INFO libpython-clj2.python - Loading python library: python3.11
15:06:52.291 [main] DEBUG libpython-clj2.python.ffi - Initializing Python C Layer
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by tech.v3.datatype.UnsafeUtil (file:/home/vmatare/.m2/repository/cnuernber/dtype-next/10.009/dtype-next-10.009.jar) to constructor java.nio.DirectByteBuffer(long,int)
WARNING: Please consider reporting this to the maintainers of tech.v3.datatype.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
15:06:52.295 [tech.resource.gc ref thread] INFO tech.v3.resource.gc - Reference thread starting
15:06:52.296 [main] DEBUG libpython-clj2.python.ffi - Python Home: /usr
15:06:53.107 [main] DEBUG libpython-clj2.codegen - Writing python module <module 'spacy' from '/home/vmatare/ces/distribution/incubator/spacy-adapter/venv/lib64/python3.11/site-packages/spacy/__init__.py'> to file src/de/cognesys/ces_spacy_adapter/python_generated/spacy.clj
Connecting to local nREPL server...
Clojure 1.10.3
nREPL server started on port 44911 on host 127.0.0.1 - nrepl://127.0.0.1:44911
(require '[de.cognesys.ces-spacy-adapter.python-generated.spacy :as spacy])
=> nil
Error updating completions:
Execution error (IllegalArgumentException) at cursive.repl.runtime/mapped-metadata (runtime.clj:9).
Don't know how to create ISeq from: clojure.lang.Symbol
(prn)
=> nil
Error updating completions:
Execution error (IllegalArgumentException) at cursive.repl.runtime/mapped-metadata (runtime.clj:9).
Don't know how to create ISeq from: clojure.lang.SymbolSo the Don't know how to create ISeq error appears after every input evaluation, but so far I see no way to investigate further.
The autogenerated namespace:
(ns de.cognesys.ces-spacy-adapter.python-generated.spacy
"No documentation provided"
(:require [libpython-clj2.python :as py]
[libpython-clj2.python.jvm-handle :refer [py-global-delay]]
[libpython-clj2.python.bridge-as-jvm :as as-jvm])
(:refer-clojure :exclude [+ - * / float double int long mod byte test char short take partition require max min identity empty mod repeat str load cast type sort conj map range list next hash eval bytes filter compile print set format compare reduce merge]))
(defonce ^:private src-obj* (py-global-delay (py/path->py-obj "spacy")))
(def ^{:doc "This class holds the model and training configuration and can load and
save the TOML-style configuration format from/to a string, file or bytes.
The Config class is a subclass of dict and uses Python's ConfigParser
under the hood.
" :arglists '[[self & [{data :data, is_interpolated :is_interpolated, section_order :section_order}]] [self & [{is_interpolated :is_interpolated, section_order :section_order}]]]} Config (as-jvm/generic-callable-pyobject (py-global-delay (py/get-attr @src-obj* "Config"))))
(def ^{:doc ""} vectors (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "vectors"))))
(def ^{:doc ""} parts_of_speech (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "parts_of_speech"))))
(def ^{:doc "Create a blank nlp object for a given language code.
name (str): The language code, e.g. \"en\".
vocab (Vocab): A Vocab object. If True, a vocab is created.
config (Dict[str, Any] / Config): Optional config overrides.
meta (Dict[str, Any]): Overrides for nlp.meta.
RETURNS (Language): The nlp object.
" :arglists '[[name & [{vocab :vocab, config :config, meta :meta}]]]} blank (as-jvm/generic-callable-pyobject (py-global-delay (py/get-attr @src-obj* "blank"))))
(def ^{:doc "A generic version of collections.abc.Iterable." :arglists '[[self origin nparams & [{inst :inst, name :name}]]]} PyIterable (as-jvm/generic-callable-pyobject (py-global-delay (py/get-attr @src-obj* "Iterable"))))
(def ^{:doc "Union type; Union[X, Y] means either X or Y.
On Python 3.10 and higher, the | operator
can also be used to denote unions;
X | Y means the same thing to the type checker as Union[X, Y].
To define a union, use e.g. Union[int, str]. Details:
- The arguments must be types and there must be at least one.
- None as an argument is a special case and is replaced by
type(None).
- Unions of unions are flattened, e.g.::
assert Union[Union[int, str], float] == Union[int, str, float]
- Unions of a single argument vanish, e.g.::
assert Union[int] == int # The constructor actually returns int
- Redundant arguments are skipped, e.g.::
assert Union[int, str, int] == Union[int, str]
- When comparing unions, the argument order is ignored, e.g.::
assert Union[int, str] == Union[str, int]
- You cannot subclass or instantiate a union.
- You can use Optional[X] as a shorthand for Union[X, None].
" :arglists '[[self getitem]]} Union (as-jvm/generic-callable-pyobject (py-global-delay (py/get-attr @src-obj* "Union"))))
(def ^{:doc ""} lang (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "lang"))))
(def ^{:doc ""} attrs (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "attrs"))))
(def ^{:doc ""} kb (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "kb"))))
(def ^{:doc ""} schemas (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "schemas"))))
(def ^{:doc ""} strings (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "strings"))))
(def ^{:doc "Get a description for a given POS tag, dependency label or entity type.
term (str): The term to explain.
RETURNS (str): The explanation, or `None` if not found in the glossary.
EXAMPLE:
>>> spacy.explain(u'NORP')
>>> doc = nlp(u'Hello world')
>>> print([w.text, w.tag_, spacy.explain(w.tag_) for w in doc])
" :arglists '[[term]]} explain (as-jvm/generic-callable-pyobject (py-global-delay (py/get-attr @src-obj* "explain"))))
(def ^{:doc "" :arglists '[[& [{gpu_id :gpu_id}]] []]} require_gpu (as-jvm/generic-callable-pyobject (py-global-delay (py/get-attr @src-obj* "require_gpu"))))
(def ^{:doc ""} lexeme (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "lexeme"))))
(def ^{:doc "
spaCy's built in visualization suite for dependencies and named entities.
DOCS: https://spacy.io/api/top-level#displacy
USAGE: https://spacy.io/usage/visualizers
"} displacy (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "displacy"))))
(def ^{:doc "PurePath subclass that can make system calls.
Path represents a filesystem path but unlike PurePath, also offers
methods to do system calls on path objects. Depending on your system,
instantiating a Path will return either a PosixPath or a WindowsPath
object. You can also instantiate a PosixPath or WindowsPath directly,
but cannot instantiate a WindowsPath on a POSIX system or vice versa.
" :arglists '[[self & [args {:as kwargs}]]]} Path (as-jvm/generic-callable-pyobject (py-global-delay (py/get-attr @src-obj* "Path"))))
(def ^{:doc ""} tokenizer (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "tokenizer"))))
(def ^{:doc ""} util (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "util"))))
(def ^{:doc "Load a spaCy model from an installed package or a local path.
name (str): Package name or model path.
vocab (Vocab): A Vocab object. If True, a vocab is created.
disable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to disable. Disabled
pipes will be loaded but they won't be run unless you explicitly
enable them by calling nlp.enable_pipe.
enable (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to enable. All other
pipes will be disabled (but can be enabled later using nlp.enable_pipe).
exclude (Union[str, Iterable[str]]): Name(s) of pipeline component(s) to exclude. Excluded
components won't be loaded.
config (Dict[str, Any] / Config): Config overrides as nested dict or dict
keyed by section values in dot notation.
RETURNS (Language): The loaded nlp object.
" :arglists 'null} load (as-jvm/generic-callable-pyobject (py-global-delay (py/get-attr @src-obj* "load"))))
(def ^{:doc "Concrete implementation of SourceLoader using the file system."} __loader__ (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "__loader__"))))
(def ^{:doc ""} about (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "about"))))
(def ^{:doc "" :arglists '[[& [{model :model, markdown :markdown, silent :silent, exclude :exclude, url :url}]] [& [{markdown :markdown, silent :silent, exclude :exclude, url :url}]]]} info (as-jvm/generic-callable-pyobject (py-global-delay (py/get-attr @src-obj* "info"))))
(def ^{:doc "Helpers for Python and platform compatibility."} compat (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "compat"))))
(def ^{:doc "Use CPU through best available backend." :arglists '[[]]} require_cpu (as-jvm/generic-callable-pyobject (py-global-delay (py/get-attr @src-obj* "require_cpu"))))
(def ^{:doc "" :arglists '[[]]} setup_default_warnings (as-jvm/generic-callable-pyobject (py-global-delay (py/get-attr @src-obj* "setup_default_warnings"))))
(def ^{:doc ""} errors (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "errors"))))
(def ^{:doc ""} __file__ "/home/vmatare/ces/distribution/incubator/spacy-adapter/venv/lib64/python3.11/site-packages/spacy/__init__.py")
(def ^{:doc "Vocab(lex_attr_getters=None, strings=tuple(), lookups=None, oov_prob=-20., vectors_name=None, writing_system={}, get_noun_chunks=None, **deprecated_kwargs)
A look-up table that allows you to access `Lexeme` objects. The `Vocab`
instance also provides access to the `StringStore`, and owns underlying
C-data that is shared between `Doc` objects.
DOCS: https://spacy.io/api/vocab
" :arglists '[[self & [args {:as kwargs}]]]} Vocab (as-jvm/generic-callable-pyobject (py-global-delay (py/get-attr @src-obj* "Vocab"))))
(def ^{:doc ""} symbols (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "symbols"))))
(def ^{:doc "Built-in mutable sequence.
If no argument is given, the constructor creates a new empty list.
The argument must be an iterable if specified."} __path__ (as-jvm/generic-python-as-list (py-global-delay (py/get-attr @src-obj* "__path__"))))
(def ^{:doc "Special type indicating an unconstrained type.
- Any is compatible with every type.
- Any assumed to have all methods.
- All values assumed to be instances of Any.
Note that all the above statements are true from the point of view of
static type checkers. At runtime, Any should not be used with instance
checks.
" :arglists '[[self & [args {:as kwargs}]]]} Any (as-jvm/generic-callable-pyobject (py-global-delay (py/get-attr @src-obj* "Any"))))
(def ^{:doc ""} vocab (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "vocab"))))
(def ^{:doc "A generic version of dict." :arglists '[[self origin nparams & [{inst :inst, name :name}]]]} Dict (as-jvm/generic-callable-pyobject (py-global-delay (py/get-attr @src-obj* "Dict"))))
(def ^{:doc "" :arglists '[[self & [args {:as kwargs}]]]} registry (as-jvm/generic-callable-pyobject (py-global-delay (py/get-attr @src-obj* "registry"))))
(def ^{:doc ""} cli (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "cli"))))
(def ^{:doc ""} ml (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "ml"))))
(def ^{:doc ""} scorer (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "scorer"))))
(def ^{:doc ""} pipe_analysis (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "pipe_analysis"))))
(def ^{:doc ""} morphology (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "morphology"))))
(def ^{:doc "The specification for a module, used for loading.
A module's spec is the source for information about the module. For
data associated with the module, including source, use the spec's
loader.
`name` is the absolute name of the module. `loader` is the loader
to use when loading the module. `parent` is the name of the
package the module is in. The parent is derived from the name.
`is_package` determines if the module is considered a package or
not. On modules this is reflected by the `__path__` attribute.
`origin` is the specific location used by the loader from which to
load the module, if that information is available. When filename is
set, origin will match.
`has_location` indicates that a spec's \"origin\" reflects a location.
When this is True, `__file__` attribute of the module is set.
`cached` is the location of the cached bytecode file, if any. It
corresponds to the `__cached__` attribute.
`submodule_search_locations` is the sequence of path entries to
search when importing submodules. If set, is_package should be
True--and False otherwise.
Packages are simply modules that (may) have submodules. If a spec
has a non-None value in `submodule_search_locations`, the import
system will consider modules loaded from the spec as packages.
Only finders (see importlib.abc.MetaPathFinder and
importlib.abc.PathEntryFinder) should modify ModuleSpec instances.
"} __spec__ (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "__spec__"))))
(def ^{:doc ""} pipeline (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "pipeline"))))
(def ^{:doc "
Instances of the Logger class represent a single logging channel. A
\"logging channel\" indicates an area of an application. Exactly how an
\"area\" is defined is up to the application developer. Since an
application can have any number of areas, logging channels are identified
by a unique string. Application areas can be nested (e.g. an area
of \"input processing\" might include sub-areas \"read CSV files\", \"read
XLS files\" and \"read Gnumeric files\"). To cater for this natural nesting,
channel names are organized into a namespace hierarchy where levels are
separated by periods, much like the Java or Python package namespace. So
in the instance given above, channel names might be \"input\" for the upper
level, and \"input.csv\", \"input.xls\" and \"input.gnu\" for the sub-levels.
There is no arbitrary limit to the depth of nesting.
"} logger (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "logger"))))
(def ^{:doc "dict() -> new empty dictionary
dict(mapping) -> new dictionary initialized from a mapping object's
(key, value) pairs
dict(iterable) -> new dictionary initialized as if via:
d = {}
for k, v in iterable:
d[k] = v
dict(**kwargs) -> new dictionary initialized with the name=value pairs
in the keyword argument list. For example: dict(one=1, two=2)"} __builtins__ (as-jvm/generic-python-as-map (py-global-delay (py/get-attr @src-obj* "__builtins__"))))
(def ^{:doc "Use GPU if it's available. Returns True if so, False otherwise." :arglists '[[& [{gpu_id :gpu_id}]] []]} prefer_gpu (as-jvm/generic-callable-pyobject (py-global-delay (py/get-attr @src-obj* "prefer_gpu"))))
(def ^{:doc ""} __version__ "3.7.4")
(def ^{:doc ""} git_info (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "git_info"))))
(def ^{:doc "A text-processing pipeline. Usually you'll load this once per process,
and pass the instance around your application.
Defaults (class): Settings, data and factory methods for creating the `nlp`
object and processing pipeline.
lang (str): IETF language code, such as 'en'.
DOCS: https://spacy.io/api/language
" :arglists '[[self & [{vocab :vocab, max_length :max_length, meta :meta, create_tokenizer :create_tokenizer, create_vectors :create_vectors, batch_size :batch_size, :as kwargs}]] [self & [{max_length :max_length, meta :meta, create_tokenizer :create_tokenizer, create_vectors :create_vectors, batch_size :batch_size, :as kwargs}]]]} Language (as-jvm/generic-callable-pyobject (py-global-delay (py/get-attr @src-obj* "Language"))))
(def ^{:doc "This module provides access to some objects used or maintained by the
interpreter and to functions that interact strongly with the interpreter.
Dynamic objects:
argv -- command line arguments; argv[0] is the script pathname if known
path -- module search path; path[0] is the script directory, else ''
modules -- dictionary of loaded modules
displayhook -- called to show results in an interactive session
excepthook -- called to handle any uncaught exception other than SystemExit
To customize printing in an interactive session or to install a custom
top-level exception handler, assign other functions to replace these.
stdin -- standard input file object; used by input()
stdout -- standard output file object; used by print()
stderr -- standard error object; used for error messages
By assigning other file objects (or objects that behave like files)
to these, it is possible to redirect all of the interpreter's I/O.
last_type -- type of last uncaught exception
last_value -- value of last uncaught exception
last_traceback -- traceback of last uncaught exception
These three are only available in an interactive session after a
traceback has been printed.
Static objects:
builtin_module_names -- tuple of module names built into this interpreter
copyright -- copyright notice pertaining to this interpreter
exec_prefix -- prefix used to find the machine-specific Python library
executable -- absolute path of the executable binary of the Python interpreter
float_info -- a named tuple with information about the float implementation.
float_repr_style -- string indicating the style of repr() output for floats
hash_info -- a named tuple with information about the hash algorithm.
hexversion -- version information encoded as a single integer
implementation -- Python implementation information.
int_info -- a named tuple with information about the int implementation.
maxsize -- the largest supported length of containers.
maxunicode -- the value of the largest Unicode code point
platform -- platform identifier
prefix -- prefix used to find the Python library
thread_info -- a named tuple with information about the thread implementation.
version -- the version of this interpreter as a string
version_info -- version information as a named tuple
__stdin__ -- the original stdin; don't touch!
__stdout__ -- the original stdout; don't touch!
__stderr__ -- the original stderr; don't touch!
__displayhook__ -- the original displayhook; don't touch!
__excepthook__ -- the original excepthook; don't touch!
Functions:
displayhook() -- print an object to the screen, and save it in builtins._
excepthook() -- print an exception and its traceback to sys.stderr
exception() -- return the current thread's active exception
exc_info() -- return information about the current thread's active exception
exit() -- exit the interpreter by raising SystemExit
getdlopenflags() -- returns flags to be used for dlopen() calls
getprofile() -- get the global profiling function
getrefcount() -- return the reference count for an object (plus one :-)
getrecursionlimit() -- return the max recursion depth for the interpreter
getsizeof() -- return the size of an object in bytes
gettrace() -- get the global debug tracing function
setdlopenflags() -- set the flags to be used for dlopen() calls
setprofile() -- set the global profiling function
setrecursionlimit() -- set the max recursion depth for the interpreter
settrace() -- set the global debug tracing function
"} sys (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "sys"))))
(def ^{:doc ""} glossary (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "glossary"))))
(def ^{:doc ""} language (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "language"))))
(def ^{:doc ""} training (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "training"))))
(def ^{:doc ""} lookups (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "lookups"))))
(def ^{:doc ""} __name__ "spacy")
(def ^{:doc ""} ty (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "ty"))))
(def ^{:doc ""} __package__ "spacy")
(def ^{:doc ""} matcher (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "matcher"))))
(def ^{:doc "" :arglists '[[self & [args {:as kwargs}]]]} Errors (as-jvm/generic-callable-pyobject (py-global-delay (py/get-attr @src-obj* "Errors"))))
(def ^{:doc ""} tokens (as-jvm/generic-pyobject (py-global-delay (py/get-attr @src-obj* "tokens"))))
libpython-cljis library to interop Python and it loads dependencies dynamically. Curisve doesn't show hints (code auto completion) when typing.Expected behaviour: code auto complete works with
libpython-clj.Can author of the library do something to make it work with Cursive?