How to use a tar.gz as a dep in another project

[wildfunctions]

I managed to build a tar.gz of a module using pygradle, and am now wanting to import it into another project and use it.

My gradle file has the following dependency from the tar.gz created before:

dependencies {
    python 'pypi:requests:2.9.1'
    python 'com.myawesomedomain.pyexample:helloworld:0.0.1' // My module
    test 'pypi:mock:1.3.0'
}

and the setup.py has the following setup portion:

setup(
    distclass=GradleDistribution,
    package_dir={'': 'src'},
    packages=find_packages('src'),
    include_package_data=True,
    entry_points={
        'console_scripts': [
            'import_example = import_example.run:main',
        ]
    }
)

However, when I build, all I get is another build/distributions/import_example-0.0.1.tar.gz which doesn't seem to contain the original helloworld-0.0.1.tar.gz package anywhere.

I do see the helloworld packages in build/venv/lib/python2.7/site-packages/helloworld, but that doesn't help with creating a publishable artifact since venv isn't a package.

Looking at the example projects provided, I get the impression that I was supposed to get a build/deployable artifact which might be what I am missing.

Also, how am I supposed to run this with the entry point? I thought it might be ./activate import_example but that clearly isn't it.

Also when I get the following errors when I try to run ./build/venv/bin/import_example

Traceback (most recent call last):
  File "./build/venv/bin/import_example", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/mnt/c/Users/myuser/Documents/dev/pyexample_import/build/venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3138, in <module>
    @_call_aside
  File "/mnt/c/Users/myuser/Documents/dev/pyexample_import/build/venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3124, in _call_aside
    f(*args, **kwargs)
  File "/mnt/c/Users/myuser/Documents/dev/pyexample_import/build/venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3151, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/mnt/c/Users/myuser/Documents/dev/pyexample_import/build/venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 661, in _build_master
    ws.require(__requires__)
  File "/mnt/c/Users/myuser/Documents/dev/pyexample_import/build/venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 962, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/mnt/c/Users/myuser/Documents/dev/pyexample_import/build/venv/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 849, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'helloworld' distribution was not found and is required by import-example

[zvezdan]

If I understand correctly, you made a library that other Python packages will depend on named helloworld. You probably used ...-sdist plugin and that produced a source distribution tarball (package) for helloworld. Other Python packages can depend on it and use it.

If you want to make an executable though, then you would use one of the other plugins, such as ...-cli or ...-web-app. In that case every entry point in your project will produce an executable referring to a PEX file (container) with your package and all its dependencies packed together. Thus, when you execute any of these, they'll be able to run and use all their dependencies independent of other Python programs. In other words, they can be shipped to any host that has Python interpreter and run. The ...-cli is intended for command-line programs and ...-web-app for apps.

Another interesting plugin is ...-pex which allows you to produce a single executable that can still be consumed similar to a library by non-Python applications that need a Python program to run integration tests, deployment hooks, etc. This is where Gradle comes in handy allowing polyglot programming.

Notice that if the package that depends on helloworld is also a library and uses ...-sdist plugin, it would again produce just a source tarball (that's what you got it seems with import_example), but it would contain helloworld in its dependencies in package metadata. So if another package now depended on it, it would pull in import_example as direct dependency and helloworld as a transitive dependency. So both of your libraries would be correctly included. The transitive dependency could be resolved through Ivy data for the artifact by Gradle, and through Python package metadata in a pure Python build environment. In other words, libraries are completely usable outside of Gradle/pygradle. Of course, some way of hosting them through Python package index would have to be provided in that case.

Let me know if I made any incorrect assumptions about your use case.

[wildfunctions]

Okay. You have definitely corrected a mistake of mine. I indeed was using ...-sdist for what I intended to be an executable. So, I made the change to ...-cli and tried to gradle build but am getting this error:

> Task :buildPex
Processing /mnt/c/Users/myuser/Documents/dev/pyexample_import
Building wheels for collected packages: import-example
  Running setup.py bdist_wheel for import-example
  Stored in directory: /mnt/c/Users/myuser/Documents/dev/pyexample_import/build/wheel-cache
Successfully built import-example
/mnt/c/Users/myuser/Documents/dev/pyexample_import/build/deployable/bin/import_example.pex
Could not satisfy all requirements for helloworld:
    helloworld(from: import-example)

Just to be sure I am organizing things correctly,

The file structure of the helloworld-0.0.1.tar.gz library is:

.
├── build.gradle
├── pinned.txt
├── PKG-INFO
├── README.md
├── settings.gradle
├── setup.cfg
├── setup.py
├── src
│   ├── helloworld
│   │   ├── __init__.py
│   │   └── main.py
│   └── helloworld.egg-info
│       ├── dependency_links.txt
│       ├── PKG-INFO
│       ├── requires.txt
│       ├── SOURCES.txt
│       └── top_level.txt
└── test
    └── test_example.py

Where ./src/helloworld/main.py is simply:

class Hello:
    def __init__(self):
        print("Hello World.")

    def greet(self):
        print("It is a pleasure to meet you.")

Then inside pyexample_import, the entry point ./src/import_example/run.py is:

from helloworld.main import Hello

def main():
    print("Running main with Hello dep")
    hello = Hello()
    hello.greet()

if __name__ == '__main__':
    print("This is not printed when run with PEX ...")
    main()

I am simply trying to import the helloworld library module Hello from the import_example executable, but seems something else is wrong.