HDFGroup / h5serv

Reference service implementation of the HDF5 REST API
Other
168 stars 35 forks source link

Make h5serv installable with pip and importable in other Python code #124

Closed telamonian closed 6 years ago

telamonian commented 6 years ago

Fixes #102, fixes #122, and fixes #123

Here's my use case for h5serv: I'm writing an HDF5 file viewer plugin for another Python program, and I want to use h5serv as a dependency. Basically, h5serv will be used to handle all of the low-level interactions between the program and a user's HDF5 files. Two things need to happen in order to make this possible:

  1. There needs to be a (very) easy way for my end-users to install h5serv (ie pip install h5serv)
  2. I need to be able to import components of h5serv into my own Python code (ie from h5serv import app), and there needs to be a way to use those components to start/stop an HDF5 server without actually having to spawn a new process.

The code in this PR aims to fix all of the things preventing 1. and 2. from happening.

telamonian commented 6 years ago

There's still a bunch of issues to fix that relate to conflicts between how this project's files/paths are structured, and the way in which setup.py and Python's setuptools "want" those files/paths to be structured.

For example, right now if I try to run the program using the h5serv command (the one that's set up by the newly fixed entry_points in setup.py), I get an error like:

FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/log/h5serv.log'

This is because the h5serv command is installed as a script at /usr/local/h5serv and h5serv defaults to saving the log file at a position relative to that script.

jreadey commented 6 years ago

This makes sense. When I original setup the project I didn't think it was important to have a setup install script.

Do you plan to update the PR for the other issues?

Also the change breaks the travis tests. e.g.: https://travis-ci.org/HDFGroup/h5serv/jobs/437807772.

The travis script just needs to use the new name for invoking the server.

telamonian commented 6 years ago

Do you plan to update the PR for the other issues?

Working on it right now!

Aside from the Travis setup, some import statements in the tests are also broken by the changes, so I'll fix those too.

There's a bunch of tricky little problems relating to config and how it's handled. Here's how I'm thinking to resolve the issues:

telamonian commented 6 years ago

I took a crack at fixing the remaining issues. At least some tests pass now. The remaining problems all have to do with how the tests... ah... "creatively leverage" the quirks of the Python import system to override h5serv.config with their own local config module. I know how to fix it, but I'll probably have to make a couple of significant changes to how config works (at least internally).

jreadey commented 6 years ago

Looks like the Travis tests are passing, that's a good sign. Are there remaining known problems with the PR?

telamonian commented 6 years ago

Just needed some updates to the docs. The PR is now finished. Some notes:

Let me know if you have any revisions @jreadey, or if the PR looks good to go!

jreadey commented 6 years ago

I tried the PR with the h5pyd test suite, and everything worked fine. Merging into develop. Thanks for the contribution!