Utilize python setuptools

[atwinkelman]

I suggest using python's setuptools to control installation of python modules such as this. This comes with several benefits

• install into your python environment using your favorite method (I used pip but I bet there's a coda-specific way),
• setuptools handles checking dependencies, which can even be a custom package,
• the packages (there are none in this module yet), scripts (waveform_collection.py in this package), and data files (avo_json/*.json) becomes available anywhere within that environment.

[liamtoney]

This seems like a logical next step for us.

[davidfee5]

Just to clarify, do I still need to do a pip install? I've haven't mixed pip and conda before in the past. Can someone update the readme to make this clear?

[liamtoney]

Yes, you should do a pip install. Instead of adding the location of waveform_collection to your path (as the README previously suggested) you use pip install . Then the code is accessible within whichever conda env you ran the pip install command in... though I'm not sure about updates. @atwinkelman would that be something like:

$ cd /path/to/waveform_collection
$ git pull
$ pip install .

to upgrade if new commits came in? I.e. just overwrite

[davidfee5]

So you have to do a "pip install ." every time there are new commits?

[liamtoney]

Yes, because it installs to your /anaconda env dir. However, there's an easy way around that which is just remembered!

$ pip install -e .

The -e option means "editable" mode. The location of the source files is simply your cloned local dir. So if you pull in changes, things will be updated. I think that's what we want and I'll update the README accordingly.

[atwinkelman]

So you have to do a "pip install ." every time there are new commits?

Yep, git pull followed by pip install .

I can do some research to see if we can install from github using conda and if conda will check github for updates to the master branch or maybe from a git release.

[atwinkelman]

As this and other projects grow, I'd like use the packages in setuptools (rather than just the script) to keep things more organized. If you look at how larger projects are organized, like obspy, there are many packages listed in their setup.py. I left the waveform_collection.py as is just to make this transition smooth.

[liamtoney]

^ your solution is cleaner, @atwinkelman!

Also doing this means that assigning a release number has no meaning. Which is fine as we develop rapidly. I've removed the version number from setup.py for the time being.

[atwinkelman]

Sure, removing the version number is fine.

[liamtoney]

As this and other projects grow, I'd like use the packages in setuptools (rather than just the script) to keep things more organized.

By that you mean refactoring this repo into a package with e.g. a waveform_collection/ dir with an __init__.py?

[liamtoney]

It looks like the pip install tries to install dependencies with pip. We're trying to use conda. So for installation purposes we'll probably want to have the user either pip install into an existing environment containing obspy, or create a new one.

We won't get the automatic dependency installation functionality that comes with using pure pip, unfortunately, unless there's a way to instruct pip install -e . to use conda for dependencies...

[atwinkelman]

Which pip are you using? Ensure which pip points to the pip in your conda environment. If it doesn't (say, it points to a pip in /usr/bin/, for example), then install pip in your conda environment with a conda activate $env and conda install pip. Then, try using pip install ..

[liamtoney]

I'm using my conda env's pip. I'm actually now getting

ImportError: No module named numpy. Please install numpy first, it is needed before installing ObsPy

even with

'install_requires': ['numpy', 'obspy']

in setup.py.

EDIT

Made a new conda env and got it working.

[atwinkelman]

That ImportError is when it tries to install obspy? I've had problems with that before when when installing obspy with conda and I got around it by manually installing numpy first.

Try

pip install numpy
conda config --add channels conda-forge
conda install obspy

Finally, does pip install waveform_collection work?

[liamtoney]

Manually installing stuff with conda beforehand works. pip install waveform_collection does not, gives

  ERROR: Could not find a version that satisfies the requirement waveform_collection (from versions: none)
ERROR: No matching distribution found for waveform_collection

[atwinkelman]

I should have written pip install waveform_collection to be cd waveform_collection/ and pip install . separately.

Maybe setuptools is complaining about a missing version number? A double check of the setup.py documentation shows the version is a required field. It also says the URL is a required field, so I don't know why setuptools doesn't care about that.

[liamtoney]

To clarify — pip install . is working. It must have been a fluke when it didn't work last time.

I've gone ahead with making this into a package. One issue is now I have to figure out how to access data_files once they've been installed...

[davidfee5]

Ok, I go this to work...thanks! However, will I have to do the install for every other conda environment (e.g. rtm?) I think so...which might be a pain if we are encouraging users to create a new environment for most of our repos.

[liamtoney]

Yes it’s environment-level install. That’s a good point about multiple install pains.

Really motivates the “single env” model. Or single env plus RTM’s more complex one.

[liamtoney]

Assuming we told folks to make a conda environment (containing ObsPy etc.) called uafinfra for everything infrasound-related in uafgeotools, then the install instructions for any of our repos — assuming they contained a setup.py — would be:

$ git clone https://github.com/uafgeotools/<package_name>.git
$ cd <package_name>
$ conda activate uafinfra  # So that pip installs into the uafinfra environment
$ pip install -e .

From then on, all the user would need to do to update their install would be git pull in the <package_name> directory (which can be anywhere on the user's system).

This seems like strong motivation to create a uafinfra environment, similar to @carltape's sln environment.