Installation: Create a setup.py file

[2320sharon]

Description

Having a setup.py file would make it easier for users to install and update to the latest version of CoastSat. I propose creating a simple setup.py file with the non-GDAL dependencies so that CoastSat can be installed with pip from GitHub pip install git+git:https://github.com/kvos/CoastSat.git. Once the setup.py is complete the further work of developing CoastSat into a package with the GDAL dependencies can be worked on.

Related Issues

55 issue: made progress in determining what it would take to create a coastsat package but the biggest obstacle is clearly installing the GDAL binaries on the user's preferred OS. I think we could mirror the approach taken by rasterio where the user has the choice to install the GDAL binaries with pip or install them manually on their OS. We should make it clear this will require users to create a virtual environment with Conda or venv. Creating only the `setup.py would not impose a rigid structure on CoastSat while making it much easier for users to install the latest version of CoastSat.

Possible Approach

Divide the development of a package into two phases

• The initial development phase would include only the setup.py file that includes the non-GDAL dependencies (eg.NumPy, pandas, etc) so that coastsat would be installable via pip install git+git:https://github.com/kvos/CoastSat.git. This first phase would leave the installation of GDAL up to the user and retain the primary way of installing CoastSat using git clone and a conda environment. The second phase would be to create an official PyPi package for CoastSat similar to rasterio that installs the bare minimum GDAL dependencies and leaves any GDAL customization up to the user. rasterio does this.

Relevant Resources

1. Older version of a CoastSat setup.py file for reference.
2. Rasterio
3. GDAL binaries
4. How to create a setup.py file guide

I'm open to any feedback or suggestions that would help with the creation of a CoastSat package. If anyone has any experience with creating packages that have GDAL as a dependency any advice would be appreciated.

[kvos]

hi @2320sharon , thanks for putting time to think about this problem. This sounds like a good way to tackle this problem of packaging coastsat.

1. I like the idea to create a setup.py file without dependencies, so we ask the users to install the coastsat environment with conda (as done now) and then run the pip command to install CoastSat only. This is because I don't think pip would do a very good job to solve conflicts for all the dependencies that are necessary.
2. We need to find out what happens to the model files under /classification, do these get downloaded by pip install?

If you want to push a setup.py file similar to #55 without dependencies, I can test it out from my already-installed coastsat environment.

[2320sharon]

So I've been working on a prototype coastsat package for a little over a week now and I've got it working and up on PyPi. This package includes the classification models, training_data, and all the coastsat code. The only caveat is that due to the dependency on geopandas which in turn relies on GDAL this package has to be installed in a conda environment in a very specific order. I have included installation instructions on the coastsat_package repo I created. Also if you notice the pypi page is missing information or has some things incorrect this version is still a prototype and has some formatting issues that still need be fixed.

I ended up choosing to create a pypi package for CoastSat instead of simply creating a setup.py because I discovered that setup.py is being deprecated and I want to make sure CoastSat is future proof. Instead all the dependencies are in a pyproject.toml file which is used to build the pypi package. The bonus of creating a pip package is that users don't need to clone the coastsat repo anymore to install the code.

If you look at the code on the pypi you will see I changed a few things to have coastsat work in package form. I placed all the code in a directory called src to make coastsat work properly with pypi and I modified SDS_shoreline to import the classification models from the directory included in the installed pip package using importlib.resources instead of reading from the current working directory as it was before. I also modified create_jpg in SDS_preprocess to have an additional parameter named create_plot that controls whether images are saved as matplotlib plots or regular jpgs.

[2320sharon]

@kvos what do you think about the idea of a pypi package for coastsat?

[kvos]

hi @2320sharon (sorry for my late reply), sounds great to make it a pypi package, I will test your coastsat_package repo to see how it works. Thanks!

[2320sharon]

No worries @kvos. I know you're very busy and thank you for trying it out. Let me know how it goes.

[kvos]

hi @2320sharon , I followed the instructions in your coastsat_package and it worked very smoothly! I downloaded the jupyter notebook separately and started running the example and it was going fine. I don't know how you did it but thanks for setting this up. So now the users still have to download the github repository to get the example notebook and example files, is that right? Here are some other comments:

• I found that the conda install pip command wasn't necessary as it was already installed.
• I'm not sure where geopandas is used, maybe to write a geojson file somewhere, but the main dependency is GDAL to manipulate the rasters and shapely to manipulate vectors.

[2320sharon]

@kvos thank you for testing the coastsat package. I'm happy it ran smoothly for you. It does appear that the users still need to install the example notebooks from the GitHub repo since it contains both the notebook and the example data. It might be useful to consider creating a new branch called download that includes only the example notebooks and associated data.

[kvos]

@2320sharon , yes that makes sense. Otherwise if they download the full repo, we risk having multiple copies of the coastsat folder on disk... I'll try creating a new branch with the downloads (excluding coastsat package) and see how it all works out. So how does the pypi package gets updated? does it automatically upload that most recent release of CoastSat?

[2320sharon]

I have to manually update the pypi package for now. But it is possible to use GitHub actions to automatically integrate changes from CoastSat. However, if there are conflicts between coastsat and the coastsat package it will require a manual update to resolve conflicts.

[dbuscombe-usgs]

Hey @2320sharon how close is https://pypi.org/project/coastsat-package/ to primetime?

@kvos I'm wondering about the roadmap towards integrating this into coastsat, and also if it would help solve individual installation errors such as https://github.com/kvos/CoastSat/issues/347?

(Mostly, I'm just trying to catch up with all the new packages that @2320sharon has made. I'll be testing the new coastseg package today!)

I'm happy to be a beta-tester!

[kvos]

it was working really well when I tested it, great work by @2320sharon. We discussed with Sharon that it was a bit risky to change the installation instructions of CoastSat now as I won't have any time to deal with any potential issues until the end of the year. But CoastSeg can already integrate with the coastsat-package and keep moving forward.

[2320sharon]

@dbuscombe-usgs thank you for reminding me about the coastsat package timeline. I apologize for the late response. Currently, the coastsat package mirrors the structure and content of coastsat very closely. I think the only things that block it from being released for real are:

1. instructions for installing and testing the package eg. how do users run the example notebook included
2. A roadmap for users to switch to the new coastsat package,
3. A developer guide that shows how to maintain the coastsat package
4. Issue templates ( we can copy the ones from coastseg)
5. (optional) github actions to run basic tests to ensure coastsat is working properly