clearwater example files

[sjsrey]

We are in the process of moving the large example datasets out of the source install for libpysal. The clearwater example data set looks to be consumed directly from mgwr rather than from libpysal:

(base) pysal/pysal - [master●] » cd model
(base) pysal/model - [master●] » grep -r clearwater .
./mgwr/tests/test_gwr.py:            os.path.dirname(__file__), 'clearwater/landslides.csv')
./mgwr/tests/test_gwr.py:                'clearwater/clearwater_BS_F_listwise.csv'))
./mgwr/tests/test_gwr.py:                'clearwater/clearwater_BS_NN_listwise.csv'))
./mgwr/tests/test_gwr.py:                'clearwater/clearwater_GS_F_listwise.csv'))
./mgwr/tests/test_gwr.py:                'clearwater/clearwater_GS_NN_listwise.csv'))
./mgwr/tests/clearwater/clearwater_BS_NN.ctl:C:\Users\IEUser\Desktop\clearwater\clearwater\landslides.csv
./mgwr/tests/clearwater/clearwater_BS_NN.ctl:summary_output: C:\Users\IEUser\Desktop\clearwater_BS_NN_summary.txt
./mgwr/tests/clearwater/clearwater_BS_NN.ctl:listwise_output: C:\Users\IEUser\Desktop\clearwater_BS_NN_listwise.csv
./mgwr/tests/clearwater/clearwater_GS_NN_summary.txt:Session control file: C:\Users\IEUser\Desktop\clearwater_GS_NN.ctl
./mgwr/tests/clearwater/clearwater_GS_NN_summary.txt:Data filename: C:\Users\IEUser\Desktop\clearwater\clearwater\landslides.csv
./mgwr/tests/clearwater/clearwater_GS_NN_summary.txt:    Listwise output file: C:\Users\IEUser\Desktop\clearwater_GS_NN_listwise.csv
./mgwr/tests/clearwater/clearwater_BS_NN_summary.txt:Session control file: C:\Users\IEUser\Desktop\clearwater_BS_NN.ctl
./mgwr/tests/clearwater/clearwater_BS_NN_summary.txt:Data filename: C:\Users\IEUser\Desktop\clearwater\clearwater\landslides.csv
./mgwr/tests/clearwater/clearwater_BS_NN_summary.txt:    Listwise output file: C:\Users\IEUser\Desktop\clearwater_BS_NN_listwise.csv

So this means for the pysal meta package we are currently installing two copies of this dataset.

I would like to propose that we rewrite the mwgr tests that use this dataset to pull from a remote repos so we don't have to include it in the source distribution.

I'm happy to do a pr into mgwr to implement that once the libpysal.examples refactor is done, but I wanted to put this on the radar screen and get feedback.

[TaylorOshan]

@sjsrey Sounds like a good plan. Is there an established place/service for external hosting.

[sjsrey]

@sjsrey Sounds like a good plan. Is there an established place/service for external hosting.

No. Just a public url with everything in the example in a single zip. I did this for clearwater at: https://github.com/sjsrey/clearwater And this gets pulled in by libpysal by updating remotes.py.

There is an example demonstrating the use of the new remotes functionality in libpysal.

[TaylorOshan]

@sjsrey Ok, so ounce the new release of libpysal is cut, I could delete the files locally and then use:

c = load_example("clearwater”)
c.load("filename” ) 

or

load_example("clearwater”)
libpysal.io.open(get_path("filename"))

[sjsrey]

New libpysal is cut.