search

ODM2 / ODM2PythonAPI

A set of Python functions that provides data read/write access to an ODM2 database by leveraging SQLAlchemy.
http://odm2.github.io/ODM2PythonAPI/
BSD 3-Clause "New" or "Revised" License
4 stars 13 forks source link

Update & add examples to work with Release 0.7.1 #157

Closed aufdenkampe closed 5 years ago

aufdenkampe commented 6 years ago

The existing examples were created by @sreeder about 1-2 years ago to work with releases 0.5 and 0.6.

It seems that none of the examples work with Release v0.7.1 and probably release 0.7.0 because of the major changes with those releases.

@emiliom, what would it take to update, replace or point to other working examples (via ReadMe) that function with the latest release?

@horsburgh or @miguelcleon, do you have any examples that work with the latest release?

cc: @Roelofversteeg

Connected to https://github.com/ODM2/ODM2PythonAPI/issues/146#issuecomment-370211959

emiliom commented 6 years ago

@aufdenkampe I can do this next week. Shouldn't be a problem -- I will test and tweak this notebook from the BiGCZ workshop.

I'm on leave today and tomorrow.

aufdenkampe commented 6 years ago

@emiliom, thanks for that offer! And thanks for the link to https://github.com/BiG-CZ/wshp2017_tutorial_content/blob/master/notebooks/ODM2_Example3.ipynb

Is this also worth updating? https://big-cz.github.io/notebook_data_demo/notebooks/2017-06-24-odm2api_sample_fromsqlite/

@miguelcleon, might you be able to update any of these four Jupyter Notebook examples to work with ODM2api v>0.7, so that we could point to them from the Examples ReadMe file?

emiliom commented 6 years ago

@aufdenkampe A question about this:

It seems that none of the examples work with Release v0.7.1 and probably release 0.7.0 because of the major changes with those releases

It sounds like you're saying this not from direct testing of every example (including Miguel's notebooks), but from interpreting the impact of changes in releases 0.7.0 and 0.7.1, specially the simplification of the module hierarchy? If so, keep in mind that those major changes were implemented in such a way that they did not break old code. We created a "transition" when the old module reference hierarchy still works but is considered deprecated (it gives warnings), and the intent to remove it altogether in some future release.

So, most of the old code should still work fine, unless it's really old and fails for reasons involving changes in even older releases! It's simply not presenting the most up-to-date and future-proof module reference style.

Regardless, I agree that we should ensure the "official" examples presented in https://github.com/ODM2/ODM2PythonAPI/tree/master/Examples should always be tested to ensure they work with the latest release; and that's probably not the case with what's currently there. I'll work on that next week, and will probably remove some of those old examples.

Is this also worth updating? https://big-cz.github.io/notebook_data_demo/notebooks/2017-06-24-odm2api_sample_fromsqlite/

Maybe. ODM2_Example3.ipynb largely superseded it, I think.

aufdenkampe commented 6 years ago

cc: @erekalper & @dougvj, who work with @roelofversteeg

aufdenkampe commented 6 years ago

@emiliom, it might also be worth adding to the top of the examples that ODM2PythonAPI is compatible with Python 3 since since Jan. 19, 2018 (Release 0.7, https://github.com/ODM2/ODM2PythonAPI/releases/tag/v0.7.0). Ideally we would ensure that the examples are also compatible with Python 3.

aufdenkampe commented 6 years ago

@emiliom, in response to your https://github.com/ODM2/ODM2PythonAPI/issues/157#issuecomment-417748062, I think @erekalper & @dougvj & @roelofversteeg did do some direct testing, but that it didn't work. However, I don't know the details.

@erekalper & @dougvj & @roelofversteeg, could you let us know what worked and what didn't?

emiliom commented 6 years ago

Pasting my comments from the email, and responding to this point:

@emiliom, it might also be worth adding to the top of the examples that ODM2PythonAPI is compatible with Python 3 since since Jan. 19, 2018 (Release 0.7, https://github.com/ODM2/ODM2PythonAPI/releases/tag/v0.7.0). Ideally we would ensure that the examples are also compatible with Python 3.

I'm tied up today and tomorrow and have only scanned these emails. But before a misunderstanding sets in, I want to clarify the status of Python 3 compatibility and ODM2PythonAPI. As far as I know it's not guaranteed to work with Python 3, nor has there been much testing against it. The text Anthony is quoting on https://github.com/ODM2/ODM2PythonAPI/releases/tag/v0.7.0 ("Python 3 Compatibility and Flake 8 Improvements", which I probably wrote myself) does not mean that it works with Python 3. Instead, it just says that improvements were made to the code to move towards Python 3 compatibility.

I don't recall anyone doing significant and sustained testing for Python 3 compatibility -- or any testing at all. You should assume that it's not compatible, and Python 2.7 is required.

emiliom commented 5 years ago

@aufdenkampe Regarding the BiGCZ workshop odm2api notebook ODM2_Example3.ipynb: I've rerun (confirmed) and made small tweaks.

I first reran it with the "odm2client" conda env created for the workshop, and everything was fine. Then I created a new conda env using the same environment file, except for an important update: I removed the reference to the retired odm2 conda channel. I reran the notebook as is with this updated environment, and all its odm2api components ran w/o problem or updates, but the Folium map statement failed b/c of an update in a recent version. So, I fixed that. Everything ran.

But I went ahead and updated the odm2api code use to reflect recommended practices, specially to get rid of things that throw a warning b/c we've slated them to be deprecated (there were a bunch, involving mainly the import module hierarchy and a couple of parameter name changes). For good measure I also made a couple of cosmetic improvements for readability. I haven't pushed my updated notebook to any github repo yet.

We're slated to chat on Monday. Let's talk about this then, to decide what path to follow in updating the odm2api examples at https://github.com/ODM2/ODM2PythonAPI/tree/master/Examples. eg, we might want to:

aufdenkampe commented 5 years ago

@emiliom, thanks for doing this and laying out possible next steps! Talk to you Monday!

emiliom commented 5 years ago

I've cleaned up the Examples folder, via PR #160 (now merged into master). Removed old sample code, and added a new notebook, WaterQualityMeasurements_RetrieveVisualize.ipynb, that's an updated and polished version of https://github.com/BiG-CZ/wshp2017_tutorial_content/blob/master/notebooks/ODM2_Example3.ipynb (works with odm2api 0.7.1, uses current best practices, added more background info). Also added auxiliary files for that notebook: the SQLite ODM2 database and conda environment files used by the notebook.

I haven't made any changes to the Sphinx documentation yet. I'll get to that in a few days.

emiliom commented 5 years ago

Added new notebook to Examples folder, via PR #162 (now merged into master). New notebook TimeSeries_RetrieveVisualize.ipynb complements the previously added one by focusing on a SQLite database with time series results. It's based on https://big-cz.github.io/notebook_data_demo/notebooks/2017-06-24-odm2api_sample_fromsqlite/ but with A LOT of cleanups, updates and polish. Made its organization and background info similar and consistent with the other notebook. Also added corresponding SQLite ODM2 database file.

emiliom commented 5 years ago

@aufdenkampe I'm done with all the updates and changes identified in this issue. The Sphinx docs are now updated. Check them out, http://odm2.github.io/ODM2PythonAPI/. If you have suggestions for further changes, let's handle them via new issues.

aufdenkampe commented 5 years ago

@emiliom, wow! Thank you for substantially enhancing the documentation! The Sphinx docs are much, much more helpful and the Example Jupyter notebooks are awesome! Thanks!