SciCatProject / pyscicat

A python client library for interacting with the SciCat data catalog backend.
Other
0 stars 9 forks source link

V4.y #38

Closed LAShemilt closed 1 year ago

LAShemilt commented 1 year ago

This is a first pass refactor to get pyscicat working with v4 of Scicat. We would like to :

LAShemilt commented 1 year ago

Now finished work on the pyscicat version for interacting with the v4 scicat backend. Please see the following important changes to pyscicat:

1) Install and Setup I have now split up the setup.py into a minimal setup.py script and a setup.cfg. This allows for easy configuration of unique setups which may require optional dependencies. To now install these please use for example: pip install -e ".[option]" so to use the hd5f plugins install with pip install -e ".hdf5"

2) Deprecation of methods I have used python's deprecation package to deprecate methods that no longer exist in the v4 backend. They will still work with the old backend, and can still be called with pyscicat as normal. However, you will get a deprecation warning when they are called.

3) Endpoint routes I had really trouble with the getter methods which was coming from urllib's urljoin. If you use a / in an endpoint definition when you join that with a base url , urljoin will treat it as an absolute path and will append it after the port number. So http://localhost:3000/api/v3 joined with /Datasets/ was becoming http://localhost:3000/Datasets/. This problem goes away when using proper routing as to not have the port in the url name, which is why I guess it has not affected production systems so far. I have removed the / at the beginning so that the append will work properly in both the test and prod cases.

4) Integration testing in github actions I have updated the github actions to include some minor integration testing for the backend-v.4 . It uses the docker image of the last stable release from https://github.com/SciCatProject/scicat-backend-next. This runs as a service and the tests run a few methods against it to ingest and retrieve data.

Any comments and review would be greatly appreciated.

nitrosx commented 1 year ago

@dylanmcreynolds why don't we align the version with the current SciCat one?

nitrosx commented 1 year ago

One more thing: given that this PR goes against the v4.x branch, should we go ahead and merge and than work on the v4.x to make it production ready? This merge would help me greatly with the current work that I'm doing now to refactor all our services to interact correctly with BE v4.x @LAShemilt , @dylanmcreynolds : thoughts?

dylanmcreynolds commented 1 year ago

Tests are failing because of the change in how requirements are listed.

dylanmcreynolds commented 1 year ago

Hmm, flake8 failures now.

LAShemilt commented 1 year ago

Please see #40