search

clstoulouse / motu

Motu is a high efficient and robust Web Server which fills the gap between heterogeneous Data Providers to End Users. Motu handles, extracts and transforms oceanographic huge volumes of data without performance collapse.
GNU Lesser General Public License v3.0
19 stars 5 forks source link

How does the authentication work with the REST-API? #3

Open SebastienDLegrand opened 4 years ago

SebastienDLegrand commented 4 years ago

Dear,

I'm a CMEMS historical user and I've been using the motu-client software for about 10 years to download product subsets.

I'm afraid I've never been a big fan of this client software. However I'm just discovering that you have implemented a REST-API that should allow me to bypass the client and directly request product download, either via a curl command or -still better- via a python script using the 'coreapi' module (I'm not sure if it is possible).

My first attempts to use the API have been done via a browser. For instance, I've cut and pasted this URL in my browser : http://nrt.cmems-du.eu/motu-web/Motu?action=productdownload&service=SST_ATL_SST_L4_NRT_OBSERVATIONS_010_025-TDS&product=IFREMER-ATL-SST-L4-NRT-OBS_FULL_TIME_SERIE&x_lo=-15&x_hi=12&y_lo=48&y_hi=60&output=netcdf&t_lo=2019-06-12+00%3A00%3A00&t_hi=2019-06-13+00%3A00%3A00 and the download of the requested product automatically started once I've provided my credentials. This was done in a second webpage. I've more or less understood that, this second webpage was storing a token in a temporary cookie that expires when the browser is closed. Unfortunately, I've not understood all the details of the authentication procedure.

My question is the following : Have you implemented basic authentication actions (like requesting a token or checking its validity) in the motu-client REST-API? If so, could you please edit the documentation to make the authentication procedure more transparent to a user of the API?

Thank you in advance, Sebastien Legrand

clstoulouse commented 4 years ago

Motu is an open source software used in several contexts. Each context has its authentication requirements. One of them is CMEMS (http://marine.copernicus.eu) and in this context accessing datasets requieres an authentication using the SSO CAS protocol (https://apereo.github.io/cas/4.2.x/protocol/CAS-Protocol.html). As it is complicated for an end user to manage this authentication directly with the REST API, we have developed a client, in python, easier to script to download datasets (https://github.com/clstoulouse/motu-client-python). Also if you develop in Java, you can easily download datasets (https://github.com/clstoulouse/motu-client-java). But using the REST API directy is a really tricky task to succeed with all redirects. Not impossible but difficult enough. To respond, no, there isn't any basic authentication action, only the SSO CAS protocol in CMEMS context.

In which language do you develop?

Best regards, CLS motu team !

SebastienDLegrand commented 4 years ago

Thank you for your swift answer!

Over the years, I've been using the python motu-clients v1.0.2, v1.0.8, v1.4.0 and v1.6.0 as black boxes. Their calls were encapsulated either in bash, perl or python-3 scripts. In the last 2 cases via a os/system call. I've never been fan of these scripts, but well, if we except the first few months of the myocean project for which products could be directly downloaded via THREDDS servers hosted in the different production units, the motu client has been being the only way to automatise the download of myOcean/CMEMS product subsets.

After having exchanged with the CMEMS service desk around a SERVICE_ID that was unknown by the motu client 1.6.0 earlier this week, I've discovered that I still have to upgrade my client to version 1.8.3. However, as other CMEMS champion users, I'm quite upset and bore to have to install and maintain a client for dialoging with a server, while more transparent solutions using REST-API (e.g. THREDDS, OPENDAP, ERDDAP,...) exist. So I've sought deeper and have discovered yesterday that motu server also has a REST API...

I'm going to further explore how to work with the SSO CAS protocol.

If I succeed to submit a download action via a curl command, I should be able to directly use the motu API with the numerous software that works with the opendap protocol.

Thanks again for the swift answer!

A last question : How stable is the API syntax?

clstoulouse commented 4 years ago

Dear, OK I understand better your usage, but you have to deal with CMEMS security. It is feasible, you just have to perfectly control the http redirection (302), cookies and tokens. But the risky development of being tedious. Good luck if you succeed, thank you for sharing your code on GitHub :-)

eblondel commented 4 years ago

Hello, is the SSO CAS protocol also the authentication method set to access data through Motu WCS server? I would like to access CMEMS data programmatically but through WCS standard protocol instead of Motu API. Thanks in advance for your advice

clstoulouse commented 3 years ago

@eblondel Yes the SSO CAS protocol is the only one to access data through Motu WCS server on CMEMS.

eblondel commented 3 years ago

Thanks @clstoulouse for your reply, I've finally figured out out to connect through the SSO CAS protocol. Last but not least: about CMEMS Motu WCS server (which is extremely useful for sustainable and standard data access), do you know if this is regularly updated with updated data coverages (eg update of spatio-temporal coverages)? and enriched with new data coverages?

clstoulouse commented 3 years ago

@eblondel temporal coverages are updated for NRT datasets. When there is a new spatio coverage, there is a new associated dataset. A dataset keep its spatio coverage fixed, only its temporal coverage is updated.

eblondel commented 3 years ago

Great, thank your prompt reply