search

unity-sds / unity-project-management

Container repo for project management (projects, epics, etc)
Apache License 2.0
0 stars 1 forks source link

Data Catalog Browser #117

Closed mike-gangl closed 2 months ago

mike-gangl commented 5 months ago

Data Catalog Browser

Deploy a data catalog browser as shared service endpoint to consume DAPA search requests.

Acceptance Criteria

Work Tickets

Link to work tickets required to implement the epic

Dependencies

Other epics or outside tickets required for this to work

Associated Risks

links to risk issues associated with this epic

GodwinShen commented 5 months ago

@rtapella is there something you need to do to support this item "Token generation method added to unity_py? Or can we integrate the authentication with a cognito login?" Or maybe @ramesh-maddegoda needs to help? I'm not sure, right now @ngachung is working this ticket but it seems like this item in particular is not in her team's scope.

rtapella commented 5 months ago

it depends on how this works. basically we need to get the STAC Browser to find/see data... so if we need to add something to Unity.py for that, then yes I agree it'd be Anil and/or Mike. If it's something that is handled directly by STAC Browser then I think Nga/team should be able to take care of it

GodwinShen commented 5 months ago

OMW, finally posting this using my JPL-related github account and on the correct ticket, sorry about all the duplicates of this comment...

@mike-gangl I'd love to hear your thoughts on this conversation, this is in regards to the token passing for STAC browser. Seems there is a fork in the road for either: (1) token generation method added to unity_py as a user-assist tool; (2) integrate with cognito login/authentication. I'm not sure the pros and cons of each and what work each option would require. What do you think?

mike-gangl commented 5 months ago

ideally there is a cognito login integrated into the stac-browser. My initial attempts at the STAC Browser allowed me to provide a token, but not integrate with an SSO provider (cognito). This is why we had the effort to create the token as a separate process so it could be used in the stac browser.

galenatjpl commented 5 months ago

@mike-gangl @ngachung . We (mostly @ramesh-maddegoda ) has proof-of-concepted a httpd integration with cognito using a cognito module in httpd. I think probably the best approach for this ticket is to add to the acceptance criteria, that this should be fronted (proxied) by a httpd proxy in the shared services venue. This is what the work in https://github.com/unity-sds/unity-cs/issues/315 represents.

ramesh-maddegoda commented 5 months ago

@mike-gangl and @galenatjpl , the Apache httpd module mod_auth_openidc that we will use, will give us a blanket authentication for all the users in a specific Cognito pool. But I can see this ticket talks about a Cognito token. The Apache httpd module will just provide blanket authentication and NOT a Cognito token. On the other hand I have developed examples to get a Cognito token in a ReachJS app https://github.com/unity-sds/unity-cs-security/tree/main/code_samples/hysds_ui_with_auth. Also, I have documented ways to get Cognito tokens in python and command line https://github.com/unity-sds/unity-cs/wiki/Getting-Cognito-JWT-Tokens-in-Command-Line

GodwinShen commented 5 months ago

@mike-gangl What would you propose as a path-forward in light of this new information from @galenatjpl and @ramesh-maddegoda ?

galenatjpl commented 5 months ago

@mike-gangl and @galenatjpl , the Apache httpd module mod_auth_openidc that we will use, will give us a blanket authentication for all the users in a specific Cognito pool. But I can see this ticket talks about a Cognito token. The Apache httpd module will just provide blanket authentication and NOT a Cognito token. On the other hand I have developed examples to get a Cognito token in a ReachJS app https://github.com/unity-sds/unity-cs-security/tree/main/code_samples/hysds_ui_with_auth. Also, I have documented ways to get Cognito tokens in python and command line https://github.com/unity-sds/unity-cs/wiki/Getting-Cognito-JWT-Tokens-in-Command-Line

Hi @ramesh-maddegoda I'm not sure if you are talking about "JWT token" when you say "Cognito token". I assumed that httpd, after authenticating using mod_auth_openidc, would then have a JWT token, and this token can be passed forward to the target of the proxy. Is that not the case?

ramesh-maddegoda commented 5 months ago

@galenatjpl , Cognito token is a JWT token. The module mod_auth_openidc is an existing Apache module which does the authentication under the hood. It has some config to setup Cognito login for a website location.The mod_auth_openidc will check if the user is coming from a valid Cognito pool and if valid, allow the user to access the website. By default, there is no token passing functionality in mod_auth_openidc.

However, there is a cache feature in mod_auth_openidc (https://github.com/OpenIDC/mod_auth_openidc/wiki/Caching). In that cache the following information is cached:

We have to implement functionality to read the cache and retrieve values.

galenatjpl commented 5 months ago

@ramesh-maddegoda thanks for the further information. I'm still looking for a sequence of events that will result in the JWT token getting passed forward to the target (for example the ALB fronting the Management Console). This is because certain applications will still want to inspect the JWT token for various purposes (like fine-grained auth stuff). So are you saying that something will be implemented that will: 1) read the cache to retrieve the JWT token 2) pass the JWT token along with the request (for example to an ALB) ? I'm sort of concerned that this seems like a custom approach, but maybe this is the standard way of doing this? Thanks!

rtapella commented 5 months ago

Thinking about this more @GodwinShen ... I am hoping the STAC browser doesn't need Unity-py to function. Not sure your perspective @mike-gangl @ngachung @anilnatha ...

It should be able to use STAC and Cognito directly. The Jobs Dashboard web app uses SPS (WPS-T) and Cognito directly AFAIK.

We could have some additions to Unity-py to work with data via DAPA or STAC (when using Jupyter/python scripts) but it probably would not be incorporated into the STAC Browser.

mike-gangl commented 5 months ago

I think we're over complicating this before we see if this is even useful or works with our STAC catalog.

  1. Deploy the STAC browser in the shared services account and make it reachable.
  2. Configure the STAC Browser to use bearer type authentication
  3. have the user, for now, use the unity-py command to generate a token to input into the form for authentication

yes, fronting this with cognito and having it be seamless would be ideal, but that's work we can do after we ensure the stac browser works and gets most of our use cases.

We have to remember that the data catalog back-end WILL care about the users groups, because it will allow or not allow access to certain collections based on the roles it has.

rtapella commented 5 months ago

Should we break all this out into separate issues so we can allocate work into sprints?

GodwinShen commented 4 months ago

@rtapella have you or anyone else broken this out into separate sprint tickets? If not, can you help do that? I like the way @mike-gangl laid out the work in his latest comment, seems like we just need to (a) create 3 new tickets and (b) assign stuccoes to each ticket.

rtapella commented 4 months ago

I made 3 tickets for this... two are in unity-data-services and I could not edit them once I put them in that repo. One is in unity-py

rtapella commented 4 months ago

Test ticket: https://github.com/unity-sds/unity-py/issues/77

rtapella commented 3 months ago

@ngachung @wphyojpl is this done ? for now at least?

ngachung commented 2 months ago

Functioning STAC Browser with token auth deployed to dev, test, prod. Future enhancements to be tracked in separate issue.