Shared Services HTTPD proxy Work (CS)

[galenatjpl]

Create, deploy and configure the Apache HTTPD proxy on shared services Unity-Prod account. Note, this is separate from the venue account HTTPD proxy. This proxy should front the UIs on the shared services. This httpd instance should also be configured with the cognito auth module.

To be specific, this is referring to the shared services account seen at this top of this diagram:

Note, this HTTPD is a dependency for accessing things in the Venue account (not shown for prod in the diagram, but the same concept as the bottom of the diagram).

[galenatjpl]

@rtapella and/or @mike-gangl : What are the UIs that will be part of this release, that need to exist on the Shared Services account? Note: in the past we have only talked about having httpd exist on the venue account, which is pointed to from the shared services CloudFront. However, in creating this ticket, I got to thinking that we also need httpd on the shared services to proxy to UIs like (Algorithm Catalog?). Please help me understand what these interfaces are.

[rtapella]

I believe all the home-grown UIs will be in venues. The Algorithm Catalog (Dockstore) (U-ADS) is one thing that may be common across venues, and possibly the Data Catalog (STAC Browser) (U-DS), depending on how we can control authorization for each of those.

[galenatjpl]

@rtapella , thanks. Based on your response, it does seem like we will have a few UIs in the shared services (e.g. dockstore, and STAC Browser). So I do think we need the shared services httpd. @ramesh-maddegoda and @buggtb do you agree?

[buggtb]

If its a simple website then you could pipe it through API Gateway potentially, or we can just chuck a HTTPD in there and proxy it all through.

On Tue, Jan 23, 2024 at 4:41 PM Galen Hollins @.***> wrote:

@rtapella https://github.com/rtapella , thanks. Based on your response, it does seem like we will have a few UIs in the shared services (e.g. dockstore, and STAC Browser). So I do think we need the shared services httpd. @ramesh-maddegoda https://github.com/ramesh-maddegoda and @buggtb https://github.com/buggtb do you agree?

— Reply to this email directly, view it on GitHub https://github.com/unity-sds/unity-cs/issues/315#issuecomment-1906475095, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAZI6APBTU3W6EWWAFHSNDYP7R3RAVCNFSM6AAAAABCCVFA32VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBWGQ3TKMBZGU . You are receiving this because you were mentioned.Message ID: @.***>

[mike-gangl]

Dockstore might require httpd integration, i know there is a lot of stuff going on in that, but we also don't have a lot of resources into that deployment right now after we lost some personnel to other projects.

[galenatjpl]

It sounds like we probably want to go the direction of this ticket, which is to install/configure httpd on the shared services, and use it for things like Dockstore.

[rtapella]

@galenatjpl is this "the" issue that is gating httpd/reference app/etc. stuff? If so can we mark it as a depedency please

[GodwinShen]

@galenatjpl and @ramesh-maddegoda ping for status.

[ramesh-maddegoda]

@galenatjpl and @ramesh-maddegoda ping for status.

@GodwinShen (CC: @galenatjpl ), I have deployed a new version of httpd in Unity Shared Services Dev account and proxied few https URLs through it. Howver, according to our design diagrams, we need https servers in venue accounts too.

I will share the same comment I posted in Slack and also in https://github.com/unity-sds/unity-project-management/issues/170, for easy reference:

I think the httpd proxy was assumed as an already existing service even before implementing and deploying it on shared service account. We had it in design "diagrams", but in fact it was deployed in Unity Dev (shared services) last week. Currently I am troubleshooting some issues related with it, when I try to write URL rewriting rules with it.

Proxying a website behind httpd is as simple as follows:
ProxyPass "/"  "http://www.example.com/"
ProxyPassReverse "/"  "http://www.example.com/"

There is a user guide on that: https://httpd.apache.org/docs/2.4/howto/reverse_proxy.html

However, when we proxy a website, in many cases usually it will have many missing links due to absolute paths used etc. So we have to use URL rewriting. URL rewriting needs some understanding of the syntax and standards. https://httpd.apache.org/docs/2.4/rewrite/intro.html

While the developers are waiting to use Unity Dev httpd server, they can try this locally using httpd docker image located at https://hub.docker.com/_/httpd

Then, they can use the local httpd to write the proxy rules and URL rewrite rules (because the URL re-writing rules can be application specific). When they are ready to deploy the app, we can slightly modify reuse the same rules in Unity Dev httpd.

[rtapella]

@anilnatha do we need to plan anything re: URL re-writes vs. React routes?

[ramesh-maddegoda]

The following list of related tickets were completed in Unity Shared Services Dev and Unity Venue Dev.

https://github.com/unity-sds/unity-cs/issues/404 https://github.com/unity-sds/unity-cs/issues/405 https://github.com/unity-sds/unity-cs/issues/406

The same setup can be deployed in Unity Prod too.

[galenatjpl]

@ramesh-maddegoda , I think we want to prove that we can connect to the STAC browser in shared services, as well as authenticate via Cognito, to be able to close this ticket off. So probably the next steps is to configure that, and demo it in an upcoming Unity meeting. Maybe @mike-gangl or @ngachung can help provide some details about what is currently deployed on Unity-Dev in terms of the STAC browser. I'm not familiar with this myself.

[galenatjpl]

@ramesh-maddegoda Can this be closed now? Any final testing that's needed?

[galenatjpl]

This is now complete. Ramesh tested access to the stac browser on unity-prod. The ALB & HTTPD EC2 is now installed.