Show backend logs in Backstage UI

pgpbpadilla commented 2 months ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

What is the outcome that you are trying to reach?

I am working on my custom template for a new component. The registration fails and I cannot quickly figure out what the error is, I want to look at the logs generated while trying to process my catalog file, but to look at the server logs I need to go into cloud watch and look at the ECS task logs, which is quite a bit of work.

Describe the solution you would like

I want to be able to see the logs for all Harmonix components in the backstage UI. I know there are some plug-ins that can be used to do this so it would be just a matter of enabling them by default.

New to backstage and to harmonics, so I leave it up to you to choose the better plug-ins to achieve the desire to result, namely, to allow debugging issues for any actions without having to leave the backstage UI.

Describe alternatives you have considered

Additional context

z-sourcecode commented 2 months ago

Hi @pgpbpadilla Thanks for reaching out. You are correct, we rarely use the ECS cluster/service/task logs for debugging development work, Instead we configure our machines for local development - you can run backstage locally with a local postgres db(docker) and still use your deployed git.yourdomain.com.

please check this documentation for running harmonix and backstage locally - https://harmonixonaws.io/docs/techdocs/customizations#running-locally

Regarding the creation of your template - if you duplicate the directory of a template - make sure the names are different and don't forget to register the new template in your git opa-admin repository: opa-admin/backstage-reference/templates/all-templates.yaml

you can use the below make command to update your local changes with the remote git. make push-backstage-reference-repo

pgpbpadilla commented 2 months ago

Hello, do you think it's possible to embed the logs from all of the deployed components and present them through the backstage UI in a standard way?

Ideally this should work no matter where the components are deployed. I can imagine that there could be adapters for different deployment environments, for example EC2, ECS, EKS, etc.

Unfortunately I don't have any examples of this type of standard interfacae but it seems to be like it would be a really good feature to have for troubleshooting live deployments.

pgpbpadilla commented 2 months ago

By the way, I would still classify this as a feature request and not as a question. Feel free to close it if this is not something that you have in your roadmap.

z-sourcecode commented 1 month ago

Hi @pgpbpadilla apologies for the delay. Regarding your question - aggregating logs from various layers of the application is possible and can be integrated in the UI. However, we do not have it planned at the moment.

Consider that the applications logs will likely to be the most interesting for the application developer(Affected by the app code changes /src), The deployment logs or mostly relevant when building new patterns or templates or for maintenance and diagnostic.

We have a customer who integrated Kubernetes cluster plugin on the provider page - thus giving platform engineering an overview of the entire cluster operations. We also received a similar requirement for ECS cluster overview (not yet available).

I'll keep this open for visibility and for updates if anything changes. thanks!

awslabs / harmonix