Bad experience when viewing console logs for a resource with a lot of logs

[JamesNK]

Is there an existing issue for this?

• [x] I have searched the existing issues

Describe the bug

Visiting the console logs page causes Aspire to load logs. For DCP resources, the host calls DCP get to get its old logs, streaming them from beginning to end, and then continuing to stream new logs as they arrive.

The problem with this the order doesn't match what is shown in the Aspire. Aspire wants to shows the latest logs by default, with a scrollbar to go back and view older logs.

By streaming logs from the beginning, the UI rapidly updates as it quickly tries to render logs from beginning to end, causing the UI to freeze, lock up, and generally behave badly. A resource with 10,000 logs does a lot of UI updates for lines 1-9900 before 9901-10000 are actually displayed.

Expected Behavior

Ideally:

1. The dashboard quickly displays the latest logs.
2. The dashboard streams new logs as they arrive.
3. The dashboard loads historical logs in the background.

A couple of solutions I see:

1. The current DCP method to get console logs is tweaked:

   1. Add a parameter to limit the amount of historical logs returned. The dashboard never attempts to show more than 10,000 logs in the past so this could be passed to provide an upper limit on the amount of data DCP attempts to return.
   2. The DCP method adds a header with a count of the current number of logs available. This could be used by Aspire to display a loading indicator in logs. The console logs wouldn't start being displayed until the amount of lines received from DCP equals this line count. This change would stop the UI from trying to display older logs as they're returned. The downside is there would be a delay while displaying the loading indicator as old logs are loaded.

2. Another option would be for there to be two calls to get log data from DCP:

   1. One call to stream the last 100 logs, plus new logs. This call would need to return a total log count in the header so the line number is accurate when logs are initially displayed.
   2. Another call to stream historical logs. It would load old logs in the background.
   3. NOTE: Aspire host doesn't use the order that logs are received for ordering, just the date. Theoritically there could be one API call that simulatanously returns logs from newest to oldest while at the same time returning latest logs.

I think option 1 is probably easy both from hosting and DCP perspective. Option 2 would create a better experience by making logs consistently load quickly.

Steps To Reproduce

No response

Exceptions (if any)

No response

.NET Version info

No response

Anything else?

No response

[JamesNK]

cc @davidfowl @mitchdenny @karolz-ms @danegsta

[mitchdenny]

We definitely don't want to see the Blazor UX freeze up, nor do we want the backend of the dashboard or the apphost to eat a lot of resources just for logs that the user doesn't want to see. So anything we can do to optimize this is worthwhile.

One thing I thought about is whether we could do this work as part of adding other value to the console logging experience. For example, showing the last x log lines is really a filter function, but I am wondering if we could add text search over the logs as part of this?

The other dimension to this is that I know that @DamianEdwards and @davidfowl want to support a model where the dashboard is a persistent resource.

[JamesNK]

Text search wouldn't be difficult. Logs are in memory after loading, so the work would mostly be in the UI. But this issue is focused on loading the logs.

Dashboard as a persistent resource doesn't matter either way. The dashboard doesn't keep all logs in-memory even when it's running. Navigating away from the console logs page clears them from memory.