Make the live build logs a stream

[fedora-copr-github-bot]

Original issue: https://pagure.io/copr/copr/issue/88 Opened: 2017-06-17 13:24:55 Opened by: ngompa

Currently, the live build logs are written in such a way that when you curl them, it reaches the end and stops. This is contrast to how OBS does it, in which when you curl an in-progress build log, curl keeps receiving data and appending to the stream (as if it's actually writing to stdout).

Please make the live build logs work that way, as it makes watching them much better.

---

praiskup commented at 2017-06-19 06:58:12:

I'm not aware of OBS details, but note that the log is transferred from builder to backend, and backend (from the end-user perspective) is "just" rock solid http server; at least nowadays. Also, builders are not accessible from the outside. Can this feature be implemented without adding specific streaming server (daemon) onto backend side, only because of live logs?

---

ngompa commented at 2017-06-19 13:19:44:

@praiskup My understanding is that what makes it a stream is that it has no EOF, so curl doesn't quit and keeps pulling stuff as it updates.

---

praiskup commented at 2017-06-20 06:22:56:

Hms, the log on backend is just result of ssh remote-builder '/usr/bin/tail -F -n +0 --pid=<PID> log_file'. This is "stream" from the OS perspective; the question is how to configure http server to allow http streaming without implementing streaming daemon.

---

msuchy commented at 2017-06-20 10:17:32:

This can be temporary workaround: https://curl.haxx.se/mail/lib-2009-04/0425.html

[praiskup]

@TomasTomecek, has something similar been investigated in Packit? Do you have some research page for live logs?

[nikromen]

I found only https://github.com/packit/research/tree/main/logs-aggregation

but I remember some discussion around live logs. Maybe remember is not a good word... It's rather feeling than a memory :D

[TomasTomecek]

We have never researched this properly.

I can only link to Testing Farm's log viewer: https://gitlab.com/testing-farm/oculus

though it doesn't seem to provide logs live: https://gitlab.com/testing-farm/oculus/-/blob/main/results.html#L122

Let me see if cockpit has something like this.

Edit: yup, cockpit seems to have code to display logs live: https://github.com/cockpit-project/cockpit/blob/13ae34675e82584bd350b8151e00b89a8c132ba1/pkg/lib/journal.js#L133

https://github.com/cockpit-project/cockpit/blob/bfbdd8bcdbd010cbcf673ec2d8b76ddd28227f1b/examples/long-running-process/index.js#L12

[TomasTomecek]

Actually zuul has a very nice live log streaming: https://opendev.org/zuul/zuul/commit/bb352a3559d75e816d1f9fd9a645fb41dee8ce10?style=unified&whitespace=ignore-change

[praiskup]

From a different issue: https://pagure.io/copr/copr/issue/1075 Opened: 2019-10-19 03:14:12 Opened by: m4rtink

Live build log in the terminal

At the moment it is rather cumbersome to watch live logs from builds with copr-cli - even though copr-cli returns returns a link to the build, one needs to click it manually and then hunt the link for the actual live build log. Also as far as I can tell the log file does not refresh automatically (like for example the live build log file does on OBS) and one has to refresh it manually.

This is especially cumbersome for packages that take a while to build as even though the the copr-cli output does inform you of the build finishing/failing, it is much less noticeable than a live log in the terminal stopping with error. Also an error can be easily inspected in this case, rather than hunting for the link and then finding the log file & inspecting it for the error message.

How this could work This should likely by optional & will require user input anyway in case the build runs on multiple chroots to set which log should be shown.

So I propose adding a new option --show-build-log to the copr-cli build sub-command:

if --show-build-log is passed and the build runs on a single chroot, the live build log from this chroot would be shown in the terminal
the --show-build-log option can also be used to pass chroot name, to select which log should be shown in case of multi-chroot builds, for example:

copr-cli build osm-scout-server --show-build-log=fedora-30-x86_64 valhalla-3.0.8-8.fc30.src.rpm

if --show-build-log is passed without chroot name and the build runs on multiple chroots one is selected automatically (say by lexicographical ordering), alternatively this could fail with error either if it is possible to detect beforehand if a build will run on multiple chroots or simply always if no chroot name is passed to --show-build-log (pretty much depends on what is the most convenient to implement in copr-cli)

if --show-build-logis not passed, nothing changes from the current behavior

[Venefilyn]

Could we do something like using SocketIO?

Would work (I think, never worked with WebSockets) like this:

• User goes to the Copr log frontend
• Frontend opens a WebSocket connection to the server
• Server, with it enabled and supported, opens the WebSocket stream
• Server send updates as soon as the log changes (with a minimum delay of 300ms to not send updates all the time)

Anyone who wants to use this could then also open their own connection using WebSockets, so it could be supported within for example Packit when viewing logs

@martinpitt you have worked with WebSockets on Cockpit right? Is the assumption I make sorta correct?

[martinpitt]

Websockets are overkill for this IMHO -- they are meant for a two-way communication channel, but for log tailing you only need the "normal" HTTP reading direction.

On the server side, you would use chunked encoding for streaming, instead of announcing Content-Length:. On the client (web page) you can then either show the log URL as embedded iframe, or use JavaScript with the fetch() API.