Logging

[AlemSnyder]

Logging needs to be integrated into Lua. The problem is that quill takes the file line and function name at compile time. (Using __FILE__, __LINE__, __FUNCTION_NAME__.) The point of using Lua is that the code is not known at compile time.

Its not just Lua where this is a problem. Errors from opengl files do not show the file. shader.cpp:142 is the line of the logger and not the vertex file with a syntax error.

The original thought was to log all this data again and log unseen chars to remove some of the message. One issue with this is that the log messages are not always the same length. This means that this character will be before the file name. This char will be loged, and may cause issues on some systems.

2023-08-18T17:19:08.670 -0700 [28541           :MainThread      ] [shader.cpp:142    ] ERROR     [shaders       ] - fragment shader error: 0:9(1): error: syntax error, unexpected '}', expecting ',' or ';'

2023-08-18T17:19:05.710 -0700 [28541           :MainThread      ] [terrain_base.cpp:109] INFO      [terrain       ] - End of land generator: place tiles.

If we are going to modify the quill handler, we should modify it in the correct way and allow multiple log formats for one output file.

The reason I wanted to use spdlog is because it allows logging to arbitrary outputs, including a dearimgui window. It was suggested that this can be done with quill, but I don't know how. The only issue is that spdlog is 100 times slower, but it can be done asynchronously so it might not matter.

The options are to

• Use spdlog and except the slowdown
• Fix the quill front end
• Use spdlog, but replace the backend

[egelja]

The reason I wanted to use spdlog is because it allows logging to arbitrary outputs, including a dearimgui window. It was suggested that this can be done with quill, but I don't know how.

Both work the same way. Declare a custom handler and pipe it to IMGUI using a thread-safe queue.

[egelja]

The original thought was to log all this data again and log unseen chars to remove some of the message. One issue with this is that the log messages are not always the same length.

Not an issue. Observe. Messages that are "special" are logged with format "\a{}", while no other messages contain the char \a. Now, with the handler implemented in pseudocode python, you can see how this works.

This is all done in the handler, and so perf doesn't matter. Regardless, strcmp is heavily optimized in glibc and we can even improve perf by advancing the ptr before the call.

[AlemSnyder]

I think we have fundamentally different ideas of what needs to be logged. If there is one log per frame then there is an error, and it doesn't matter how slow the program is as it's about to crash.

1. Is it helpful to log 2.2 X 10^6 messages in one second? No
2. Is it helpful to write a log in 20 vs 200 ns? Not really
3. Is it necessary to know when a lua log was written? probably
4. Is it necessary to know from which thread a lua log was written? probably

3, and 4 cannot be done in the way you have suggested. One would have to do something like this:

msg = msg[0:END_OF_PREAMBLE] + msg[loc+1:]

But I shouldn't worry about the performance as

This is all done in the handler, and so perf doesn't matter. Regardless, strcmp is heavily optimized in glibc and we can even improve perf by advancing the ptr before the call.

This is fantastic as the program will be able to crash one millisecond faster.

Lets be realistic and estimate a chance of adding a bug. You want to do byte compares on an array. You need to make sure that you don't exceed the length of this message. How about if you accidentally log something that is much longer than you would expect. Now the handler needs to search the entire message. In addition you are forced to change the log format.

[egelja]

3, and 4 cannot be done in the way you have suggested.

Yes they can because they will be logged in Lua. So you have two preambles, a C++ and a Lua one. Then you strip the C++ one off.

You want to do byte compares on an array. You need to make sure that you don't exceed the length of this message.

memchr is safe.

How about if you accidentally log something that is much longer than you would expect. Now the handler needs to search the entire message.

It won't be longer than a kB. All of this is peanuts compared to I/O time.

In addition you are forced to change the log format.

Nope. And changing out your logging infra will take longer.

[AlemSnyder]

Yes they can because they will be logged in Lua. So you have two preambles, a C++ and a Lua one. Then you strip the C++ one off.

The reason quill is fast is because almost everything is inlined or known at compile time. This information must be collected in the calling thread, removing the other benefit of quill.

memchr is safe.

Feel free to implement this, but this will be relatively buggy. I just want it to look nice as most people don't log across languages, but it will probably look bad and barely function. If it does function it will be horrible code, and drowning in technical debt.

Nope. And changing out your logging infra will take longer.

You want to change the quill source code, but can't even compile it. The logging infrastructure is maybe two files.

If you want to implement this char find as part of the logging library, go ahead, but this method seems slow and bug prone.