Help setting up log for Laravel?

[hailwood]

Hey there!

Great plugin, super useful for quickly viewing log files... Well it was. But it seems Laravel doesn't match the default provided formats anymore so I was hoping we could get some help.

The general format looks like this:

[2018-08-02 03:03:14] local.ERROR: SQLSTATE[42P01]: Undefined table: 7 ERROR:  relation "product_id" does not exist
LINE 1: insert into "product_id" ("customer_id", "product_id") value...
                    ^ (SQL: insert into "product_id" ("customer_id", "product_id") values (1838, 2149)) {"exception":"[object] (Illuminate\\Database\\QueryException(code: 42P01): SQLSTATE[42P01]: Undefined table: 7 ERROR:  relation \"product_id\" does not exist
LINE 1: insert into \"product_id\" (\"customer_id\", \"product_id\") value...
                    ^ (SQL: insert into \"product_id\" (\"customer_id\", \"product_id\") values (1838, 2149)) at /app/vendor/laravel/framework/src/Illuminate/Database/Connection.php:664, Doctrine\\DBAL\\Driver\\PDOException(code: 42P01): SQLSTATE[42P01]: Undefined table: 7 ERROR:  relation \"product_id\" does not exist
LINE 1: insert into \"product_id\" (\"customer_id\", \"product_id\") value...
                    ^ at /app/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php:144, PDOException(code: 42P01): SQLSTATE[42P01]: Undefined table: 7 ERROR:  relation \"product_id\" does not exist
LINE 1: insert into \"product_id\" (\"customer_id\", \"product_id\") value...
                    ^ at /app/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php:142)
[stacktrace]
#0 /app/vendor/laravel/framework/src/Illuminate/Database/Connection.php(624): Illuminate\\Database\\Connection->runQueryCallback('insert into \"pr...', Array, Object(Closure))
#1 /app/vendor/laravel/framework/src/Illuminate/Database/Connection.php(459): Illuminate\\Database\\Connection->run('insert into \"pr...', Array, Object(Closure))

Of note is that the new log message always starts with [{date} {time}] {category}.{severity}: Also, there are some lines that start with a [ that are not new entries e.g. [stacktrace].

Another example of an info log entry would be:

[2018-08-02 03:02:42] custom.INFO: Successfully imported customer 123456
[2018-08-02 03:02:42] custom.INFO: Successfully imported customer 653512

So, I'd love some assistance in what patterns I should use to support this format?

[atamansv]

Found describing log format for another tools for symfony logs (similar to laravel) https://github.com/n-educatio-pl/logstash-symfony-pattern/blob/master/patterns/symfony

^\[%{TIMESTAMP_ISO8601:date}\] %{SYMFONY_LOG}: %{SYMFONY_LOG_MESSAGE:log_message} (\[\]|%{SYMFONY_CONTEXT:log_context}) (\[\]|%{SYMFONY_REQUEST_PARAMETERS:log_request}) need to replace constants SYMFONY_LOG, SYMFONY_LOG_MESSAGE...

[atamansv]

Try this: Message pattern: ^\[(.*)\] (.+?)\.([A-Z]+): (.*) Message start pattern: ^\[ Time format: yyyy-MM-dd HH:mm:ss Time capture group: 1 Severity capture group: 3 Category capture group: 2

[hailwood]

@atamansv Thanks for the attempt, it looks like it should work, unfortunately though it doesn't seem to be matching anything. Not too sure why as it looks like it should.

[dcdiogo]

This is the config that I'm using.. not the best I think, but it works in almost cases..

[hailwood]

@dcdiogo pretty sure that's going to act super strange on stacktraces in the log though yeah?

[dcdiogo]

yeah, unfortunately, but, like I said on other thread, not the best, but work in almost cases (I use for Laravel, and it prints colored first line os stack trace only, witch is the word ERROR, WARNING, INFO, etc)

[ccondrup]

I used what @dcdiogo wrote, with these changes: Log Format Message pattern: ^\[([^]]+)]\s?([^.]+)\.([\w]+):(.*)$ Error pattern: ^[^.]+\.(ERROR):

These are crude but quite fast on my Ubuntu 18.04, PhpStorm 2018.3

[alperyazgan]

Log Format Message pattern: ^\[([\d\s:-]+)\] ([^.]+)\.([^:]+): (.*)$ Time Capture Group 1, Severity Group 3, Category Group 2 Time Format:yyyy-MM-dd HH:mm:ss Patterns: (as @ccondrup recommends) ^[^.]+\.(ERROR): ^[^.]+\.(WARNING): etc. with highlight line option selected
and for stack lines ^(Stack|\#\d)

[litingjun2015]

Try methods provided above, not work in my PhpStorm 2019.1.3 Build #PS-191.7479.51 Ideolog v191.0.12.0

[hatamiarash7]

I can't use them in PhpStorm 2019 ... Any new suggestion ?

[aledmb]

I can't use them in PhpStorm 2019 ... Any new suggestion ?

same here.

any help?

[ielhag]

I can't use them in PhpStorm 2019 ... Any new suggestion ?

same here.

any help?

+1

[jezmck]

+1

Please use the reaction button instead of spamming us all.

[Eric-Ma-Q]

This is the config that I'm using.. not the best I think, but it works in almost cases..

thank you

[Michael-Stokoe]

I'm using Laravel-Log-Enhancer to add extra info to my logs.

Format with enhanced logs:

^\[([\d\s:-]+)\] ([^.]+)\.([^:]+): (.*)(\{*\})$

Format without:

^\[([\d\s:-]+)\] ([^.]+)\.([^:]+): (.*)$

Patterns:

^[^.]+\.(ERROR): ^[^.]+\.(WARNING): ^[^.]+\.(INFO): ^[^.]+\.(ALERT): ^[^.]+\.(DEBUG): ^[^.]+\.(CRITICAL):

[makuro]

To update this here, because I just stumbled upon this.

here is what I use in Laravel 6 to filter the default laravel.[DATE].log:

Message pattern: ^(\[([12]\d{3}-(0[1-9]|1[0-2])-(0[1-9]|[12]\d|3[01])\s([0-1][0-9]|[2][0-3]):([0-5][0-9]):([0-5][0-9]))\])\s*(\w*)\.(\w*):(\w*)(.+)$ Message start pattern: ^\[ Time Format: yyyy-MM-dd HH:mm:ss

this leaves the groups:

Time capture: 2 Severity capture: 9 Category capture: 8 // will be environment

As for the patterns:

.ERROR: .WARNING: .INFO:

[mvalitov]

I set all the settings exactly the same as @makuro , but the error "Log format not recognized" does not disappear

[makuro]

@mvalitov I see in your screenshot you have a full debug output including the stack trace. This is what keeps the error appearing. It needs a specific amount of occurances within a couple of lines to be recognized as a log format.

Try removing all stack trace and have a look at it again, the error message should disappear.

You have different kind of options to populate the informations from within laravel. You can set up your own channel: https://laravel.com/docs/6.x/logging#building-log-stacks or you update app/Exceptions/Handler.php, that is if you are using Laravel 6, and use Monolog ( https://github.com/Seldaek/monolog ) to populate your message to the log file but without the stacktrace.

[mvalitov]

@makuro thank you very much, I set up the log output format, but it is still not recognized

[makuro]

@mvalitov you Need at least 5 recognized lines

[joseffb-mla]

Any one figure out the wordress debug log format?

[14-Apr-2020 21:56:03 UTC] /var/www/html/wp-content/themes/themename/functions.php/287: functions file loaded (File load)

[jezmck]

@joseffb-mla Please make your own new issue for that.

[tflori]

it is still required to have 5 lines in the beginning of the file, right? it is not enough to find the pattern 5 times in the file. it has to be the first 5 lines? or are 5 matches in the first 20 lines ok?

where I'm using ideolog? in the IDE what logs do I watch? from the local environment how big are the logs? huge - it is debug logging how many log lines I'm expecting before the exception with trace? none - I directly try the malfunctioning feature the customer reported

so my local logs are rotated on a daily base and usually start with:

[yyyy-MM-dd HH:mm:ss] local.ERROR: <message>
Stack trace:
#0 <file>(<line>): <call>
#1 <file>(<line>): <call>
#2 <file>(<line>): <call>
#3 <file>(<line>): <call>
#4 <file>(<line>): <call>

IMO it does not make sense to limit it to a specific amount of lines. As this is only checked once the file get's opened it we could run it for the first 20 kb maybe.. and instead of searching for the 5 lines we could have a rank and say: "Monolog" matched two times, "JLogger" matched 1 times, others didn't match - lets use monolog and show a warning "switch log format"

[joseffb-mla]

I figured it out based on the Laravel example. Thanks!

On Wed, Apr 15, 2020 at 3:56 AM Jez McKean notifications@github.com wrote:

@joseffb-mla https://github.com/joseffb-mla Please make your own new issue for that.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/JetBrains/ideolog/issues/42#issuecomment-613879781, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKXFTEZMGBVRYZEJHGBF76LRMVSBBANCNFSM4FNUGILQ .

[mehranabi]

Hello everyone, Is there any recap of this issue? which works the best so far? thanks

[lderkzen]

Hello everyone, Is there any recap of this issue? which works the best so far? thanks

I went through all of them, none of them appear to work. Have you found anything since asking?

[mehranabi]

Hello everyone, Is there any recap of this issue? which works the best so far? thanks

I went through all of them, none of them appear to work. Have you found anything since asking?

Hey, no not really. I haven't found anything and gave up...

[tflori]

I don't know what they did. But I removed that plugin and have a good view on log files now.

[makuro]

Hello everyone, Is there any recap of this issue? which works the best so far? thanks

I went through all of them, none of them appear to work. Have you found anything since asking?

Hey, no not really. I haven't found anything and gave up...

the results here are for Laravel 6 and below. At the moment, this is 2 major versions behind.

in my experience the main issue is, that there are not enough lines available for the color to show up. you would need at least 5 consecutive lines for the plugin to trigger.

try creating a file with 5 lines without linebreaks between and see if the plugin lights up the colors.

if not, please provide a bit more intel here.

[lderkzen]

Hello everyone, Is there any recap of this issue? which works the best so far? thanks

I went through all of them, none of them appear to work. Have you found anything since asking?

Hey, no not really. I haven't found anything and gave up...

the results here are for Laravel 6 and below. At the moment, this is 2 major versions behind.

in my experience the main issue is, that there are not enough lines available for the color to show up. you would need at least 5 consecutive lines for the plugin to trigger.

try creating a file with 5 lines without linebreaks between and see if the plugin lights up the colors.

if not, please provide a bit more intel here.

The logging format hasn't changed too much since then as far as I know. But I could be wrong.

Anyways, I ended up spending some time editing / making my own regex and everything works fine now. Although I do still get the notification about Ideolog not recognizing the format, which I just ignore.

[mehranabi]

@iderkzen Can u please share your work with us?? Really appreciated.

[lderkzen]

@mehranabi Here is the setup I'm currently using. It seems to work for all use-cases I've tried so far.

Log Format

• Message pattern: ^(\[([12]\d{3}-(0[1-9]|1[0-2])-(0[1-9]|[12]\d|3[01])\s([0-1][0-9]|[2][0-3]):([0-5][0-9]):([0-5][0-9]))\])\s*(\w*)\.(\w*):(\w*)(.+)$
• Message start pattern: ^\[
• Time format: yyyy-MM-dd HH:mm:ss
• Time capture group: 2
• Time capture group: 9
• Time capture group: 8

Patterns

Critical: ^[^.]+\.(CRITICAL): Error: ^[^.]+\.(ERROR): Warning: ^[^.]+\.(WARNING): Info: ^[^.]+\.(INFO):

I would love to give credit where credit is due, but I can't remember exactly which parts were edited and which were found somewhere on the internet (maybe even in this issue).

[tflori]

without that plugin and even when the first lines are not log lines it looks like that now:

[Dagsi]

在这里更新这个，因为我只是偶然发现了这个。

这是我在 Laravel 6 中用于过滤默认 Laravel.[DATE].log 的内容：

消息模式： ^(\[([12]\d{3}-(0[1-9]|1[0-2])-(0[1-9]|[12]\d|3[01])\s([0-1][0-9]|[2][0-3]):([0-5][0-9]):([0-5][0-9]))\])\s*(\w*)\.(\w*):(\w*)(.+)$ 消息开始模式：^\[ 时间格式：yyyy-MM-dd HH:mm:ss

这使小组：

时间捕获：2 严重性捕获：9 类别捕获：8 // 将是环境

至于图案：

.ERROR: .WARNING: .INFO:

I want to know how your log file can automatically wrap when the # sign is encountered. Can you help me?

[Dagsi]

没有那个插件，即使第一行不是日志行，它现在看起来像这样：

I want to know how your log file can automatically wrap when the # sign is encountered. Can you help me?

[jyxjjj]

What can i do

[ArtemKar123]

Added in #168. Will be released in the next version. Thanks for your feedback!