robustness in laravel queue - handling corrupted payload

[kikkauz]

Back in December I opened the bug https://github.com/laravel/framework/issues/22513 basically when the paylod of the job is corrupted, the job is tried indefinitely till reaching the max size of the DB field. Some error handling in the payload parsing should do the trick.

[mfn]

the job is tried indefinitely till reaching the max size of the DB field

I guess you mean when using the queue via a DB driver? (ok, it's more clear when reading the referenced issue).

The redis driver would have not such a limit, but probably one when the max representable Integer was reached.. which would have worsened your experience I guess.

You wrote in the other issue (the title)

max retry not working when the payload is not parsable

But how did the payload actually got corrupted? I undertand, the payload was too big but didn't the DB (MySQL?) failed to completely write the record? Did the DB silently truncate it, making it invalid?

[kikkauz]

The payload got corrupted because of a missing pdo configuration that caused the content to be truncated at 1mb. So this was clearly a mistake on my side - but it took a while to understand it as the job was retried till the DB field size was reached.

If I was using Redis I would have not had the issue at all as the whole payload would have been stored. But if for whatever reason the payload was corrupted it would have been impossible to understand what was happening.

My point is simply that there is a part of the code where possible errors are not well handled. Let's say that a language like java would have not allowed it because there would have been an exception not correctly handled.

[sisve]

@mfn You have way to high thoughts about mysql. Mysql will truncate your data, it will accept invalid values, and it will laugh at you while doing it. It's only with the recently introduced strict mode that mysql admits that "aha, maybe I actually need to care about these things".

Another related problem is that people now disable the strict mode because it interferes in how they write their group by clauses. Related: (My)SQL mistakes. Do you use GROUP BY correctly?

So, in short, it's very easy to have mysql truncate your strings without throwing any exceptions.

Strict mode controls how MySQL handles invalid or missing values in data-change statements such as INSERT or UPDATE. A value can be invalid for several reasons. For example, it might have the wrong data type for the column, or it might be out of range. A value is missing when a new row to be inserted does not contain a value for a non-NULL column that has no explicit DEFAULT clause in its definition. (For a NULL column, NULL is inserted if the value is missing.) Strict mode also affects DDL statements such as CREATE TABLE.

If strict mode is not in effect, MySQL inserts adjusted values for invalid or missing values and produces warnings (see Section 13.7.5.40, “SHOW WARNINGS Syntax”). In strict mode, you can produce this behavior by using INSERT IGNORE or UPDATE IGNORE.

Source: https://dev.mysql.com/doc/refman/5.7/en/sql-mode.html#sql-mode-strict, my emphasis

[mfn]

@sisve thanks, I'm a (happy!) recent MySQL→PgSQL convert and know of such problems 😄

Just wanted to better understand the issue. I'm also a "mid-volume" background Worker user of Laravel and we also have seen a few problems (also with Horizon) so I'm observing this problem space diligently to learn about these problems.