Closed data-doge closed 1 month ago
It looks like we're seeing the error again β this specific veteran was able to submit successfully though (selected no evidence as a part of their form, so no evidence lost π )
Another instance of it
Recent occurrence
Investigation in the works
Investigation notes:
Suggestions/possible next steps:
rescue
block to the PersonalInformationLog#create!
call so that we can properly handle whatever error is being thrown there if there is one. It's possible that isn't exactly where the recursion is happening, but it might be a start.
irb(main):019> PersonalInformationLog.where('error_class LIKE ?', "%V1::SupplementalClaimsController%").order(created_at: :desc).count
=> 1
In code review. Poking platform.
PR was merged, working on validating but may take a few days as we don't see this error too frequently
Saw an instance of an error that leads to the stack level too deep
:
Exception occurred while submitting Supplemental Claim: 1 validation error detected: Value at 'plaintext' failed to satisfy constraint: Member must have length less than or equal to 4096
Not sure what this means, but it's a start
Going to add the extra logging to NOD and HLR too since we saw a couple of stack level too deep
errors for those forms as well π©
May open a new ticket to investigate errors from the logs
May have figured out what is possibly causing the stack level too deep
error in Supplemental Claims, which is the form that most frequently encounters it, so I'll make a ticket to resolve it. But not sure what have caused it in NOD and HLR, but we can monitor for it now that we've added the extra logging
Haven't yet encountered the stack level too deep
error from NOD and HLR since we've added the logging but upon inspecting their traces, they both happened around the same time and both share this error:
PG::UndefinedColumn: ERROR: column form_submissions.in_progress_form_id does not exist
LINE 1: ...submissions" SET "in_progress_form_id" = $1 WHERE "form_subm...
^
/usr/local/bundle/cache/ruby/3.3.0/gems/ddtrace-1.23.2/lib/datadog/tracing/contrib/pg/instrumentation.rb:35:in `exec_params': ERROR: column form_submissions.in_progress_form_id does not exist
LINE 1: ...submissions" SET "in_progress_form_id" = $1 WHERE "form_subm...
^
(PG::UndefinedColumn)
from /usr/local/bundle/cache/ruby/3.3.0/gems/ddtrace-1.23.2/lib/datadog/tracing/contrib/pg/instrumentation.rb:35:in `block in exec_params'
...
So going to look into this a bit and see if there are any other instances of this error π
Gonna close this since there's a new ticket to cover the fix for SC and we haven't seen the stack level too deep
error in HLR and NOD occur again with the new logging (yet) β if we do encounter the error for those forms in the future, we can create a ticket as we'll be able to find more details around what may be causing it for them.
In Dec. 2023, we began to receive "SystemStackError: stack level too deep" errors on the Supplemental Claims form. See Sentry. This error is thrown, synchronously, at some during the creation/submission of a Supplemental Claim. Which makes it a submission-blocking error, but not a silent error. Interestingly, the error we see here is the result of our error handling code erroring. So we are unable to see the underlying reason for these errors. There is some discussion of this error here: https://dsva.slack.com/archives/C05FCEH2NRG/p1704901615183509.
An update (Feb 13), this error no longer appears to be occurring.
Another update: this error does appear to be occurring again.
Update 4/17/24: Refer to Derek's comment below for more details, but it looks like it's also happening in HLR
Tasks