search

vercel / next.js

The React Framework
https://nextjs.org
MIT License
126.71k stars 26.94k forks source link

How to catch and handle errors to report logs on server side #1852

Closed acanimal closed 2 years ago

acanimal commented 7 years ago

Hi, I'm in the situation where we want to sent errors, both on server and client side, to Sentry tool.

Our app uses Express as a custom server. Basically we create an express app, apply some middlewares but delegate all the real job to the next.js handle:

  const app = nextJs({ dev: process.env.NODE_ENV !== 'production' });
  const handler = routes.getRequestHandler(app);
  const expressApp = express();

  ...
  ...

  expressApp.use(morgan('combined', { stream: logger.stream }));
  expressApp.use(statsdMiddleware);

  // Add security
  expressApp.use(helmet());

  // Sentry handler
  expressApp.use(sentry.requestHandler());

  // Load locale and translation messages
  expressApp.use(i18n);

  // Next.js handler
  expressApp.use(handler);

  // Sentry error handler.
  // MUST be placed before any other express error handler !!!
  expressApp.use(sentry.errorHandler());

With this approach next.js takes control over the rendering process and any error is catch by next.js and the only way I have to process it is overriding the _error.js page file.

Within that _error.js file I need a universal way to report errors to Sentry. Currently there are two libraries (raven for node and raven-js por javascript). The proble is I can't import both of them because raven works for SSR but fails when webpack builds the bundle, and also raven-js fails due XMLHTTPRequest dependency too.

Is there a way I can be notified for next.js error on server side?

sheerun commented 5 years ago

The only thing one needs in next.js is ability to add custom middleware in next.config.js and something like next.browser.js for universal (isomorphic) plugin configuration (next.config.js is used among others for webpack configuration, which means other things defined in this file cannot be used in application code, because it would cause circular dependency).

For next.browser.js next.js could define configuration like decorator for App or Document components. This way I could implement sentry integration entirely as a plugin.

EDIT: I don't know whether there is ticket for this, but I think @timneutkens is already extracting next server into separate packages. I think best plugin interface would be something like:


module.exports = {
  server: server => {
    server.use(Sentry.Handlers.errorHandler())
  }
}
sheerun commented 5 years ago

I've implemented proposal of such API in #6922

It allows to add decorators to custom server code because contract is changed to one that doesn't automatically call .listen() so it allows next.js to decorate it further before instantiating

leerob commented 5 years ago

I was having a lot of trouble using the with-sentry example, so I opened up a PR for a much more simple example. It doesn't have all the bells and whistles, but it's been working for me.

https://github.com/zeit/next.js/pull/7119

Janealter commented 5 years ago

Try this in custom _document.js:

import React from "react";
import Document, {
  Html,
  Head,
  Main,
  NextScript,
} from "next/document";
import { NodeClient } from "@sentry/node";

const { default: getConfig } = require("next/config");
const { publicRuntimeConfig: { sentryDSN } } = getConfig();

let sentry = null;
if (sentryDSN) {
  sentry = new NodeClient({ dsn: sentryDSN });
}

export default class MyDocument extends Document {
  static async getInitialProps(ctx) {
    const initialProps = await Document.getInitialProps(ctx);
    if (ctx.err && sentry) sentry.captureException(ctx.err);
    return { ...initialProps };
  }

  render() {
    return (
      <Html>
        <Head />
        <body>
          <Main />
          <NextScript />
        </body>
      </Html>
    );
  }
}

And this in custom _app.js:

import * as Sentry from "@sentry/browser";

const { default: getConfig } = require("next/config");
const { publicRuntimeConfig: { sentryDSN } } = getConfig();

if (sentryDSN) {
  Sentry.init({ dsn: sentryDSN });
}

It helps me.

abraxxas commented 5 years ago

Tbh i am very confused on what the correct places to catch errors for next.js are right now. There are _app.tsx, _error.tsx and _document.tsx there are some comments saying that some stuff should be caught in _error.tsx (like: https://github.com/zeit/next.js/pull/5727/files#r235981700) but the current examples use either _app.tsx or _app and _document.

I tried both ways now and it seems that some errors that happen during ssr are not getting caught for me. Which places are now the correct ones to check for errors? The most logical one for me seems to check in the componentDidCatch of _app and the getInitialProps of the _error as this one is being served in cases of errors. I am a bit confused about the whole process.on part in the _document

Could someone with more in depth knowledge please resolve this question?

leerob commented 5 years ago

@abraxxas process.on in _document.js is catching errors that happen on the server. As a recap,

Therefore, when an error happens on the server it should throw an error to Sentry via process.on and then the client will render the default error page or _error.js.

Hopefully this will help: https://leerob.io/blog/configuring-sentry-for-nextjs-apps/

timneutkens commented 5 years ago

_app.js is not client-side only, however componentDidCatch is

leerob commented 5 years ago

@timneutkens Okay, that wrinkles my brain a little bit. I think the docs have been updated since the last time I looked. Could you explain how _app.js is not client-side only in more detail?

abraxxas commented 5 years ago

@timneutkens could you maybe explain where and why you would catch errors? There seem to be so many opinions.

raphaelpc commented 5 years ago

@timneutkens Okay, that wrinkles my brain a little bit. I think the docs have been updated since the last time I looked. Could you explain how _app.js is not client-side only in more detail?

Hello leerob! I just posted a question on the merge request of your example: https://github.com/zeit/next.js/pull/7360#issuecomment-514318899

Adding to that question... I also tried to throw an error in render() on the server side (i started the state with raiseErrorInRender: true, so the first render, that is server sided, would already throw an error), and that error also wasn't captured by Sentry.

Have you tested that in your application, and are those server sided errors really being captured? I'm using Next 9.

If that really is the case, and only the client side errors are being captured, there would also be no reason to override the _document.js file.

Thanks in advance for any help!

raphaelpc commented 5 years ago

@timneutkens Okay, that wrinkles my brain a little bit. I think the docs have been updated since the last time I looked. Could you explain how _app.js is not client-side only in more detail?

Answering your question, _app is where an App component is defined to initialize pages. We would override it in cases like the need for a persisting layout between page changes (all pages utilize the same _app), or to make use of redux. So, even the first render, that occurs on server side, will render the content of _app (it is not client side only).

abraxxas commented 5 years ago

Adding to that question... I also tried to throw an error in render() on the server side (i started the state with raiseErrorInRender: true, so the first render, that is server sided, would already throw an error), and that error also wasn't captured by Sentry.

Have you managed in any other way to get that error to show up in sentry? I tried capturing it in the getInitialProps of _error but that also does show nothing in sentry. Currently i haven't found one reliable way to capture errors on the server, some of them (mostly if they are connected to api failures) they show up but i have not managed to get a simple error from the errortestpage to show up in sentry

raphaelpc commented 5 years ago

Adding to that question... I also tried to throw an error in render() on the server side (i started the state with raiseErrorInRender: true, so the first render, that is server sided, would already throw an error), and that error also wasn't captured by Sentry.

Have you managed in any other way to get that error to show up in sentry? I tried capturing it in the getInitialProps of _error but that also does show nothing in sentry. Currently i haven't found one reliable way to capture errors on the server, some of them (mostly if they are connected to api failures) they show up but i have not managed to get a simple error from the errortestpage to show up in sentry

Unfortunately not. The solution i think i'm going to implement is to use the example (minus the overrided _document.js file) as my template to capture errors that occur on the client side, since those are the ones that i have no control and way to know about, unless users report then to me. On the server side, i think i wil just redirect any log line directly to a log file. That is definitely not the best solution since i wanted to have all errors on the same place, but doing it that way i will at least have information about any error that might occur on the application.

What do you think of that? Thanks!

abraxxas commented 5 years ago

Adding to that question... I also tried to throw an error in render() on the server side (i started the state with raiseErrorInRender: true, so the first render, that is server sided, would already throw an error), and that error also wasn't captured by Sentry.

Have you managed in any other way to get that error to show up in sentry? I tried capturing it in the getInitialProps of _error but that also does show nothing in sentry. Currently i haven't found one reliable way to capture errors on the server, some of them (mostly if they are connected to api failures) they show up but i have not managed to get a simple error from the errortestpage to show up in sentry

Unfortunately not. The solution i think i'm going to implement is to use the example (minus the overrided _document.js file) as my template to capture errors that occur on the client side, since those are the ones that i have no control and way to know about, unless users report then to me. On the server side, i think i wil just redirect any log line directly to a log file. That is definitely not the best solution since i wanted to have all errors on the same place, but doing it that way i will at least have information about any error that might occur on the application.

What do you think of that? Thanks!

That is exactly what we are doing right now, it's a bit clunky but the best we could come up. But since some server-side errors show up in sentry for us there must be something going on either with sentry or next.js i think. I tried with both the simple and the more complex example and both of them behave the same so i am at least somewhat confident that this behaviour is not related to our setup

WestonThayer commented 5 years ago

Folks in this thread might be interested in https://github.com/zeit/next.js/pull/8684 and related bugs. It has 12 different tests of unhandled exceptions that you can play with to gain an understanding of what exceptions Next.js handles for you and what it doesn't.

oswaldoacauan commented 5 years ago

Any news regarding this issue?

luffyZh commented 5 years ago

My solution is use redux save the node fetch data error in state。Then in the componentDidMount of _app.js do something(alert for user or post error req to backend). The code is in there next-antd-scaffold_server-error.

========> state

import {
  SERVER_ERROR,
  CLEAR_SERVER_ERROR
} from '../../constants/ActionTypes';

const initialState = {
  errorType: []
};

const serverError = (state = initialState, { type, payload }) => {
  switch (type) {
    case SERVER_ERROR: {
      const { errorType } = state;
      errorType.includes(payload) ? null : errorType.push(payload);
      return {
        ...state,
        errorType
      };
    }
    case CLEAR_SERVER_ERROR: {
      return initialState;
    }
    default:
      return state;
  }
};

export default serverError;

=======> action

import {
  SERVER_ERROR
} from '../../constants/ActionTypes';

export default () => next => action => {
  if (!process.browser && action.type.includes('FAIL')) {
    next({
      type: SERVER_ERROR,
      payload: action.type 
    });
  }
  return next(action);
};

=======> _app.js

...
componentDidMount() {
    const { store: { getState, dispatch } } = this.props;
    const { errorType } = getState().serverError;
    if (errorType.length > 0) {
      Promise.all(
        errorType.map(type => message.error(`Node Error, Code：${type}`))
      );
      dispatch(clearServerError());
    }
  }
...
saltycrane commented 4 years ago

Thanks to https://github.com/zeit/next.js/issues/1852#issuecomment-353671222 I got an example working with server-side Rollbar error reporting.

codemilli commented 4 years ago

Gotta love unortodox solutions

function installErrorHandler(app) {
  const _renderErrorToHTML = app.renderErrorToHTML.bind(app)
  const errorHandler = rollbar.errorHandler()

  app.renderErrorToHTML = (err, req, res, pathname, query) => {
    if (err) {
      errorHandler(err, req, res, () => {})
    }

    return _renderErrorToHTML(err, req, res, pathname, query)
  }

  return app
}
// ¯\_(ツ)_/¯

This method cannot be applied to 404 error because the err argument is going to be null when 404 error.

jooj123 commented 4 years ago

I solved this for the Elastic APM node client (https://www.npmjs.com/package/elastic-apm-node)

For anyone interested:

In next.config.js:

webpack: (config, { isServer, webpack }) => {
      if (!isServer) {
        config.node = {
          dgram: 'empty',
          fs: 'empty',
          net: 'empty',
          tls: 'empty',
          child_process: 'empty',
        };

        // ignore apm (might use in nextjs code but dont want it in client bundles)
        config.plugins.push(
          new webpack.IgnorePlugin(/^(elastic-apm-node)$/),
        );
      }

      return config;
}

Then in _error.js render func you can call captureError manually: 


function Error({ statusCode, message, err }) {

const serverSide = typeof window === 'undefined';

  // only run this on server side as APM only works on server
  if (serverSide) {
    // my apm instance (imports elastic-apm-node and returns  captureError)
    const { captureError } = require('../src/apm'); 

    if (err) {
      captureError(err);
    } else {
      captureError(`Message: ${message}, Status Code: ${statusCode}`);
    }
  }

}
sarapowers commented 4 years ago

Hey all! We just launched an NPM library for Express style architecture in Next.js without adding an Express server. It might be helpful for your server error handling challenges! Check it out if you're interested. https://github.com/oslabs-beta/connext-js

stephankaag commented 4 years ago

Anyone had success catching (and handle) exceptions that occur in getServerSideProps?

j-quelly commented 4 years ago

Anyone had success catching (and handle) exceptions that occur in getServerSideProps?

It's unclear, you'd think a framework would provide an idiomatic way to log errors both client and server 🤷‍♂️

ghost commented 4 years ago

Anyone had success catching (and handle) exceptions that occur in getServerSideProps?

@stephankaag

yes, something like this worked for me:

first create a middleware to handle the crash reporting. I've wrapped Sentry inside the CrashReporter class because simply returning Sentry will not work (i.e. req.getCrashReporter = () => Sentry).

// crash-reporter.js

const Sentry = require("@sentry/node");

class CrashReporter {
  constructor(){
    Sentry.init({ dsn: process.env.SENTRY_DSN });
  }
  captureException(ex){
    return Sentry.captureException(ex);
  }
}

function crashReporterMiddleware(req, res, next) {
  req.getCrashReporter = () => new CrashReporter();
  next();
}

module.exports = crashReporterMiddleware;

next, of course, load the middleware into your app before the Next.js request handler is set:

// server.js

const crashReporterMiddleware = require("./middleware/crash-reporter")

...

app.use(crashReporterMiddleware);

...

setHandler((req, res) => {
  return handle(req, res);
});

then wherever you call getServerSideProps:

// _error.js

export async function getServerSideProps({req, res, err}) {
  const statusCode = res ? res.statusCode : err ? err.statusCode : 404;
  const crashReporter = req.getCrashReporter();
  const eventId = crashReporter.captureException(err);
  req.session.eventId = eventId;
  return {
    props: { statusCode, eventId }
  }
}
Vadorequest commented 3 years ago

I'm not sure what's meant to be done here (why is this issue still open? What's the plan?)

But error handling with Next was very complicated, there are a lot of use cases and places to catch errors from.

I added Sentry built-in in Next Right Now, if you're struggling about how to configure Sentry on either backend or frontend, you might want to take a look. All Next.js errors are caught and sent to Sentry, whether client side, server side, build issue, page loading crashes, CSR crashes, etc.

If you think I missed anything, tell me!

Kinda spent a few months improving those bit by bit, should be a great starting point.

1ambda commented 3 years ago

I use getServerSideProps only for fetching initialData (e.g, for SWR). Just throwing error would block layout rendering since _error.tsx will be rendered instead of.

So I made two small utility functions. 1) HOC wrapping page components 2) container wrapping getServerSideProps


/**
 * HOC handle `failure` prop by wrapping page components.
 */
export const withExceptionCatcher = (WrappedComponent) => {
    const wrapper = (props) => {

        // if there is no failure, render normal page component :)
        if (!props.failure) {
            return <WrappedComponent {...props} />
        }

        const { ... } = props.failure

        return (
            <div>
                <FailureSection
                    ...
                />
            </div>
        )
    }

    return wrapper
}

/**
 * Handle exceptions in original page components from getServerSideProps
 * `failure` prop will be passed if error occurs.
 */
export function getServerSidePropsWithCatcher(getServerSidePropsFunc: Function) {
    return async (ctx) => {
        let props = {}
        try {
            props = await getServerSidePropsFunc(ctx)
        } catch (error) {
            return { props: { failure: error } }
        }

        return { ...props }
    }
}

Here is usage

// MyPage.tsx

function MyPage() {
  ...
}

export const getServerSideProps = getServerSidePropsWithCatcher(async (ctx) => {
    const result = await fetchData()
    return {
        props: { ... },
    }
})

export default withExceptionCatcher(MyPage)

Hope this might help.

justjake commented 3 years ago

I've been struggling with monkey-patching all the different parts of Next.js that log errors on the path to using Next.js for Notion's new marketing site. It's been a frustrating experience; I'm up to patching global.console.log = myCustomLoggingFunction at this point.

oswaldoacauan commented 3 years ago

It's super frustrating that a basic need like this still open after 4 years and 10 major releases.

Custom logging and error handling are basic features being neglected from the Next.js team sadly 🥲

Vadorequest commented 3 years ago

Despite having spent quite a lot of time configuring Sentry, I also feel like it can still be improved and I'm afraid I'm not catching every case. (I'm talking about error catching only)

Regarding logging, I'm also "frustrated" with it and not satisfied of the solution I came up with using Winston (too heavy, not flexible enough, etc.)

likerRr commented 3 years ago

Same to you folks. with-sentry example doesn't allow lazy loading, as well as it doesn't catch errors before app initialization. Moreover there are can be errors in your custom ssr server which are also want to be tracked with sentry. So these days I see such issues with setup of sentry error tracking with nextjs which are not covered by any guide:

  1. Lazy loading
  2. Errors during client\server rendering
  3. Errors raised on custom ssr server
kelly-tock commented 3 years ago

I feel like this and lack of guides/prebuilt integrations for apm and error reporting services are a big gap for next.js. I hope this is high on the roadmap. how are large companies even running next.js without these 2 basic things? I guess custom server and prayers 😢

leerob commented 3 years ago

@kelly-tock Large companies using Next.js do have error handling, logging, and monitoring set up! I agree with your statement though, and we're working on better integrations (e.g. Sentry, Datadog) with Vercel, as well as overall better guidance for "going to production" with Next.js (logging, monitoring).

kelly-tock commented 3 years ago

@kelly-tock Large companies using Next.js do have error handling, logging, and monitoring set up! I agree with your statement though, and we're working on better integrations (e.g. Sentry, Datadog) with Vercel, as well as overall better guidance for "going to production" with Next.js (logging, monitoring).

Glad to hear that! I'm currently working on a proposal for switching to next, and performance monitoring and error handling will be key. Any ideas on timelines? And for both APM and error logging services, we're left to do custom servers for now right?

leerob commented 3 years ago

It's definitely a priority! And no, you wouldn't need a custom server:

Hope this helps in the meantime.

justjake commented 3 years ago

@kelly-tock if you have control of Next’s invocation, you can do this without a “custom server” using a “preload”. If you can invoke next like node -r yourCrazyPreloadMonkeyPatches.js node_modules/.bin/next you can preload arbitrarily code without losing automatic static optimization. At Notion we run Next on AWS Beanstalk and use this technique to load Datadog APM into the process, and monkey-patch global.console.log and global.console.error to use our error reporting logic.

leerob commented 3 years ago

@justjake Yes, absolutely 😄

kelly-tock commented 3 years ago

we need a yourCrazyMonkeyPatches.js guide 😄 . do you have an example repo somewhere @justjake ? thanks for all the input!

my project would fall into

so would like to integrate a datadog/appsignal/etc into it. as well as rollbar.

looking forward to more guides and resources.

Vadorequest commented 3 years ago

@kelly-tock Next Right Now uses Sentry for monitoring (front + back + api) https://github.com/UnlyEd/next-right-now/

Running about 20 projects using Next.js (most of them using NRN) I totally feel you regarding monitoring, it's a must-have for confidence.

Regarding logging though, I haven't found a solution that satisfies me. I hope it comes built-in.

justjake commented 3 years ago

@kelly-tock okay, I wrote up a quick note about how we use preloading with NextJS: https://jake.tl/notes/2021-04-04-nextjs-preload-hack

shem8 commented 3 years ago

Hi, 2 things that I still don't understand regarding this issue:

  1. You keep mentioning middleware as a way to solve it, but from what I see in Next.js you have to call the middleware in every API call. This is not very useful. In Express you set the middleware once on the server and it called for every request.
  2. There're many more monitoring tools out there, you're very focused on Sentry and give a built-in solution for integrating with this, but the platform has to give so a way for developers to customize the error handling so we'll be able to use other tools too (Rollbar in my example).
thomasboyt commented 3 years ago

@justjake Thanks for the excellent writeup! While having to monkey-patch logging is a shame, I think using --require for adding instrumentation/error handling is fairly "elegant" as far as these things go.

Sentry just released a new official Next.js SDK that has an interesting approach: it actually injects Sentry setup into the Webpack bundles for your app, both the client and server (https://github.com/getsentry/sentry-javascript/blob/master/packages/nextjs/src/utils/config.ts#L154-L155). Unfortunately, we're using Rollbar at my company, and Rollbar doesn't have any Next.js integration, so we'll have to do a manual setup. The --require route seems much easier, but I wonder if maybe the Webpack solution has some other advantages I'm missing (besides not having to remember to use different command line arguments).

In general, I think Next.js docs should incorporate discussion of error reporting. I had deferred adding error handling to the "last 10% work" of a new project launch and spent yesterday in a bit of a panic trying to figure out how on earth I was going to add it. I had just assumed it'd be as easy as adding Sentry/Rollbar/Bugsnag/etc usually is - just add it to the entry point - until I realized Next has no "entry point" without a custom server. If the use of --require to register such things is the "happy path" for Next apps, I think it absolutely needs to be documented, rather than be buried here in a 4 year old GitHub issue.

Vadorequest commented 3 years ago

@thomasboyt Thanks for pointing out Sentry released something for Next.js! I'm gonna take a look at it very soon!

For anyone interested: https://sentry.io/for/nextjs/

Also, we mentionned logging above, and we've released https://github.com/UnlyEd/simple-logger a few days ago, which is 1kB an universal. After trying out a lot of open source alternatives, we couldn't find something that felt just right. Maybe you'll find it as useful as we do!

hems commented 3 years ago

@kelly-tock okay, I wrote up a quick note about how we use preloading with NextJS: https://jake.tl/notes/2021-04-04-nextjs-preload-hack

thanks for writing that out but indeed we need a generic "error handler" for the server like pretty much any express.js production app will end up having so unexpected errors can be caught, a user-friendly 500 can be return and the error can be sent to any error reporting library.

fingers crossed this will be sorted out soon as it's rather important.

DzTheRage commented 3 years ago

Same waiting for update on this.

meotimdihia commented 3 years ago

@kelly-tock okay, I wrote up a quick note about how we use preloading with NextJS: https://jake.tl/notes/2021-04-04-nextjs-preload-hack

I tried with this preloading script. It printed error messages but missed request URL and stack trace. It is super hard or impossible to know where the error was happening.

Edit: I have ended up using https://github.com/getsentry/sentry-javascript/tree/master/packages/nextjs

goforbg commented 3 years ago

The errors details are also available on the stderr stream. process.stderr.write = error => yourErrorLog(error);

Worked for me!

bitworking commented 2 years ago

We need a server side ErrorBoundary 🤗