opennextjs / opennextjs-aws

Open-source Next.js adapter for AWS
https://opennext.js.org
MIT License
4.12k stars 126 forks source link

instrumentation does not work during deployment #606

Open abaybektursun opened 1 week ago

abaybektursun commented 1 week ago

I am trying to get OpenTelemtry working with my app (For debugging LLMs). It works when I run it locally, however it stops working after deploying with SST v3 ("sst": "3.2.12") I have this in my next.config.js

module.exports = {
  experimental: {
    instrumentationHook: true
  }
}

instrumentation.ts

// Ensure this is in the root of your project, not in app or pages directories
export async function register() {
  if (process.env.NEXT_RUNTIME === 'nodejs') {
    await import('./instrumentation.node')
  }
}

instrumentation.node.js

import { NodeSDK } from '@opentelemetry/sdk-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { Resource } from '@opentelemetry/resources';
import { SemanticResourceAttributes } from '@opentelemetry/semantic-conventions';
import { SimpleSpanProcessor } from '@opentelemetry/sdk-trace-node';
import { diag, DiagConsoleLogger, DiagLogLevel } from '@opentelemetry/api';

// Enable //console.logging for OpenTelemetry
diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.DEBUG);

console.log('Starting OpenTelemetry initialization');

const isProduction = process.env.NODE_ENV === 'production';
console.log(`(instrumentation.node.js) Environment: ${isProduction ? 'Production' : 'Development'}`);

const traceExporter = new OTLPTraceExporter({
  url: '.......',
});

//console.log(`Trace exporter URL: ${traceExporter.url}`);

const sdk = new NodeSDK({
  resource: new Resource({
    [SemanticResourceAttributes.SERVICE_NAME]: 'tracing',
    [SemanticResourceAttributes.DEPLOYMENT_ENVIRONMENT]: isProduction ? 'production' : 'development',
  }),
  spanProcessors: [new SimpleSpanProcessor(traceExporter)],
});

// Start the SDK without assuming it returns a Promise
try {
  sdk.start();
  console.log('OpenTelemetry SDK started successfully');
} catch (error) {
  console.error('Error starting OpenTelemetry SDK:', error);
}

process.on('SIGTERM', () => {
  sdk.shutdown()
    .then(() => console.log('Tracing terminated'))
    .catch((error) => console.log('Error terminating tracing', error))
    .finally(() => process.exit(0));
});

console.log('OpenTelemetry setup script finished');

This entire file seems to be ignored since there are no logs from instrumentation.node.js in CloudWatch or SST.

conico974 commented 1 week ago

Have you tested the solution in #520 ? It uses @vercel/otel instead

My guess is that one of the import is failing for some reason. Does it work with the standalone output ?

And BTW this cannot properly work on lambda, you're not awaiting things properly here, the last exported trace of a request will likely never get sent.

oldbettie commented 5 hours ago

The solution in my post did not work I spent about a week trying different methods and conico was helpful but everything I tried resulted in the same failure. We have switched to railway for deployment until opennext supports Otel

conico974 commented 10 minutes ago

The solution in the other post did work for me with @vercel/otel. If it still doesn't work, you'll need to provide a reproduction