Unable to find newrelic-log-ingestion lambda due to appended digits

[Jamchello]

Description

The current version of this tool has an issue with the instrumentation of lambda layers. Initially, an outdated version of the log-ingestion lambda was in use. To address this, the outdated NewRelicIngestion stack was removed, and a new one was created using the serverlessrepo. The new stack is labeled as serverlessrepo-NewRelic-log-ingestion, and the corresponding deployed lambda function is identified by newrelic-log-ingestion-129616fa9bcd. Consequently, the following stackname argument is now included during the installation of the integration:

newrelic-lambda integrations install <NR account ID, API key etc> --stackname serverlessrepo-NewRelic-log-ingestion

However, I'm facing an issue where there doesn't seem to be a CLI argument to specify the actual function. This results in the error:

Creating newrelic-log-ingestion Lambda function in AWS account
✖️ CloudFormation Stack NewRelicLogIngestion exists (status: CREATE_COMPLETE), but newrelic-log-ingestion Lambda function does not.
Please manually delete the stack and re-run this command.

Additionally, attempts to resolve this by deleting the stack and running the tool without specifying any stack leads to another error:

Setting up CloudFormation Stack NewRelicLogIngestion in region: us-east-1
Fetching new CloudFormation template url
✖️ CloudFormation Stack NewRelicLogIngestion exists (status: None).
Please manually delete the stack and re-run this command.

Steps to Reproduce

1. Deploy the NewRelic-log-ingestion application using the serverless repository with the ARN arn:aws:serverlessrepo:us-east-1:463657938898:applications/NewRelic-log-ingestion.
2. Try installing the lambda integration and instrumenting a lambda using the layer ARN arn:aws:lambda:us-east-1:451483290750:layer:NewRelicPython310:7.

Expected Behaviour

The process should either create a new log-ingestion function or utilize an existing one within the stack.

Relevant Logs / Console output

The errors and output described in the initial section indicate issues with stack creation and lambda function recognition.

[chaudharysaket]

Hi @Jamchello The message printed by the CLI suggests that there exists newrelic-log-ingestion lambda associated with the --stackname serverlessrepo-NewRelic-log-ingestion. Thus that particular last step which typically installs a new log-ingestion-lambda has failed. In case you don't provide stackname it will create by default NewRelicLogIngestion stack, if that already exists then you will receive following message ✖️ CloudFormation Stack NewRelicLogIngestion exists (status: None). Please manually delete the stack and re-run this command. You can use other commands such as subscriptions install to use the new stackname/log-ingestion-lambda

[Jamchello]

Hey @chaudharysaket, thanks for looking into this promptly. I tried manually deleting the stack and re-running as mentioned above!

Here is the logs that came from that:

Creating newrelic-log-ingestion Lambda function in AWS account
Setting up CloudFormation Stack NewRelicLogIngestion in region: us-east-1
Fetching new CloudFormation template url
✖️ CloudFormation Stack NewRelicLogIngestion exists (status: None).
Please manually delete the stack and re-run this command.
✖️ Install Incomplete. See messages above for details.

Adfter logging this, it then crashes and fails too.

[chaudharysaket]

Hi @Jamchello Could you please provide the logs after it crashes? The above logs only indicate the steps that failed.

[Jamchello]

This is all I have @chaudharysaket

Creating newrelic-log-ingestion Lambda function in AWS account
Setting up CloudFormation Stack NewRelicLogIngestion in region: us-east-1
Fetching new CloudFormation template url
✖️ CloudFormation Stack NewRelicLogIngestion exists (status: None).
Please manually delete the stack and re-run this command.
✖️ Install Incomplete. See messages above for details.
Error: Process completed with exit code 1.

[Jamchello]

And to follow up here - it is not creating the stack and then crashing, its simply crashing without ever creating the stack or lambda.

[Jamchello]

I have managed to resolve this, solution was to install the new-relic integration via this CLI tool locally (not on the CLI) - this created the lambda log ingestion function & stack without crashing the process as in our CI, and without appending characters to the names of the created resources.

It might be worth looking into why this entire flow broke down with an older version of the log-ingestion stack, as we have had this automated for many months as the latest release of this tool became incompatible with the version of the log-ingestion stack we had deployed!

[chaudharysaket]

Hi @Jamchello, This was expected that the latest version v0.9.0 of CLI will not be able to support the log-ingestion-lambda [without suffix] created by older versions of the CLI. The latest changes were made in order to bring parity between the changes in serverlessrepo repo. In case you want to instrument the older version of log-ingestion-lambda, I would suggest using older version of the CLI in the CI workflow.

[Jamchello]

I see, was this published anywhere?!

[Jamchello]

I'm also confused as to why the installation of the integration is failing in the CI, but works fine when I run it locally from my machine with the same commands

[chaudharysaket]

Thanks for the suggestion. I have updated the release notes for v0.9.0. I would check the CLI version in the CI, if it is not the latest version v0.9.0 and you have created the stack using serverless repo then it might not work.