googleapis / nodejs-logging

Node.js client for Stackdriver Logging: Store, search, analyze, monitor, and alert on log data and events from Google Cloud Platform and Amazon Web Services (AWS).
https://cloud.google.com/logging/
Apache License 2.0
168 stars 99 forks source link

Not able to retreive log entries #1411

Open deama opened 1 year ago

deama commented 1 year ago

So I have the following function here:

const functions = require('@google-cloud/functions-framework');
const { Storage } = require('@google-cloud/storage');
const readline = require('readline');
const { Logging } = require('@google-cloud/logging');
const storage = new Storage();
const logging = new Logging();

async function searchLogs(date, timeStart, timeEnd, user) {
  const logFilter = `logName="timestamp>="2023-03-01T11:55:00Z" AND timestamp<="2023-03-03T11:55:51Z"`;
  const resourceNames = ['projects/production-logs'];
  const [entries] = await logging.getEntries({ resourceNames, filter: logFilter });

  console.log(`Searching logs with filter: ${logFilter}`);
  console.log(`Found ${entries.length} entries.`);

  for (const entry of entries) {
    const message = entry.data.jsonPayload.message;
    const ipAddress = message.match(/Ip Address:\s+(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/);
    if (ipAddress) {
      return ipAddress[1];
    }
  }

  console.log(`----`);
  return null;
}

//... rest of code

What I'm having an issue is I have my production logs stored in a logging bucket called "prod", however the above code only works on the project scope rather than on a per-bucket scope, so it basically can't find the logs, because it can't see the logging bucket.

Is there a way to direct it so that it sends the request to a specific logging bucket? I can't find much online, and I have found a function in the source code for the loggin API about retreiving a list of logging buckets in the project, which works, but how do I set it to use that bucket now?

Btw, I hardcoded the timestamp for now to keep it simple.

deama commented 1 year ago

I've tried setting the resourceNames to view:

const resourceNames = ['projects/production-logs/locations/global/buckets/prod/views/_AllLogs'];

And whilst that is what is said in the documentation: https://github.com/googleapis/nodejs-logging/blob/main/src/v2/logging_service_v2_client.ts [line 772]

It compiles fine, and it gives me the entries, however if I now try and make a filter specific to the bucket in question, e.g a filter that only that bucket should contain, it gives me 0 entries, even though manually checking it displays actual entries. So looks like it still applies the getEntries on the project only rather than the logging bucket.