hello i am trying to install saleforce event logs on new relic . where in saleforce do i edit the config.yaml file.

dlux29 commented 1 year ago

Description

Steps to Reproduce

Expected Behavior

Relevant Logs / Console output

Your Environment

ex: Browser name and version:
ex: Operating System and version:

Additional context

shahramk commented 1 year ago

@dlux29, salesforce offers their docs for enabling and using logfile monitoring on an account. In salesforce you need to make sure that the account is enabled to generate logfiles, and create a"connected app" (or use an existing one) to obtain your consumer key and secret.

Salesforce

accessing this object requires View Event Log Files and API Enabled user permissions. users with "View All Data" permission can view event log files. make sure you have a salesforce user with proper access
in salesforce menu go to Events->Event Manager and enable streaming for all required event types
make note of your security token. if you don't have it, go to 'Settings', and in the top left (Quick Find) type 'Reset'. you should get an option 'Reset My Security Token'. select that and reset your token. it'll be emailed to you.
from the left nav menu go to "Apps->App Manager".
pick a connected app, or you can go to "Apps->Manage Connected Apps" and create a new app.
on the far right of the app click on the arrow and select "View"
copy "Consumer Key" and "Consumer Secret" (or client_id & client_secret)

New Relic logfile agent

now go to config.yml from the repo you downloaded and add these keys and secrets.
note that you need to append the access token to your password
add new relic account id
if your "data_format" is set to "logs" for "license_key" put your NR license key. if "data_format" is set to "events", then set "license_key" to your account's API Key

dlux29 commented 1 year ago

run this locally?

shahramk commented 1 year ago

Either local or on some machine outside of salesforce. The agent tries to login to your salesforce account and access the log files.

dlux29 commented 1 year ago

What is the process to set up this integration.? Do i need to stand up a linux server to have the main.py file run the integration?

shahramk commented 1 year ago

if you are testing the agent, you should be able to run it from your desktop of laptop. if you run it 24/7 for production use, then I recommend that you use some small server (preferably linux) in the cloud (AWS EC2, GCP Compute Engine, etc.)

dlux29 commented 1 year ago

ok i have now set up the server . downloaded the file from repo. Now i want to edit the config.yml file . for Authentication with jw2 auth: { "grant_type": "urn:ietf:params:oauth:grant-type:jwt-bearer", "client_id": "", "private_key": "path_to_private_key", "subject": "", "audience": "https://login.salesforce.com" }

What and where do i devolope my private key? /home/users/private_key?

shahramk commented 1 year ago

as long as you have obtained your private key in a file, you can specify the path to the file as "private_key" in config.yml file. just replace "path_to_private_key" with the path to the location of your private key file.

dlux29 commented 1 year ago

Private ssh key? On Monday, February 6, 2023 at 02:24:04 PM EST, Shahram Khazai @.***> wrote:

as long as you have obtained your private key in a file, you can specify the path to the file as "private_key" in config.yml file. just replace "path_to_private_key" with the path to the location of your private key file.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

shahramk commented 1 year ago

This is documented on salesforce docs site. Perhaps these 2 document links should get you going:

OAuth 2.0 JWT Bearer Flow for Server-to-Server Integration Create a Private Key and Self-Signed Digital Certificate

additional references: OpenSSL Create a Connected App in Your Org Authorize an Org Using the JWT Bearer Flow

dlux29 commented 1 year ago

ok i did not use the bearar flow method. I used the password method. But receive this error ""salesforce token request failed. status-code:400, reason: Bad Request"" On Monday, February 6, 2023 at 03:25:27 PM EST, Shahram Khazai @.***> wrote:

This is documented on salesforce docs site. Perhaps these 2 document links should get you going:

OAuth 2.0 JWT Bearer Flow for Server-to-Server Integration Create a Private Key and Self-Signed Digital Certificate

additional references: OpenSSL Create a Connected App in Your Org Authorize an Org Using the JWT Bearer Flow

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

shahramk commented 1 year ago

@dlux29 please send me the content of you config.yml file. mask token/password/secret values before sending.

dlux29 commented 1 year ago

using program arguments []Running query SELECT+Id,+EventType,+CreatedDate,+LogDate,+Interval,+LogFile,+Sequence+From+EventLogFile+Where+LogDate>=2023-02-07T11:17:27.557Z+AND+LogDate<2023-02-07T12:17:27.785Z+AND+Interval='Hourly' error when trying to run SOQL query. message: salesforce event log query failed. status-code:400, reason: Bad Request response: [{"message":"\nEventType, CreatedDate, LogDate, Interval, LogFile, Sequence From\n ^\nERROR at Row:1:Column:45\nNo such column 'Interval' on entity 'EventLogFile'. If you are attempting to use a custom field, be sure to append the '__c' after the custom field name. Please reference your WSDL or the describe call for the appropriate names.","errorCode":"INVALID_FIELD"}] Traceback (most recent call last): File "/home/vffadmin/newrelic-logs-salesforce-eventlogfile/src/main.py", line 102, in main() File "/home/vffadmin/newrelic-logs-salesforce-eventlogfile/src/main.py", line 79, in main integration.run() File "/home/vffadmin/newrelic-logs-salesforce-eventlogfile/src/newrelic_logging/integration.py", line 60, in run self.process_events(logs, labels) File "/home/vffadmin/newrelic-logs-salesforce-eventlogfile/src/newrelic_logging/integration.py", line 83, in process_events for log_file_obj in logs:TypeError: 'NoneType' object is not iterable

On Tuesday, February 7, 2023 at 11:54:49 AM EST, Shahram Khazai ***@***.***> wrote:

@dlux29 please send me the content of you config.yml file. mask token/password/secret values before sending.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

shahramk commented 1 year ago

I need the config.yml file.

On Tue, Feb 7, 2023, 9:25 AM dlux29 @.***> wrote:

using program arguments []Running query SELECT+Id,+EventType,+CreatedDate,+LogDate,+Interval,+LogFile,+Sequence+From+EventLogFile+Where+LogDate>=2023-02-07T11:17:27.557Z+AND+LogDate<2023-02-07T12:17:27.785Z+AND+Interval='Hourly' error when trying to run SOQL query. message: salesforce event log query failed. status-code:400, reason: Bad Request response: [{"message":"\nEventType, CreatedDate, LogDate, Interval, LogFile, Sequence From\n ^\nERROR at Row:1:Column:45\nNo such column 'Interval' on entity 'EventLogFile'. If you are attempting to use a custom field, be sure to append the '__c' after the custom field name. Please reference your WSDL or the describe call for the appropriate names.","errorCode":"INVALID_FIELD"}] Traceback (most recent call last): File "/home/vffadmin/newrelic-logs-salesforce-eventlogfile/src/main.py", line 102, in main() File "/home/vffadmin/newrelic-logs-salesforce-eventlogfile/src/main.py", line 79, in main integration.run() File "/home/vffadmin/newrelic-logs-salesforce-eventlogfile/src/newrelic_logging/integration.py", line 60, in run self.process_events(logs, labels) File "/home/vffadmin/newrelic-logs-salesforce-eventlogfile/src/newrelic_logging/integration.py", line 83, in process_events for log_file_obj in logs:TypeError: 'NoneType' object is not iterable

On Tuesday, February 7, 2023 at 11:54:49 AM EST, Shahram Khazai @.***> wrote:

@dlux29 please send me the content of you config.yml file. mask token/password/secret values before sending.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/newrelic-experimental/newrelic-logs-salesforce-eventlogfile/issues/5#issuecomment-1421155094, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDU3LHD3FDZWLLCTK2AUTTWWKAPPANCNFSM6AAAAAAUHY7OBU . You are receiving this because you commented.Message ID: <newrelic-experimental/newrelic-logs-salesforce-eventlogfile/issues/5/1421155094 @github.com>

dlux29 commented 1 year ago

integration_name: com.newrelic.labs.sfdc.eventlogfilesrun_as_service: Falsecron_interval_minutes: 60service_schedule: { "hour": "*", "minute": "30" }instances: - name: sfdc-logs arguments: token_url: "https://valleyforgefabrics.my.salesforce.com/services/oauth2/token" auth: { "grant_type": "password", "client_id": "3agjdlajdgajgj;aj", "client_secret": "jdjddj", "username": "valleyforge.com", "password": "" } cache_enabled: False redis: { "host": "", "port": "", "db_number": 0, "password": "" } date_field: "LogDate" generation_interval: "Hourly" time_lag_minutes: 300 labels: environment: productionnewrelic: data_format: "events" api_endpoint: "US" account_id: "" license_key: ""

On Tuesday, February 7, 2023 at 12:41:13 PM EST, Shahram Khazai ***@***.***> wrote:

I need the config.yml file.

On Tue, Feb 7, 2023, 9:25 AM dlux29 @.***> wrote:

using program arguments []Running query SELECT+Id,+EventType,+CreatedDate,+LogDate,+Interval,+LogFile,+Sequence+From+EventLogFile+Where+LogDate>=2023-02-07T11:17:27.557Z+AND+LogDate<2023-02-07T12:17:27.785Z+AND+Interval='Hourly' error when trying to run SOQL query. message: salesforce event log query failed. status-code:400, reason: Bad Request response: [{"message":"\nEventType, CreatedDate, LogDate, Interval, LogFile, Sequence From\n ^\nERROR at Row:1:Column:45\nNo such column 'Interval' on entity 'EventLogFile'. If you are attempting to use a custom field, be sure to append the '__c' after the custom field name. Please reference your WSDL or the describe call for the appropriate names.","errorCode":"INVALID_FIELD"}] Traceback (most recent call last): File "/home/vffadmin/newrelic-logs-salesforce-eventlogfile/src/main.py", line 102, in main() File "/home/vffadmin/newrelic-logs-salesforce-eventlogfile/src/main.py", line 79, in main integration.run() File "/home/vffadmin/newrelic-logs-salesforce-eventlogfile/src/newrelic_logging/integration.py", line 60, in run self.process_events(logs, labels) File "/home/vffadmin/newrelic-logs-salesforce-eventlogfile/src/newrelic_logging/integration.py", line 83, in process_events for log_file_obj in logs:TypeError: 'NoneType' object is not iterable

On Tuesday, February 7, 2023 at 11:54:49 AM EST, Shahram Khazai @.***> wrote:

@dlux29 please send me the content of you config.yml file. mask token/password/secret values before sending.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/newrelic-experimental/newrelic-logs-salesforce-eventlogfile/issues/5#issuecomment-1421155094, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDU3LHD3FDZWLLCTK2AUTTWWKAPPANCNFSM6AAAAAAUHY7OBU . You are receiving this because you commented.Message ID: <newrelic-experimental/newrelic-logs-salesforce-eventlogfile/issues/5/1421155094 @github.com>

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

dlux29 commented 1 year ago

integration_name: com.newrelic.labs.sfdc.eventlogfiles run_as_service: False cron_interval_minutes: 60 service_schedule: { "hour": "*", "minute": "30" } instances:

name: sfdc-logs arguments: token_url: "https://valleyforgefabrics.my.salesforce.com/services/oauth2/token" auth: { "grant_type": "password", "client_id": ".PM5mSRN7y1ZU7yLMp0.25__r", "client_secret": "DB", "username": "dwillm", "password": "" } cache_enabled: False redis: { "host": "", "port": "", "db_number": 0, "password": "" } date_field: "LogDate" generation_interval: "Hourly" time_lag_minutes: 300 labels: environment: production newrelic: data_format: "events" api_endpoint: "US" account_id: "" license_key: "e"

dlux29 commented 1 year ago

BELOW IS THE ERROR I GET

SELECT+Id,+EventType,+CreatedDate,+LogDate,+Interval,+LogFile,+Sequence+From+EventLogFile+Where+LogDate>=2023-02-06T20:29:58.243Z+AND+LogDate<2023-02-07T15:29:58.450Z+AND+Interval='Hourly'error when trying to run SOQL query. message: salesforce event log query failed. status-code:400, reason: Bad Request response: [{"message":"\nEventType, CreatedDate, LogDate, Interval, LogFile, Sequence From\n ^\nERROR at Row:1:Column:45\nNo such column 'Interval' on entity 'EventLogFile'. If you are attempting to use a custom field, be sure to append the '__c' after the custom field name. Please reference your WSDL or the describe call for the appropriate names.","errorCode":"INVALID_FIELD"}] Traceback (most recent call last): File "/home/vffadmin/newrelic-logs-salesforce-eventlogfile/src/main.py", line 102, in main() File "/home/vffadmin/newrelic-logs-salesforce-eventlogfile/src/main.py", line 79, in main integration.run() File "/home/vffadmin/newrelic-logs-salesforce-eventlogfile/src/newrelic_logging/integration.py", line 60, in run self.process_events(logs, labels) File "/home/vffadmin/newrelic-logs-salesforce-eventlogfile/src/newrelic_logging/integration.py", line 83, in process_events for log_file_obj in logs: TypeError: 'NoneType' object is not iterable

shahramk commented 1 year ago

The code fetches the queued logs from salesforce at line 56 of "integration.py":

logs = client.fetch_logs(sfdc_session)

Based on the error you are getting at line 83 of "integration.py", it means that the logs variable does not contain the expected object, which tells me that you account either has not generated any logs, or it may not have been configured properly so the returned data doesn't have the correct format.

That is out of my ability. perhaps you can contact your salesforce support, or your admin.

As a debugging measure you can insert a print line right after line 56 of "integration.py" to see what the logs variable contains right after logs are fetched.

Note: Insert the line with exactly 12 leading spaces

            print("vars(sfdc_session): ", vars(sfdc_session))

and run the code to see what salesforce returns.

dlux29 commented 1 year ago

do i need to have event-log monitoring license in salesforce activated ?

shahramk commented 1 year ago

I think salesforce support could answer that question better than I can.

On Mon, Feb 27, 2023 at 9:44 AM dlux29 @.***> wrote:

do i need to have event-log monitoring license in salesforce activated ?

— Reply to this email directly, view it on GitHub https://github.com/newrelic-experimental/newrelic-logs-salesforce-eventlogfile/issues/5#issuecomment-1446766680, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDU3LAYRKR2PGT3WWV6KOTWZTRWPANCNFSM6AAAAAAUHY7OBU . You are receiving this because you commented.Message ID: <newrelic-experimental/newrelic-logs-salesforce-eventlogfile/issues/5/1446766680 @github.com>

newrelic / newrelic-salesforce-exporter