http code: 431 error while reading data from salesforce account table

[KANAK-G]

Describe the bug Salesforce account object has 754 columns with 730 custom columns and the rest are default columns. While fetching data from this object getting 431 error.

Steampipe version (steampipe -v) v0.21.2

Plugin version (steampipe plugin list) v0.7

To reproduce Pass the credentials in the config file

Expected behavior Should be queryable

Additional context No

[ParthaI]

Hello @KANAK-G, thank you for bringing this issue to our attention. I'm sorry to hear about the difficulties you're encountering.

I've been investigating the problem and found that our custom Salesforce table salesforce_api__c has approximately 10,422 rows and 28 columns. I can successfully retrieve results when querying this table.

The error code you mentioned (431) suggests an issue with Request Header Fields Too Large.

Could you provide more information on how you're utilizing the tables and the specific query you're executing? This information will assist us in replicating and troubleshooting the issue.

Additionally, could you try executing a query that fetches only one or two columns from the table causing the 431 error and let us know if the problem persists?

Thank you!

[KANAK-G]

Hello @ParthaI, thank you for promptly addressing this issue.

I'm also successfully retrieving data from all my custom objects. However, I'm encountering an issue with the static table 'salesforce_account.' I could query it before, but there was a schema change.

Is there a workaround or solution to overcome this issue?

I'm providing credentials in the config as per the documentation, and when attempting to query through the terminal using select * from salesforce_account limit 10, I'm facing an error.

I've also tried queries with 1-2 columns, but the error persists.

[ParthaI]

Hello @KANAK-G, thank you for providing the information.

Currently, I'm unable to reproduce the error on my end when querying the salesforce_account table, as it consistently returns results for me. To further investigate this issue, I have a few questions:

1. Is this error specific to the salesforce_account table, or have you encountered it with other tables too?
2. Regarding the salesforce_account table, does this error occur every time you query it, or do you sometimes receive results?
3. Could you please provide the log details found in ~/.steampipe/logs/plugin-2024-01-1*.log? Before sharing, ensure any sensitive information is redacted.
4. It would also be helpful if you could share the .spc file for the Salesforce plugin, with sensitive information removed.

Your responses to these queries will be invaluable in resolving this issue. Thank you!

[KANAK-G]

Hi @ParthaI ,

1. No, I'm not encountering this error with other tables.

2. No, it's occurring each time and I have not received any results so far.

3. 2024-01-17 11:39:36.302 UTC [109755] LOG: connection received: host=127.0.0.1 port=52558 2024-01-17 11:39:36.306 UTC [109755] LOG: connection authorized: user=steampipe database=steampipe application_name=steampipe_client_system_fee2 SSL enabled (protocol=TLSv1.2, cipher=ECDHE-RSA-AES256-GCM-SHA384, bits=256) 2024-01-17 11:39:36.309 UTC [109676] LOG: could not receive data from client: Connection reset by peer 2024-01-17 11:39:36.309 UTC [109677] LOG: could not receive data from client: Connection reset by peer 2024-01-17 11:39:36.309 UTC [109676] LOG: disconnection: session time: 0:00:08.401 user=root database=steampipe host=127.0.0.1 port=52528 2024-01-17 11:39:36.309 UTC [109677] LOG: disconnection: session time: 0:00:08.390 user=root database=steampipe host=127.0.0.1 port=52534 2024-01-17 11:39:41.156 UTC [WARN] hub: stream receive error rpc error: code = Unknown desc = sfcrmuat: [simpleforce] Error. http code: 431 Error Message: Error Code: (0xc000bfe000) 2024-01-17 11:39:41.156 UTC [109753] ERROR: rpc error: code = Unknown desc = sfcrmuat: [simpleforce] Error. http code: 431 Error Message: Error Code: 2024-01-17 11:39:41.156 UTC [109753] STATEMENT: select * from salesforce_account limit 10 2024-01-17 11:40:18.779 UTC [WARN] hub: stream receive error rpc error: code = Unknown desc = sfcrmuat: [simpleforce] Error. http code: 431 Error Message: Error Code: (0xc000bfe0c0) 2024-01-17 11:40:18.779 UTC [109753] ERROR: rpc error: code = Unknown desc = sfcrmuat: [simpleforce] Error. http code: 431 Error Message: Error Code: 2024-01-17 11:40:18.779 UTC [109753] STATEMENT: select * from salesforce_account limit 10 2024-01-17 11:40:28.105 UTC [WARN] hub: stream receive error rpc error: code = Unknown desc = sfcrmuat: [simpleforce] Error. http code: 431 Error Message: Error Code: (0xc000bfe180) 2024-01-17 11:40:28.106 UTC [109753] ERROR: rpc error: code = Unknown desc = sfcrmuat: [simpleforce] Error. http code: 431 Error Message: Error Code: 2024-01-17 11:40:28.106 UTC [109753] STATEMENT: select * from salesforce_account limit 10 2024-01-17 11:41:27.960 UTC [109687] LOG: disconnection: session time: 0:01:59.999 user=root database=steampipe host=127.0.0.1 port=52538 2024-01-17 15:55:14.909 UTC [WARN] hub: stream receive error rpc error: code = Unknown desc = sfcrmuat: [simpleforce] Error. http code: 431 Error Message: Error Code: (0xc000bfe300) 2024-01-17 15:55:14.909 UTC [109753] ERROR: rpc error: code = Unknown desc = sfcrmuat: [simpleforce] Error. http code: 431 Error Message: Error Code: 2024-01-17 15:55:14.909 UTC [109753] STATEMENT: select * from salesforce_account limit 10

4. connection "sfcrmuat" { plugin = "salesforce" url = "xxxxxxx" username = "xxxxxx" password = "xxxxxxx" token = "xxxxxxx" objects = ["UserRole","User","Proposal_WorkOrder__c","Product_Deal_Detail__c","Opportunity_WorkOrder__c","Opportunity_Objective__c", "Case","Proposal__c","Product_Deal__c", "Workorder","Proposal_item__c"]}

[bigdatasourav]

Hey @KANAK-G, we have tried to reproduce the above error but failed to do so, an HTTP 431 error indicates that the request header fields are too large. This can happen if the Salesforce API request includes too much data in the headers, possibly due to a large set of columns requested.

https://salesforce.stackexchange.com/questions/310014/how-to-fix-error-431-request-header-fields-too-large-http-code431-in-anonymo

Could you please try to execute the below code and let us know if it is working for you?

package main

import (
    "fmt"

    "github.com/simpleforce/simpleforce"
)

func main() {
    // Initialize a new Salesforce client
    sf := simpleforce.NewClient(
        "https://yourInstance.salesforce.com/", // Replace with your Salesforce instance URL
        "steampipe", // Replace with your Client ID
        "43.0",
    )

    // Login with Salesforce credentials
    err := sf.LoginPassword("yourUsername", "yourPassword", "yourToken")
    if err != nil {
        fmt.Printf("Error logging in to Salesforce: %s\n", err)
        return
    }

    // Query to fetch accounts with specific fields
    query := "SELECT Id, Name, AnnualRevenue, Industry, OwnerId, Type FROM Account"

    // Execute the query
    result, err := sf.Query(query)
    if err != nil {
        fmt.Printf("Error executing query: %s\n", err)
        return
    }

    // Iterate over the query results and print the fields
    for _, record := range result.Records {
        fmt.Printf("Id: %s, Name: %s, AnnualRevenue: %v, Industry: %s, OwnerId: %s, Type: %s\n",
            record["Id"], record["Name"], record["AnnualRevenue"], record["Industry"], record["OwnerId"], record["Type"])
    }
}

Also please try the below query and see if it is working for you -

select
  *
from
  salesforce_account
where
  id = '243242343';  

[bigdatasourav]

This PR may solve the above problem.

@KANAK-G, If you get a chance, could you please try out this PR as well?

1. Clone the repository: git clone https://github.com/turbot/steampipe-plugin-salesforce.git
2. Change directory: cd steampipe-plugin-salesforce
3. Switch to the specific branch: git checkout issue-51
4. Execute the make command: make
5. Run your query

[bigdatasourav]

@KANAK-G,closing this issue for now as we haven't received further feedback. The above PR might resolve the problem. Please feel free to reopen this issue if you have any more to share or if the issue persists. Thank you!