forcedotcom / dataloader

Salesforce Data Loader
BSD 3-Clause "New" or "Revised" License
480 stars 294 forks source link

Cannot load files to ContentVersion in Recent versions of Dataloader #1189

Closed jclark-dot-org closed 3 months ago

jclark-dot-org commented 3 months ago

Describe the bug Per Salesforce Help (https://help.salesforce.com/s/articleView?id=000382372&type=1), Dataloader can load files into ContentVersion by including the full path to the local file in the FirstPublishLocationId field. This worked prior to recent changes to Dataloader to better support External Ids, but no longer works. It works as recently as Dataloader v57, and perhaps newer, but definitely does not work in Dataloader v60.0, 61.0, or 61.1. When trying to load files with these versions of Dataloader, every row fails with the error:

"Error converting value to correct data type: Cannot invoke ""String.equalsIgnoreCase(String)"" because ""mimeType"" is null"

To Reproduce Steps to reproduce the behavior:

  1. Create a CSV for a ContentVersion as outlined in https://help.salesforce.com/s/articleView?id=000382372&type=1. I'm not including an example here because the local paths and Ids (OwnerId, FirstPublishLocationId) will differ.
  2. Attempt to insert the file using Dataloader.
  3. Process fails with error as outlined above.

Expected behavior New ContentVersion records are created.

Screenshots

image

Desktop (please complete the following information):

Additional context While we can use older versions of Dataloader in most cases, those versions do not correctly support using an External Id for FirstPublishLocationId, so this error many use cases, unless we first export and map a cross reference of object ids, which we'd like to avoid.

sdl.log

ashitsalesforce commented 3 months ago

Hi @jclark-dot-org , here are the steps I took to try and reproduce the issue with Data Loader v61.1.0:

I am able to successfully import it using SOAP API and default batch size of 200. Here is a screenshot of Related tab in the Account object with external id "uo1" showing that FirstPublishLocation relationship was created correctly using external id.

Screenshot 2024-06-28 at 8 47 02 AM

Can you check if you are using SOAP API or Bulk in Settings dialog? If you are using SOAP, share your config.properties and csv files after removing any personal or company-sensitive information for further troubleshooting.

R0Wi commented 3 months ago

I'm facing the same issue. In my opinion this has been introduced in https://github.com/forcedotcom/dataloader/commit/b42979e1e92c7b23700857a478bef94127a433bb (v60 onwards). The error originates from this line: https://github.com/forcedotcom/dataloader/blob/7a0f31e08b1cec3439b46e233038707fc7f5d3d7/src/main/java/com/salesforce/dataloader/dyna/FileByteArrayConverter.java#L109

The variable mimeType (which is a String) is null and therefore line 111 throws the mentioned exception. I digged a bit deeper and found out that Files.probeContentType is not able to determine the mimetype for files without a file extension. In my case I'm using files named exactly the same as the Id of the ContentVersion object without any file extension

image

I think this causes the issue. IMHO a simple change like this should fix it:

 if (mimeType != null // add this ...
            && mimeType.equalsIgnoreCase("text/plain")
             && config != null
             && config.getBoolean(Config.LOAD_PRESERVE_WHITESPACE_IN_RICH_TEXT)
             && "ContentNote".equalsIgnoreCase(config.getString(Config.ENTITY))) {
// ...
}
ashitsalesforce commented 3 months ago

thanks @R0Wi for identifying the root-cause and fixing the issue. I have merged your PR. The next release of data loader will contain the fix.