forcedotcom / cli

Salesforce CLI
https://developer.salesforce.com/docs/atlas.en-us.sfdx_cli_reference.meta/sfdx_cli_reference/
BSD 3-Clause "New" or "Revised" License
491 stars 78 forks source link

Issue when deploying Custom Indexes that have commas in their name #2928

Closed pabesphoy closed 1 month ago

pabesphoy commented 3 months ago

We are trying to deploy several metadata files to an organization, specifically several custom indexes with the following name, automatically generated by salesforce: Account.RD3_Direccionc,FR_RowID_Magiac.indx-meta

This is in the customindex file. We know for a fact that it is possible to deploy metadata with commas in its name, because we already have custom indexes in the organization such that if we download the metadata, we find commas in their name. However, whenever we try to deploy this file, we get this error: $ sf project convert source --source-dir 'force-app/main/default/customindex/Account.RD3_Direccionc,FR_RowID_Magiac.indx-meta.xml' --json { "code": 1, "context": "Source", "commandName": "Source", "message": "force-app/main/default/customindex/Account.RD3_Direccion__c: File or folder not found", "name": "SfError", "status": 1, "stack": "SfError: force-app/main/default/customindex/Account.RD3_Direccion__c: File or folder not found\n at assertFileExists (/Users/ea_enel/.local/share/sf/client/2.43.7-085948d/node_modules/@salesforce/source-deploy-retrieve/lib/src/collections/componentSetBuilder.js:185:15)\n at validateAndResolvePath (/Users/ea_enel/.local/share/sf/client/2.43.7-085948d/node_modules/@salesforce/source-deploy-retrieve/lib/src/collections/componentSetBuilder.js:182:59)\n at Array.map ()\n at ComponentSetBuilder.build (/Users/ea_enel/.local/share/sf/client/2.43.7-085948d/node_modules/@salesforce/source-deploy-retrieve/lib/src/collections/componentSetBuilder.js:65:44)\n at Source.convert (file:///Users/ea_enel/.local/share/sf/client/2.43.7-085948d/node_modules/@salesforce/plugin-deploy-retrieve/lib/commands/project/convert/source.js:93:55)\n at async Source.run (file:///Users/ea_enel/.local/share/sf/client/2.43.7-085948d/node_modules/@salesforce/plugin-deploy-retrieve/lib/commands/project/convert/source.js:71:9)\n at async Source._run (/Users/ea_enel/.local/share/sf/client/2.43.7-085948d/node_modules/@oclif/core/lib/command.js:311:22)\n at async Config.runCommand (/Users/ea_enel/.local/share/sf/client/2.43.7-085948d/node_modules/@oclif/core/lib/config/config.js:433:25)\n at async run (/Users/ea_enel/.local/share/sf/client/2.43.7-085948d/node_modules/@oclif/core/lib/main.js:92:16)", "exitCode": 1, "warnings": [] } 14:20:35 ~/Documents/01_repo/b2b-crm (merge)

We understan this is a bug, because custom indexes would not be able to be deployed without convertir comma-containing source files. Has anyone ever faced this issue?

The sf version we are using is: $ sf --version
@salesforce/cli/2.43.7 darwin-arm64 node-v20.13.1

{ "architecture": "darwin-arm64", "cliVersion": "@salesforce/cli/2.46.6", "nodeVersion": "node-v20.14.0", "osVersion": "Darwin 23.5.0", "rootPath": "/Users/ea_enel/.local/share/sf/client/2.46.6-191291b", "shell": "zsh", "pluginVersions": [ "@oclif/plugin-autocomplete 3.1.2 (core)", "@oclif/plugin-commands 4.0.2 (core)", "@oclif/plugin-help 6.2.1 (core)", "@oclif/plugin-not-found 3.2.3 (core)", "@oclif/plugin-plugins 5.2.4 (core)", "@oclif/plugin-search 1.1.2 (core)", "@oclif/plugin-update 4.3.5 (core)", "@oclif/plugin-version 2.2.3 (core)", "@oclif/plugin-warn-if-update-available 3.1.5 (core)", "@oclif/plugin-which 3.2.2 (core)", "@salesforce/cli 2.46.6 (core)", "apex 3.1.18 (core)", "auth 3.6.21 (core)", "data 3.4.6 (core)", "deploy-retrieve 3.9.4 (core)", "info 3.3.6 (core)", "limits 3.3.12 (core)", "marketplace 1.2.12 (core)", "org 4.2.1 (core)", "packaging 2.4.6 (core)", "schema 3.3.11 (core)", "settings 2.3.1 (core)", "sobject 1.4.11 (core)", "source 3.4.2 (core)", "telemetry 3.4.0 (core)", "templates 56.2.11 (core)", "trust 3.7.5 (core)", "user 3.5.13 (core)", "sfdx-plugin-source-read 1.2.0 (user) published 239 days ago (Tue Oct 24 2023)" ] }

github-actions[bot] commented 3 months ago

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

github-actions[bot] commented 3 months ago

Hello @pabesphoy :wave: It looks like you didn't include the full Salesforce CLI version information in your issue. Please provide the output of version --verbose --json for the CLI you're using (sf or sfdx).

A few more things to check:

Thank you!

github-actions[bot] commented 3 months ago

Hello @pabesphoy :wave: None of the versions of sf you shared match the latest release.

Shared: 2.43.7, 2.43.7 Latest: 2.46.6

Update to the latest version of Salesforce CLI (docs) and confirm that you're still seeing your issue. You can also try the rc and nightly releases! (docs)

After updating, share the full output of sf version --verbose --json

shetzel commented 3 months ago

specifically several custom indexes with the following name, automatically generated by salesforce

How were these created? A comma in the API name is a very bad idea. The CLI uses a comma character to split on multiple entries and it would be a huge breaking change to lots of people if that were changed.

edencrash commented 3 months ago

As far as we know, administrators do not choose the API name that Salesforce use to refer to Custom Indexes. When refering to Two-Column custom indexes, the autogenerated API Name contains both API Names sepparated by a comma

shetzel commented 3 months ago

As a workaround, are you able to deploy the CustomIndex metadata with sf project deploy start -m CustomIndex?

shetzel commented 3 months ago

Another workaround would be to deploy the customindex directory: sf project convert source --source-dir 'force-app/main/default/customindex

Alfystar commented 3 months ago

hi @shetzel, I'm from the team that found this problem. We have an automated pipeline that works for single assets, loading the entire directory or selecting metadata with the "-m" option is not a valid workaround in our workflow.

Now in an emergency we are releasing them manually but we would like to resume using the pipeline as soon as possible

shetzel commented 3 months ago

@Alfystar - thanks, that makes sense. I've asked the team who owns the CustomIndex metadata type to clarify some things, including whether they can change how the API names are generated. If that's not possible, then the only solution I can think of is to allow the definition of a different split character, which would likely cause other problems.

Alfystar commented 3 months ago

I think a simpler solution is to add an escape character in the cli, such as the '\' before the problematic character.

Or else do, as for example in argvparse, that the lists declare themselves by putting spaces and if the parameter has spaces inside, then use double quotation marks

Alfystar commented 3 months ago

Any news?

mshanemc commented 2 months ago

@Alfystar @edencrash @pabesphoy I think these will deploy just fine using sf project deploy ...

I think the issue is with the project convert source command , which kept support for commas in the multiple flag scenario to avoid it being a breaking change from the old force:source:convert.

yeah, it's probably possible to fix that, maybe even without breaking something else 😄

Why do you need to convert to mdapi format to do a deployment? Maybe there's a better way than waiting on us to do a code change?

Alfystar commented 2 months ago

Hi @mshanemc In our pipeline we convert the package for deployment into source-format to allow us to compare it with the repository. I realize that on this particular metadata it might not be useful, but I'm sure you'll agree with me that a pipeline should be generic.

However, we also did some deployment tests without converting and in any case the cli gives an error, unfortunately

Edit

Looking at the code more closely, I also saw that the problem does not occur with the "source-dir" flag, but with "root-dir" and therefore I cannot even skip the offending directory in the convert

mkMetaCmd = f"sf project convert source --root-dir {sourceFolderShort} --output-dir {metaFolderShort} --json"
git2gus[bot] commented 2 months ago

This issue has been linked to a new work item: W-16343745

mdonnalley commented 2 months ago

@Alfystar I'm working on a fix for this now. In your pipeline are you able to insert a \ into the flag value to escape the comma?

It would look something like this:

sf project convert source --source-dir 'force-app/main/default/customindex/Account.RD3_Direccion__c\,FR_RowID_Magia__c.indx-meta.xml' --json

If that doesn't work for you, we'll have to consider another solution that won't break other customers. Perhaps an environment variable that disables the splitting.

Thanks!

Alfystar commented 2 months ago

Yes! I can easily implement a fix like this, but inside the pipeline I would use your feature --flag-dir, would there be any problems?

As far as I'm concerned I can replace all the , in '\,' both on terminal and on flag-file, if this is the change

Alfystar commented 1 month ago

So from which version of sf the fix will start to work?? Thank you!