force:source:push gacks when force:source:legacy:push succeeds

[aidan-harding]

Summary

I have an unlocked package repo that I haven't built for a while. When setting up my scratch org for work recently, I ran sfdx force:source:push. The response from the CLI was:

sfdx force:source:push --forceoverwrite 
*** Pushing with SOAP API v51.0 ***
DEPLOY PROGRESS | ████████████████████████████████████████ | 49/49 Components
Updating source tracking... done

However, when I go to the scratch org, the deployment status says that it failed with UNKNOWN_EXCEPTION: An unexpected error occurred. Please include this ErrorId if you contact support: 2080389041-229584 (208639346). And nothing is deployed.

When I run the legacy command, it succeeds:

 sfdx force:source:legacy:push
WARNING: The command "SourcePushCommand" has been deprecated and will be removed in v56.0 or later. Use "force:source:push" instead.
*** Deploying with SOAP ***
Job ID | 0Af3G00000P5yoiSAB
SOURCE PROGRESS | ████████████████████████████████████████ | 363/363 Components
*** Deploying with SOAP ***
Job ID | 0Af3G00000P5yuqSAB
SOURCE PROGRESS | ████████████████████████████████████████ | 8/8 Components
=== Pushed Source

I was able to get the non-legacy command to work by adding "pushPackageDirectoriesSequentially": true to my sfdx-project.json file.

Steps To Reproduce:

The metadata causing this issue is in a private repository. If the gack code and other details are not enough, then I may be able to share the code with individuals at Salesforce but I cannot post it here. If that's not workable, I could perhaps try to trim it down to a minimal reproduction, but I don't have much time to spare on that since I have viable workarounds for the customer (use legacy mode or the extra property in sfdx-project.json).

Expected result

Source should have deployed, as per the legacy push. Or, at least, provided an error message in the CLI.

Actual result

No error message, no code deployed.

System Information

• Which shell/terminal are you using? (e.g. bash, zsh, powershell 5, powershell 7, cmd.exe, etc.)

zsh on Mac.

sfdx version --verbose --json
{
        "cliVersion": "sfdx-cli/7.158.1",
        "architecture": "darwin-x64",
        "nodeVersion": "node-v16.15.1",
        "pluginVersions": [
                "@dx-cli-toolbox/sfdx-toolbox-package-utils 0.8.3",
                "@oclif/plugin-autocomplete 1.3.0",
                "@oclif/plugin-commands 1.3.0 (core)",
                "@oclif/plugin-help 5.1.12",
                "@oclif/plugin-not-found 1.2.6 (core)",
                "@oclif/plugin-plugins 1.10.11 (core)",
                "@oclif/plugin-update 1.5.0 (core)",
                "@oclif/plugin-warn-if-update-available 1.7.3 (core)",
                "@oclif/plugin-which 1.0.4 (core)",
                "@salesforce/analytics 1.0.9",
                "@salesforce/sfdx-plugin-lwc-test 1.0.0 (core)",
                "alias 2.1.0 (core)",
                "apex 0.13.0 (core)",
                "auth 2.2.2 (core)",
                "community 2.0.0 (core)",
                "config 1.4.12 (core)",
                "custom-metadata 2.0.0 (core)",
                "data 2.0.4 (core)",
                "generator 2.0.2 (core)",
                "info 2.0.1 (core)",
                "limits 2.0.1 (core)",
                "org 2.0.1 (core)",
                "salesforce-alm 54.6.2 (core)",
                "schema 2.1.1 (core)",
                "sfdmu 4.15.0",
                "sfdx-cli 7.158.1 (core)",
                "sfdx-hardis 2.90.0",
                "shane-sfdx-plugins 4.43.0",
                "├─ @mshanemc/sfdx-sosl 1.1.0",
                "└─ @mshanemc/plugin-streaming 1.1.7",
                "signups 1.2.0 (core)",
                "source 2.0.3 (core)",
                "telemetry 2.0.0 (core)",
                "templates 55.0.0 (core)",
                "trust 2.0.1 (core)",
                "user 2.1.0 (core)"
        ],
        "osVersion": "Darwin 21.5.0"
}

[github-actions[bot]]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[git2gus[bot]]

This issue has been linked to a new work item: W-11410766

[maggiben]

Hello, @aidan-harding I opened a new bug W-11410766 where you can privately upload a sample org for us to repro, thanks !

[mshanemc]

based on the gack, the metadata API is complaining that received two objects that are exactly the same (==) which is why sequential deploy fixed it.

So I see potentially 2 bugs here

1. new push is not handling duplicated components as nicely as legacy did. Maybe that's a bug?
2. it's reporting success, but it definitely seems not successful (❗ )

[aidan-harding]

Hi @maggiben, thanks for that!

When I open the link to the bug, it invites me to sign in to Okta. I guess I sign in with my Salesforce credentials from our org? I tried that but it won't accept it.

So, I've put a zip file up on our Google Drive. If you use this link, it'll ask me to approve and I'll grant it https://drive.google.com/file/d/1-X_LFVHkhM0WndyQjqPq_ygYw_jekj__/view?usp=sharing

You'll need to install the package dependencies before pushing. Those are just packages listed in the sfdx-project.json, and I think that anyone will have the required access to do that. They don't have installation keys.

@mshanemc That makes sense. The project is an unlocked package where some data is in an unpackaged directory. The unpackaged elements are things that are separate files in sfdx format, but I think they are the same files in old-school XML. I think it all becomes XML for the deployment, so maybe that's the issue. It's a legit use-case because I need stuff like page layouts to test my package in the scratch org. But I don't want the package to control/overwrite what the subscriber org does with layouts.

[mshanemc]

@aidan-harding thanks for the metadata!

The issue is definitely the duplicate for those 2 layouts, which is why sequential helped. To check my understanding, you want to have 1 layout that gets packaged, and then another layout (same obj, same name) that is just for testing in your scratch org, presumably more cases than the "basic" on that gets packaged?

I think that's a good use case for sequential (which is what legacy push was doing) OR a follow-on deploy of just that folder (ex: sfdx force:source:deploy -p unpackaged --tracksource)

Alternatively, you could have it be a standalone page layout (same obj, different page layout name) so the conflict isn't an issue. You could probably even update Profile.PageLayoutAssignments to default your scratch to the 'alternative' layouts.

I'm gonna close the issue because other than the gack, it's working like it's supposed to (not allowing duplicate metadata in the same deployment)

[aidan-harding]

Ha... thanks for looking into it @mshanemc. The layout is not supposed to be in the package at all. It must have been pulled down by accident 😳

I'm happy to take your opinion on closing this issue, but I would say that this is exactly the sort of behaviour from the CLI that makes it hard for people to adopt. Confusing or missing error messages can end up with the user having to spend a lot of time finding the source of their own mistake (and blaming the tool along the way!).

[mshanemc]

100% with you that gacks are uncool and that a better error message from the server would have been nice--the CLI could then tell you what component(s) caused the issue.

the CLI (and SDR library) could maybe produce warnings about duplicate components (we expect customObjects to be duplicated across packages sometimes, but most types shouldn't), which would have helped even with the gack.