Closed aheber closed 3 years ago
Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.
thanks for the cool repro for proving an intermittent bug (repeat until failure).
There's a big revision in progress to the deploy, which won't use that force.js file.
We can use this issue as a test case (mock html from the api AND proper exit code)
Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.
This issue has been linked to a new work item: W-8910494
For any who come after. I opened support case number 40502104 for the problem with the REST API and they indicated that it will be handled as part of the internal work item referenced above, W-8910494.
I am not sure if my issue is related to this but minutes after updating SFDX to sfdx-cli/7.88.4-3b2e55c3f1 win32-x64 node-v14.15.4
I created a new scratch org and then did a push -f.
I get
Deploying with REST SyntaxError: Unexpected token < in JSON at position 1
Then a handful of errors.
A fix for this is in progress. Also, the source commands are switching back to SOAP in the next latest-rc release (probably in an hour or 2). We'll add an env var to use the REST API, which will eventually become the default.
@shetzel will the mentioned change restore the config flag or be limited to a new environment variable?
We're thinking it would be a new env var that would only apply to the source commands.
I don't have a big objection but going that route means I'm going to have a more difficult transition period where I have to sync source code updates with DX version updates.
Right now my scratch org build script flips between REST and SOAP deployments a couple of times mid-script. Of course that doesn't work right now for anyone that has updated. I'm going to have to update that script once the new release comes out then push the team to update DX as well as rebase/update all feature branches to be based off of an updated root branch. I'll also have to update our build server to handle the new version and CI/CD runs won't work while that is transitioning.
It isn't a deal breaker for me but will take some additional effort to make work well. I don't know if it will have a bigger impact on others as I don't think the REST API is deliberately leveraged all that often.
I think I'll watch the RC branch and get the ENV name from there. Then I'll have my script setting both of those for the time being, getting that in about a week early will help a lot in that transition.
Let me clarify because you shouldn't have to modify your scripts too much with the change going into latest-rc today.
Right now in latest:
mdapi:deploy
uses REST but you can switch to SOAP with the --soapdeploy
flag
source:deploy
and source:push
use REST and you can't switch to SOAP (regression)
Coming in latest-rc today:
mdapi:deploy
is unchanged
source:deploy
and source:push
use SOAP and you can't switch to REST (matches v50 behavior)
Coming in a future latest-rc: An environment variable to control SOAP/REST for the source commands. Use this to switch the source commands to use REST, which will eventually be the default but probably not until v52.
I don't know your CI scripts but I think you could either pin to some v50 version temporarily or use the latest-rc version coming out today. If you're able to, it would probably be good to have CI that uses latest and latest-rc, with errors from latest-rc providing an early warning. Ideally, we could jump on early warnings so that they never become bugs in latest.
@shetzel this might be better to have some direct communication but I'll try and explain what I can. Let me know if it would help to setup a meeting and to walk you through any of this.
If you change source:deploy
and source:push
so I can't get to the REST API then you will break me. My current scripts do most of my building using source:deploy
using SOAP and flipping it to REST for my StaticResource deploys, I'm over the 50MB total file size limit for SOAP and have been using REST for those items. I have a workaround that loads them via the tooling API that I took out of the system when we started using the REST API and I could add that back if needed. I think I still have the custom plugin installed on users machines.
For the environment variable, I'm going to have to coordinate between where we actively flip the configuration attribute to moving to actively flipping the environment variable. Right now to deal with this I've added the --soapdeploy
flag to my source:deploy
command and adjusted the script version on my machine only, now I'm building scratch orgs for anyone on my team that needs them because my manipulated version is the only one that is functional. Getting everyone on the same version of source and the same version of SFDX at the same time will be difficult and I'm going to have a problem window where some users will be broken because of this.
My primary complaint is that the existing mechanism is being removed and a replacement is being added without a crossover between those two states to allow customers a window to change and adapt. You're making it very likely that I will have at least a small window where things are broken for some of my users. Add to that it also sounds like you're planning a larger gap where neither of these methods will work. Hopefully you can see where I'm coming from and why I'm raising the problem flag.
I see where you're coming from but your workarounds include a modified core plugin so how can we possibly account for all the various ways people can mod plugins? That said, I can modify the code to look for a soapdeploy option and use that if defined.
I don't think we're quite aligned yet. I've only modified the core plugin this week due to the changes we've discussed.
Up until this recent release I've been using the config attribute applied via sfdx force:config:set restDeploy=true
and conversely sfdx force:config:set restDeploy=false
as that was officially supported and subsequently removed without warning in this release. That config attribute changed the behavior of all of the deployments via the CLI. The linked repo to replicate the problem at the base of this issue uses this configuration attribute to force the REST deploy exactly how I've been discussing.
My concern is that my existing scripts use that configuration attribute and you're talking about having a period of time where neither that attribute is available nor the proposed environment variable, we will have lost the ability to force the REST API for the source:deploy
command entirely and that causes problems for me. I'm asking for a world where both the configuration attribute and the environment variable exist simultaneously and both are respected (might be a problem where the configuration attribute can disable as well as enable) and the configuration attribute receives a proper deprecation cycle.
My modified core is only being used now because I have no other way to force source:deploy
to leverage the REST API. I'm in no way asking you to respect my changes, only mentioning them so you understand how much the currently released version of SFDX has broken our ability to get work done. My ask is that we take the way the world used to work and provide a smooth transition to the way you're proposing it should work.
If it helps, the block of code I'm really talking about is in the mdapiDeployApi.js
file, as recently as 7.85.1
as best I can tell.
async _sendMetadata(zipPath, options) {
zipPath = path.resolve(zipPath);
const zipStream = this._createReadStream(zipPath);
if (await mdApiUtil_1.MetadataTransportInfo.isRestDeploy()) {
this._log('*** Deploying with REST ***');
return this.force.mdapiRestDeploy(this.scratchOrg, zipStream, convertParamsToDeployOptions(options));
}
else {
return this.force.mdapiSoapDeploy(this.scratchOrg, zipStream, convertParamsToDeployOptions(options));
}
}
As of 7.88.4
that method now looks like this:
async _sendMetadata(zipPath, options) {
zipPath = path.resolve(zipPath);
const zipStream = this._createReadStream(zipPath);
// REST is the default path, unless specified with the soapddeploy flag on the command.
if (options.soapdeploy) {
this._log('*** Deploying with SOAP ***');
return this.force.mdapiSoapDeploy(this.scratchOrg, zipStream, convertParamsToDeployOptions(options));
}
else {
this._log('*** Deploying with REST ***');
return this.force.mdapiRestDeploy(this.scratchOrg, zipStream, convertParamsToDeployOptions(options));
}
}
Chasing the isRestDeploy
method will take you back to the configuration option that I've been referencing.
I should probably add I totally understand this should have been a fully transparent change and would have been successful if the REST API hadn't had a problem. This entire topic seems to be caused by that dependency failure and I get that, otherwise I would be in full support and never look back.
@heber - thanks for the explanation. I think I'm clear about the state of things now and I have to undo that change I talked about above. I definitely want a smoother transition for this.
I completely forgot about the restDeploy SFDX config value and when you were mentioning a config setting I mistakenly thought you were referring to your soapdeploy command parameter modification. Still, I thought (incorrectly) that only applied to mdapi commands and not source commands. That definitely changes the fix for this.
The restDeploy config needs to be respected and will be the way to toggle soap/rest for all deploys. I'll make some changes and post more about each usecase here when it's ready. You may know this already but all SFDX config settings are also available as env vars, so setting SFDX_REST_DEPLOY
should toggle between soap/rest as well, which should be an easier change to make in CI.
RE: your last post, yes that was the assumption but unfortunately it didn't work out that way. There will still be the serverside JSON problem until that is fixed in the core app. So I'm only addressing what can be fixed in the CLI and working with the appropriate team for the other fix(es).
Awesome, thanks for bearing with me while I figured out the details we needed.
I did not realize that config values could be managed via env vars, that will come in handy.
Looking forward to the upcoming fix.
@aheber and everyone from #884
Let me try to summarize what happened and the CLI fix that will be applied.
With the v50.0 --> v51.0 release several weeks ago there were 2 bugs introduced that affect mdapi and source deploys:
restDeploy
SFDX config variable was no longer being respected by the commands, which meant there was no way to switch from the new default (REST API) back to the SOAP API using that config var. A CLI fix is coming in latest-rc for this.Here are details on what to expect with the restDeploy
CLI fix:
source:deploy
, source:push
, and mdapi:deploy
should all use REST by default.mdapi:deploy
should use SOAP if the soapdeploy command flag is set regardless of the restDeploy
config var.source:deploy
and source:push
should use SOAP if restDeploy config var is set to false either with config:set
command or the SFDX_REST_DEPLOY
env var.config:set
command or the SFDX_REST_DEPLOY
env var.@shetzel, this all looks really good to me and that plan satisfies all my needs.
Is there a reason not to give the source:deploy
command a soapdeploy
flag? It wouldn't be necessary for anything I'm doing but I could see it being useful to keep some flag parity between the two deploy types. This is mostly a curiosity question and I'm not making a strong case that it is needed.
No reason that I know of other than command and plugin ownership internals within SFDC. We generally like to have flag parity between mdapi: and source: so definitely something we could add. Someday it'll all be open source and then anyone can submit a PR. :)
For all these issues, is there a hotfix pathway?
The CLI fix is available now in sfdx-cli@latest-rc, which will become latest this Thursday. I don't know enough about the serverside bugs to provide useful fix information. It's with another team and they're working on it so pretty sure it will be fixed and rolled out soon.
@shetzel installed sfdx-cli@latest-rc
(currently in sfdx-cli/7.90.1) and seems like this is still breaking for force:source:deploy
or force:source:push
?
SyntaxError: Unexpected token < in JSON at position 1 at JSON.parse (<anonymous>)
@pserquiz - The fix I put in latest-rc was to restore the behavior of the restDeploy
config variable so that people could set it to false and use SOAP as a workaround. There will be more fixes going into the CLI and the core app. The error you're seeing is due to the server sending HTML or XML but the CLI is expecting JSON. That will be another fix to come soon, as will the incorrect exit code problem.
oh I see, thanks @shetzel and sorry for the confusion. There is any workaround for this right now then? not being able to deploy / push anything since yesterday. what I noticed in the html is that it shows us a message for Illegal Request
and You have sent us an Illegal URL or an improperly formatted request.
@pserquiz - If you update to the latest-rc of the CLI you can then use the restDeploy
config variable to switch the source commands back to SOAP. That version will be promoted to latest tomorrow so alternatively you can wait for the release tomorrow. Another option would be to install an earlier CLI version where the default is still SOAP.
Not sure if this is an option for you but you could also combine source:convert
with mdapi:deploy
to do something similar. E.g.:
sfdx force:source:convert -d md_src
sfdx force:mdapi:deploy -d md_src --soapdeploy
Updated to 7.90.2-b8f9206a5c win32-x64 node-v14.15.4. I have set restDeploy=false. This error is STILL occurring.
@shetzel yeah couldn't really do that in my current project, I just uninstalled it and installed an older version for now. Thanks
@CrownSeven make sure your salesforcedx is on latest and isn't still the downgraded plugin.
With the latest version (7.91.0-6a6ed69ebe), I am getting ` PS C:\xxxx> sfdx force:source:push -u MyScratchOrg Deploying with REST ERROR running force:source:push:
Illegal Request |
You have sent us an Illegal URL or an improperly formatted request. |
`
I am also seeing this error when deploying.
sfdx force:source:deploy --sourcepath c:\ --json --loglevel fatal ended with exit code 1
<html>
<body>
<table cellspacing=10>
<tr>
<td><span style="font-weight: bold; font-size: 12pt;">Illegal Request</span></td>
</tr>
<tr>
<td>You have sent us an Illegal URL or an improperly formatted request.
</td>
</tr>
<tr>
<td>
</td>
</tr>
</table>
<!-- Body events -->
<script type="text/javascript">function bodyOnLoad(){if(window.PreferenceBits){window.PreferenceBits.prototype.csrfToken="null";};}function bodyOnBeforeUnload(){}function bodyOnFocus(){}function bodyOnUnload(){}</script>
</body>
</html>
<!--
...................................................................................................
...................................................................................................
...................................................................................................
...................................................................................................
-->
SFDX Version: sfdx-cli/7.91.0-6a6ed69ebe win32-x64 node-v14.15.4
I am getting this 'Illegal Request' html error myself, its actually very random. When I restart my VS Code for first time it works but is back again.
same issue as @mhamzas and @MJ12358 but with the command "sfdx force:source:push"
The only solution that worked for me is: https://github.com/forcedotcom/cli/issues/884#issuecomment-785927296 @ricofairsailcom @irfanKhanSage @MJ12358
The issue is only for Windows machine apparently, just downgrading to v7.81 should work. npm install --global sfdx-cli@7.81.0
For those of you seeing this error: You have sent us an Illegal URL or an improperly formatted request.
Please provide as many details as you can. At a minimum, CLI version and salesforcedx plugin version. OS. The command with parameters (values can be obfuscated or fake). If possible, get the URL that was used in the request. Choose whatever tool you like to do that or optionally debug the code.
As was stated earlier in this thread you can switch to use SOAP instead of REST by sfdx config:set restDeploy=false
or the SFDX_REST_DEPLOY
env var to workaround any REST deploy errors.
Hi @shetzel,
I'm having the same issue with the illegal URL. Here are my details - let me know if you need any more details:
OS: Windows 10 Pro Build 19042.867
sfdx plugins --core
@oclif/plugin-autocomplete 0.3.0 (core)
@oclif/plugin-commands 1.3.0 (core)
@oclif/plugin-help 3.2.2 (core)
@oclif/plugin-not-found 1.2.4 (core)
@oclif/plugin-plugins 1.10.0 (core)
@oclif/plugin-update 1.4.0-3 (core)
@oclif/plugin-warn-if-update-available 1.7.0 (core)
@oclif/plugin-which 1.0.3 (core)
@salesforce/sfdx-trust 3.6.0 (core)
alias 1.1.8 (core)
auth 1.5.1 (core)
config 1.2.7 (core)
generator 1.1.5 (core)
salesforcedx 51.5.0 (core)
├─ schema 1.0.4 (core)
├─ limits 1.0.4 (core)
├─ apex 0.1.4 (core)
├─ org 1.5.0 (core)
├─ custom-metadata 1.0.11 (core)
├─ templates 51.3.0 (core)
├─ user 1.2.0 (core)
├─ @salesforce/sfdx-plugin-lwc-test 0.1.7 (core)
└─ salesforce-alm 51.6.3 (core)
sfdx-cli 7.93.0 (core)
telemetry 1.1.1 (core)
sfdx force:source:push -f --json --loglevel=trace
{
"status": 1,
"name": "SfdxError",
"message": "\n\n\n\n\n\n<html>\n<head><title>An internal server error has occurred</title></head>\n<body>\n\n\n<div style=\"display:none;\" id=\"errorTitle\">An internal server error has occurred</div>\n<div style=\"display:none;\" id=\"errorDesc\">An error has occurred while processing your request. The salesforce.com support team has been notified of the problem. If you believe you have additional information that may be of help in reproducing or correcting the error, please contact <a href=\"https://help.salesforce.com/apex/hthome\">Salesforce Support</a>. Please indicate the URL of the page you were requesting, any error id shown on this page as well as any other related information. We apologize for the inconvenience. <br/><br/>Thank you again for your patience and assistance. And thanks for using salesforce.com!</div>\n<table cellspacing=10>\n<tr><td><span style=\"font-weight: bold; font-size: 12pt;\">An internal server error has occurred</span></td></tr>\n<tr><td>\nAn error has occurred while processing your request. The salesforce.com support team has been notified of the problem. If you believe you have additional information that may be of help in reproducing or correcting the error, please contact <a href=\"https://help.salesforce.com/apex/hthome\">Salesforce Support</a>. Please indicate the URL of the page you were requesting, any error id shown on this page as well as any other related information. We apologize for the inconvenience. <br/><br/>Thank you again for your patience and assistance. And thanks for
using salesforce.com!\n<br><br>\nError ID: 1104561287-91770 (1534884652)\n</td>\n</tr>\n<tr><td>\n<br clear=\"all\"><br><br>\n\n\n</td></tr>\n</table>\n\n</td></tr>\n</table>\n\n\n\n</body>\n</html>\n",
"exitCode": 1,
"commandName": "SourcePushCommand",
"stack": "SfdxError: \n\n\n\n\n\n<html>\n<head><title>An internal server error has occurred</title></head>\n<body>\n\n\n<div style=\"display:none;\" id=\"errorTitle\">An internal server error has occurred</div>\n<div style=\"display:none;\" id=\"errorDesc\">An error has occurred while processing your request. The salesforce.com support team has been notified of the problem. If you believe you have additional information that may be of help in reproducing or correcting the error, please contact <a href=\"https://help.salesforce.com/apex/hthome\">Salesforce Support</a>. Please indicate the URL of the page you were requesting, any error id shown on this page as well as any other related information. We apologize for the inconvenience. <br/><br/>Thank you again for
your patience and assistance. And thanks for using salesforce.com!</div>\n<table cellspacing=10>\n<tr><td><span style=\"font-weight: bold; font-size: 12pt;\">An internal server error has occurred</span></td></tr>\n<tr><td>\nAn error has occurred while processing your request. The salesforce.com support team has been notified of the problem. If you believe you have additional information that may be of help in reproducing or correcting the error, please contact <a href=\"https://help.salesforce.com/apex/hthome\">Salesforce Support</a>. Please indicate the URL of the page you were requesting, any error id shown on this page as well as any other related information. We apologize for the inconvenience. <br/><br/>Thank you again for your patience and assistance. And thanks for using salesforce.com!\n<br><br>\nError ID: 1104561287-91770 (1534884652)\n</td>\n</tr>\n<tr><td>\n<br clear=\"all\"><br><br>\n\n\n</td></tr>\n</table>\n\n</td></tr>\n</table>\n\n\n\n</body>\n</html>\n\n at Function.wrap (C:\\Users\\chris\\AppData\\Local\\sfdx\\client\\7.93.0-9e2365b\\node_modules\\@salesforce\\core\\lib\\sfdxError.js:169:20)\n at Request._callback (C:\\Users\\chris\\AppData\\Local\\sfdx\\client\\7.93.0-9e2365b\\node_modules\\salesforce-alm\\dist\\lib\\core\\force.js:587:41)\n
at Request.self.callback (C:\\Users\\chris\\AppData\\Local\\sfdx\\client\\7.93.0-9e2365b\\node_modules\\request\\request.js:185:22)\n at Request.emit (node:events:369:20)\n at Request.<anonymous> (C:\\Users\\chris\\AppData\\Local\\sfdx\\client\\7.93.0-9e2365b\\node_modules\\request\\request.js:1154:10)\n at Request.emit (node:events:369:20)\n at IncomingMessage.<anonymous> (C:\\Users\\chris\\AppData\\Local\\sfdx\\client\\7.93.0-9e2365b\\node_modules\\request\\request.js:1076:12)\n at Object.onceWrapper (node:events:475:28)\n at IncomingMessage.emit (node:events:381:22)\n at endReadableNT (node:internal/streams/readable:1307:12)\n at processTicksAndRejections (node:internal/process/task_queues:81:21)",
"warnings": []
}
Edit: As mentioned, using sfdx config:set restDeploy=false
works. These errors occur when using REST.
@shetzel I'm getting the illegal URL error, but I've only seen this using the built-in VS Code command palette push function. Behaviours I've seen:
sfdx force:source:push
to scratch org using command line: Workssfdx force:source:push --json --loglevel fatal
to scratch org using command line: WorksI've also seen the outcomes described in 2-4 while working on a scratch org created by another user which was added through authentication.
Quite elusive but hope this is helpful. I'm not sure how I can view the URL used but happy to try if there's a way to see this detail
Error output:
Starting SFDX: Push Source to Default Scratch Org
11:54:51.869 sfdx force:source:push --json --loglevel fatal
11:55:31.654 sfdx force:source:push --json --loglevel fatal
ended with exit code 1
SfdxError:
<html>
<body>
<table cellspacing=10>
<tr>
<td><span style="font-weight: bold; font-size: 12pt;">Illegal Request</span></td>
</tr>
<tr>
<td>You have sent us an Illegal URL or an improperly formatted request.
</td>
</tr>
<tr>
<td>
</td>
</tr>
</table>
<!-- Body events -->
<script type="text/javascript">function bodyOnLoad(){if(window.PreferenceBits){window.PreferenceBits.prototype.csrfToken="null";};}function bodyOnBeforeUnload(){}function bodyOnFocus(){}function bodyOnUnload(){}</script>
</body>
</html>
<!--
...................................................................................................
...................................................................................................
...................................................................................................
...................................................................................................
-->
SFDX core, OS and node versions:
sfdx-cli/7.93.1-762bca056d win32-x64 node-v14.15.4
(Windows 10.0.19042)
SFDX plugin versions:
@dxatscale/sfpowerscripts 3.0.6
@oclif/plugin-autocomplete 0.3.0 (core)
@oclif/plugin-commands 1.3.0 (core)
@oclif/plugin-help 3.2.2 (core)
@oclif/plugin-not-found 1.2.4 (core)
@oclif/plugin-plugins 1.10.0 (core)
@oclif/plugin-update 1.4.0-3 (core)
@oclif/plugin-warn-if-update-available 1.7.0 (core)
@oclif/plugin-which 1.0.3 (core)
@salesforce/sfdx-trust 3.6.0 (core)
alias 1.1.8 (core)
auth 1.5.1 (core)
bbdoc 3.4.3
config 1.2.7 (core)
generator 1.1.5 (core)
salesforcedx 51.5.0 (core)
├─ schema 1.0.4 (core)
├─ limits 1.0.4 (core)
├─ apex 0.1.4 (core)
├─ templates 51.3.0 (core)
├─ org 1.5.0 (core)
├─ custom-metadata 1.0.11 (core)
├─ @salesforce/sfdx-plugin-lwc-test 0.1.7 (core)
├─ user 1.2.0 (core)
└─ salesforce-alm 51.6.3 (core)
sfdmu 3.9.4
sfdx-cli 7.93.1 (core)
sfpowerkit 2.8.1 (2.8.1)
telemetry 1.1.1 (core)
Salesforce CLI Integration VS Code plugin version v51.6.0
On VS Code we can run command: sfdx config:set restDeploy=false
As of the 51.7.1 release yesterday the force:source:*
and force:mdapi:*
commands are back to using SOAP as the default as well as appropriately exiting and printing HTML errors to the terminal when they come back from the REST API.
Summary
Deploying to a fresh scratch org using the REST Metadata API sometimes results in deployment errors, often ~15-30 minutes after org creation, it works until it doesn't. Those errors aren't handled well and the result ended in "success" exit code (0) when it should be error (1+). Errors in parsing a JSON message that is actually an HTML message causes everything to break and not handle the error correctly.
SyntaxError: Unexpected token < in JSON at position 1
(The REST API crashing soon after creation I will be opening a support case for, this is specifically about the DX failings)
Steps To Reproduce:
Repository to reproduce: sfdx-deployfailure-example
I have an example repository and the README has steps to run a script that creates an org, sets it to deploy via REST, then deploys until it breaks. It doesn't break 100% of the time but I'm probably seeing ~70+% failure rates, though each run could take 30+ minutes to test before giving up on it.
Expected result
Edge cases should be handled properly and when SFDX crashes it should result in a non-zero exit code.
Actual result
SFDX crashes out and prints to the STDERR channel but returns an exit code of 0 reporting success. This has been a huge problem is automated scripts that don't realize a deployment failed before moving on to the next step.
Additional information
See the README in the repo.
SFDX CLI Version(to find the version of the CLI engine run sfdx --version):
sfdx-cli/7.85.1-2fb9e41053 win32-x64 node-v12.18.3
SFDX plugin Version(to find the version of the CLI plugin run sfdx plugins --core)
sfdx-cli 7.85.1
OS and version: Windows 10