Closed sheldonleelazarus closed 4 years ago
@sheldonleelazarus Could you please try to play with it to make it reproducible?
So what I've worked on was parsing large files referenced in a git diff (e.g. a CustomObject > 1 MB). Now this reads as if the diff itself causes problems...
@amtrack sent you an email with the steps to replicate
Hi, I have the same issue in my local, I the issue is inside the diff.js
file in the Diff.stream
function, when parseDiff hasn't at the beginning git diff header, then it can fail.
A sample of git diff header is:
diff --git a/src/classes/sample.cls b/src/classes/sample.cls
index af7583d3..aa3a9968 100644
--- a/src/classes/sample.cls
+++ b/src/classes/sample.cls
I rewrited the Diff.stream
code in my local like:
Diff.stream = function(opts) {
var git = new Git(process.cwd());
var unpackagedPath = path.join(process.cwd(), 'src');
opts = opts || {};
return miss.through.obj(function(diff, enc, cb) {
var strDiff = diff.toString();
var index = strDiff.indexOf('diff --git a/');
if (index <= 0) return cb();
strDiff = strDiff.substring(index);
if (!strDiff) return cb();
var files = parseDiff(strDiff);
if (files.length <= 0) return cb();
var containerFrom = new MetadataContainer();
var containerTo = new MetadataContainer();
files.forEach(function(file) {
...
});
cb(null, {
source: containerFrom,
target: containerTo
});
});
I think this code doesn't resolve completely the issue, for example when the git diff header is spliced between different streams.
@amtrack, does this helps? do you need help with the issue?
Regards
@froucher Yes, please help me on this. If possible: Can we treat the diff as a string and not as a stream at all?
@sheldonleelazarus, yesterday I do a pull request which can resolve this issue, do you know how to test my branch?
Thanks
@froucher yes please let me have a look
@froucher i done a quick test and from what i can see when I create changes on an object eg creating new fields, updating field settings on an object more than 3MB the command git diff master testing | force-dev-tool changeset create testingr does not pick up any changes on the object i changed and therefore package resolves to a blank package.xml
I am still running more tests but this is the issue i found with regards to your changes
Hi, @sheldonleelazarus, Can you provide a test case that checks this issue?
Hello @froucher - I am also facing same issue as @sheldonleelazarus , I tried to use changeset and ended with only blank pacakge.xml.
I would like to use the git diff and create deployment method for salesforce ci/cd in bitbucket pipelines, I am stuck with creating deployment for the changed meta data.
@Snehil03 Could you please describe what commands exactly you are running and what version of force-dev-tool you're using?
Also, if possible the output (redacted) of the git diff
command would be interesting.
@amtrack
git diff --no-renames feature/lets_fix_cicd master | force-dev-tool changeset create aexp -f Manifest: <?xml version="1.0" encoding="UTF-8"?>
exported metadata container to ../../config/deployments/aexp
force-dev-tool --version 0.0.0-development
Hi,
Please, can you try with last version 2.8.4?
And if it still does not work, then can you send us the "git diff" result?
git diff --no-renames feature/lets_fix_cicd master > output.diff
If you prefer, then you can send this attachment with a private email.
Thanks
Hello ,
Kindly find below diff :-
diff --git a/mdapi/src/classes/BORPO_DTO_Account.cls b/mdapi/src/classes/BORPO_DTO_Account.cls index 952b881a1..274602803 100644 --- a/mdapi/src/classes/BRPO_DTO_Account.cls +++ b/mdapi/src/classes/BRPO_DTO_Account.cls @@ -18,7 +18,6 @@ global class BRPO_DTO_Account { private String country;
Thanks, Snehil Belekar
On Thu, 15 Oct 2020 at 16:02, Felipe Roucher notifications@github.com wrote:
Hi,
Please, can you try with last version 2.8.4?
And if it still does not work, then can you send us the "git diff" result?
git diff --no-renames feature/lets_fix_cicd master > output.diff
If you prefer, then you can send this attachment with a private email.
Thanks
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/amtrack/force-dev-tool/issues/263#issuecomment-709346743, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMCU44P2P5AGIXEVQSQRXHTSK36H3ANCNFSM4PTATQMA .
@Snehil03 Looking at your diff, I see that your Metadata lives in mdapi/src
.
force-dev-tool expects the Metadata to live in src
.
You can work around this limitation by changing the working directory (to mdapi
) and adding a --relative
argument to the git command:
cd mdapi
git diff --relative --no-renames feature/lets_fix_cicd master | force-dev-tool changeset create aexp -f
Hey, Thanks it worked !
On Thu, 15 Oct 2020 at 18:13, Matthias Rolke notifications@github.com wrote:
@Snehil03 https://github.com/Snehil03 Looking at your diff, I see that your Metadata lives in mdapi/src. force-dev-tool expects the Metadata to live in src.
You can work around this limitation by changing the working directory and adding a --relative argument to the git command:
cd mdapigit diff --relative --no-renames feature/lets_fix_cicd master | force-dev-tool changeset create aexp -f
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/amtrack/force-dev-tool/issues/263#issuecomment-709433013, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMCU44IMNHO32YTLWLH2TU3SK4NTRANCNFSM4PTATQMA .
-- Thanks, Snehil Belekar
Hi @amtrack started getting this error message after your last commit when i try to parse a diff command through the force-dev-tool, it doesnt happen on every diff just on some diffs
/usr/local/lib/node_modules/force-dev-tool/node_modules/parse-diff/index.js:81 return file.chunks.push(current); ^
TypeError: Cannot read property 'chunks' of null at Array.chunk (/usr/local/lib/node_modules/force-dev-tool/node_modules/parse-diff/index.js:81:17) at parse (/usr/local/lib/node_modules/force-dev-tool/node_modules/parse-diff/index.js:139:13) at module.exports (/usr/local/lib/node_modules/force-dev-tool/node_modules/parse-diff/index.js:147:5) at DestroyableTransform._transform (/usr/local/lib/node_modules/force-dev-tool/lib/diff.js:65:15) at DestroyableTransform.Transform._read (/usr/local/lib/node_modules/force-dev-tool/node_modules/readable-stream/lib/_stream_transform.js:184:10) at DestroyableTransform.Transform._write (/usr/local/lib/node_modules/force-dev-tool/node_modules/readable-stream/lib/_stream_transform.js:172:83) at doWrite (/usr/local/lib/node_modules/force-dev-tool/node_modules/readable-stream/lib/_stream_writable.js:428:64) at writeOrBuffer (/usr/local/lib/node_modules/force-dev-tool/node_modules/readable-stream/lib/_stream_writable.js:417:5) at DestroyableTransform.Writable.write (/usr/local/lib/node_modules/force-dev-tool/node_modules/readable-stream/lib/_stream_writable.js:334:11) at Socket.ondata (_stream_readable.js:716:22) Running Validation of directory config/deployments/51379_release_validation to remote RELEASE Error: Validation failed.