diff resulting to high number of files takes a long time

[okram999]

hi @amtrack my git diff returns like 565 files, and the generating the package.xml like force-dev-tool changeset create devDeploy takes forever, it is still running.. is there a way how i can make this faster. May be feed in more memory to the node??

force-dev-tool@0.13.1

[amtrack]

Sounds wrong to me. Did you really pipe in the git diff?

[okram999]

@amtrack , i wish i am wrong. As i find your tool so cool and really want to give it a try. Here is what i have as a part of the shell script

node --version npm install --global force-dev-tool force-dev-tool --help git fetch --all git branch -a git checkout master git checkout config2 git diff master config2 | force-dev-tool changeset create devDeploy

But if i do the same, for a diff, that returns just 4-5 files, it generates the package.xml. The other thing which contradicts is that, the same generate the destructivepackage.xml with all the other (higher count), just fine.

Do you see any issue with the above script?

[okram999]

So, i just removed the force-dev-tool 's changeset create section from the pipe, and ran just the diff, the diff is okay.

node --version npm install --global force-dev-tool git fetch --all git branch -a git checkout master git checkout config2 git diff master config2 #| force-dev-tool changeset create devDeploy

I am going to save the diff into a file and pipe it in the force-dev-tool, and see what happens?

[okram999]

The output file from the git diff is 108MB and there are 3005097 lines.. is this the reason? Won't it better if, you pipe in the git diff --name-only ? (But looks like you are using a nodejs package for the git-diff)

I am dealing with these many files, as i am pulling in the config changes done by the admins on the sandboxes into a branch and trying to generate the package.xml for deploying the changes

[okram999]

We removed a component (object translation) from versioning. It was contributing to around 359 files in the git diff. Now force-dev-tool is generating the package.xml.

Can you pls confirm if this is the due to the enormous # of lines presented to the git-diff node package that your app depends on?

[amtrack]

OK, so i didn't run any performance tests but as far as i can tell parsing the diff shouldn't be the bottleneck. However I can confirm that parsing and preparing the files seems to be not so performant at the moment. Do you want to contribute on this issue by creating a test repo or improving the code itself?

[okram999]

yes, i will have to see how i can get a similarly sized repo, where i am having the issue by scrapping any confidential info

[amtrack]

Sounds good! Let me know if I can help. I would be happy to talk to you about your use cases btw to see what else can be improved :-)

[okram999]

anything in the v1.0, which you think have improved the "parsing and preparing the files "

[okram999]

i am not there yet, but i am kind of thinking this might be due to the objectTranslation 's content and the regex that you defined to build the package.xml ... will update more

[amtrack]

Closed by #99 .