package is showing changes not included in `git diff`

[bartoszsliwinski]

When using git diff | force-dev-tool changeset create on Azure DevOps some XML files sometimes include non-existing changes. It usually happens for Admin.profile and SObjects. For example diff below:

diff --git a/src/profiles/Admin.profile b/src/profiles/Admin.profile
index 47b8c622..e7558afc 100644
--- a/src/profiles/Admin.profile
+++ b/src/profiles/Admin.profile
@@ -750,6 +750,10 @@
         <apexClass>BLD_GlobalClientRevenue</apexClass>
         <enabled>true</enabled>
     </classAccesses>
+    <classAccesses>
+        <apexClass>BLD_Knowledge</apexClass>
+        <enabled>true</enabled>
+    </classAccesses>
     <classAccesses>
         <apexClass>BLD_KYC_Status</apexClass>
         <enabled>true</enabled>

Produces this Admin.profile:

<?xml version="1.0" encoding="UTF-8"?>
<Profile xmlns="http://soap.sforce.com/2006/04/metadata">
    <classAccesses>
        <apexClass>BLD_Knowledge</apexClass>
        <enabled>true</enabled>
    </classAccesses>
    <tabVisibilities>
        <tab>SingletrackCMS__Activity_Types</tab></tabVisibilities>
</Profile>

[ohtormen]

Hi! We have the same issue. Any suggestion?

[sheldonleelazarus]

@froucher @amtrack have you experienced this as well?

[amtrack]

@sheldonleelazarus No and I'm surprised, but I'll try to reproduce this when I find some time.

[froucher]

Hi,

Sorry, I have not experienced the issue, @sheldonleelazarus, @mentux, @bartoszsliwinski maybe you can help force-dev-tool support, starting a branch with a test example of this issue, based in the following test folder & behaviour:

https://github.com/amtrack/force-dev-tool/tree/master/test-functional/data/simple-metadata/profile-custom-settings

https://github.com/amtrack/force-dev-tool/blob/master/test-functional/feature/changeset-simple-metadata.feature#L14

Here you can find how functional tests works:

https://github.com/amtrack/force-dev-tool/blob/master/test-functional/README.md

Anyway, when I have time I will try to help @amtrack trying to reproduce this issue.

Regards

[sheldonleelazarus]

@froucher thanks for this will have a look

[ohtormen]

Guys!

The problem occurs because the package.xml file is missing from the src directory. I added the file and the deployment was successful. Worth the try @bartoszsliwinski .

That solves for me.

Thanks to all, especially to @ercarval!

p.s.: @amtrack maybe put a more explicit message about the missing file. o/

(@dieffrei :rofl:)

[sheldonleelazarus]

Hi @froucher your information is very useful, i am trying to debug my scenario however i am getting the attached error log when trying to run npm run test:functional on my machine, do you know what could be the problem 2020-07-07T08_20_10_757Z-debug.log

[froucher]

Hello,

There's not to much information. Logs says:

18 verbose node v12.18.0
19 verbose npm  v6.14.5
20 error code ELIFECYCLE
21 error errno 1
22 error force-dev-tool@2.7.1 test:functional: `cucumber-js test-functional/feature/*.feature --tags 'not @skipped' --require test-functional/step-definitions/changeSet.js --format ./node_modules/cucumber-pretty`
22 error Exit status 1
23 error Failed at the force-dev-tool@2.7.1 test:functional script.
23 error This is probably not a problem with npm. There is likely additional logging output above.

Maybe it is not a npm issue, but in my local, I run the tests with the below versions:

$ node -v
v10.4.0
$ npm -v 
6.9.0

Please, can you try to launch directly without npm with the following command?

./node_modules/.bin/cucumber-js test-functional/feature/*.feature --tags 'not @skipped' --require test-functional/step-definitions/changeSet.js --format ./node_modules/cucumber-pretty

Thanks.

[amtrack]

@bartoszsliwinski Do you have Metadata in other directories than src? If so, you could try to limit the diff to explicitly the src directory using git diff -- src.

[bartoszsliwinski]

Hey @amtrack I see your point but actually not - we don't have metadata in other directories. Even if there would be something else, doesn't FDL work based on just the results of diff? And diff does not contain anything other than just my single file with single change. It doesn't seem like that's the issue here

[amtrack]

@bartoszsliwinski Is the solution @mentux provided working for you?

[bartoszsliwinski]

@amtrack unfortunately not, we already do have the package.xml file. Is there any way other then already mentioned here that I could help debugging?

[amtrack]

@bartoszsliwinski Could you please share your Azure DevOps configuration, especially how you launch git diff | force-dev-tool changeset create?

This sounds as strange as #233 which was also an issue only on Azure DevOps.

[bartoszsliwinski]

Well, this is actually exactly the same repository, we are having these issues for a while now but I wasn't aware @sheldonleelazarus created the issue already. Guess we can consolidate those two. Anyway, here's the config:

echo Creating Package
git diff testPipeline
git diff testPipeline  | force-dev-tool changeset create _release_validation                  
cat config/deployments/_release_validation/profiles/Admin.profile
if [ -f config/deployments/$(Build.BuildId)_release_validation/destructiveChanges.xml ]; then   
   mv config/deployments/$(Build.BuildId)_release_validation/destructiveChanges.xml config/deployments/$(Build.BuildId)_release_validation/destructiveChangesPost.xml
fi
force-dev-tool remote add RELEASE $(sfdc_username_rel) $(sfdc_password_rel)
force-dev-tool deploy -c RELEASE -d config/deployments/$(Build.BuildId)_release_validation

cat command already shows that Admin.profile has that additional tabVisibilities node

[amtrack]

@bartoszsliwinski Would you mind showing me this live by sharing your screen in a video conference?

[sheldonleelazarus]

@amtrack i dont mind setting up a videa conference with yourself to show you the issue as well

[amtrack]

@sheldonleelazarus As discussed in the call, I will try to reproduce this issue and we currently assume it is related to files larger than 1 MB.

However my test case with a CustomObject having 10.000 CustomFields (~3.5 MB) where I change the label of one field passes. Can you please post the numbers of CustomFields, ListViews,... of your CustomObject and explain again the scenario which specific change causes force-dev-tool to fail?

[amtrack]

@sheldonleelazarus UPDATE: Re-running the build a couple of times I was able to reproduce a failure:

Failures:
1) Scenario: Child metadata are added and/or updated # test-functional\feature\changeset-complex-metadata.feature:14
   √ Given a list of "CustomField" metadata in "complex-metadata/customField-updated-large-object" folder which has been added and updated in a git repository # test-functional\step-definitions\changeSet.js:10
   √ When a user launches a change set with force-dev-tool # test-functional\step-definitions\changeSet.js:26
   × Then it will create a change set with the list of "CustomField" metadata # test-functional\step-definitions\changeSet.js:31
       AssertionError: Only in C:\Users\travis\AppData\Local\Temp\tmp-4644-1lc0aoLH044G\config\deployments\test: destructiveChanges.xml
       diff -u -r "C:\\Users\\travis\\AppData\\Local\\Temp\\tmp-4644-1lc0aoLH044G\\config\\deployments\\test/objects/LargeCustomObjectTest__c.object" "C:\\Users\\travis\\build\\amtrack\\force-dev-tool\\test-functional\\data\\complex-metadata\\customField-updated-large-object\\expected/objects/LargeCustomObjectTest__c.object"
       --- "C:\\Users\\travis\\AppData\\Local\\Temp\\tmp-4644-1lc0aoLH044G\\config\\deployments\\test/objects/LargeCustomObjectTest__c.object"  2020-07-27 12:18:55.128165100 +0000
       +++ "C:\\Users\\travis\\build\\amtrack\\force-dev-tool\\test-functional\\data\\complex-metadata\\customField-updated-large-object\\expected/objects/LargeCustomObjectTest__c.object" 2020-07-27 12:17:59.296354200 +0000
       @@ -12,13 +12,4 @@
                <type>Text</type>
                <unique>false</unique>
            </fields>
       -    <fields>
       -        <fullName>Test2778__c</fullName>
       -        <externalId>false</externalId>
       -        <label>Test</label>
       -        <length>200</length>
       -        <required>false</required>
       -        <trackFeedHistory>false</trackFeedHistory>
       -        <trackHistory>false</trackHistory>
       -        <trackTrending/></fields>
        </CustomObject>
       diff -u -r "C:\\Users\\travis\\AppData\\Local\\Temp\\tmp-4644-1lc0aoLH044G\\config\\deployments\\test/package.xml" "C:\\Users\\travis\\build\\amtrack\\force-dev-tool\\test-functional\\data\\complex-metadata\\customField-updated-large-object\\expected/package.xml"
       --- "C:\\Users\\travis\\AppData\\Local\\Temp\\tmp-4644-1lc0aoLH044G\\config\\deployments\\test/package.xml"  2020-07-27 12:18:55.128165100 +0000
       +++ "C:\\Users\\travis\\build\\amtrack\\force-dev-tool\\test-functional\\data\\complex-metadata\\customField-updated-large-object\\expected/package.xml" 2020-07-27 12:17:59.296354200 +0000
       @@ -2,7 +2,6 @@
        <Package xmlns="http://soap.sforce.com/2006/04/metadata">
            <types>
                <members>LargeCustomObjectTest__c.Test0815__c</members>
       -        <members>LargeCustomObjectTest__c.Test2778__c</members>
                <name>CustomField</name>
            </types>
            <version>46.0</version>
       : expected 1 to equal 0
           + expected - actual
           -1
           +0
           at Object.directoryContentEquals (C:\Users\travis\build\amtrack\force-dev-tool\test-functional\lib\diff.js:6:64)
           at World.<anonymous> (C:\Users\travis\build\amtrack\force-dev-tool\test-functional\step-definitions\changeSet.js:33:8)
   - And excluding any "CustomObject" metadata in the change set # test-functional\step-definitions\changeSet.js:37
   - And the change set could be deployed correctly # test-functional\step-definitions\changeSet.js:59
   √ After # test-functional\step-definitions\changeSet.js:69

[amtrack]

@froucher I could use your helping hand here if you have some time. I cannot reproduce this on my machine, only in CI sporadically. The interesting thing is that in the test above, it seems to fail for CustomFields with an index of ~2.700 of 10.000. Do you have any idea?

[froucher]

Sure @amtrack, I will try to reproduce now this error in my local.

[sheldonleelazarus]

Hi @amtrack for some stats, I have over 50 lists views, 200+ fields, 13+ Recordtypes with atleast 20+ picklist fields assigned to the recordtypes

[froucher]

In my local, macOs Catalina, it seems that always works, I have even run the test more than 10 times, but has never failed.

But I will continue to think which could be the reason of that unexpected behaviour.

[froucher]

I don't know, but this could be due to a synchronization issue?

[amtrack]

:tada: This issue has been resolved in version 2.8.3 :tada:

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot :package::rocket: