martinvonz / jj

A Git-compatible VCS that is both simple and powerful
https://martinvonz.github.io/jj/
Apache License 2.0
8.36k stars 287 forks source link

Spurious `jj status`/`jj diff` due to case-insensitivity, such as for gecko-dev #1737

Open arxanas opened 1 year ago

arxanas commented 1 year ago

Description

Steps to Reproduce the Problem

  1. Use Git to clone https://github.com/mozilla/gecko-dev/commit/f0967c33054b3d7c5f9b5e4cf4ae1a85d9ded553
  2. Initialize colocated repo with jj init --git-repo .
  3. Run jj status

Expected Behavior

Fresh clone should be clean.

Actual Behavior

git status is clean:

$ git status
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

But not jj:

Status ``` Parent commit: f0967c33054b Backed out changeset 8be4693aecbb (bug 1754905) for causing bustages in xpcAccessiblePivot.h. CLOSED TREE Working copy : 4354f8335918 (no description set) Working copy changes: A testing/web-platform/meta/WebIDL/current-realm.html.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/allow-resizable.html.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/builtin-function-properties.any.js.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/class-string-interface.any.js.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/class-string-iterator-prototype-object.any.js.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/class-string-named-properties-object.window.js.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/constructors.html.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/default-iterator-object.html.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/default-toJSON-cross-realm.html.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/es-exceptions/DOMException-constructor-and-prototype.any.js.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/es-exceptions/DOMException-constructor-behavior.any.js.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/es-exceptions/DOMException-custom-bindings.any.js.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/es-exceptions/exceptions.html.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/global-immutable-prototype.any.js.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/global-object-implicit-this-value-cross-realm.html.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/global-object-implicit-this-value.any.js.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/has-instance.html.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/interface-object-set-receiver.html.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/interface-object.html.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/interface-prototype-constructor-set-receiver.html.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/interface-prototype-object.html.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/invalid-this-value-cross-realm.html.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/iterator-invalidation-foreach.html.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/iterator-prototype-object.html.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/legacy-callback-interface-object.html.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/legacy-factor-function-subclass.window.js.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/no-regexp-special-casing.any.js.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/observable-array-no-leak-of-internals.window.js.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/observable-array-ownkeys.window.js.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/sequence-conversion.html.ini A testing/web-platform/meta/WebIDL/ecmascript-binding/window-named-properties-object.html.ini A testing/web-platform/meta/WebIDL/idlharness-shadowrealm.window.js.ini A testing/web-platform/meta/WebIDL/idlharness.any.js.ini R testing/web-platform/meta/webidl/current-realm.html.ini R testing/web-platform/meta/webidl/ecmascript-binding/allow-resizable.html.ini R testing/web-platform/meta/webidl/ecmascript-binding/builtin-function-properties.any.js.ini R testing/web-platform/meta/webidl/ecmascript-binding/class-string-interface.any.js.ini R testing/web-platform/meta/webidl/ecmascript-binding/class-string-iterator-prototype-object.any.js.ini R testing/web-platform/meta/webidl/ecmascript-binding/class-string-named-properties-object.window.js.ini R testing/web-platform/meta/webidl/ecmascript-binding/constructors.html.ini R testing/web-platform/meta/webidl/ecmascript-binding/default-iterator-object.html.ini R testing/web-platform/meta/webidl/ecmascript-binding/default-toJSON-cross-realm.html.ini R testing/web-platform/meta/webidl/ecmascript-binding/es-exceptions/DOMException-constructor-and-prototype.any.js.ini R testing/web-platform/meta/webidl/ecmascript-binding/es-exceptions/DOMException-constructor-behavior.any.js.ini R testing/web-platform/meta/webidl/ecmascript-binding/es-exceptions/DOMException-custom-bindings.any.js.ini R testing/web-platform/meta/webidl/ecmascript-binding/es-exceptions/exceptions.html.ini R testing/web-platform/meta/webidl/ecmascript-binding/global-immutable-prototype.any.js.ini R testing/web-platform/meta/webidl/ecmascript-binding/global-object-implicit-this-value-cross-realm.html.ini R testing/web-platform/meta/webidl/ecmascript-binding/global-object-implicit-this-value.any.js.ini R testing/web-platform/meta/webidl/ecmascript-binding/has-instance.html.ini R testing/web-platform/meta/webidl/ecmascript-binding/interface-object-set-receiver.html.ini R testing/web-platform/meta/webidl/ecmascript-binding/interface-object.html.ini R testing/web-platform/meta/webidl/ecmascript-binding/interface-prototype-constructor-set-receiver.html.ini R testing/web-platform/meta/webidl/ecmascript-binding/interface-prototype-object.html.ini R testing/web-platform/meta/webidl/ecmascript-binding/invalid-this-value-cross-realm.html.ini R testing/web-platform/meta/webidl/ecmascript-binding/iterator-invalidation-foreach.html.ini R testing/web-platform/meta/webidl/ecmascript-binding/iterator-prototype-object.html.ini R testing/web-platform/meta/webidl/ecmascript-binding/legacy-callback-interface-object.html.ini R testing/web-platform/meta/webidl/ecmascript-binding/legacy-factor-function-subclass.window.js.ini R testing/web-platform/meta/webidl/ecmascript-binding/no-regexp-special-casing.any.js.ini R testing/web-platform/meta/webidl/ecmascript-binding/observable-array-no-leak-of-internals.window.js.ini R testing/web-platform/meta/webidl/ecmascript-binding/observable-array-ownkeys.window.js.ini R testing/web-platform/meta/webidl/ecmascript-binding/sequence-conversion.html.ini R testing/web-platform/meta/webidl/ecmascript-binding/window-named-properties-object.html.ini R testing/web-platform/meta/webidl/idlharness-shadowrealm.window.js.ini R testing/web-platform/meta/webidl/idlharness.any.js.ini ```

The diff appears to show removing and adding the same file contents:

Diff ``` Added regular file testing/web-platform/meta/WebIDL/current-realm.html.ini: 1: [current-realm.html] 2: [getImageData] 3: expected: FAIL Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/allow-resizable.html.ini: 1: [allow-resizable.html] 2: [APIs without [AllowResizable\] throw when passed resizable ArrayBuffers] 3: expected: FAIL 4: 5: [APIs with [AllowShared\] but without [AllowResizable\] throw when passed growable SharedArrayBuffers] 6: expected: FAIL Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/builtin-function-properties.any.js.ini: 1: [builtin-function-properties.any.html] 2: 3: [builtin-function-properties.any.worker.html] 4: expected: 5: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/class-string-interface.any.js.ini: 1: [class-string-interface.any.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] 4: 5: [class-string-interface.any.worker.html] 6: expected: 7: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/class-string-iterator-prototype-object.any.js.ini: 1: [class-string-iterator-prototype-object.any.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] 4: 5: [class-string-iterator-prototype-object.any.worker.html] 6: expected: 7: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/class-string-named-properties-object.window.js.ini: 1: [class-string-named-properties-object.window.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/constructors.html.ini: 1: [constructors.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/default-iterator-object.html.ini: 1: [default-iterator-object.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/default-toJSON-cross-realm.html.ini: 1: [default-toJSON-cross-realm.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/es-exceptions/DOMException-constructor-and-prototype.any.js.ini: 1: [DOMException-constructor-and-prototype.any.worker.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] 4: 5: [DOMException-constructor-and-prototype.any.html] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/es-exceptions/DOMException-constructor-behavior.any.js.ini: 1: [DOMException-constructor-behavior.any.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] 4: 5: [DOMException-constructor-behavior.any.worker.html] 6: expected: 7: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/es-exceptions/DOMException-custom-bindings.any.js.ini: 1: [DOMException-custom-bindings.any.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] 4: 5: [DOMException-custom-bindings.any.worker.html] 6: expected: 7: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/es-exceptions/exceptions.html.ini: 1: [exceptions.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/global-immutable-prototype.any.js.ini: 1: [global-immutable-prototype.any.worker.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] 4: 5: [global-immutable-prototype.any.serviceworker.html] 6: expected: 7: if (os == "android") and fission: [OK, TIMEOUT] 8: 9: [global-immutable-prototype.any.html] 10: expected: 11: if (os == "android") and fission: [OK, TIMEOUT] 12: 13: [global-immutable-prototype.any.sharedworker.html] 14: expected: 15: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/global-object-implicit-this-value-cross-realm.html.ini: 1: [global-object-implicit-this-value-cross-realm.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/global-object-implicit-this-value.any.js.ini: 1: [global-object-implicit-this-value.any.worker.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] 4: 5: [global-object-implicit-this-value.any.serviceworker.html] 6: expected: 7: if (os == "android") and fission: [OK, TIMEOUT] 8: 9: [global-object-implicit-this-value.any.html] 10: expected: 11: if (os == "android") and fission: [OK, TIMEOUT] 12: 13: [global-object-implicit-this-value.any.sharedworker.html] 14: expected: 15: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/has-instance.html.ini: 1: [has-instance.html] 2: prefs: [dom.webidl.crosscontext_hasinstance.enabled:false] 3: expected: 4: if (os == "android") and fission: [TIMEOUT, OK] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/interface-object-set-receiver.html.ini: 1: [interface-object-set-receiver.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/interface-object.html.ini: 1: [interface-object.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/interface-prototype-constructor-set-receiver.html.ini: 1: [interface-prototype-constructor-set-receiver.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/interface-prototype-object.html.ini: 1: [interface-prototype-object.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/invalid-this-value-cross-realm.html.ini: 1: [invalid-this-value-cross-realm.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/iterator-invalidation-foreach.html.ini: 1: [iterator-invalidation-foreach.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/iterator-prototype-object.html.ini: 1: [iterator-prototype-object.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/legacy-callback-interface-object.html.ini: 1: [legacy-callback-interface-object.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/legacy-factor-function-subclass.window.js.ini: 1: [legacy-factor-function-subclass.window.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/no-regexp-special-casing.any.js.ini: 1: [no-regexp-special-casing.any.worker.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] 4: 5: [no-regexp-special-casing.any.html] 6: expected: 7: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/observable-array-no-leak-of-internals.window.js.ini: 1: [observable-array-no-leak-of-internals.window.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/observable-array-ownkeys.window.js.ini: 1: [observable-array-ownkeys.window.html] 2: prefs: [layout.css.constructable-stylesheets.enabled:true] 3: expected: 4: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/sequence-conversion.html.ini: 1: [sequence-conversion.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] Added regular file testing/web-platform/meta/WebIDL/ecmascript-binding/window-named-properties-object.html.ini: 1: [window-named-properties-object.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] 4: [[[OwnPropertyKeys\]\]] 5: expected: FAIL Added regular file testing/web-platform/meta/WebIDL/idlharness-shadowrealm.window.js.ini: 1: [idlharness-shadowrealm.window.html] 2: prefs: [javascript.options.experimental.shadow_realms:true] 3: [DOMException interface: existence and properties of interface object] 4: expected: FAIL 5: 6: [DOMException interface object length] 7: expected: FAIL 8: 9: [DOMException interface object name] 10: expected: FAIL 11: 12: [DOMException interface: existence and properties of interface prototype object] 13: expected: FAIL 14: 15: [DOMException interface: existence and properties of interface prototype object's "constructor" property] 16: expected: FAIL 17: 18: [DOMException interface: existence and properties of interface prototype object's @@unscopables property] 19: expected: FAIL 20: 21: [DOMException interface: attribute name] 22: expected: FAIL 23: 24: [DOMException interface: attribute message] 25: expected: FAIL 26: 27: [DOMException interface: attribute code] 28: expected: FAIL 29: 30: [DOMException interface: constant INDEX_SIZE_ERR on interface object] 31: expected: FAIL 32: 33: [DOMException interface: constant INDEX_SIZE_ERR on interface prototype object] 34: expected: FAIL 35: 36: [DOMException interface: constant DOMSTRING_SIZE_ERR on interface object] 37: expected: FAIL 38: 39: [DOMException interface: constant DOMSTRING_SIZE_ERR on interface prototype object] 40: expected: FAIL 41: 42: [DOMException interface: constant HIERARCHY_REQUEST_ERR on interface object] 43: expected: FAIL 44: 45: [DOMException interface: constant HIERARCHY_REQUEST_ERR on interface prototype object] 46: expected: FAIL 47: 48: [DOMException interface: constant WRONG_DOCUMENT_ERR on interface object] 49: expected: FAIL 50: 51: [DOMException interface: constant WRONG_DOCUMENT_ERR on interface prototype object] 52: expected: FAIL 53: 54: [DOMException interface: constant INVALID_CHARACTER_ERR on interface object] 55: expected: FAIL 56: 57: [DOMException interface: constant INVALID_CHARACTER_ERR on interface prototype object] 58: expected: FAIL 59: 60: [DOMException interface: constant NO_DATA_ALLOWED_ERR on interface object] 61: expected: FAIL 62: 63: [DOMException interface: constant NO_DATA_ALLOWED_ERR on interface prototype object] 64: expected: FAIL 65: 66: [DOMException interface: constant NO_MODIFICATION_ALLOWED_ERR on interface object] 67: expected: FAIL 68: 69: [DOMException interface: constant NO_MODIFICATION_ALLOWED_ERR on interface prototype object] 70: expected: FAIL 71: 72: [DOMException interface: constant NOT_FOUND_ERR on interface object] 73: expected: FAIL 74: 75: [DOMException interface: constant NOT_FOUND_ERR on interface prototype object] 76: expected: FAIL 77: 78: [DOMException interface: constant NOT_SUPPORTED_ERR on interface object] 79: expected: FAIL 80: 81: [DOMException interface: constant NOT_SUPPORTED_ERR on interface prototype object] 82: expected: FAIL 83: 84: [DOMException interface: constant INUSE_ATTRIBUTE_ERR on interface object] 85: expected: FAIL 86: 87: [DOMException interface: constant INUSE_ATTRIBUTE_ERR on interface prototype object] 88: expected: FAIL 89: 90: [DOMException interface: constant INVALID_STATE_ERR on interface object] 91: expected: FAIL 92: 93: [DOMException interface: constant INVALID_STATE_ERR on interface prototype object] 94: expected: FAIL 95: 96: [DOMException interface: constant SYNTAX_ERR on interface object] 97: expected: FAIL 98: 99: [DOMException interface: constant SYNTAX_ERR on interface prototype object] 100: expected: FAIL 101: 102: [DOMException interface: constant INVALID_MODIFICATION_ERR on interface object] 103: expected: FAIL 104: 105: [DOMException interface: constant INVALID_MODIFICATION_ERR on interface prototype object] 106: expected: FAIL 107: 108: [DOMException interface: constant NAMESPACE_ERR on interface object] 109: expected: FAIL 110: 111: [DOMException interface: constant NAMESPACE_ERR on interface prototype object] 112: expected: FAIL 113: 114: [DOMException interface: constant INVALID_ACCESS_ERR on interface object] 115: expected: FAIL 116: 117: [DOMException interface: constant INVALID_ACCESS_ERR on interface prototype object] 118: expected: FAIL 119: 120: [DOMException interface: constant VALIDATION_ERR on interface object] 121: expected: FAIL 122: 123: [DOMException interface: constant VALIDATION_ERR on interface prototype object] 124: expected: FAIL 125: 126: [DOMException interface: constant TYPE_MISMATCH_ERR on interface object] 127: expected: FAIL 128: 129: [DOMException interface: constant TYPE_MISMATCH_ERR on interface prototype object] 130: expected: FAIL 131: 132: [DOMException interface: constant SECURITY_ERR on interface object] 133: expected: FAIL 134: 135: [DOMException interface: constant SECURITY_ERR on interface prototype object] 136: expected: FAIL 137: 138: [DOMException interface: constant NETWORK_ERR on interface object] 139: expected: FAIL 140: 141: [DOMException interface: constant NETWORK_ERR on interface prototype object] 142: expected: FAIL 143: 144: [DOMException interface: constant ABORT_ERR on interface object] 145: expected: FAIL 146: 147: [DOMException interface: constant ABORT_ERR on interface prototype object] 148: expected: FAIL 149: 150: [DOMException interface: constant URL_MISMATCH_ERR on interface object] 151: expected: FAIL 152: 153: [DOMException interface: constant URL_MISMATCH_ERR on interface prototype object] 154: expected: FAIL 155: 156: [DOMException interface: constant QUOTA_EXCEEDED_ERR on interface object] 157: expected: FAIL 158: 159: [DOMException interface: constant QUOTA_EXCEEDED_ERR on interface prototype object] 160: expected: FAIL 161: 162: [DOMException interface: constant TIMEOUT_ERR on interface object] 163: expected: FAIL 164: 165: [DOMException interface: constant TIMEOUT_ERR on interface prototype object] 166: expected: FAIL 167: 168: [DOMException interface: constant INVALID_NODE_TYPE_ERR on interface object] 169: expected: FAIL 170: 171: [DOMException interface: constant INVALID_NODE_TYPE_ERR on interface prototype object] 172: expected: FAIL 173: 174: [DOMException interface: constant DATA_CLONE_ERR on interface object] 175: expected: FAIL 176: 177: [DOMException interface: constant DATA_CLONE_ERR on interface prototype object] 178: expected: FAIL Added regular file testing/web-platform/meta/WebIDL/idlharness.any.js.ini: 1: [idlharness.any.worker.html] 2: expected: 3: if (os == "android") and fission: [OK, TIMEOUT] 4: 5: [idlharness.any.html] 6: expected: 7: if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/current-realm.html.ini: 1 : [current-realm.html] 2 : [getImageData] 3 : expected: FAIL Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/allow-resizable.html.ini: 1 : [allow-resizable.html] 2 : [APIs without [AllowResizable\] throw when passed resizable ArrayBuffers] 3 : expected: FAIL 4 : 5 : [APIs with [AllowShared\] but without [AllowResizable\] throw when passed growable SharedArrayBuffers] 6 : expected: FAIL Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/builtin-function-properties.any.js.ini: 1 : [builtin-function-properties.any.html] 2 : 3 : [builtin-function-properties.any.worker.html] 4 : expected: 5 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/class-string-interface.any.js.ini: 1 : [class-string-interface.any.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] 4 : 5 : [class-string-interface.any.worker.html] 6 : expected: 7 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/class-string-iterator-prototype-object.any.js.ini: 1 : [class-string-iterator-prototype-object.any.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] 4 : 5 : [class-string-iterator-prototype-object.any.worker.html] 6 : expected: 7 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/class-string-named-properties-object.window.js.ini: 1 : [class-string-named-properties-object.window.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/constructors.html.ini: 1 : [constructors.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/default-iterator-object.html.ini: 1 : [default-iterator-object.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/default-toJSON-cross-realm.html.ini: 1 : [default-toJSON-cross-realm.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/es-exceptions/DOMException-constructor-and-prototype.any.js.ini: 1 : [DOMException-constructor-and-prototype.any.worker.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] 4 : 5 : [DOMException-constructor-and-prototype.any.html] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/es-exceptions/DOMException-constructor-behavior.any.js.ini: 1 : [DOMException-constructor-behavior.any.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] 4 : 5 : [DOMException-constructor-behavior.any.worker.html] 6 : expected: 7 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/es-exceptions/DOMException-custom-bindings.any.js.ini: 1 : [DOMException-custom-bindings.any.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] 4 : 5 : [DOMException-custom-bindings.any.worker.html] 6 : expected: 7 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/es-exceptions/exceptions.html.ini: 1 : [exceptions.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/global-immutable-prototype.any.js.ini: 1 : [global-immutable-prototype.any.worker.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] 4 : 5 : [global-immutable-prototype.any.serviceworker.html] 6 : expected: 7 : if (os == "android") and fission: [OK, TIMEOUT] 8 : 9 : [global-immutable-prototype.any.html] 10 : expected: 11 : if (os == "android") and fission: [OK, TIMEOUT] 12 : 13 : [global-immutable-prototype.any.sharedworker.html] 14 : expected: 15 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/global-object-implicit-this-value-cross-realm.html.ini: 1 : [global-object-implicit-this-value-cross-realm.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/global-object-implicit-this-value.any.js.ini: 1 : [global-object-implicit-this-value.any.worker.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] 4 : 5 : [global-object-implicit-this-value.any.serviceworker.html] 6 : expected: 7 : if (os == "android") and fission: [OK, TIMEOUT] 8 : 9 : [global-object-implicit-this-value.any.html] 10 : expected: 11 : if (os == "android") and fission: [OK, TIMEOUT] 12 : 13 : [global-object-implicit-this-value.any.sharedworker.html] 14 : expected: 15 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/has-instance.html.ini: 1 : [has-instance.html] 2 : prefs: [dom.webidl.crosscontext_hasinstance.enabled:false] 3 : expected: 4 : if (os == "android") and fission: [TIMEOUT, OK] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/interface-object-set-receiver.html.ini: 1 : [interface-object-set-receiver.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/interface-object.html.ini: 1 : [interface-object.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/interface-prototype-constructor-set-receiver.html.ini: 1 : [interface-prototype-constructor-set-receiver.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/interface-prototype-object.html.ini: 1 : [interface-prototype-object.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/invalid-this-value-cross-realm.html.ini: 1 : [invalid-this-value-cross-realm.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/iterator-invalidation-foreach.html.ini: 1 : [iterator-invalidation-foreach.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/iterator-prototype-object.html.ini: 1 : [iterator-prototype-object.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/legacy-callback-interface-object.html.ini: 1 : [legacy-callback-interface-object.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/legacy-factor-function-subclass.window.js.ini: 1 : [legacy-factor-function-subclass.window.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/no-regexp-special-casing.any.js.ini: 1 : [no-regexp-special-casing.any.worker.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] 4 : 5 : [no-regexp-special-casing.any.html] 6 : expected: 7 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/observable-array-no-leak-of-internals.window.js.ini: 1 : [observable-array-no-leak-of-internals.window.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/observable-array-ownkeys.window.js.ini: 1 : [observable-array-ownkeys.window.html] 2 : prefs: [layout.css.constructable-stylesheets.enabled:true] 3 : expected: 4 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/sequence-conversion.html.ini: 1 : [sequence-conversion.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] Removed regular file testing/web-platform/meta/webidl/ecmascript-binding/window-named-properties-object.html.ini: 1 : [window-named-properties-object.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] 4 : [[[OwnPropertyKeys\]\]] 5 : expected: FAIL Removed regular file testing/web-platform/meta/webidl/idlharness-shadowrealm.window.js.ini: 1 : [idlharness-shadowrealm.window.html] 2 : prefs: [javascript.options.experimental.shadow_realms:true] 3 : [DOMException interface: existence and properties of interface object] 4 : expected: FAIL 5 : 6 : [DOMException interface object length] 7 : expected: FAIL 8 : 9 : [DOMException interface object name] 10 : expected: FAIL 11 : 12 : [DOMException interface: existence and properties of interface prototype object] 13 : expected: FAIL 14 : 15 : [DOMException interface: existence and properties of interface prototype object's "constructor" property] 16 : expected: FAIL 17 : 18 : [DOMException interface: existence and properties of interface prototype object's @@unscopables property] 19 : expected: FAIL 20 : 21 : [DOMException interface: attribute name] 22 : expected: FAIL 23 : 24 : [DOMException interface: attribute message] 25 : expected: FAIL 26 : 27 : [DOMException interface: attribute code] 28 : expected: FAIL 29 : 30 : [DOMException interface: constant INDEX_SIZE_ERR on interface object] 31 : expected: FAIL 32 : 33 : [DOMException interface: constant INDEX_SIZE_ERR on interface prototype object] 34 : expected: FAIL 35 : 36 : [DOMException interface: constant DOMSTRING_SIZE_ERR on interface object] 37 : expected: FAIL 38 : 39 : [DOMException interface: constant DOMSTRING_SIZE_ERR on interface prototype object] 40 : expected: FAIL 41 : 42 : [DOMException interface: constant HIERARCHY_REQUEST_ERR on interface object] 43 : expected: FAIL 44 : 45 : [DOMException interface: constant HIERARCHY_REQUEST_ERR on interface prototype object] 46 : expected: FAIL 47 : 48 : [DOMException interface: constant WRONG_DOCUMENT_ERR on interface object] 49 : expected: FAIL 50 : 51 : [DOMException interface: constant WRONG_DOCUMENT_ERR on interface prototype object] 52 : expected: FAIL 53 : 54 : [DOMException interface: constant INVALID_CHARACTER_ERR on interface object] 55 : expected: FAIL 56 : 57 : [DOMException interface: constant INVALID_CHARACTER_ERR on interface prototype object] 58 : expected: FAIL 59 : 60 : [DOMException interface: constant NO_DATA_ALLOWED_ERR on interface object] 61 : expected: FAIL 62 : 63 : [DOMException interface: constant NO_DATA_ALLOWED_ERR on interface prototype object] 64 : expected: FAIL 65 : 66 : [DOMException interface: constant NO_MODIFICATION_ALLOWED_ERR on interface object] 67 : expected: FAIL 68 : 69 : [DOMException interface: constant NO_MODIFICATION_ALLOWED_ERR on interface prototype object] 70 : expected: FAIL 71 : 72 : [DOMException interface: constant NOT_FOUND_ERR on interface object] 73 : expected: FAIL 74 : 75 : [DOMException interface: constant NOT_FOUND_ERR on interface prototype object] 76 : expected: FAIL 77 : 78 : [DOMException interface: constant NOT_SUPPORTED_ERR on interface object] 79 : expected: FAIL 80 : 81 : [DOMException interface: constant NOT_SUPPORTED_ERR on interface prototype object] 82 : expected: FAIL 83 : 84 : [DOMException interface: constant INUSE_ATTRIBUTE_ERR on interface object] 85 : expected: FAIL 86 : 87 : [DOMException interface: constant INUSE_ATTRIBUTE_ERR on interface prototype object] 88 : expected: FAIL 89 : 90 : [DOMException interface: constant INVALID_STATE_ERR on interface object] 91 : expected: FAIL 92 : 93 : [DOMException interface: constant INVALID_STATE_ERR on interface prototype object] 94 : expected: FAIL 95 : 96 : [DOMException interface: constant SYNTAX_ERR on interface object] 97 : expected: FAIL 98 : 99 : [DOMException interface: constant SYNTAX_ERR on interface prototype object] 100 : expected: FAIL 101 : 102 : [DOMException interface: constant INVALID_MODIFICATION_ERR on interface object] 103 : expected: FAIL 104 : 105 : [DOMException interface: constant INVALID_MODIFICATION_ERR on interface prototype object] 106 : expected: FAIL 107 : 108 : [DOMException interface: constant NAMESPACE_ERR on interface object] 109 : expected: FAIL 110 : 111 : [DOMException interface: constant NAMESPACE_ERR on interface prototype object] 112 : expected: FAIL 113 : 114 : [DOMException interface: constant INVALID_ACCESS_ERR on interface object] 115 : expected: FAIL 116 : 117 : [DOMException interface: constant INVALID_ACCESS_ERR on interface prototype object] 118 : expected: FAIL 119 : 120 : [DOMException interface: constant VALIDATION_ERR on interface object] 121 : expected: FAIL 122 : 123 : [DOMException interface: constant VALIDATION_ERR on interface prototype object] 124 : expected: FAIL 125 : 126 : [DOMException interface: constant TYPE_MISMATCH_ERR on interface object] 127 : expected: FAIL 128 : 129 : [DOMException interface: constant TYPE_MISMATCH_ERR on interface prototype object] 130 : expected: FAIL 131 : 132 : [DOMException interface: constant SECURITY_ERR on interface object] 133 : expected: FAIL 134 : 135 : [DOMException interface: constant SECURITY_ERR on interface prototype object] 136 : expected: FAIL 137 : 138 : [DOMException interface: constant NETWORK_ERR on interface object] 139 : expected: FAIL 140 : 141 : [DOMException interface: constant NETWORK_ERR on interface prototype object] 142 : expected: FAIL 143 : 144 : [DOMException interface: constant ABORT_ERR on interface object] 145 : expected: FAIL 146 : 147 : [DOMException interface: constant ABORT_ERR on interface prototype object] 148 : expected: FAIL 149 : 150 : [DOMException interface: constant URL_MISMATCH_ERR on interface object] 151 : expected: FAIL 152 : 153 : [DOMException interface: constant URL_MISMATCH_ERR on interface prototype object] 154 : expected: FAIL 155 : 156 : [DOMException interface: constant QUOTA_EXCEEDED_ERR on interface object] 157 : expected: FAIL 158 : 159 : [DOMException interface: constant QUOTA_EXCEEDED_ERR on interface prototype object] 160 : expected: FAIL 161 : 162 : [DOMException interface: constant TIMEOUT_ERR on interface object] 163 : expected: FAIL 164 : 165 : [DOMException interface: constant TIMEOUT_ERR on interface prototype object] 166 : expected: FAIL 167 : 168 : [DOMException interface: constant INVALID_NODE_TYPE_ERR on interface object] 169 : expected: FAIL 170 : 171 : [DOMException interface: constant INVALID_NODE_TYPE_ERR on interface prototype object] 172 : expected: FAIL 173 : 174 : [DOMException interface: constant DATA_CLONE_ERR on interface object] 175 : expected: FAIL 176 : 177 : [DOMException interface: constant DATA_CLONE_ERR on interface prototype object] 178 : expected: FAIL Removed regular file testing/web-platform/meta/webidl/idlharness.any.js.ini: 1 : [idlharness.any.worker.html] 2 : expected: 3 : if (os == "android") and fission: [OK, TIMEOUT] 4 : 5 : [idlharness.any.html] 6 : expected: 7 : if (os == "android") and fission: [OK, TIMEOUT] ```

It looks like the difference is due to casing, i.e. testing/web-platform/meta/WebIDL/current-realm.html.ini vs testing/web-platform/meta/webidl/current-realm.html.ini. Interestingly, all of the problematic files are .ini.

Specifications

martinvonz commented 1 year ago

We have this TODO about that: https://github.com/martinvonz/jj/blob/d01ecc5c46043351ea0a3ab982bb2fd606d81ea3/lib/src/working_copy.rs#L60-L61

I'm not sure if there's a better solution. Since we have recently talked about making TreeState more of a pure cache than it currently is, maybe we should not store the information there (as that TODO suggests).

ilyagr commented 8 months ago

This is not a fix, but a comment to https://www.mgaudet.ca/technical/2023/11/23/exploring-jujitsu-jj suggested a workaround of creating a case-sensitive APFS volume, which is cheap on a Mac. Unfortunately, it seems like many Mac-native apps are untested and buggy on a case-sensitive volume, so you can't live your life on such a volume.

I'm unsure about Windows, the analogue might require repartitioning there.

mhammond commented 7 months ago

I've added a simple test case for this. I also added the Option<PathBuf> to FileState, but I'm not sure when it would get set other than None :) I'll have more of a poke, but thought I might as well share this now for comment.

https://github.com/martinvonz/jj/commit/ecfbb2531df39d369903eb4d6922c2141400b3dc

martinvonz commented 7 months ago

I've added a simple test case for this. I also added the Option<PathBuf> to FileState, but I'm not sure when it would get set other than None :) I'll have more of a poke, but thought I might as well share this now for comment.

I think I was thinking that we would set it to the actual path on disk if it was different from the RepoPath but I don't think I spent much time thinking about how it would work in detail.

Also note this comment from above:

Since we have recently talked about making TreeState more of a pure cache than it currently is, maybe we should not store the information there (as that TODO suggests).

Maybe it's better to walk the directory tree on disk and then check what's already in the base tree to decide which path to write to the new tree. For example, if we see a file called "foo" on disk and there's an existing file called "Foo" in the tree, we update that path instead (but only if we know we're on a case-insensitive file system, of course).

We will also have to decide what to do if there are multiple paths with different case in the tree. We will probably want to make snapshotting consistent with what we do on checkout. For example, if there's both "Foo" and "foo" in the tree (commit), we might decide that we put the "Foo" path in the working copy. Then, when we later snapshot the working copy, we should make sure we write a new tree where we update the "Foo" path with any changes to the "foo"/"Foo" file on disk.

mhammond commented 7 months ago

Thanks for the comment. I suspect that "full" support for case-insensitive file-systems will be a multi-pronged effort. In this first instance I'd be very happy with just allowing a tree to be checked out and considered clean, allowing the rest of the repo to be used. That fact things might fall apart if you try and change such files could be a followup.

I was wondering if this use-case might be able to avoid a new configuration option - after writing the file "FOO/bar" we could ask the file-system what the actual name is, and when told it's actually "foo/bar" (due to that directory already existing) we could adjust things accordingly.

I'm very new to jj, and even newer to the code and my grasp of the terminology is going to be tenuous, so apologies for any confusion, but:

Maybe it's better to walk the directory tree on disk and then check what's already in the base tree to decide which path to write to the new tree. For example, if we see a file called "foo" on disk and there's an existing file called "Foo" in the tree, we update that path instead

In an initial checkout we obviously can't look at the file-system before any operations. In my example, I suspect that whichever file is processed first out of "FOO" and "foo", the file-system will match that case. Then when the second file attempts to create "foo/bar" (or "FOO/bar") it will succeed, but the "foo" part will have an unexpected case.

Does that make sense? Is that just re-phrasing what you were saying, or am I missing the bigger picture?

mhammond commented 7 months ago

Maybe it's better to walk the directory tree on disk and then check what's already in the base tree to decide which path to write to the new tree.

I've looked at this some more and dug into how the tree-walking works for a snaphot. Like ecfbb25, consider snapshotting the pathological:

IIUC, as TreeState::visit_directory() is visiting A/b/d, it sets up the state for that file in the tree here - but all we have there is the path to the file (ie, the A/b/d) and not the path we need, a/B/d. Because this path is a flat key from the root of the repo, I can't see an efficient way of discovering a/B/d given just A/b/d (ie, an efficient way to "walk up" the tree to resolve each of the path components individually)

And without that we have the current situation: jj believes a/B/d is missing while A/b/d is a new untracked file.

Is my above understanding correct? Any suggestions for how I might proceed with this experiment?

martinvonz commented 7 months ago
  • repo paths: A/b/C, a/B/d
  • which was written into a wc as: A/b/C, A/b/d

Good example!

I actually wonder if it's even correct to write the tree repo paths to the working copy in that way. Maybe we should skip the entire a/ directory since we had already written the A/ directory (assuming we let the first entry in lexicographical order win). It seems odd to combine them into a single directory. That would likely break builds. Of course, completely skipping a/ would also break the build, but at least the build targets inside A/ would be more likely to work.

What do you think about skipping entries in a Tree object if we've already visited an entry that maps to the same path? But how do we even know if they map to the same path? Does that depend on the file system? I know the unicode rules for lowercasing are complicated. Hmm, a quick web search led to this FAQ, which seems to say that HFS+ and APFS do it differently. So I suppose we have to open the file for writes and then ask the file system for the name to see if it matches another path in the tree?

Btw, do we want to suppose only case-insensitive file systems that are at least case-preserving (I think some filesystems will give you a file called "A" if you write a file called "a")? Maybe it's not much work to support both.

Do you have thoughts on when we would detect a case insensitive file system? I'm thinking we would do that when we initialize the TreeState and then store a flag in it.

Might be worth looking into how other tools (e.g. Git and Mercurial) handle these things. It seems pretty thorny.

mhammond commented 7 months ago

What do you think about skipping entries in a Tree object if we've already visited an entry that maps to the same path?

I think there are 2 questions here - how should we write a working-tree vs how do we snapshot an existing working tree. In the original example we are snapshotting a working-tree created by git. ie, we are walking the file-system and discover the directory testing/web-platform/meta/WebIDL without knowing the repo has testing/web-platform/meta/WebIDL and testing/web-platform/meta/webidl

Might be worth looking into how other tools (e.g. Git and Mercurial) handle these things.

When it comes to writing the working tree, git does what my example above will do - ie, create all the files in a working-tree that doesn't exactly match the repo. Things will quickly fall apart if there's A/b and a/b, but I think that's OK (ie, such a tree would simply not be supported on a case-insenstive fs)

In my experiments, that's also what jj does - IOW, jj and git seem to agree on what a working-tree looks like when creating it. It's been years since I used hg, but I'd be super surprised if it doesn't do the same thing when writing a tree, simply because that's what a naive implementation does automatically, and also because gecko-dev works with hg on mac/windows and any other behaviour (ie, not checking out the mis-matched files etc) would not.

Btw, do we want to suppose only case-insensitive file systems that are at least case-preserving

I think it's probably fine to assume case-preserving these days (even fat32 is iiuc), but I don't see how that changes things though - case-preserving means the first-writer wins but the problem is the "losing" writers. Non case-preserving would just mean that first-writer also becomes a "loser" in this regard, but we need to deal with losers anyway.

Do you have thoughts on when we would detect a case insensitive file system?

auto-detecting would be fantastic. I'd be happy to lean into this once things are in-place to work with it.

poliorcetics commented 7 months ago

auto-detecting would be fantastic. I'd be happy to lean into this once things are in-place to work with it.

The most reliable and FS-independant way I know of to detect such systems (and all future case insensitive FS) is to create a file/or dir and check it's existence out in another casing:

std::fs::create_dir("<path to .jj>/CaSe-deTECtion/")?;
assert_eq!(Path::new("<path to .jj>/cAsE-DEtectTION").try_exists(), Ok(true))

If we store this as metadata we'll need to recheck it periodically, since a jj repo could be moved to another FS. The simplest way for that is probably to save our current absolute path and if it changes, recompute the case sensitivity ?

sunshowers commented 7 months ago

Auto-detection is definitely part of the story, but each filesystem has its own case-folding and normalization rules on top (and the normalization rules apply even on some case-sensitive filesystems).

For example, as documented here, APFS uses Unicode NFD for the most part, but there are exceptions. There's also some weirdness around zero-width characters that I don't quite recall at the moment.

And this applies to case-sensitive APFS as well.

We could probably handle simple cases with a flag, but in general this is a hugely difficult problem.

mhammond commented 7 months ago

(and the normalization rules apply even on some case-sensitive filesystems).

I guess those case-sensitive file-systems will have the same problem - if walking the fs provides paths which aren't identical to the original tree, jj will get confused about what files are missing/new etc?

but in general this is a hugely difficult problem.

Indeed, but is there a way to make small steps here? If we scale the problem down to, say "just ascii case normalization" (or more abstractly "support reading a working tree created by git") we'd probably solve the vast majority of these cases in the wild without compromising the ability to go further in the future?

martinvonz commented 7 months ago

Indeed, but is there a way to make small steps here? If we scale the problem down to, say "just ascii case normalization"

That's fine with me.

(or more abstractly "support reading a working tree created by git") we'd probably solve the vast majority of these cases in the wild without compromising the ability to go further in the future?

Matching Git is a bit more limiting but probably fine. Do you want to investigate what Git does? We would also need to be able to write a working copy ourselves, and I suppose we would want Git to be able to read the working copy after we write it then?

mhammond commented 7 months ago

Matching Git is a bit more limiting but probably fine.

When I said "matching git" I meant in a very loose way. git doesn't necessarily do the best thing, just a minimal "good enough" thing. For example, doing git log A/b (matching the file-system but not the repo) gives zero log entries, whereas git log a/B (ie, matching the repo but not the file-system) shows the actual log. I don't think we should necessarily aspire to that level of "matching git"

Also, to be very explicit, I really don't think we should see this capability as a "feature" but instead as a work-around. I've never seen a case-mismatch like this which was intentional, but instead it was accidentally introduced and likely to be "fixed" at some point for (some revisions of) that tree anyway. So IMO it's OK for jj to not aim for the best thing to do in some of these cases, but instead just avoid an obviously bad thing.

Do you want to investigate what Git does?

git has some interesting behaviour when adding files. eg:

let's add A/b/C to a repo

$ git init foo && cd foo
$ mkdir -p A/b & touch A/b/C
$ git add A
$ git ls-files
A/b/C
$ git ci -am 1

all good. Now let's nuke the files and try to add a/B/c

$ rm -rf A
$ mkdir -p a/B & touch a/B/d
$ # verify the file-system has the case we specified
$ find a
a
a/B
a/B/d
$ # Add the new file
$ git add a/B/d
$ # restore the file we removed.
$ git restore A
$ # The repo
$ git ls-files
A/b/C
A/b/d

Note that git created the new file with a path that matches the repo and not the file-system.

$ # The file-system
$ find a
a
a/B
a/B/C
a/B/d

git status is clean. Further experiments show that I can manually make a working tree with all case combinations and git doesn't care - git status is always clean regardless of case-mismatches here.

If I trick git into allowing mixed-case tree entries via, eg, merging a branch, git does get a little confused:

eg, create the main branch:

$ git init foo && cd foo
Initialized empty Git repository in /private/tmp/foo/foo/.git/
$ touch README && git add README && git ci -am README

create a branch with one variation of case:

$ git co -b b1
Switched to a new branch 'b1'
$ mkdir -p A/b && touch A/b/C && git add A/b/C && git ci -am "first c"

another branch with another variation, then merge the branches:

$ git co -b b2 main
Switched to a new branch 'b2'
$ mkdir -p a/B && touch a/B/c && git add a/B/c && git ci -am "second c"
$ git merge b1
Merge made by the 'ort' strategy.
 A/b/C | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 A/b/C

git now has two variations of the same file:

$ git ls-files
A/b/C
README
a/B/c
$ git st
On branch b2
nothing to commit, working tree clean

However, touching the file-system doesn't impact git:

$ echo new > a/B/C 
$ git st
On branch b2
nothing to commit, working tree clean

(ie, presumably you'd expect one of the variations of C to be dirty, but the tree is clean. It's not clear if this is intentional or an implementation accident, but regardless, there's no objectively "correct" thing to do.)

eg, if I check-out that tree:

$ rm -rf A
$ ls
README
$ git co .
Updated 2 paths from the index

git says it updated 2 paths, but it didn't - obviously there's only 1 c file on disk.

We would also need to be able to write a working copy ourselves, and I suppose we would want Git to be able to read the working copy after we write it then?

My experiments show that git really doesn't care what the case of the file-system is vs the tree. Even if git and jj would end up creating different file-systems but where the only difference is the case, then git will be happy. Or if not happy, at least no more confused than it can make itself.

I'll try and get some insight into what git actually does with the core.ignorecase setting too, but I suspect I'll find that code-base fairly intractable.

mhammond commented 7 months ago

I'll try and get some insight into what git actually does with the core.ignorecase setting too, but I suspect I'll find that code-base fairly intractable.

heh, I was wrong. This comment describes what I saw, and the implementation is here

They also probe the file-system to automatically set core.ignorecase

mhammond commented 7 months ago

FWIW, https://hg.mozilla.org/mozilla-central/rev/3b6c10d28209 just landed, which means the current tip of gecko no longer exhibits this problem (although obviously it may come back at any time and still exists for older revisions). It might be worth updating the title of this issue to be more generic?

sunshowers commented 7 months ago

I guess those case-sensitive file-systems will have the same problem - if walking the fs provides paths which aren't identical to the original tree, jj will get confused about what files are missing/new etc?

Yeah, any kind of normalization needs to be accounted for. (From what I remember, Git wasn't quite perfect either.)

Indeed, but is there a way to make small steps here? If we scale the problem down to, say "just ascii case normalization" (or more abstractly "support reading a working tree created by git") we'd probably solve the vast majority of these cases in the wild without compromising the ability to go further in the future?

Yeah, though even, say codebases with accents, which I'm going to guess are pretty common, are going to run into Unicode normalization issues. For example, if a filename is stored in Unicode NFC, on macOS it'll get written out as NFD -- and we'd need a way to match up the names internal to jj with the ones on disk. So it's worth at least thinking more holistically about the problem, even if the initial solution chosen isn't complete.