wgrep-bom-with-multibyte and wgrep-bom-with-unibyte failing

With f0ef9bf, wgrep-bom-with-multibyte and wgrep-bom-with-unibyte still often fail. The test passes on some CI infrastructure but not others, so it looks like a flaky test or a corner-case to me. eg: our CI does various forms of minor fuzzing for basic QA. If it looks more like a problem in our infrastructure (rare, but sometimes happens) please let me know why, and I'll contact the people who are responsible for this. At this time, tests are run with Emacs 26.3, but we'll most likely be switching to 27.1 sometime in the next few months. I've also confirmed test pass on my local system. By the way, I sincerely appreciate that you're providing these tests :-)

Were I to hasard a guess as to why the tests are failing on the buildd network, but not the DebCI network, it would be because buildd sets LANG='C' and LANGUAGE='en_US:en', and maybe wgrep assumes a UTF-8 locale? I'm sorry I can't be of more help at this time. Assuming it's not a subtle dynamic binding issue, I suspect the solution will be something like modifying wgrep-mode to use one of these https://www.gnu.org/software/emacs/manual/html_node/elisp/Converting-Representations.html on non-UTF8 systems.

Here's the data:

Running 7 tests (2021-09-28 02:02:28-1200)
  skipped  1/7  wgrep-ag-normal

Grep finished with no matches found
Press C-x C-s when finished or C-c C-k to abort changes.
Test wgrep-bom-with-multibyte backtrace:
  signal(ert-test-failed (((should (re-search-forward (concat (regexp-
  ert-fail(((should (re-search-forward (concat (regexp-quote file) ":[
  (if (unwind-protect (setq value-173 (apply fn-171 args-172)) (setq f
  (let (form-description-175) (if (unwind-protect (setq value-173 (app
  (let ((value-173 (quote ert-form-evaluation-aborted-174))) (let (for
  (let* ((fn-171 (function re-search-forward)) (args-172 (condition-ca
  (lambda (file) (wgrep-test--grep (concat "grep -nH -e 'あ' -A 2 " fil
  funcall((lambda (file) (wgrep-test--grep (concat "grep -nH -e 'あ' -A
  (unwind-protect (funcall body-fn file) (wgrep-test--cleanup-file fil
  (let ((file (concat (make-temp-name "test-data") ".txt"))) (cond ((s
  (let ((default-directory (file-name-as-directory test-directory))) (
  (let ((test-directory (expand-file-name "test-work" default-director
  wgrep-test-fixture(("あ\nい\nう\n" utf-8-with-signature) (lambda (file)
  (progn (wgrep-test-fixture (quote ("あ\nい\nう\n" utf-8-with-signature)
  (let ((wgrep-change-readonly-file nil) (wgrep-auto-save-buffer nil))
  (lambda nil (let ((wgrep-change-readonly-file nil) (wgrep-auto-save-
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name wgrep-bom-with-multibyte :documentati
  ert-run-or-rerun-test(#s(ert--stats :selector t :tests [#s(ert-test 
  ert-run-tests(t #f(compiled-function (event-type &rest event-args) #
  ert-run-tests-batch(nil)
  ert-run-tests-batch-and-exit()
  eval((ert-run-tests-batch-and-exit))
  command-line-1(("-l" "package" "--eval" "(add-to-list 'package-direc
  command-line()
  normal-top-level()
Test wgrep-bom-with-multibyte condition:
    (ert-test-failed
     ((should
       (re-search-forward
    (concat ... ":[0-9]+:.*\\(あ\\)$")
    nil t))
      :form
      (re-search-forward "test-dataxLD6Qt\\.txt:[0-9]+:.*\\(あ\\)$" nil t)
      :value nil))
   FAILED  2/7  wgrep-bom-with-multibyte

Grep finished with matches found
Press C-x C-s when finished or C-c C-k to abort changes.
Writing 1 files, 0 files are left...
There is an unapplied change. (0 changed)
No buffer has been saved.
Test wgrep-bom-with-unibyte backtrace:
  signal(ert-test-failed (((should (equal "ABCD\nb\n" (wgrep-test--get
  ert-fail(((should (equal "ABCD\nb\n" (wgrep-test--get-contents file)
  (if (unwind-protect (setq value-193 (apply fn-191 args-192)) (setq f
  (let (form-description-195) (if (unwind-protect (setq value-193 (app
  (let ((value-193 (quote ert-form-evaluation-aborted-194))) (let (for
  (let* ((fn-191 (function equal)) (args-192 (condition-case err (let 
  (lambda (file) (wgrep-test--grep (concat "grep -nH -e 'a' -A 2 " fil
  funcall((lambda (file) (wgrep-test--grep (concat "grep -nH -e 'a' -A
  (unwind-protect (funcall body-fn file) (wgrep-test--cleanup-file fil
  (let ((file (concat (make-temp-name "test-data") ".txt"))) (cond ((s
  (let ((default-directory (file-name-as-directory test-directory))) (
  (let ((test-directory (expand-file-name "test-work" default-director
  wgrep-test-fixture(("a\nb\n" utf-8-with-signature) (lambda (file) (w
  (progn (wgrep-test-fixture (quote ("a\nb\n" utf-8-with-signature)) (
  (let ((wgrep-change-readonly-file nil) (wgrep-auto-save-buffer nil))
  (lambda nil (let ((wgrep-change-readonly-file nil) (wgrep-auto-save-
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name wgrep-bom-with-unibyte :documentation
  ert-run-or-rerun-test(#s(ert--stats :selector t :tests [#s(ert-test 
  ert-run-tests(t #f(compiled-function (event-type &rest event-args) #
  ert-run-tests-batch(nil)
  ert-run-tests-batch-and-exit()
  eval((ert-run-tests-batch-and-exit))
  command-line-1(("-l" "package" "--eval" "(add-to-list 'package-direc
  command-line()
  normal-top-level()
Test wgrep-bom-with-unibyte condition:
    (ert-test-failed
     ((should
       (equal "ABCD
b
"
          (wgrep-test--get-contents file)))
      :form
      (equal "ABCD
b
" "a
b
")
      :value nil :explanation
      (arrays-of-different-length 7 4 "ABCD
b
" "a
b
" first-mismatch-at 0)))
   FAILED  3/7  wgrep-bom-with-unibyte

Full build logs for all tested architectures available here (all failing): https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/emacs-wgrep.html

And the logs from our second CI network (all passing, except wgrep-ag-normal which is skipped due to #73) https://ci.debian.net/packages/e/emacs-wgrep/testing/amd64/ https://ci.debian.net/data/autopkgtest/testing/amd64/e/emacs-wgrep/6761710/log.gz

mhayashi1120 / Emacs-wgrep

wgrep-bom-with-multibyte and wgrep-bom-with-unibyte failing #78