metamath / metamath-knife

Metamath-knife can rapidly verify Metamath proofs, providing strong confidence that the proofs are correct.
Apache License 2.0
25 stars 9 forks source link

Discouraged file produced by `--discouraged` does not pass verifiers #154

Closed GinoGiotto closed 6 months ago

GinoGiotto commented 6 months ago

Being able to update the discouraged file automatically is a very nice feature, unfortunately it is currently not usable due to incorrect output generation.

I created a new branch in my set.mm fork called discouraged-test. I updated that branch with the most recent version of set.mm, then I executed:

metamath-knife set.mm --discouraged discouraged

After committing the newly produced discouraged file, these errors appear:

https://github.com/GinoGiotto/set.mm/actions/runs/7398403197/job/20127485347

Checking discouraged file:
--- discouraged 2024-01-03 14:19:14.827913367 +0000
+++ discouraged.new 2024-01-03 14:19:58.135999643 +0000
@@ -17831,0 +17832 @@
+Proof modification of "19.21a3con13vVD" is discouraged (107 steps).
@@ -17869,0 +17871 @@
+Proof modification of "3impexpbicomiVD" is discouraged (28 steps).
@@ -18260,0 +18263 @@
+Proof modification of "con3ALTVD" is discouraged (51 steps).
@@ -18261,0 +18265 @@
+Proof modification of "con5VD" is discouraged (47 steps).
@@ -18266,0 +18271,4 @@
+Proof modification of "csbeq2gVD" is discouraged (61 steps).
+Proof modification of "csbfv12gALTVD" is discouraged (322 steps).
+Proof modification of "csbima12gALTVD" is discouraged (148 steps).
+Proof modification of "csbingVD" is discouraged (256 steps).
@@ -18267,0 +18276,5 @@
+Proof modification of "csbresgVD" is discouraged (195 steps).
+Proof modification of "csbrngVD" is discouraged (266 steps).
+Proof modification of "csbsngVD" is discouraged (204 steps).
+Proof modification of "csbunigVD" is discouraged (302 steps).
+Proof modification of "csbxpgVD" is discouraged (538 steps).
@@ -18395,0 +18409 @@
+Proof modification of "e2ebindVD" is discouraged (109 steps).
@@ -18458,0 +18473 @@
+Proof modification of "ee33VD" is discouraged (68 steps).
@@ -18527,0 +18543 @@
+Proof modification of "equncomiVD" is discouraged (19 steps).
@@ -18572,0 +18589 @@
+Proof modification of "exbiriVD" is discouraged (70 steps).
@@ -18799,0 +18817 @@
+Proof modification of "hbra2VD" is discouraged (26 steps).
@@ -18809,0 +18828 @@
+Proof modification of "idiVD" is discouraged (6 steps).
@@ -18823,0 +18843 @@
+Proof modification of "imbi13VD" is discouraged (67 steps).
@@ -19023,0 +19044 @@
+Proof modification of "notnotrALTVD" is discouraged (34 steps).
@@ -19030,0 +19052 @@
+Proof modification of "onfrALTVD" is discouraged (152 steps).
@@ -19031,0 +19054 @@
+Proof modification of "onfrALTlem1VD" is discouraged (105 steps).
@@ -19032,0 +19056 @@
+Proof modification of "onfrALTlem2VD" is discouraged (377 steps).
@@ -19033,0 +19058 @@
+Proof modification of "onfrALTlem3VD" is discouraged (218 steps).
@@ -19034,0 +19060 @@
+Proof modification of "onfrALTlem4VD" is discouraged (116 steps).
@@ -19035,0 +19062 @@
+Proof modification of "onfrALTlem5VD" is discouraged (320 steps).
@@ -19046,0 +19074 @@
+Proof modification of "ordelordALTVD" is discouraged (202 steps).
@@ -19131,0 +19160 @@
+Proof modification of "relopabVD" is discouraged (354 steps).
@@ -19173,0 +19203 @@
+Proof modification of "rspsbc2VD" is discouraged (90 steps).
@@ -19194,0 +19225 @@
+Proof modification of "sb5ALTVD" is discouraged (110 steps).
@@ -19216,0 +19248 @@
+Proof modification of "sbc3orgVD" is discouraged (180 steps).
@@ -19220,0 +19253 @@
+Proof modification of "sbcbiVD" is discouraged (59 steps).
@@ -19223,0 +19257 @@
+Proof modification of "sbcim2gVD" is discouraged (139 steps).
@@ -19226,0 +19261 @@
+Proof modification of "sbcoreleleqVD" is discouraged (176 steps).
@@ -19227,0 +19263 @@
+Proof modification of "sbcssgVD" is discouraged (229 steps).
@@ -19266,0 +19303,2 @@
+Proof modification of "simplbi2VD" is discouraged (24 steps).
+Proof modification of "simplbi2comtVD" is discouraged (42 steps).
@@ -19302,0 +19341 @@
+Proof modification of "ssralv2VD" is discouraged (147 steps).
@@ -19307,0 +19347,2 @@
+Proof modification of "sucidALTVD" is discouraged (28 steps).
+Proof modification of "sucidVD" is discouraged (24 steps).
@@ -19341,0 +19383 @@
+Proof modification of "tratrbVD" is discouraged (395 steps).
@@ -19344,0 +19387 @@
+Proof modification of "trintALTVD" is discouraged (211 steps).
@@ -19345,0 +19389 @@
+Proof modification of "trsbcVD" is discouraged (396 steps).
@@ -19350,0 +19395 @@
+Proof modification of "truniALTVD" is discouraged (220 steps).
@@ -19357,0 +19403 @@
+Proof modification of "undif3VD" is discouraged (295 steps).
@@ -19415,0 +19462 @@
+Proof modification of "vk15.4jVD" is discouraged (268 steps).
Error: Process completed with exit code 1.

Basically metamath-knife incorrectly removes some lines about proof modification. I observed that these theorems contain an HTML table in their comments, which might be related to why the (Proof modification is discouraged.) tag isn't being read for them.

digama0 commented 6 months ago

Could you try this on the latest version? I was not able to reproduce (running the latest metamath-knife on your branch produces a discouraged file containing Proof modification of "19.21a3con13vVD" is discouraged (107 steps). among other things).

GinoGiotto commented 6 months ago

I believe I already had the most recent metamath-knife version, but to be sure I did the following:

What I've been done is somewhat redundant considering that the last commit of metamath-knife fixes issue https://github.com/metamath/metamath-knife/issues/147#issuecomment-1829350039, so once I tested that metamath-knife demo0.mm --dump-formula produced the fixed output I knew that it must be the newest version.

digama0 commented 6 months ago

Could you try the following:

git clone https://github.com/metamath/set.mm
cd set.mm
git checkout b8e5ee3affd6f73b88a656600d2cb5b92c13ef65 # the parent of discouraged-test2
cd ..

git clone https://github.com/metamath/metamath-knife
cd metamath-knife
git checkout ee02227e5feddb0d60133ed27f43de3a49f6008b # current main
cargo run --release -- ../set.mm/set.mm --discouraged ../set.mm/discouraged2
diff ../set.mm/discouraged ../set.mm/discouraged2

The final diff should be empty, and when I do this I get an empty diff (and the resulting file is 19532 lines long according to wc -l)

GinoGiotto commented 6 months ago

git clone https://github.com/metamath/set.mm

My internet connection is giving me problems, hope it's ok if I did git clone --depth 1 https://github.com/metamath/set.mm (b8e5ee.. is still the latest commit, so I think it's ok for now).

diff ../set.mm/discouraged ../set.mm/discouraged2

I am on Windows, so I did fc discouraged discouraged2

The final diff should be empty, and when I do this I get an empty diff (and the resulting file is 19532 lines long according to wc -l)

The final diff is not empty for me, and with a quick look it seems to be the same as the github diff. The file "discouraged" has 19532 lines + a blank line, while "discouraged2" has 19485 lines + a blank line.

GinoGiotto commented 6 months ago

I think I'm getting somewhere. The metamath.exe rewrapping algorithm does not break inside a (Proof modification is discouraged.) tag or inside a (New usage is discouraged.) tag, but for these theorems the metamath.exe rewrapping algorithm fails (I still have to figure out why, but it seems related to the presence of the html tables), so for these theorems the first tag gets breaked (and usually the (Proof modification is discouraged.) tag is written before the (New usage is discouraged.) one, so it does not happen that the second one gets breaked as well since it's not long enough). Since metamath-knife assumes that those tags are not breaked, it cannot read the (Proof modification is discouraged.) one because in these cases it is breaked.

In this minimal test case metamath-knife cannot read the (Proof modification is discouraged.) tag, can you replicate this?

$c A $.

axiom $a A $.

$( (Proof modification
is discouraged.) (New usage is discouraged.) $)
theorem $p A $= axiom $.

Generated discouraged:

New usage of "theorem" is discouraged (0 uses).
digama0 commented 6 months ago

No, metamath-knife understands line breaking and this should not be a problem... unless the set.mm file you have uses \r\n newlines instead of \n!

GinoGiotto commented 6 months ago

No, metamath-knife understands line breaking and this should not be a problem... unless the set.mm file you have uses \r\n newlines instead of \n!

Ah, so the problem might be related to the fact that I'm on Windows then.

GinoGiotto commented 6 months ago

It works now! Thank you.

A question: what I mentioned before about metamath-exe rewrapping does happen tho. If I read the following minimal example and run write source test1.mm /rewrap then the comment above theorem is not rewrapped. Is this a bug or is it done on purpose to not mess up the tables?

Input:

  $c A $.

  axiom $a A $.

  ${
    $( <HTML> <TABLE>
       </TABLE> </HTML>
       (Contributed by GG, 03-Jan-2023.)  (Proof modification is discouraged.)  (New usage is discouraged.) $)
    theorem $p A $= axiom $.
  $}

Output:

  $c A $.
  $( PLEASE PUT DESCRIPTION HERE. $)
  axiom $a A $.

  ${
    $( <HTML> <TABLE>
       </TABLE> </HTML>
       (Contributed by GG, 03-Jan-2023.)  (Proof modification is discouraged.)  (New usage is discouraged.) $)
    theorem $p A $= axiom $.
  $}

digama0 commented 6 months ago

I think metamath-exe will deliberately not rewrap comments containing <HTML> to avoid breaking manual formatting.

GinoGiotto commented 6 months ago

I think metamath-exe will deliberately not rewrap comments containing <HTML> to avoid breaking manual formatting.

Then it's ok with me.