Leo failing to recognize change in external at-file node

[maphew]

Recipe

1. Set local git repo to match "commit b380d72f300c2c534cd8cb687f9e5b0a03cd5ed5 (HEAD -> desktop-links, origin/desktop-links)". Change active directory to .\leo\scripts\win

2. run pasteurize -w unregister-leo.leox && pasteurize -w register-leo.leox (c.f. https://python-future.org/pasteurize.html#backwards-conversion)

3. run leo ..\scripts.leo and look at at-file nodes for the .leox scripts:

   .../scripts.leo#Scripts-->@file leoScripts.txt-->Windows-only scripts-->@file register-leo.leox
   .../scripts.leo#Scripts-->@file leoScripts.txt-->Windows-only scripts-->@file unregister-leo.leox

   Notice that Leo does not show the changes in either of these files.

4. Use "Refresh from disk" on both nodes --> unregister shows the inserted lines, register does not.

[maphew]

Noticing that pasteurize seemed to begin placing its changes after the first docstring in the file I hard reset checkout to the beginning, inserted a docstring in register-leo.leox after at-file node beginning and before at-language etc., re-ran pasteurize. True to theory, that's where the pasteurize changes started. And Leo picked up the changes (though I still needed to use Refresh-from-disk, which I consider an additional problem).

#@+leo-ver=5-thin
#@+node:maphew.20130611235500.2658: * @file register-leo.leox
'''test placement of first docstring in at-file node (edited outside Leo)'''
#@@language python
#@@tabwidth -4
#@+others

[edreamleo]

@maphew Matt, I'd like your help with this as follows:

1. Make sure that @bool check-for-changed-external-files is True.

2. Presumably, the fault lies in leoExternalFiles.py. Rev 1e918e in devel adds a disabled trace in efc.idle_check_at_file_node. Please find that method and enable the trace as shown. The idea is just to trace the file of interest to you.

Please report what the trace shows. Thanks.

[maphew]

Verify changed files is true

Alt-X >> show-settings reports: @bool checkforchangedexternalfiles = True

((aside: wish that "alt-x: show-settings sometext" would emit list for 'sometext'))

Ensure workspace is in clean state and go through bug recipe:

Commands only:

pushd d:\code-maphew\leo-editor\leo\scripts
git checkout 1e918e && git reset --hard
:: Add trace
pasteurize -w win-unregister-leo.leox && pasteurize -w win-register-leo.leox
leo-m scripts.leo
:: refresh from disk
git diff

Commands with output:

$ pushd d:\code-maphew\leo-editor\leo\scripts

$ git checkout 1e918e && git reset --hard

HEAD is now at 1e918e95 Added a disabled trace for Matt Wilke in efc.idle_check_at_file_node
HEAD is now at 1e918e95 Added a disabled trace for Matt Wilke in efc.idle_check_at_file_node

Manually add trace lines and save >> show diff:

$ git diff

diff --git a/leo/core/leoExternalFiles.py b/leo/core/leoExternalFiles.py
index 516c992c..90c14499 100644
--- a/leo/core/leoExternalFiles.py
+++ b/leo/core/leoExternalFiles.py
@@ -191,7 +191,8 @@ class ExternalFilesController(object):
     #@+node:ekr.20150403044823.1: *5* efc.idle_check_at_file_node
     def idle_check_at_file_node(self, c, p):
         '''Check the @<file> node at p for external changes.'''
-        trace = False
+        #trace = False
+        trace = ['@file win-unregister-leo.leox', '@file win-register-leo.leox']
             # Matt, set this to True, but only for the file that interests you.\
             # trace = p.h == '@file unregister-leo.leox'
         path = g.fullPath(c, p)

Make some changes to .leox files:

$ pasteurize -w win-unregister-leo.leox && pasteurize -w win-register-leo.leox

{...snip...}
RefactoringTool: Files that were modified:
RefactoringTool: win-unregister-leo.leox

{...snip...}
RefactoringTool: Files that were modified:
RefactoringTool: win-register-leo.leox

Open leo with scripts.leo >> Refresh from disk the 2 .leox files >> Close Leo:

$ leo-m scripts.leo

Leo 5.8.1-b2 devel, build 20190227233831, Wed Feb 27 23:38:31 PST 2019
livecode.py: can not import meta
pip install meta
livecode.py: can not import meta
pip install meta
idle_check_at_file_node changed False @file gen_blender_docs.py
idle_check_at_file_node changed False @file leoScripts.txt
idle_check_at_file_node changed False @file gen_blender_docs.py
idle_check_at_file_node changed False @file leoScripts.txt
idle_check_at_file_node changed False @file gen_blender_docs.py
idle_check_at_file_node changed False @file leoScripts.txt
idle_check_at_file_node changed False @file gen_blender_docs.py
idle_check_at_file_node changed False @file leoScripts.txt
idle_check_at_file_node changed False @file gen_blender_docs.py
idle_check_at_file_node changed False @file leoScripts.txt
idle_check_at_file_node changed False @file gen_blender_docs.py
idle_check_at_file_node changed False @file leoScripts.txt
idle_check_at_file_node changed False @file gen_blender_docs.py
idle_check_at_file_node changed False @file leoScripts.txt
idle_check_at_file_node changed False @file gen_blender_docs.py
idle_check_at_file_node changed False @file leoScripts.txt
idle_check_at_file_node changed False @file gen_blender_docs.py
idle_check_at_file_node changed False @file leoScripts.txt
idle_check_at_file_node changed False @file gen_blender_docs.py
idle_check_at_file_node changed False @file leoScripts.txt
idle_check_at_file_node changed False @file gen_blender_docs.py
idle_check_at_file_node changed False @file leoScripts.txt
idle_check_at_file_node changed False @file gen_blender_docs.py
idle_check_at_file_node changed False @file leoScripts.txt
idle_check_at_file_node changed False @file gen_blender_docs.py
idle_check_at_file_node changed False @file leoScripts.txt
idle_check_at_file_node changed False @file gen_blender_docs.py
idle_check_at_file_node changed False @file leoScripts.txt
idle_check_at_file_node changed False @file gen_blender_docs.py
idle_check_at_file_node changed False @file leoScripts.txt
wrote recent file: C:/Users/mhwilkie/.leo/.leoRecentFiles.txt

Git reported changes in file system:

$ git diff

diff --git a/leo/core/leoExternalFiles.py b/leo/core/leoExternalFiles.py
index 516c992c..ce49b9b1 100644
--- a/leo/core/leoExternalFiles.py
+++ b/leo/core/leoExternalFiles.py
@@ -191,7 +191,8 @@ class ExternalFilesController(object):
     #@+node:ekr.20150403044823.1: *5* efc.idle_check_at_file_node
     def idle_check_at_file_node(self, c, p):
         '''Check the @<file> node at p for external changes.'''
-        trace = False
+        #trace = False
+        trace = '@file win-register-leo.leox'
             # Matt, set this to True, but only for the file that interests you.\
             # trace = p.h == '@file unregister-leo.leox'
         path = g.fullPath(c, p)
diff --git a/leo/scripts/win-register-leo.leox b/leo/scripts/win-register-leo.leox
index 56b1cb9c..db833699 100644
--- a/leo/scripts/win-register-leo.leox
+++ b/leo/scripts/win-register-leo.leox
@@ -1,16 +1,22 @@
+from __future__ import unicode_literals
+from __future__ import print_function
+from __future__ import division
+from __future__ import absolute_import
 #@+leo-ver=5-thin
 #@+node:maphew.20130611235500.2658: * @file win-register-leo.leox
 #@@language python
 #@@tabwidth -4
 #@+others
 #@+node:mhw-debug-1081.20190216130451.1: ** imports
+from future import standard_library
+standard_library.install_aliases()
 import leo
 import leo.core.leoGlobals as g
 import os
 import sys
 import shutil
 try:
-    from _winreg import * #py2
+    from winreg import * #py2
 except ModuleNotFoundError as err:
     from winreg import * #py3
 #@+node:maphew.20130510011223.1645: ** docstring
diff --git a/leo/scripts/win-unregister-leo.leox b/leo/scripts/win-unregister-leo.leox
index 8d741414..54c10d54 100644
--- a/leo/scripts/win-unregister-leo.leox
+++ b/leo/scripts/win-unregister-leo.leox
@@ -3,15 +3,21 @@
 #@@language python
 #@@tabwidth -4
 '''Remove registry key for Leo file type and .leo file association'''
+from __future__ import unicode_literals
+from __future__ import print_function
+from __future__ import division
+from __future__ import absolute_import
 #@+others
 #@+node:mhw-debug-1081.20190216130451.1: ** imports
+from future import standard_library
+standard_library.install_aliases()
 import leo
 import leo.core.leoGlobals as g
 import os
 import sys
 import shutil
 try:
-    from _winreg import * #py2
+    from winreg import * #py2
 except ModuleNotFoundError as err:
     from winreg import * #py3
 #@+node:mhw-debug-1081.20190216145532.1: ** unregister_leo

[maphew]

...forgot to confirm that yes I still see the reported problem with this latest run through.

[edreamleo]

Thanks for the new work. However, I wasn't clear about how to enable tracing for files that interest you. The code should be: trace = p.h == '@file win-register-leo.leox'

Your code always enables tracing, which produces too much.

Please rerun with the new trace.

[maphew]

Recipe:

pushd d:\code-maphew\leo-editor
git checkout 1e918e && git reset --hard
git apply ..\case1100-enable-trace.diff
pushd leo\scripts
pasteurize -w win-unregister-leo.leox && pasteurize -w win-register-leo.leox
leo-m scripts.leo

Patch for step#3: case1100-enable-trace-diff.txt

In Leo navigate to nodes below and "Refresh from disk":

• scripts.leo#Scripts-->@file leoScripts.txt-->Command-line scripts-->Windows-only scripts-->@file win-register-leo.leox
  • scripts.leo#Scripts-->@file leoScripts.txt-->Command-line scripts-->Windows-only scripts-->@file win-unregister-leo.leox

Save scripts.leo and Exit

Console log of this session:

$ leo-m scripts.leo

Leo 5.8.1-b2 devel, build 20190227233831, Wed Feb 27 23:38:31 PST 2019
livecode.py: can not import meta
pip install meta
livecode.py: can not import meta
pip install meta
wrote recent file: C:/Users/mhwilkie/.leo/.leoRecentFiles.txt

Leo log of this session:

Leo Log Window
Leo 5.8.1-b2 devel, build 20190227233831, Wed Feb 27 23:38:31 PST 2019
Git repo info: branch = None, commit = 1e918e9
Python 3.7.1, PyQt version 5.11.2
Windows 10 AMD64 (build 10.0.17134) SP0
isPython3: True
current dir: d:/code-maphew/leo-editor/leo/scripts
load dir: D:/code-maphew/leo-editor/leo/core
global config dir: D:/code-maphew/leo-editor/leo/config
home dir: C:/Users/mhwilkie
reading settings in D:/code-maphew/leo-editor/leo/config/leoSettings.leo
reading settings in d:/code-maphew/leo-editor/leo/scripts/scripts.leo
livecode.py: can not import meta
pip install meta
reading settings in d:/code-maphew/leo-editor/leo/scripts/scripts.leo
livecode.py: can not import meta
pip install meta
read 2 files in 0.06 seconds
read outline in 0.11 seconds

wrote recent file: C:/Users/mhwilkie/.leo/.leoRecentFiles.txt
saved: scripts.leo

[maphew]

I believe the problem arises from nested @file nodes. Leo doesn't seem to register changes past the first one.

-- Root node
    \-- a child node
          \-- @file foo.txt
                  \-- @file bar.py

Enable trace on bar.py: nothing happens.

Move bar.py up to child node:

-- Root node
    \-- a child node
          \-- @file foo.txt
    \-- @file bar.py

...and console log starts emitting idle_check_at_file_node changed messages, and Leo's body pane shows changes in the external file without asking it to Refresh-from-disk.

...

How I figured this out:

• git reset hard
• enable trace
• leo-m scripts.leo (no idle_check messages emitted)
  • move @file win-unregister-leo.leox outside of parent at-file node (idle_check messages start)
  • save .leo file
  • copy .leo file outside of git repo
• git reset hard
• enable trace
• change *.leox files outside of Leo
• copy stashed .leo over git's .leo
• run leo-m scripts.leo
  • the changes to external .leox files show in Leo without refresh-from-disk

Bugs/issues/recommendations:

1. Nested @files nodes work, but probably shouldn't be used. Leo could/should issue warning if they're used. (Case in point: the nested nodes here were created up to 6 years ago, and neither myself, you or Terry saw a problem.)

2. Even when not nested, content above the opening Leo sentinel #@+leo-ver=5-thin is invisible to Leo, or at least not shown in the body pane. Here that means all the from __future__ lines:

from __future__ import unicode_literals
from __future__ import print_function
from __future__ import division
from __future__ import absolute_import
#@+leo-ver=5-thin
#@+node:maphew.20130611235500.2658: * @file win-register-leo.leox
{...snip...}

[edreamleo]

On Thu, Apr 18, 2019 at 10:39 PM matt wilkie notifications@github.com wrote:

I believe the problem arises from nested @file nodes. Leo doesn't seem to register changes past the first one.

Thanks, Matt. This looks like the crucial clue. This will be fixed before the b2 release.

Edward

[edreamleo]

Fixed, I think, in both devel and py3. Please reopen if problems remain.

[maphew]

15 lines code to 3. Nice ;-) Thanks Edward

[edreamleo]

On Tue, Apr 23, 2019 at 1:44 PM matt wilkie notifications@github.com wrote:

15 lines code to 3. Nice ;-)

Only if it works. I take it it does?

Edward

[maphew]

more testing needed for true confidence. not sure when I'll get to that!

[maphew]

I think there's still something lurking here, but I haven't been able to build a clean test case to reproduce what I thought I saw. Leaving closed until I can do that, but at the same time am not devoting attention to that task until something changes. There are bigger fish to seek.