TODO item does not link to correct line

[DeeEmm]

When clicking on the item in the sidebar the editor changes to the correct file, however the highlighted line is incorrect.

Please see above screenshot.

• Highlighted line (100) is incorrect
• Actual line that TODO refers to is line 124

This issue is consistent throughout my projects however the number of lines that the pointer is out varies.

/DM

[jasonplatts]

Hi DM,

Thanks for letting me know about the problems you are having with the extension. I'm having a hard time replicating the issue on my end.

If you press the "Refresh" icon in the top right hand corner of the TODO panel, does it then provide you the correct position of the tag? When you first open the file, are the locations correct?

Thanks! Jason

[DeeEmm]

Hi Jason,

I was hoping it might be a simple Schoolboy error on my part.

Unfortunately, refreshing the list / reopening Nova does not make any difference. Nor does expanding or regrouping the list. I have also tried reinstalling the plugin.

One thing of note is that when I initially open Nova the TODO list only shows one file and will not expand by clicking on the expand arrow next to the file name. I have to click on the group-by button to get the list to expand. After this point l can filter / group the list as expected. I suspect that this is related as it does not seem like designed behaviour.

I know it's a hard trying to debug remotely with minimal info and no access to my machine and can appreciate that it's hard to replicate what is quite likely an environment issue so if there's anything i can do to assist just let me know.

If it helps I'm running Nova 7.1 Build 293145 on Big Sur 11.5 (20G71)

/DM

[DeeEmm]

...as a comparison, here's what the list looks like after I hit the group button, as you can see it does not display the second file in the project until I hit the button

[jasonplatts]

Thanks for the additional information DM.

A couple of thoughts...

1. If you go to the 'Extension' menu and select 'Show Extension Console', do you see any errors?
2. Are you working with a very large project? If so, does it help if you add irrelevant directories to the ignored list in the extension preferences?

If these suggestions don't help and you are able to provide me with a file that consistently produces this behavior, I would be happy to take a look. I understand that may not be possible, but I'm not sure what else to try at this point.

Wish I could be more help. I will let you know if I have any more ideas.

Thanks!

[DeeEmm]

Hi Jason.

I do (did) have a python error showing in the Extensions console...

Python Language Server[23:37:56.399000] 2021-07-27 23:37:56,398 UTC - WARNING - pyls.config.config - Failed to load pyls entry point 'pyls_mypy': No module named 'future'

So I've disabled all extensions apart from yours and restarted Nova. The error message has cleared but the sidebar behaviour described above along with the reported issue is still the same. I assume the error is form a different plugin (I will investigate this further)

If you want to try with the files I'm working with, you can download the project I'm working on from GitHub...

https://github.com/DeeEmm/DIY-Flow-Bench/tree/ESP32

I'm a bit ahead of the repo as I have some uncommitted changes but it's not massively different. There are one or two additional local projectio files that are used for compiling and transferring but these are not open in the editor.

Thinking on this...I actually wonder if this might be a nova/project/theia related issue. There is some project behaviour that seems a bit flakey to me. I migrated the projects from Coda when Nova launched so perhaps this could be related?? I might try creating a new project space.

/DM

[DeeEmm]

(Confirmed: the error above is from a Python plugin)

[jasonplatts]

That was very helpful! Thanks. I am thinking it might be related to your environment. When I tested the files in a new project, all positions appeared correct.

If you decide to create a new project space, let me know if it fixes the problem!

[DeeEmm]

Creating a new test project and copying the project contents across fixed the issue. Yay :)

I've now deleted the original project, dumped the original folder and copied everything back into a new project (of the same name) and everything is working fine. I had to copy some hidden folders back for git/pio but they have not brought back the problem. So I suspect that it was Nova project / workspace related.

I also note that I have lots more TODO's listed now as well. I have added no additional TODO's since the original screenshot. I guess I'm further behind than I thought lol.

There must have been something in the original project that stopped it parsing.

Thanks for your assistance Jason. Very much appreciated.

P.S.

Just a heads-up. I'm showing more TODO's than you are. I think the reason for this is that when you parse you actually check for 'TODO:' not just 'TODO' which is the format that I have traditionally used throughout my code. So I did a search and destroy on the local code in the project to add the colons, but I'd rather not have to do that to all my projects.

Is there any reason that you require the colon?? Dropping this requirement will obviously match more entries. I would venture to say that it's not really needed - 'TODO' should not return any false positives as it's pretty unambiguous.

Could I therefore suggest an enhancement; making it a delimiter that is user configurable via text box in the settings dialog? This way users can tailor it to their particular coding style (TODO- TODO> TODO... TODO-- (TODO) [TODO] etc/whatever). The semantics of some languages tend to favour different styles as well, not to mention that may coders will adopt the style of the code author if they are working on libraries extending existing code.

Cheers. And thanks for a great plugin.

/DM

[jasonplatts]

That's great! Happy to help.

Yes, the reason for the colon is simply to try to reduce false positives. While 'TODO' may not return many false positives, I was concerned that 'NOTE' and others might. I also plan to add a feature that would allow users to add their own tags, which I thought might cause more issues. Great suggestion though! I can see how it would be useful. I'll work on adding some configuration options in an upcoming release.

You're welcome!

[DeeEmm]

I also plan to add a feature that would allow users to add their own tags, which I thought might cause more issues.

Haha, yes. just one escape char away from kernel panic lol.

[DeeEmm]

Hi Jason,

I have experienced this issue again so I think that the cause is not as previously suspected. I do however have another idea...

I actually develop on two machines. The iMac in my office and my Macbook at home. I sync all of my data using iCloud. The documents folders on both machines point to my iCloud account data. So when I update a file on my iMac, the same file update is automagically updated on the Macbook.

I suspect that therein lies this issue... When I am editing on one machine, it TODO is somehow picking up a cached version of the file.

The previous 'fix' of recreating the project, simply renewed all cached files and so the problem went away.

I have not dug into this any further but noticed this when I went to review my TODO's.

Any suggestions??

/DM

[DeeEmm]

As a quick test I added a TODO comment to force a refresh of the list.

(I know that in an Nova extension I wrote that the sidebar is updated pretty much instantly.)

Whilst the new TODO appeared in the list on the document save, the position was still incorrect.

[DeeEmm]

I tried closing and reopening Nova and the line numbers were then correct.

Previously...

• Nova was open on both machines
• I would work on both machines alternately
• Line number issue present

What I just tried...

• Closing Nova on Macbook (issue still present on iMac)
• Adding TODO on iMac to force TODO list refresh (issue still present on iMac)
• Restarting Nova on iMac (issue not present)

Can I ask what file does the TODO extension parse as it does not appear to be parsing the live file that is open in Nova. Are you using a cached file, or a Nova file reference that might point to a cached file?

Happy to try out some additional tests if you cannot replicate this at your end.

/DM

[DeeEmm]

Further testing & observations...

Just back working on my iMac. The Macbook still has the Nova project open.

Line numbers on the iMac are incorrect. Simply restarting Nova on the iMac fixes this.

Definitely suspect some kind of cache issue

/DM

[jasonplatts]

Hi DM,

Sorry you are still having issues with the line numbers.

To answer your question, when the extension first loads in a local workspace, it runs the OS "egrep" command in a process to determine what files should be parsed. The parsing of these identified files then occurs using the file related Nova API methods. Once this initial loading is complete, the extension monitors files that are opened and saved in Nova itself. If a tag is found when an opened file is saved, the editor related Nova API methods are used to parse the contents of the open editor.

Caching could certainly be the root cause of the problem. I wonder if the issue would still occur when using another cloud storage provider, such as Dropbox? I also use the Alfred Mac app and I know they recommend to avoid syncing using iCloud because of problems that can occur.

Unfortunately, if it is at the OS level, I'm not sure there is much that can be done at the extension level to fix the problem. Especially since after the initial loading, the extension only ever reads content provided directly from the Nova API. I will reach out to Panic, direct them to this issue, and see if they have any ideas or suggestions. And, I will certainly try to fix the problem if it is fixable at the extension level.

Thanks again, Jason

[DeeEmm]

Hi Jason,

Thanks for getting back to me.

If it is using the Nova API hooks I wonder if the cache issue is related to panic sync and not the physical documents, in which case using a third party provider to sync the documents may still not resolve the issue as the Nova cache would still not be updated.

Looking at what I have found with the tests I have carried out I think that the Cache may be released / updated when the program is closed down or perhaps only read when the program is opened. Perhaps the cache files are locked and unable to be updated by the second instance.

Obviously I do have a workaround - making sure I close down the app before I switch machines, but I do tend to jump from machine to machine if I'm using my macbook at the workshop, which is fine in everything except for this issue with the TODO extension. The actual files themselves are updated on both machines without intervention.

I'm going to try and force a panic sync and see if that has any effect. If that doesn't work I might look at a third party sync as a test, however my preference is to keep the native icloud sync as it works well and I have over 100GB of critical data that I would prefer not to mess with.

I'll report back with what I find.

/DM

[DeeEmm]

Ok.

So Panic Sync made no difference.

On restarting Nova on my iMac I have noticed a delay in the Todo list updating. This may be unrelated but I'll mention it just in case...

The list is blank and I receive the message above

Then after clicking the list (or maybe it was a period of time I was unable to replicate to test further) I then see the list, but it only shows the file names, not the code snippet.

If I click on 'Group By', the snippet then appears

This behaviour does not seem to occur on the Macbook and also not every time on the iMac, however I have noticed it several times. Unfortunately I've not been able to replicate this behaviour to identify why it happens.

I did do some additional testing on the line numbers and can replicate the issue fairly consistently...

• Close down both apps (iMac and Macbook)
• Open both apps
• All line numbers are in sync
• Make an edit on one machine (add a line mid-file)
• Go to second machine
• The file is updated (new line is present within open file in Nova editor), but line numbers are one line out from the inserted line onwards
• Remove line on second machine to reverse the change
• Expectation is that the line numbers are then back in sync
• Line numbers DO NOT revert back to being correct, they appear to be randomly worse (some out by one line, others by several lines within the same file)
• Other files are still okay.
• Closing the file and reopening it remedies the issue (it is not necessary to reopen the app itself)
• Hitting the TODO's refresh button also remedies the issue.

This last solution was different from the original problem, which was not recoverable and I now suspect was indeed an unrelated problem. Given that the refresh fixes the issue I would say that whilst the line number behaviour is not ideal, there is a simple fix and forcing a refresh brings everything back in line.

I'm not really sure if it is feasible to force a refresh as the only event that would make sense would be onFocus, which may make the behaviour appear to be glitchy (the list refreshes as you click on it.)

However, if it is possible to refresh the list when the sidebar tab is clicked on, this would address the issue for my workflow as I only view the sidebar when I need to look at the next task to work on. The rest of the time I have the symbol list open. Obviously other users workflows and sidebar layout will differ and so this might not be a one-size-fits solution.

/DM

[jasonplatts]

Hi DM,

I appreciate you taking the time to test and try to replicate the issue.

It isn't currently possible to detect when the sidebar tab is clicked. Others have asked for this functionality to be added to the Nova API, so hopefully it will be added at some point. However, based on past experiences with the extension, I would be hesitant to refresh on this type of event because I think it could end up causing some performance issues in larger projects.

I am happy to hear that the manual refresh seems to resolve the problem you. When I have some more time to put into the extension, I will take a look at this again and see if any improvements can be made.

Thanks again, Jason

[DeeEmm]

That's all good. I can definitely live with the manual refresh. It's no big deal.

I'll close this out.

Thanks once again for your help.

/DM