Tail functionality

[kofifus]

A few related questions ..

1. Looking here https://github.com/rizonesoft/Notepad3/issues/456 it seems there should be an eye icon to stop/start tail however on my version:

Notepad3 (64-bit) v5.19.402.1671 develop Scintilla v414+ (RegEx: Onigmo v6.2.0) Compiler: MS Visual C++ 2019 16.0.0 Prev 4.(3-4) (VC v1920) [en-US] English(amer.) text by Flo Balmer et al.

This icon is missing.

2. When choosing Auto-reload (unmodified) is the file actually reloaded every time it changes ? this would make it terribly inefficient for tailing large logs .. or is it that just the added tail is loaded ?

Thanks!

[RaiKoHoff]

1.) Are you able to add the icon: right mouse-button on tool-bar should open a context-menu with entry Customize Toolbar... ? If answer is 'YES', then it was just missing in your configuration (maybe, because it is a new feature, which has been not known by old configurations.

2.) Notepad3 auto reloads the complete file, which makes it indeed very inefficient for tailing large logs, but a text-editor is not supposed to be used as a "large log file tail growing viewer" 😉 , there are better tools around for this purpose...

[blackcrack]

:+1: http://tailforwin32.sourceforge.net/ https://www.heise.de/download/product/tailforwindows-92939 http://www.baremetalsoft.com/wintail/

huhh was this an a.. kick ? *lol*

damnn.. notepad3 is grown adipose/fat/fatty / huge enough.. there dont need a tail funktion.. best Blacky

[RaiKoHoff]

Notepad2/3 already had a "reopen file automatically on change" functionality, so the (inefficient) "tail" functionality was just a natural extension. Keeping Notepad3 "mean and lean" but providing sensible functionality is always a trade-off 😐

[kofifus]

Ok I reinstalled notepad3 and the button is now there.

Basically if I understand correctly what we have is an 'auto reload' functionality, not a tail functionality.

Is there a difference between:

settings -> file change notifications -> Auto reload

and

view -> display -> Doc tail chasing (or pressing the toolbar button)

?

IMO The feature/name/icon 'Doc Tail Chasing' is misleading, it obviously makes you think of tail functionality which as you said above it is not.

I suggest that the icon and feature are removed. And settings -> file change notifications -> Auto reload is used for that if needed.

It would be awesome to have Notepad3 provide true trail functionality, I have been unhappy with all other options I know (wintail/baretail/glogg) but this would prob be big work ...

Thanks!

[RaiKoHoff]

If I remember correctly, this button has been implemented on an user request. He knows about the Auto-Reload-Functionality but he misses the "Jump to the End" of the reloaded file and he want's all that be bundled on one button press. So we should stay with this functionality. We may think about an option for "chasing" the "head" or the "tail" of the (log-)file.

[hpwamr]

If I remember correctly, this button has been implemented on an user request.

Yes you're right, this "Tail Chasing" feature and button were at the request of @skest3qc (see: #456 ) 😏

[kofifus]

I see 3 ways forward:

1. Leave things as they are - this is my least favorite, basically ATM Notepad3 presents as if it has real tail functionality with option and icon where in fact this is a very inefficient auto reload + scroll to end which to me is misleading.

2. Remove the option from view -> display -> Doc tail chasing and the icon and add a tick box next to settings -> file change notifications -> Auto reload for 'Auto Scroll to End' or something like that. That will make the existing functionality possible while not advertising it as a proper Tail.

3. Work on a true tail functionality. Tail assumes the file was only appended to since the last time it was loaded. Instead of a reload, tail only appends whatever was added (based on file size). This will IMO be a great addition to Notepad3 and enough to present as Tail. A further addition to that would be to only view a certain amount of text from the end which will allow tailing very big files but that will probably be more complicated.

[RaiKoHoff]

A true "tail" functionallity is out of scope (near future). Removing existing functionality (icon) is a critical path. We can do a renaming "Doc Tail Chasing" -> "Auto Reload - Watch Tail(or End)" and hide the icon in default setup/installation (must be activated using "Customizing Toolbar...") ?

[hpwamr]

1. Leave things as they are - this is my least favorite,

Leave ALL things as they are is my preference !

Let's see the vote or reactions of other users of Notepad3. 🤔 I vote to keep it unchanged ! 😃

[kofifus]

Yes @RaiKoHoff both sounds great, and any wording that doesn't have the word tail in it is better .. as a developer myself, tail means something else than the current functionality..

Thanks !

[RaiKoHoff]

I vote for renaming (suggestion: "Auto Reload - Watch End" (any better/shorter suggestion?)) and initial default Toolbar configuration without :eye:-Button.

[kofifus]

+1

[craigo-]

Agree with the discussion to remove the word 'tail'. Happy with @RaiKoHoff's naming suggestion (I played around in my head with the phrase 'Focus On End', but it was longer and more unwieldy so I'm not advocating that).

NB for useful tail functionality on Windows I'm still using unxutils' tail (by way of a batch file) - 16 years since last update!

[hpwamr]

The majority wins. 😃

I propose to rename it to:

• "Focusing" or "Focusing Log" or "Focusing Doc" or "Focusing View"
• "Watching" or "Watching Log" or "Watching Doc" or "Watching View"
• "Monitoring" or "Monitoring Log" or "Monitoring Doc" or "Monitoring View" 🤔

[RaiKoHoff]

I vote for Monitoring Log, cause the verb "monitoring" describes very well the functionality and the "log" implies, that 99% of files, which got a frequent update (minutes, second or milliseconds) at the "tail", are log-files. I am unsure about the need to add the Auto-Reload aspect in the name :thinking:

[hpwamr]

I agree 👍 , "Monitoring Log", I already use it in my French, Spanish and Dutch translations ... 😃

[kofifus]

All sounds good, more important imo is removing it from defaults, as I said above my vote is for this to be removed altogether .. why is it there ? If someone wants to tail a file use a tail tool not a text editor..

[RaiKoHoff]

@kofifus : One use-case to think of is: Monitoring a log for a while (time or event), stop monitoring and use the functionality of this editor to , e.g., search with Regex power, copy text snippets, save as, using focused view on selection/finding, count occurrences, ...

[kofifus]

For sure, but then you would also want true tail functionality which I hope will some day come to Notepad3 :)

[ltguillaume]

I've been checking out this feature, but I don't understand its use for the following reasons:

• Apparently it doesn't work well with files that are currently opened by another process. Only a manual refresh works, otherwise the automatic refresh can take minutes, even though the log file has been updated constantly.
• Currently, a reload/refresh will put the cursor back at the first line. I understand that some people would like it to scroll to the last line automatically, which I don't get at all. I'd think ideally it would just refresh every x second and you get to control wherever the cursor is. Either way, jumping back to the top doesn't make sense, I'd say.
• It would be really cool to highlight new lines (and keep on doing that) until the user places the cursor on or past these lines. From that moment on, all highlighted lines would be considered "seen" and new lines from then on will be highlighted again.
• The latter could even be expanded to "highlight all changes".

[kofifus]

You are confusing notepad3's auto-refresh for tail functionality. Auto refresh is more useful on small files that may change by some other program (configuration files etc). It is not suited for tailing logs etc for the reasons you mentioned and others. Try a dedicated tail tool (my favorite is glogg)

[ltguillaume]

No, I'm not, I just don't get these two things:

1. Why wouldn't the cursor be kept in the original place, instead of it going to the top line?
2. Why don't the Monitor Log and the File Change Notification feature work on log files that are currently in use by the program creating the log?

[kofifus]

1. I agree that can possibly be changed, but tricky, cursor position may become irrelevant in the refreshed copy

2. Not sure, probably has to do with getting OS read/write access to the file, maybe someone else can comment here

[hpwamr]

No, I'm not, I just don't get these two things:

Hello @ltGuillaume , Notepad3 is an Open Source Project supported only by volunteers... Feel free to make a fork and develop in your fork all the "bells and whistles" that you want ! 😉

[ltguillaume]

I'm not asking for features in https://github.com/rizonesoft/Notepad3/issues/1114#issuecomment-528148432, I'm asking why something works like it does 😜

[RaiKoHoff]

The "tail" functionality is based on the "File Changed Notification" feature of Notepad3 (which is not the same as a tail() utility performs). For the File Changed Notification feature, Notepad3 uses a timer (timeout set by [Settings2]FileCheckInverval=2000 [milliseconds]) to check file modifications every check-interval, using FindNextChangeNotification() and checking differences between currently edited file and file on disk by CompareFileTime() comparing "WriteTimes" and "FileSizes"(Low & High). I am not sure, if a logger, while continuously logging, updates this properties correctly 😲 🤔 .

[RaiKoHoff]

Maybe we should remove that "tail functionality" - other utilities do a much better job here 🤔

[hpwamr]

Maybe we should remove that "tail functionality" - other utilities do a much better job here 🤔

Maybe a vote to keep or remove the "Monitoring Log" ("tail functionality") in Notepad3 ?

• to keep --> 👍
• to remove --> 👎

[kofifus]

I vote to remove the entire "Monitoring Log" feature, to me it does not make sense in a text editor thanks

[ltguillaume]

Oof, I sincerely hope that the File Change Notification stays put. I make use of it all the time (for example. when merging two files with WinMerge or synchronizing two directories, while I also have the file opened in Np3). I think the file change notification is something that can't be missed in any proper editor.

As for "Monitoring log", I'd love to keep it in, but as simple as possible:

• Keep the file read-only
• Actively reload the file every 2 seconds (even if the modified timetamp hasn't changed), because a log that's in use by a program is buffered/cached, and will not be changed on disk, unless an open/read request is made.
• When reloading, keep the cursor in the same position if possible, just like the File Change Notification feature.
• No tailing

I think that'd be a great feature and I would make use of it on a daily basis.

(File Change Notification could be further improved with instant notifications and no need for active polling with the FileCheckInverval timer by making use of WaitForSingleObject - see https://github.com/ProgerXP/Notepad2e/issues/129 and https://github.com/ProgerXP/Notepad2e/blob/c724fc0ca8c837c6f67d18425edf104620b47f61/src/Notepad2.c#L8050)

[kofifus]

Sorry @ltGuillaume I may have confused you, we are talking about the View -> Display -> Monitoring Log functionality not the File Change Notification

[RaiKoHoff]

Of cause, the File Change Notification functionality will not be removed, it is essential. But i think the "tail" functionality based on this File Change Notification is a bad design. In general, a tool which supports tail functionality (watching the growing of a file) should also provide some text filter and alarm pattern configuration/settings - this is beyond the scope of a text editor, especially Notepad3. I think there are many (free) tools out there, which do a much better job than Notepad3 could do. Once the log stream is closed, the file maybe a candidate for Notepad3 ...

[RaiKoHoff]

Okay, two requests here:

1. Change File Change Notification from an interval check to direct change notification
2. Keep File Monitoring (remove 'tail' or 'log' strings) with:
   • forced (no check) reload interval (configurable)
   • read only
   • keep caret position (no tailing) (Remark: the keep caret position is easy for files growing at the end, problematic for files growing at the beginning (putting latest news/log-entries at the top)).

[hpwamr]

Hi @craigo- , @kofifus , @blackcrack , @ltGuillaume , @skest3qc , ...

https://github.com/rizonesoft/Notepad3/issues/1114#issuecomment-528103315 Try a dedicated tail tool (my favorite is glogg)

https://github.com/rizonesoft/Notepad3/issues/1114#issuecomment-486719745 damnn.. notepad3 is grown adipose/fat/fatty / huge enough.. there dont need a tail funktion.. best Blacky

See comment above (voting) : https://github.com/rizonesoft/Notepad3/issues/1114#issuecomment-529214476

Some specialized alternative:

• BareTail
• glogg
• LogExpert
• LogFusion
• LogViewer
• LogViewPlus
• mTAIL
• SnakeTail
• Tail for Win32
• TailExpert
• TailForWindows
• unxutils
• Wintail
• etc...

[RaiKoHoff]

I voted a :+1: , cause I can see a use-case for "File Monitoring: forced reload interval, read-only, keep caret position" in my daily workflow ... :astonished:

[craigo-]

@ltGuillaume, @RaiKoHoff, what is this use case that you speak of? Maybe I’m missing out and will change my vote!

[hpwamr]

I voted a 👍 , cause I can see a use-case for "File Monitoring: forced reload interval, read-only, keep caret position" in my daily workflow ... 😲

I've change my vote because you can use it in your daily workflow. 😉

[ltguillaume]

@ltGuillaume, @RaiKoHoff, what is this use case that you speak of? Maybe I’m missing out and will change my vote! @craigo-

1. Keeping the caret where you left it is really nice, because when you put it at the end right before starting a procedure in the program generating the log, you can easily go through everything new with arrow down, or copy everything below the caret for evaluation/reporting to the author.
2. When putting the log file in Np3 and the program generating the log side by side, you could just hover over Np3 to scroll down and read the new entries while keeping the program that generates the log active (you don't have to Alt-Tab, so to speak).
3. There's value in having a notification that the file has changed with the regular File Changed Notification feature, whereas tracking a log file is nicer to have without that notification. With a Monitor Log toggle button, you can easily have both these features at hand, without being confused about what the current setting for File Changed Notification is.
4. Forced auto-reload (without checking if the timestamp has changed) allows for log files in use to still be monitored. File Changed Notification does not work like that, so you won't get a notification for log files in use.

[kofifus]

@ltGuillaume I again think you insist on using the wrong tool for the job, what you describe is tailing and np3 is very inefficient and limited at at that, its like using np3 for javascript when you can use VScode .. have you tried a dedicated tail tool like baretail or glogg ?

[RaiKoHoff]

Yes, ltGuillaume describes the workflow I used too and I like to do it using my favorite text handling tool.

The worst thing on NP3's tail functionality is the reload of the complete file instead of adding the difference only at the end. I would like to remove the impression, that this is a real "tail monitoring" feature. Maybe a solution would be:

Using the File Changed Notification feature only (but offering 2 flavors/options):

1. Immediate reload of the file when changed on disk (with or without user notification)
2. Forced (timeout interval) reload of the file regardless of modified-flag (e.g. a log stream)

• Yes, the 2nd flavor/option would be similar to current "tail monitoring", but not named as such.
• For the 2nd flavor/option, it makes sens to keep the the file read-only.
• Both flavors will keep the caret at the given position (character count from the beginning, not text semantic related, means independent from changed text).
• The "Monitoring-Button" will toggle both flavors (but 2nd flavor is changed to 1st flavor on session end).

@ltGuillaume : Notepad3 uses the same mechanism as ProgerXP\Notepad2e (your referenced issue is still open and not implemented). The reason is: You can not block the main thread while WaitForSingleObject(), NP3 would stop reacting. So the interval polling (WaitForSingleObject() with a zero timeout) is the only solution for a single threaded system. You could wait on a separate thread by WaitForSingleObject() with infinite timeout, but this thread has to signal the main threat about occurring events somehow and doing this by interrupts is a bad idea, i.e. if the main thread is in a Scintilla method. So polling is a pragmatic solution, maybe the interval can be reduced to 100 ms (depending on the performance of your system).

So providing a polling interval input box on File Changed Notification maybe an option?

Did I miss something?

[ltguillaume]

@RaiKoHoff I think it's best to keep them separate: there's merit in keeping all File Change Notification (FCN) choices as is, but seeing the "Monitor File" (my preferred name for it) as a separate feature that happens to override the FCN preference. If you just add the current "Monitor File" as a choice within FCN, then you get a cycle option between four choices, because there should also be an option to completely disable FCN. A cycle option would defeat the purpose of quickly turning monitoring on and off. Plus, how would the current "Auto-reload (unmodified)" fit in? This feature also makes perfect sense.

As such, I'd say a separate toggle between "Monitor File" ON/OFF, like it is now but with caret position saving and forced auto-reload, would be best (and the least work to implement).

your referenced issue is still open and not implemented

It looks like that when you look at the issue, but I think the code and the fact that the following is already in its README.md suggest otherwise:

Replaced polling File Change Notification mechanism with a proper instant change listener, making the program suitable for watching log files (tail -f-style). https://github.com/ProgerXP/Notepad2e/issues/129

@kofifus I honestly think that by that way of reasoning, Notepad3 wouldn't be fit for anything. It's not best at stuff, its strength lies in the fact that it's a compact and fast little gem that helps you out just enough for a wide range of operations. If you adhere to the Linux one-tool-for-one-job philosphy, I think Notepad3 is not for you. Just my 2 cents.

[kofifus]

@RaiKoHoff fair enough, my main thinking here is that if np3 have this option people will use it for tailing without really being aware of the poor results they are getting. No other text editor I know does tailing (but maybe I'm not aware of some). Well that's my 2cents anyways ..

[RaiKoHoff]

@kofifus : Looking through the code of "File Change Notification", I found some inconsistencies and issues where the implementation is broken and does not follow the obvious intention.. need some time to check this. The "tailing" functionality has been implemented on user request (#456), if I remember correctly, cause of Notepad++ has a similar feature. Digging deeper, I found discussions similar to this issue and content (https://notepad-plus-plus.org/community/topic/10832/tailing-in-notepad) and (https://github.com/notepad-plus-plus/notepad-plus-plus/issues/4955). I don't like to discard this feature - I am going to rework it to have a similar functionality as Notepad++ (using above suggestions), but I have no clue how to avoid a comparison with real tailing tools like "tail -f", "glogg" or "TailSharp" :thinking:

[blackcrack]

by an other to taking my Words.. :

#1114 (comment) damnn.. notepad3 is grown adipose/fat/fatty / huge enough.. there dont need a tail funktion.. best Blacky it is now in.. and it stay in.. you, someone whant it, someone whant it not.. well, now it is in.. and should stay in as fiture (or how can named it) because, we should at the others not fall short ...

these other Wintail/tail for win and so on, be nice standalone programms, but why should we install 1000 other programms if we can make 2 things in one.. so.. bugfixing and hold this fiture in future .. To stay shoulder at shoulder with at.. example N++ and other multifunctional Programs..

anything else ? hpwamr

[kofifus]

@blackcrack

why should we install 1000 other programms if we can make 2 things in one

This comment is exactly why I vote to remove this feature - you assume that np3 has tail functionality and that the discussion here is about removing it just because we think it does not belong in np3. With this assumption you will use np3 to tail log files etc..

However the assumption is wrong, np3 does not have tail functionality. It has an auto-refresh feature meant for other purposes (which I'm not totally clear of) which can be abused as a very poor man's tail...

Imagine I would suggest adding the option to connect tractor attachments to a Ford Fiesta Hatch .. I assume you'll say it's a bad idea because it doesn't make sense and worse - someone somewhere will surely connect a fork and drive that lame thing around .. if you need a tractor get a tractor not a Fiesta ..

[hpwamr]

Hello @kofifus , Thanks for your interest and your contributions to Notepad3. 😃 I would like to add you in the "Acknowledgments" list of Notepad3. Do you agree ? 😉

[kofifus]

Sure, NP3 is a favorite SW, glad I can help ! Just be aware that I never contributed to the codebase.

[hpwamr]

Just be aware that I never contributed to the codebase.

We are also very happy to count you among the team of testers. 😃

[engelhro]

[NP3…] has an auto-refresh feature meant for other purposes (which I'm not totally clear of) which can be abused as a very poor man's tail...

Example use case (for me): I often have a file open both in a diff tool (e.g. old vs. new version) and in NP3 at the same time. I do some modifications in the diff tool (e.g. copying over certain changes from the one file to the other), and in parallel some modifications in NP3 (which are easier to achieve there than in the diff tool).

I love the ability of both programs to recognize changes which happened "outside", execute an automatic reload in this case and always see the most current state in both tools. Of course one needs to make sure that changes are always saved in one tool and get visible in the other one before performing changes there in order to avoid any conflicts.

So I use the automatic reload for what its name implies, but not a tail functionality.

[ProgerXP]

I might be late to the party but I've just noticed this.

Notepad3 uses the same mechanism as ProgerXP\Notepad2e (your referenced issue is still open and not implemented).

It's actually implemented (and works very well) in scope of https://github.com/ProgerXP/Notepad2e/issues/241, just not closed. We discovered that the detection mechanism inherited from Notepad2 is really unstable outside of the local system, and even within the local system if "funny" mount points are present. For example, it works in VMware but not VirtualBox, and even in VMware it works only if mounted as a network drive, not over a UNC path (\\comp\...) - although I might be messing up the specific error cases right now.

We also tried the approach of using a separate thread but it complicated the code due to synchronization and a bunch of new events, need to free all threads we've created, repeated stacked prompts when a watched file was deleted, etc. We don't have these changes in git anymore but I'm attaching the file if anyone is interested (changes from IsRunningWatcher() to FileWatcherThread() near the end).

Notepad2.c.txt

So in the end we decided to forego separate threads and only simplified WatchTimerProc(). Now it works on every file system providing last change time (which is virtually any, certainly more than those that work with ReadDirectoryChangesW - again, see https://github.com/ProgerXP/Notepad2e/issues/241) and no need for synchronization-related bugs (especially in the future when you forget that Notepad2 has now multiple threads!) and memory leaks.

I found this approach to be very workable for log files of a few MiB's in size - it doesn't eat the CPU during an active watching (and you can customize the watch delays in the INI) and reloading happens fast. Anything more complex like appending loaded data to the end of the buffer rather than reloading it, or filtering it (like you'd do with tail -f | grep foo) is way beyond the scope of Notepad2 and forks like @RaiKoHoff said - you just need to make the reloading actually reliable and then you will cover the majority of cases. Well, for me at least.