In #332, the reason given for not adding Miller columns is the redundant disk reads:

The fundamental issue with Miller View is too many redundant disk reads due to reading of subdir contents on opening or minor navigation in parent. On a remote mount this subdir content load will also slow you down.

Originally posted by @jarun in https://github.com/jarun/nnn/issues/332#issuecomment-534528993

This is a very valid reason. Having 3 columns used at all times is wasteful.

However, I believe a Miller-like mode can be done just with the existing information, without extra disk reads or CPU cycles waste, simply by making the numbers of columns not fixed to 3 at all times (which starts by wasting time at startup on acquiring information on 2 levels above) but dynamic (which further avoids having to refresh minor view when navigating in parent)

It would simply mean to erase stale information and not show anything until the navigation is confirmed by entering into a folder, like by pressing on the left or right key - which would keep the performance constant, while enriching the navigation with context about the content of the folder previously visited in the filesystem hierarchy.

Here is a description of how adding an option for dynamic Miller columns would work.

For the "going into subfolders" situation:

nnn would start exactly as it does now, with the exact same look and feel
but upon opening a child folder, the screen is not redrawn the same: just keep the information of the parent folder on the left hand side (where it already is!), while supplementing it on a middle column now displayed (to the right of the parent folder) with the information of the child folder, which now becomes navigable: 2 columns are now displayed, the middle column is navigable
if another child folder is opened, the same applies, except this time it gets displayed in the right column (.), with the immediate parent in the middle (..), and the parent own parent in the left column (../..) : 3 columns are now displayed, the right column is navigable
when going further in, the 2 rightmost columns are moved to the left: 3 columns are still displayed, the right column is navigable
when leaving the child folder, the child content is erased (the right column is cleared), its parent becomes navigable again, but no other folder is read: 2 columns are now displayed, the center column is navigable
if going back again, likewise, the center column is erased, its parent become navigable: only 1 column is now shown

For "going into parent folders" situations:

likewise, start as is
when pressing on the left key, redraw the current folder content in a middle column, while drawing the parent content folder now in the left column: 2 columns are now displayed, the left column is navigable
when pressing again on the left, same thing: 3 columns are now displayed, the left column is navigable
when starting navigation on the left column: the stale content of the middle and right column is erased. Only one column is shown and navigable. This allows to goes back to the "going into subfolders" situation, without visual distraction.

It could be argued that knowing which column is navigable can be hard, as when showing >1 column it is not always the same column. However, the column being navigable can be identified by the presence file selector, which also moves when pressing on up/down, while other non navigable columns would have no such visual identifier.

If keeping multiple lists of the files available in the displayed folders when having >1 column is a concern for the memory used, the previous lists can be discarded, keeping instead only the information displayed on the screen: this means that when coming back to a previously visited folder, instead of having the information cached and immediately available, it will simply have to be read from disk - which is preferable anyway, as new file may have been created, and which would be identical performance-wise to the current situation.

I don't think it's reasonable to expect every file manager to become ranger-like because you prefer Miller Columns. You can come up with many visual benefits of Miller Columns but thousands of users find nnn useful as it is.

If you really like Miller Columns so much, I would suggest you stick to ranger. It's also a very good file manager. If you find something useful in nnn and not available in ranger, probably add the functionality in ranger and make it more useful. For example, you can add your suggested logic for lesser memory usage in ranger and it would be a good improvement.

Alternatively, you can also choose lf or hunter which are very good file managers too.

From nnn perspective - we want to keep it simple. Both the light and detail mode work fine on smaller form-factors like mobiles and that's where we stop spending our free time at.

I don't think it's reasonable to expect every file manager to become ranger-like because you prefer Miller Columns

And I don't think it's reasonable to avoid adding a feature that would not increase either the CPU usage or the memory footage.

If you really like Miller Columns so much, I would suggest you stick to ranger.

Both ranger and lf have several bad design choices. nnn is the first file manager I find decent enough to work with.

If you find something useful in nnn and not available in ranger, probably add the functionality in ranger and make it more useful

Actually, I think I'll just fork nnn and add the functionalities and features I want!

I'll be starting with dynamic Miller columns first, because that the # 1 missing feature, and as explained above it is conceptually simple with very little overhead. nnn is fast and will be a much better starting basis than ranger.

The second feature will be a shell, as forking with ! is far too slow on Windows (a known limitation with all cygwin), so a quake-style terminal, or another form of persisting terminal immediately accessible and always running in the background will be a great addtion.

Thanks anyway!

Actually, I think I'll just fork nnn and add the functionalities and features I want!

That's the best solution if you can spare the time.

I refrain from requesting people to contribute because most people don't want to get their hands dirty in C. Despite a dev cycle of 3+ years, we only have 4 steady contributors. If you want to, feel free to raise a PR anytime. In any case, if you need any help with the existing implementation, do reach out.

most people don't want to get their hands dirty in C

Unfortunately, most people are lazy and entitled.

On the other hand, some maintainers also have a very strict and unmovable idea of what their software should and should not do: they will not add features, even when it makes no sense to refuse a feature, like when they are provided a premade patch that does not cause any regression, loss of functionality, wasted RAM or CPU cycles.

I do care a lot about both usability (and nnn nailed it, with various features such as type-to-nav) and what some people may consider more "obscure" features, such as sixels. I consider sixels a far better option than say kitty graphics, if only because of their wide availability and compatibility even in a default xterm, over ssh, etc.

In the past, I've had to do forks to work around hostile maintainers - one such example being tmux.

The maintainer explicitly said several times that even if a patch was provided to support sixels, the functionality would never be added to tmux, and it would have to be a fork https://github.com/tmux/tmux/issues/44#issuecomment-119755304 -- also, tmux intercept and improperly rewrites various escapes codes, breaking other things https://github.com/tmux/tmux/issues/1391#issuecomment-403267557

However, a fork is often an admission of failure, as the functionality will not be made available to the greater number of users who would benefit from it.

I thought it was going to be such a case again, as the reply made little sense considering it would be an optional feature that would cause a gain in functionality without drawbacks.

Now with the context you have provided, I'm sorry if I misjudged the situation.

Despite a dev cycle of 3+ years, we only have 4 steady contributors.

I would be happy to help. I provided a fix to compile under msys2 and a detailed analysis of why it failed because I want to help!

As there's no nnn package in pacman, feel free to start by taking the msys2 binaries for nnn and nnn "nerd mode" from: https://github.com/csdvrx/nnn/blob/master/nnn.exe

I will later contribute a patched iosevka font, for people who want to immediately start playing with icons.

The license is BSD, so it's perfectly compatible with nnn. I believe it may be better than giving instructions and scripts to patch a font: I stick to the belief that most people are lazy and entitled, so they are more likely to use the font that's provided to them!

If you want to, feel free to raise a PR anytime.

Just to clarify: if you are provided with a patch adding Miller columns, is there a chance the feature will be included in nnn -- or do you just hate the feature and will never include it because you do not want to see Miller columns?

I'm asking, because I can see different ways to do that based on the probability of inclusion: either a "lightweight dynamic" Miller column mode, harder to implement, or a "typical 3 columns" Miller column mode, faster to code (just always scan and draw 3 columns, no need for cache or context)

I am also considering adding other features later, however, they depend on Miller columns, the #1 thing I want to add, and also the one with the least impact on nnn.

So this question also applies to other features, which may have more impact. It's mostly to work around the slow fork in Windows: for example, if later I add a preview column, calling plugins will make it impossible to have a quick/responsive navigation in Windows, as starting a plugin often takes longer than what the plugin does.

A good example would be rendering images in the console: having the C functionality inside nnn (whether in unicode blocks or sixels) would make the difference between an "instant" rendering when pressing the down arrow, and a slow unresponsive navigation wasting many seconds on each directory.

It is a well known and documented issue, cf https://github.com/microsoft/WinDev/issues/15 especially if you don't want a full WSL but a more minimalistic solution like cygwin or msys2 https://stackoverflow.com/questions/985281/what-is-the-closest-thing-windows-has-to-fork/985525#985525 ; it's due to many things but essentially boiling down to fork() ; this mostly affects compilation time https://esp32.com/viewtopic.php?t=4166 but here it would also affect nnn especially on "quick browsing" like pressing the Down arrow several times.

It's not that I want to clutter the nnn codebase, it's just that there is no other solution but to have the preview functionality provided within nnn, as otherwise it will be practically unusable.

Could you consider an optional compile-time dependency:

for unicode rendering, on the libraries required to process image formats that are often available (libjpeg, libpng)
for sixel rendering, on the above, and libsixel too (that would save me time and effort, also it's BSD licensed)

In any case, if you need any help with the existing implementation, do reach out.

Thanks a lot! My first analysis points to printent() as the place to modify, by adding a column counter.

There would also be global variables, such as a cache of the directories previously read when navigating up or down the same tree (left/right key action) that would be discarded upon jump (going back to 1 column display), along with a left and right key history (to know which columns should be drawn and given the focus.

As there's no nnn package in pacman, feel free to start by taking the msys2 binaries for nnn and nnn "nerd mode" from

If it goes fine, see if you can add the package nnn to msys2 official repos.

is there a chance the feature will be included in nnn -- or do you just hate the feature and will never include it because you do not want to see Miller columns?

There would also be global variables, such as a cache of the directories previously read when navigating up or down the same tree

I still have my reservations because I am sure this is going to shoot up memory usage and disk reads. And I can't guess by how much right now. As you are going to implement it anyway, I would suggest you add a program option and keep the implementation separate from printent().

other features

We need to discuss case by case. Rendering images natively is not our top priority. A lot of work has gone into plugins already to keep the core program minimal.

@leovilok @KlzXS @0xACE what's your opinion on Miller view?

This really sound like you want a total overhaul of nnn. It would certainly add a lot of complexity to the already somewhat complex code.

I think that it's a must for the new code to be compile-in. Included in a completely separate file(s). Organized like a user patch to nnn. There's been some talk about such patches previously.

The details of said organization of code need to be worked out. Also issues with your patches go to you. I'm sure we'll help out, but I'm also sure they wouldn't be our top priority.

TL:DR;

Sort of regret typing all this but maybe i made some sense?

Hack away at your feature, I've been carrying mine myself for a while. Maybe you are lucky: some of the features i carried in my personal branch eventually got adopted upstream as other contributors simply submitted pull requests out of the thin air rather than trying discussing it first...

ramble(); // I'm sorry for this nonsense

The maintainer explicitly said several times that even if a patch was provided to support sixels, the functionality would never be added to tmux, and it would have to be a fork

FYI, I think there exists a fork of tmux that supports sixel...

However, a fork is often an admission of failure, as the functionality will not be made available to the greater number of users who would benefit from it.

My personal branch differs from master, it's not more than my own "configuration", I don't consider it a failure...

I thought it was going to be such a case again, as the reply made little sense considering it would be an optional feature that would cause a gain in functionality without drawbacks.

I don't think he is directly opposing your feature request... e.g. In my case it's more that I'm "afraid" of the whip (the labor), I'm roughly 200 commits behind master and I don't even have the energy to merge... The program runs fine on my devices as is...

I am also considering adding other features later, however, they depend on Miller columns, the #1 thing I want to add, and also the one with the least impact on nnn.

My personal branch of nnn has been different from master since day 1. After a couple of months of maintaining my extra features I got tired of keeping up with merge conflicts with master so I dropped most of the complex ones and I'm keeping it relatively simple today. Therefore my advice/tip is beware of merge conflicts, nnn code base in need of refactoring... Though lately it seems like nnn has settled on it's features so it may not be so bad. But i still fear what will happen with:

3d71uj

but in a good way :) ...

Thanks a lot! My first analysis points to printent() as the place to modify, by adding a column counter.

Iirc, printent() had 3 annoying merge conflicts for me the past year, but this time it seems like they settled down (I still haven't merged with master). Anyhow if custom columns are added to printent_long() or w/e it's called I'd suggest you add your own seperate printent() function and another function to handle the up/down movement (there's some function that only updates 2 affected lines unless you are scrolling)... Oh and the current mouse support is very basic and assuming everything is seperated by 1 line and that they cover the entire width... I don't even know if that's enough to implement your Miller columns... Geez, may the spaghetti monster be on your side...

If it goes fine, see if you can add the package nnn to msys2 official repos.

@jarun will do

I still have my reservations because I am sure this is going to shoot up memory usage and disk reads.

But how?

And I can't guess by how much right now.

Let's look at all the possible cases then, for example, starting inside the folder A of A/B/C/D, and navigating down the tree, then in the opposite direction (as starting from D and navigating left is the same problem in reverse)

If you keep going in the same direction (ex: right,right,right,right...) it will only cache at most 2 levels left of the current position (and same in the opposite or when changing direction). As soon as you go "beyond" the maximum amount of columns (3) than must be available to be drawn on screen, extra entries in the cache are flushed. As soon as you switch away from the linear path, the extra entries are pruned from the cache. As soon as you jump to a different directory, the full cache is emptied. So I really can't foresee anything bad.

But more precisely, let's look at that step by step starting inside A :

start by listing the content of A (then save it in cache) in one left column: same io and memory usage as before
press right: keep displaying the content of A on the left (no disk reads, use the cache) , then in the center column, read then display the content of B (then save it in cache): A B are displayed in 2 columns, no more disk read than usual, just 1 extra level in cache
press right again: keep displaying the content of A on the left (no disk reads, use the cache), keep displaying the content of B in the center (no disk reads, use the cache), read then display the content of C: A B C are displayed in 3 columns, no more disk read than usual, just 2 extra levels in cache
press right again: prune A from cache, display B on the left and C on the center using the cache, read then display D on the right (then save it in cache): B C D are displayed in 3 columns, no more disk read that usual, just 2 extra level in cache
press left in a sudden change in direction: drop D from the cache (could be kept, but let's simplify the algorithm). display B on the left and C in the center using the cache, (C could be read again in case new files have been created) : B C are displayed in 2 columns, maybe less disk read that usual, in the worst case (C read again) no more disk read than usual, just 1 extra levels in cache
press left again: drop C from cache, display B on the left (either read it again or use the cache): B is displayed in 1 column, maybe less disk read that usual, in the worst case (B read again) no more disk read than usual, no extra levels in cache

press left again: read A and display it on the left (then save it in cache), display B in the center: A B are displayed in 2 columns, no more disk read than usual, just 1 extra level in cache

Now let's assume at each level there are sister folders: (ex A1,A2.. at the first level, B1, B2 at the second level)

If instead of the last step (left again from B to A), from B you go to B1, B is dropped from cache (which only keeps linear path), A is displayed on the left using cache, B1 is read: again, no more disk read than usual, just 1 extra level in cache.

I keep thinking but I just can't see how it can underperform: all I can see is at most 2 extra levels in cache, just as many disk read as now if a revisited folder is read again, and fewer reads if the cache is used (in theory a bad idea as new files could have been created inbetween, it could still be doneby using inotify to subscribe to file creations and invalidate the cache... but it may be a pain)

Maybe I haven't gone deep enough into nnn, but really, I can't see it. Can you please give a case where it would underperform compared to now?

As you are going to implement it anyway, I would suggest you add a program option and keep the implementation separate from printent().

Can do - an alternate definition of printent #ifdef'ed away when not being requested at compile time.

This really sound like you want a total overhaul of nnn. It would certainly add a lot of complexity to the already somewhat complex code.

@KlzXS Uh, no. I'm trying to keep the changes to a minimum. I'm even thinking about a "unidirectional" miller column (ex: only when going say max 2 levels to the right) by not doing a screen redraw and only writing over the empty space until it just can't fit in. It would be dirty but do the job.

I think that it's a must for the new code to be compile-in. Included in a completely separate file(s).

Can do - as a separate .c, not linked or even compiled unless the right #defines are made or -D flags are passed to make

The details of said organization of code need to be worked out.

Hence this discussion

Also issues with your patches go to you. I'm sure we'll help out, but I'm also sure they wouldn't be our top priority.

Of course! If I add something, I take care of it!

FYI, I think there exists a fork of tmux that supports sixel...

@0xACE Indeed. I made the fork :)

Joke aside, it wasn't the first time - see also https://github.com/yatli/tmux and https://github.com/ChrisSteinbach/tmux-sixel that were made before. And actually, that's the problem: the wheel was reinvented.

My personal branch differs from master, it's not more than my own "configuration", I don't consider it a failure...

For me, it was, because all 3 forks are now bit rotting. Guess why I didn't use an existing fork? backporting new mainline features was too much work, so it was easier to fork the mainline and add the sixel support.

I'm sure someone needing sixel now would prefer to do a 4th fork, as it is hard to keep current with any main line, especially when you don't need to, like when you have the features you need for yourself.

After a couple of months of maintaining my extra features I got tired of keeping up with merge conflicts with master so I dropped most of the complex ones and I'm keeping it relatively simple today

Exactly the point!!! This is why I prefer to talk first then code later, to avoid having to redo the same thing again in the future

Anyhow if custom columns are added to printent_long() or w/e it's called I'd suggest you add your own seperate printent() function and another function to handle the up/down movement

There's not enough space on screen to have Miller columns in the -d detailed mode.

For now, it would only be shown in non-detailed mode.

(there's some function that only updates 2 affected lines unless you are scrolling)...

Yes, to highlight the current entry I guess

Oh and the current mouse support is very basic and assuming everything is seperated by 1 line and that they cover the entire width

That would have to stop at the end of the column

Geez, may the spaghetti monster be on your side...

Indeed, it may be slightly spaghetti-code depending on the constraints for how to do it - but all options are on the table.

As changing printent is apparently too much, it will be an #ifdef, in its own C file

I don't even know if that's enough to implement your Miller columns...

What do you think of the logic proposed above?

I don't see anything funky: it just caches the content of the folder previously visited, and prunes it when it get stale (ie more than 2 levels away from the current position)

The only difficulties are decisions related to the display, like what else should be cached if say filtering A then going into B: should the left column show the filtered A, or the "full" content of A?

By default I would stick to whatever was shown, to minimize visual changes, and restore the view "as is" when returning to A - this means caching not just the content of A, but also some states.

I emulated a kind of Miller view with a second nnn instance in the preview-tabbed plugin, so I think it can be usefull.

I'm curious about how it would be implemented in nnn's core, but it can't use more memory than 2 nnn processes!

@csdvrx quoting you

There would also be global variables, such as a cache of the directories previously read when navigating up or down the same tree

If you mean caching dir as well as the contents in them i think it will use more memory. Anyway, when your fork is ready, let us know for testing.

I'm sure someone needing sixel now would prefer to do a 4th fork, as it is hard to keep current with any main line, especially when you don't need to, like when you have the features you need for yourself.

Yeah, you have got a point... Also, thinking about it now, I don't use neovim yet it still affects me... I guess you could call your fork neonnn or nnnn for short :P

Exactly the point!!! This is why I prefer to talk first then code later, to avoid having to redo the same thing again in the future

Yeah, that's what I did: I discussed it first and I got denied, so I maintained it locally for months. Then another person just pushed a PR for the exact same request without discussing it first and it got approved. In my experience jarun isn't happy with increased memory usage nor complications, I understand it's vague, but miller columns has been requested previously, so maybe it's good after all? If it makes browsing faster for me, I can see myself using it as half the screen isn't used anyhow...

There's not enough space on screen to have Miller columns in the -d detailed mode.

For now, it would only be shown in non-detailed mode.

Oh sorry, I really shouldn't answer issues when I'm exhausted... What I meant was that if detailed mode gets modular columns, it would be wise to not tangle with printent() (now as I'm typing it, modular columns shouldn't affect it, I guess I was premature with that idea)... Anyhow the problem becomes irrelevant if your just provide your own printent_miller()

Yes, to highlight the current entry I guess

Correct, it can be found somewhere along the SEL_(UP|DOWN) (up/down movement)

That would have to stop at the end of the column

Yeah, I guess mousehandling should be moved to it's own function so you can have it replaced dynamically.

I might supply a patch because nnn mousehandling on my pc is similiar to master but on my phone it's modded to work like Apples Ipods (scroll item with finger, click on right side to go in, click on left side to go out. It could be a swiping action but iirc we cheated with the MOUSE_CLICKED, so I kept it simple instead) I find it useful when using nnn while not standing still as I can accurately hit the correct file.

Indeed, it may be slightly spaghetti-code depending on the constraints for how to do it - but all options are on the table.

I don't know what I thought would be the spaghetti for you other than the mousehandling, so it may not be too bad after all...

As changing printent is apparently too much, it will be an #ifdef, in its own C file

It makes more sense to provide your own printent_miller() because the program is easily capable of changing the printent_ptr to point to your function when needed rather than handling merge conflicts...

What do you think of the logic proposed above?

I think it looks fine...

The only difficulties are decisions related to the display, like what else should be cached if say filtering A then going into B: should the left column show the filtered A, or the "full" content of A?

By default I would stick to whatever was shown, to minimize visual changes, and restore the view "as is" when returning to A - this means caching not just the content of A, but also some states.

I don't know how miller columns works, but I think it makes sense that it's in the state that the user left it, but I would also find it confusing if im browsing upwards in the tree structure and then suddenly im in search mode or can't see the entire folder...

Also note that resizing may also be a point where the screen can be updated...

Spent too much time on this and I got to go, but go ahead and tinker with it, maybe you will be happy with the results :)

If you mean caching dir as well as the contents in them i think it will use more memory.

@jarun : yes, it will use more, but it's capped by the number of levels kept in cache

I emulated a kind of Miller view with a second nnn instance in the preview-tabbed plugin, so I think it can be usefull.

@leovilok : indeed, it may sound stupid, but to me the biggest advantage of Miller columns is to get a better mental picture of the folder free!

I understand it's vague, but miller columns has been requested previously, so maybe it's good after all? If it makes browsing faster for me, I can see myself using it as half the screen isn't used anyhow...

@0xACE : Yes, it's also going to put the wasted space to a practical use. So overall, I see that as an innocent feature with very few drawbacks IF it's done properly.

As for making nnn faster, actually, that's possible too: the cache could be implemented separately from the Miller column mode, so it can be used by normal nnn browsing: if you are revisiting a directory traversed linearly before, no need to do disk IO - just use the cache!

Which makes me think: there IS a good case for a directory content cache, even without talking about Miller columns. I understand that small devices like raspberries are one of your important usecases, as memory is cheaper than disk IO, especially on slow devices or remote mounts.

Also note that resizing may also be a point where the screen can be updated...

Yes, I've toyed with such screen update issues on a related project (a timestamped bash prompt, to be released soon)

Spent too much time on this and I got to go, but go ahead and tinker with it, maybe you will be happy with the results :)

You're right, there's only so much that can be discussed: some details can only be seen when playing with a working implementation!

@jarun : as the directory content caching feature is what will consume memory, but also has potential to speed up nnn when used without Miller column, especially on devices with slow disk IO, what do you think about it being a standalone feature? If you like it, may I ask you to do it your own way? (ex: you may want ionotify to invalidate the cache... or not, you may want a common global cache shared between contexts... or not, and other design decisions you may take better than me as you are more familiar with the codebase and the constraints)

As long as the cache exists in one way or another, now that the idea and the details of the implementation are defined (printent_ptr, separe C file etc) seem generally ok, I can figure a way to do Miller columns.

And since the cache is what will bring the most changes in nnn core, it may as well be done in the most polished way, by taking into account nnn details I can't yet understand

jarun / nnn

Support for minimalistic dynamic Miller columns #794

TL:DR;

ramble(); // I'm sorry for this nonsense