Alexander-Miller commented 6 years ago

Hello everyone,

the next big thing I am teaching treemacs to do is the ability to display multiple file trees at the same time. This feature may sound fairly simple, but will require serious amounts of changes and redesign, such that the next iteration of treemacs is an ideal point to introduce further breaking changes and break old habits.

So this thread serves the purpose of getting community feedback not just for my current multiroot design, but also for treemacs as a whole. If there is some default behaviour or feature or keybind that you want to see changed, now is your chance.

Let's also tag some people who have previously provided valuable feedback before and who might be interested in this: @seagle0128 @duianto @rswgnu @ambihelical @piotrpalek @sdwolfz @timofreiberg @bitterblue

Multiroot design

What I have so far

Current design looks like this. Code's on the multiroot branch. The icons and faces are, of course negotiable.
The fileroots will be called "projects". Each project has a filepath, but also a name, which will be displayed in treemacs, so you can call a project whatever you like.
Treemacs will ask for a name when adding the project, but the name can be changed any time.
Projects are displayed in the order they are entered, but I'll probably add the option to change their order around.
Project nodes behave much like directories, so expanded/collapsed on TAB & RET.
There are dedicated keys to move to the next/previous project node.
The minor modes should eventually (as of posting this they don't) work as expected with no changes for users.

Local Keys

My current prefix choice is C-p. Currently the keymap looks like this:

Key	Description
C-p a	add project
C-p d	delete project
C-p r	rename project
C-j/n	goto next project
C-k/p	goto prev project

Global Keys

SPC f T and SPC f P need to be changed to run something like "add current dir/project" to the workspace. Otherwise the global keymap looks as follows:

Function	Keybind
treemacs	SPC f T
treemacs-toggle	SPC f t
treemacs-projectile	SPC f P
treemacs-projectile-toggle	SPC f p
treemacs-find-file	SPC f C-t
treemacs-find-tag	-
treemacs-bookmark	-
treemacs-select-window	M-0 + SPC0
treemacs-delete-other-windows	-

Session Persistence

Let us call the projects you've added to treemacs your workspace. I expect people will naturally want their workspaces to be persistent. I think that my current approach of integrating with desktop-save-mode is no longer the best fit, since 1) desktop-mode can be lazy. If you want your workspace to show up immediately desktop-mode needs to be bypassed. 2) This in turn is an issue for treemacs being frame-local. Since frames might not be restored before treemacs is initialized.

So there is some serious dissonance between treemacs' workspaces, its frame-locality and desktop-save-mode.

My idea to untangle this is as follows: desktop-save-mode integration is removed, it's pretty hacky as it is anyway. What treemacs will always be persisting instead is your workspace. Treemacs will remain frame-local, but your workspace will be global. Each frame will have its own treemacs buffer offering a unique view on the workspace, but adding/removing/renaming projects will be a global action.

Persisting, using and switcing between multiple workspaces is something I'm saving for later.

Issues

Project path independence

There is one limitation I need to enforce which may or may not annoy you, depending on your workflow. Project paths must be fully independent of each other, it will not be possible to add a project which is a subdirectory of another project. Doing so will break a very important invariant that treemacs relies on heavily: each node has a unique natural key in its absolute path. If I do away with that I wreck my backing data structures and turn (tag-)follow-mode and filewatch-mode into undefined behaviour.

No more root changing

Arbitrarily changing the root doesn't really fit into the new concept of mostly static projects file trees, especially with the above-mentioned restriction, so I've removed that option.

Future Options

And finally a list of things I plan to get around to after multiroot are done, which I'll prioritize based on feedback.

Dynamic Module

I'll rewrite treemacs in rust! At the very least I'll try and offer a native option for the current python-based tasks (git & directory-collapsing). If that goes well I'll write a native implementation of as much of the hot path (basically what happens when you press TAB) as possible.

Improved performance

Native modules aside I am already pretty much at the limit of what can be done, as far as just the rendering is concerned. One avenue still open to me is make filewatch-mode smarter with how it runs its updates. Its current approach is to refresh directories with changes (basically closing and reopening them). Depening on the amount of work it's probably much cheaper to make changes to single lines.

Mouse Faces

The nodes in treemacs currently all use the same generic mouse-over face (I think it's region). I want to try and give them mouse faces which are correct w.r.t. both your current theme and the nodes' actual faces, including their git state.

Better tag options - tag hook

If treemacs gets its tags for a file that is not open in emacs it will use the default imenu implementation since mode-hooks are disabled to prevent bugs and improve performance. There should be an option to define some hook treemacs should use in this case when the default won't do.

Indent Guide

I want to optionally give each indent step in treemacs a consecutively darker background to make indentation better visible.

MaskRay commented 6 years ago

Not directly related.. but yesterday I borrowed some ideas from your amazing treemacs cquery-tree.el Which was my attempt to help https://github.com/emacs-lsp/lsp-ui/issues/73#issuecomment-368392665 If your library can be used on objects other than filesystem, it'd be great...

The multiroots idea sounds like it aligns with LSP 3.6's new method workspace/workspaceFolders, which has not been implemented in lsp-mode yet

Alexander-Miller commented 6 years ago

If your library can be used on objects other than filesystem, it'd be great...

Huh, I actually thought about adding my "treelib" idea to the future option list, but figured there'd be no demand. Treemacs can certainly be used on nested objects other than the file system, at least its building blocks can, with only minor adjustments. In fact I built a basic showcase that'll display your buffers, segregated based on major mode. The example is run with treelib-showcase, TAB is used to open/close nodes.

That code you see is mostly the result of copy-pasting my current treemacs macros into a new file, so there is still plenty of room for taking out and automating many manual parts.

If the lsp project can use a tree-builder library (I assume the protocol has features like call/inheritance hierarchies) I'd be happy to get this rolling. There are certainly other libraries out there, so what a hypothetical treelib what offer is 1) all the navigation you already know from treemacs, 2) speed - I am very conscious about performance and try to render as fast as possible, and 3) I could probably also extract my introspective faculties.

Alexander-Miller commented 6 years ago

The issue has disappeared page 2, so I'd like to give a status update, lest it looks like nothing came of all those paragraphs I spent on my grand design.

As of now I am more than 40 commits in, and making good progress. A basic workspace, its automatic persistence, adding, removing and renaming projects, refreshing, follow-mode, tag-follow-mode and their manual invocations, as well as the file management basics are more or less done, though of course mostly untested.

There are 3 major blocks still left: filewatch-mode, the test suite and documentation. There are also some minor issues, like how to treat treemacs-bookmark. And that:

❯❯❯ rg TODO | wc -l
24

I am hoping to polish things enough to have a version worth testing by others sometime next week.

Participation in the ticket has so far been less than I had hoped for. I prefer to be optimistic and assume it's because I'm doing something right, nonetheless if someone could help me fill up that global keymap that'd be great.

piotrpalek commented 6 years ago

@Alexander-Miller I can't talk for everybody but I mostly was pretty busy and what I've read at the time sounded good to me (and I didn't have anything worthwhile to add) so I kept silent :)

One thing that bother me right now are the semantics around SPC f t and SPC f p, for eg:

I (mistakely) open treemacs via SPC f t (I wanted the project view)
I do use it though because the file that interests me is in the current directory
I close treemacs (whichever way, ft or fp)
I open treemacs using SPC f p but it doesn't open projectile view. I specifically have to invoke SPC f P or kill the buffer and do SPC f p

In general though I don't see the value in having bindings for treemacs and treemacs-projectile and the separate treemacs-projectile-* mode as well. Imo SPC f t (or SPC f p) should do the right thing depending on context. If I'm in a project -> open the project root and highlight my current file. If I'm not then open the current directory and highlight my file. SPC 0 already activates and opens treemacs so all usecases (at least all I'm using) are covered.

I'm pretty sure I'm missing something, because I remember SPC f T was brought back not so long ago. So there must be a reason :)

Alexander-Miller commented 6 years ago

Now that's the kind of answer I was looking for.

First of all it's important to move away from the idea of having a single file tree whose root you arbitrarily change. The new treemacs is all about multiple static and persistent project trees, like the project explorer in eclipse. So instead of changing your root it's about adding and removing projects from your workspace.

At any rate you're right on ft/T and fp/P, they are are indeed too similar, especially with M-0 thrown into the mix. Right now it's even worse with a workspace, since with multiple file trees you won't be changing roots, wich is what T & P are all about. Let's try to come up with something more consistent and intuitive:

no mode fT
no more fP
ft will toggle visibility if a treemacs buffers exists for the current frame
If no treemacs buffer exists it will create it and load the projects in your workspace.
If the workspace is empty (mostly if you start treemacs for the first time) it will ask you for the first project root
fp will be like fP with a prefix arg - select a projectile project to add it to the workspace. The current project is preselected if possible
What's missing is a global keybind to add an arbitrary path as a new project, but that's already covererd by either ft or a binding inside treemacs (C-p a from the table above).

What do you think?

axelson commented 6 years ago

Throwing my 2 cents in real quick. I'd prefer to keep a binding of fT but I don't think we need to keep the current behavior of changing the root (which as you point out doesn't match with these changes). Instead as a user I would expect fT to open treemacs (if it isn't already open) and select/show the current file in treemacs (regardless of the treemacs settings that toggles if the current file is always selected in treemacs).

Alexander-Miller commented 6 years ago

Sounds like you're describing treemacs-find-file (there is also find-tag). It'll find and focus the current file among the projects in your workspace, or log that the current file is not part of any project (including visual feedback via pulse.el).

It's already around on f C-t, so I don't think putting it on fT is much of an advantage - you're pressing a modifier in both cases.

Alexander-Miller commented 6 years ago

I am now at a point where, except for filewatch-mode, the new code is ready to be properly tested. If anyone wants to lend me a hand just grab the multiroot branch. To run a local installation just clone the package and pass its path to :load-path in use-package.

piotrpalek commented 6 years ago

@Alexander-Miller that sounds really great to me. There's only one point I'd like to possibly see streamlined:

If the workspace is empty (mostly if you start treemacs for the first time) it will ask you for the first project root

If I've already opened a projectile wouldn't it make sense to add the root of the current project automatically and open treemacs? Or maybe if that's suboptimal for some reason, at least add it as a config option.

Regarding @axelson's comment: wouldn't this use case be covered by SPC 0? And if the user has follow-mode or tag-follow-mode disabled then there are the corresponding methods which can select the intended node in treemacs (as Alex noted).

btw I'll try to test your branch this week :+1:

Alexander-Miller commented 6 years ago

If I've already opened a projectile wouldn't it make sense to add the root of the current project automatically and open treemacs?

I can try integrating something like this in treemacs-projectile.

Filewatch-mode should work too now, integrating it turned out to be much easier than expected. I did however discover an issue in filewatch-mode that causes git-mode to be inaccurate. Filewatch-mode tries to be smart and do as little work as needed - for example if you change /foo/bar/baz/file.el the update code will basically just moves point to /foo/bar/baz and hits TAB twice. Directories to be updated are marked in O(1) and updated in O(log n), a clean and efficient process all around.

Except that with extended git-mode /foo and /foo/bar and /foo/bar/baz now need to be fontified as (un)modified as well. Now that really puts a spoke in my wheel ... Long story short I'll need to launch another python process to take care of just that.

Here's what I have on the matter. If anyone knows how to do it better, either the part of python part, I am all ears.

IS_UNTRACKED_CMD = 'git ls-files --error-unmatch '
IS_CLEAN_CMD     = 'git diff --quiet '
IS_IGNORED_CMD   = 'git check-ignore --quiet '

def get_status(path):
    is_ignored_proc = Popen(IS_IGNORED_CMD + path, shell=True, bufsize=100)
    is_ignored_proc.communicate()
    if is_ignored_proc.returncode == 0:
        return "I"

    is_untracked_proc = Popen(IS_UNTRACKED_CMD + path, shell=True, bufsize=100)
    is_untracked_proc.communicate()
    if is_untracked_proc.returncode == 1:
        return "U"

    is_clean_proc = Popen(IS_CLEAN_CMD + path, shell=True, bufsize=100)
    is_clean_proc.communicate()
    if is_clean_proc.returncode == 0:
        return "C"

    return "D"

for path in sys.argv[1:]:
    state = get_status(path)
    print("State of {} is {}".format(path, state))

bitterblue commented 6 years ago

Sorry, @Alexander-Miller, I've been too busy to follow along, and this is some serious commitment you are showing here. Really looking forward to this next iteration of treemacs and how it will work out for Clojure development!

Alexander-Miller commented 6 years ago

and this is some serious commitment you are showing here

Yeah, well, some people got me into that software craftsmanship mindset and I'm pretty sure you know the rest.

how it will work out for Clojure development!

I don't think I'll be doing anything clojure-specific. That kind of specialization is not webscale.

piotrpalek commented 6 years ago

@Alexander-Miller I wanted to checkout your branch and did the following:

cloned the repo to ~/.emacs.d/private/treemacs and changed the branch to multiroot
added (push "~/.emacs.d/private/treemacs/" load-path) to the dotspacemacs/init function in init.el

It doesn't seem to be loaded (for eg. I can still change roots), pretty sure I'm doing something wrong here :sweat_smile:

Alexander-Miller commented 6 years ago

Putting it on the load path just means that require can find it, but you still need to (require 'treemacs) yourself. If you've a previous installation you'll need to remove that as well.

Alexander-Miller commented 6 years ago

Small update, since it's been quiet for a while. I've simply taken a small break and development is back on track now. The git stuff from above will be done later (otherwise I'll never finish this), so I'll be moving on to tests and docs soon enough.

@piotrpalek Any news on your end, did you get it working?

Alexander-Miller commented 6 years ago

Almost done now. Should only be minor cleanup left. I've rewritten the readme and added section on projects and contributing as requested on gitter.

@piotrpalek First starting treemacs with an empty workspace should now preselect the current projectile project if treemacs-projectile is loaded (I just haven't tried this yet).

Other than that treemacs-projectile now only contains a function to add a projectile project to the treemacs workspace (bound to C-p p), making it awfully empty in my eyes.So if anyone has an idea for another way treemacs can integrate with projectile I'm all ears.

myrgy commented 6 years ago

I would like to remain one feature - keep only one open project and sweetch between it on project switch.

Alexander-Miller commented 6 years ago

Well now that's the complete opposite of what I've been doing all this time. I don't even know what exactly you mean by project switch, but the way things are now it's probably only possible by butchering half my code base.

You know that one scene in the new DOOM where the robot dude tells you that "this is the cost of progress" as you stand there surrounded by corpses? It's kind of like that. The new design is not without its limitations, however jumping between projects is perfectly achievable by adding the relevant projects to the workspace and treemacs-follow-mode.

myrgy commented 6 years ago

@Alexander-Miller , is it possible to expand active project and collapse all other during project switch?

Alexander-Miller commented 6 years ago

Yes that's doable. I already have a similar feature in tag-follow-mode, so this shouldn't be a problem.

Alexander-Miller commented 6 years ago

@myrgy I've created an issue that you can track at https://github.com/Alexander-Miller/treemacs/issues/169 so you know when it's done. Right now it's not on the top of my priorities, I expect to mostly focus on bug fixes and stability once multiroot is released, but it'll be implemented in the medium-term.

myrgy commented 6 years ago

Thanks a lot!

Alexander-Miller commented 6 years ago

The changes are up on master now, so I'll close up shop here. If you've bug reports or feature requests with the new setup just open up an issue over at treemacs.

I'll update the layer when I find the time, though seeing how it should be just changing some keybinds at most anyone who's interested can make that PR.

syl20bnr / spacemacs

Let's talk about treemacs and spacemacs Part 2: feedback and future plans #10432

Multiroot design

What I have so far

Local Keys

Global Keys

Session Persistence

Issues

Project path independence

No more root changing

Future Options

Dynamic Module

Improved performance

Mouse Faces

Better tag options - tag hook

Indent Guide