philosophy/design: user interface and task representation, idea for harpoon-like behavior

[matu3ba]

Coming from harpoon, harpoon tends to have a global index of buffers and terminals, which one can lazily add (stored as json on disk) and which one can open via numbering as index into the global table (0,1,...).

Harpoon does not support command formats and neither supports persistent buffers, which leads to a very cluttered session file. Neither does harpoon provide a cleanup function for unused indexed files etc.

What is the philosophy/design of overseer in how 1. tasks and 2. cmds and 3. the toggle interface ought to work?

• 1. Should tasks be uniquely identifiable (in what context) and should they be numbered, ie as table?
• 2. Should tasks have (the option for) a related buffer in neovim outside of terminal as content backup and/or real file?
• 3. Should task output be a terminal and/or a simple file as output, if the former is unfeasible (security, workarounds etc)?
• 4. Should cmds only support the basics and everything else be done via toggle interface to be "more graphical", should both interfaces complement another or should one be the extension of the other? (related to accessing tasks (names,indices) + functionality via cmd)
• 5. what is the maximum number of options the toggle interface should provide and why is it not merely a supervision window with the cmds providing access to the underlying tests etc?
• 6. "Note: both tasks and components are designed to be serializable. They avoid putting things like functions in their constructors, and as a result can easily be serialized and saved to disk." has no link to an example how to serialize and deserialize the state for a use case. Are you referring here maybe to the output of the task?

[stevearc]

I'm not sure exactly what you mean by "2. cmds". In general I think I'm only understanding 60-80% of what you're asking, but I'll do my best to answer.

Should tasks be uniquely identifiable (in what context) and should they be numbered, ie as table?

Tasks do each have a unique ID. Fetching tasks by ID is not an official part of the public API, but if you have need of it I could add it. Currently the main way to access tasks is the list_tasks() method, which returns a list of tasks ordered by creation time. It's not stable though, because tasks get bumped back to the top when they are restarted.

Should tasks have (the option for) a related buffer in neovim outside of terminal as content backup and/or real file?

I don't think this should be built in to the task, but certainly this sort of thing could be done with a component

Should task output be a terminal and/or a simple file as output, if the former is unfeasible (security, workarounds etc)?

I'm not sure what you mean by this? The task output is just the combined stdout/stderr of the process, and by default we display it in a Neovim integrated terminal. As mentioned above, this output could also be captured and saved to disk, but is not done by default.

Should cmds only support the basics and everything else be done via toggle interface to be "more graphical", should both interfaces complement another or should one be the extension of the other? (related to accessing tasks (names,indices) + functionality via cmd)

By "cmds" do you mean the Overseer* commands? In terms of interacting with tasks, the things you would want to do to a task are modeled as actions. There are multiple ways of selecting a task and choosing an action to perform on it. You could use the OverseerTaskAction to select a task + action, OverseerQuickAction to run an action on the most recent task, you could use the sidebar and your cursor to select a task with <CR> and then choose an action, or you could do it all programmatically with overseer.list_tasks and overseer.run_action. These all do the same thing, but with different ergonomics that might be more or less desirable in different situations.

what is the maximum number of options the toggle interface should provide and why is it not merely a supervision window with the cmds providing access to the underlying tests etc?

Are you talking about the action menu when you select a task in the sidebar? There's no maximum number of actions and I don't...really know why there would be? I'm also not sure what you mean by a "supervision window" or "underlying tests"

"Note: both tasks and components are designed to be serializable. They avoid putting things like functions in their constructors, and as a result can easily be serialized and saved to disk." has no link to an example how to serialize and deserialize the state for a use case. Are you referring here maybe to the output of the task?

No, not the output. Since the tasks themselves are (usually) created with only serializable data, it's possible to save and restore them. This is used in the OverseerSaveBundle / OverseerLoadBundle commands and their associated API methods. This is also used for session manager plugins so you can save & restore your tasks.

[matu3ba]

Fetching tasks by ID is not an official part of the public API, but if you have need of it I could add it.

I'd like to be able to script certain behavior of the task runners for long running tasks as CI-like behavior, ie to git push to master after 1. successful run, 2. successful run after modifications and testing only the files with failures. I'd further like the ability to check if the current command (name, id) 1. name/id was not changed (since last start [+finish] to put them into a wait queue for execution and 2. be able to do loading of tasks and execution in separate steps and/or 3. reading via ipc/socket etc. (just for playing around if the design would allow it etc)

Context of the thing I'd like to do: Nice (minimal) CI with 1. timeline view and 2. collection of started program arguments and 3. show currently started programs, 4. dependency resolution why what things were started. One would ideally with attach the relevant metadata to reproduce the program behavior, but often that is not feasible and/or too tedious. As example, one could write a neovim-buffer backed storage for the output (to workaround lua being racy on garbage collection or too slow/unfeasible, see https://github.com/nvim-telescope/telescope.nvim/issues/647#issuecomment-1532244593).

[matu3ba]

I think the design as it is now is very good and my motivation was more that I felt like the user interface feels abit slowish without being able to have keybindings like harpoon to do things on currently available tasks more quickly.

Did you try harpoon yet?

[stevearc]

This sounds like a potentially very complex and very custom workflow. Nothing like this is directly built-in, but there are some building blocks that you could use that might get you there. First, check out the orchestrator strategy. If you're trying to run a sequence of tasks with various dependencies and want some sort of top-level UI, this is probably the closest we have. Check out test_orchestrator.lua for an example.

I haven't tried harpoon, but I use a custom bufferline that supports pinning and switching with <leader>1-9, which I think is fairly similar. I don't think it makes sense for something like that to be built in to overseer, but you could build something similar. Here's an idea to get you started

local function get_task_callback(i)
  return function()
    local tasks = require("overseer").list_tasks({ recent_first = true })
    local task = tasks[i]
    if task then
      require("overseer").run_action(task)
    end
  end
end
for i = 1, 9 do
  vim.keymap.set("n", "<leader>" .. i, get_task_callback(i), { desc = string.format("Run an action on task #%d", i) })
end

Lastly you can also build your own custom components and add them to your tasks. That would allow you to hook into the task lifecycle and, for example, register each task with your own centralized registry that you can use for hot switching, display, or anything else.

LMK if you have specific requests for an API or questions about how to best accomplish something.

[matu3ba]

Thanks, I'll close this for now and keep the info.

I'll start once I have my basic nixos setup fixed (keyboard on arcan works, lol) and #201 is fixed for at least the buffer-backed case (currently it makes me feeling uneasy about the dropping content thing, because I did also run into #209).