various enhancements

[zhubonan]

I think it would be a good time to raise and PR, as discussed in the meeting I have added a few commands and I will add more/change the existing accordingly.

This PR is a work-in-progress and mainly act as a potential place of discussion and progress tracking.

Changes:

• [x] Access to verdi commands - I have made the entire verdi command accessible in the node shell using the native click interface. It is very fast, but there is no autocompletion. I also added expansion so {} expands to the current node's pk and {-1} expands to that of the last node in the history.
• [x] Access to verdi shell using vshell command. This start an embedded IPython shell with pre-loaded aiida objects in local scope similar to that of verdi shell
• [x] cd and pwd command for switching directories
• [x] do_in , do_out stable order and rename -l/--follow-link-id and coloring
• [x] make nodehist_show display relationships
• [x] add group_list and group_belong commands
• [x] -set options for label and description commands
• [x] reload command for reloading the node and needs_new_node decorator so some commands can automatically reload unsealed ProcessNode.
• [x] comment_add and comment_show commands.

Konwn Issues:

• verdi status will surge CPU usage is rabbitmq is not running - this is probably a problem in the actual verdi status command since it was assumed that the python process should terminate after the command execution.

This PR address:

• 1

• 2

• 3

[zhubonan]

@giovannipizzi @bosonie I have made some major improvement based on the meeting notes. Could you please have the new version in this PR a try and give me some feedback?

Obviously there is no automated testing at the moment... So feedback would be very welcome.

A major improvement that I think people will like is that the entire verdi command line suite can be used with much better responsiveness than that from the shell. In addition, we have command substitution using the currently loaded node or reference to history. So you can use the comand like this:

verdi calcjob outputcat {-1} | less

without the need to remember the pks.

There is no tab completion though, and it is unlikely to have an easy solution, but people are free to add aliases. For example, I use vpl for verdi process list.

We can close #2 and #3 once this is merged.

[bosonie]

Hi @zhubonan. This looks great! Seriously! Only one minor comments. The graphics of nodehist_show doesn't satisfy me fully. At the moment I see:

    (Dict<21668>@first) nodehist_show
    WorkChainNode<21662> Li 
    🢃  ---  [iteration_01] CALL_CALC
    CalcJobNode<21665>  
    🢃  ---  [output_parameters] CREATE
    Dict<21668>  <-- We are here
    (Dict<21668>@first)

it would be nice to have an indentation or a bar that helps to distinguish between the output of the nodehist_show command and the shell prompt. Something like:

    (Dict<21668>@first) nodehist_show
      |  WorkChainNode<21662> Li 
      |  🢃  ---  [iteration_01] CALL_CALC
      |  CalcJobNode<21665>  
      |  🢃  ---  [output_parameters] CREATE
      |  Dict<21668>  <-- We are here
    (Dict<21668>@first)

I'm not very familiar with groups therefore I didn't test this part extensively... the rest seems perfect to me. Thanks!

[zhubonan]

@bosonie Thanks, I agree it looks much better with the indentation! I will add it soon.

[giovannipizzi]

Indeed, this really looks great! I will now give some feedback on the PR.

There is no tab completion though, and it is unlikely to have an easy solution, but people are free to add aliases. For example, I use vpl for verdi process list. I just created a PR on your PR branch (I'm trying to create circular dependencies in GitHub ;-) ) that introduces fast bash completion for verdi! With this, the tool is perfect, I would say :-D

Actually, I would maybe thing to rename it to verdi powershell? In principle even if you don't care about the rest of the commands, now you can just call verdi powershell and then continue working with verdi commands, that now become super fast?

[giovannipizzi]

In general it looks great! I didn't try out the comments interface but we can improve later in case it's needed.

Instead, I have a final comment on the groups.

• group_belong is useful, with the comment I left in
• group_list is probably not useful now that we have verdi working and with TAB completion

What I had in mind was an interface mirroring the new verdi group path (pinging @chrisjsewell for feedback). The key idea is that if you have the following groups a, a/b, a/c, a/c/d, e, they behave as folders, so if you do verdi group path ls you see only a and e, if you do verdi group path ls a you see b and c, and so on.

Maybe we could do the following (in the following I speak of groups but I actually refer to group paths):

• keep internally, as we do for the nodes, a current 'group path' and maybe even a history of the recently visited group paths. The default at load would be /. If we do a history, it could expose grouphist_* commands similar to the nodehist_* commands/
• define commands group_cd, group_pwd (and aliases gcd, gpwd) to change group using relative paths, or starting with / to get a top group.
• make sure gcd has working TAB completion
• define command group_ls (gls) command to show subgroups (with TAB completion)
• have finally an option to show the nodes in the current group (group_show, gshow). @chrisjsewell by the way, is this possible to do in the current verdi command line interface? What is the command? I couldn't find it. Also, is the documentation already accessible? It could be useful for @zhubonan and @bosonie to understand the underlying logic.

One important thing to decide is how to deal with group types. Should we limit to only "user types"? Can we define a syntax (do we already have one?) to load a group of a different type, e.g. group_name:group_type?

Since this discussion might become long: maybe let's just have a simple group_belong in this PR, and we move possible discussions on the group path functionality in an issue (@zhubonan if you see that there starts to be discussion, feel free to move to a separate issue).

[zhubonan]

@giovannipizzi Thanks for the comments. I will change the codes accordingly.

I think perhaps it is good to leave the group related stuff to the next PR. I am not familiar with the group path concept yet, since I have not upgraded aiida-core to v1.2.0 (need to test the plugins with the new changes in v1.2.0 for safety).

[zhubonan]

@giovannipizzi All done. It is ready to be merged now.

[chrisjsewell]

pinging @chrisjsewell for feedback

I think what you said makes sense btw. But yes I'd have to have a play around with this feature to understand better, and happy to have a stab at it in a further PR 👍

[zhubonan]

@giovannipizzi Thanks, all done, please do check if the colors are OK on OSX!

[giovannipizzi]

Thanks a lot!!!!