Engineering feedback (Griz and VisIt)

[aowen87]

I recently spoke with a couple of engineers who work with both Griz and VisIt to get some insight into the GrizIt endeavor and, more generally, what they actually need to start using VisIt as their primary tool.

Surprisingly, they both separately gave me very similar (almost identical) feedback, and they expressed that these opinions were shared by most of their colleagues.

I interviewed them both separately, and I went in with the following questions in mind:

1. Where are VisIt's shortcomings in their workflow?
2. What advantages does Griz have over VisIt?
3. Will the GrizIt interface be helpful/used?
4. What are the largest barriers keeping these user from VisIt?

General Takeaways

1. The engineers have a specific set of operations that they perform regularly, and they want to be able to perform these steps with as much ease as possible.
2. VisIt has a few significant shortcomings that have a large impact in their workflow.
3. I did not get the impression that they want a Griz clone. That is, moving forward, I don't believe we should try to mimic Griz in every detail.
4. Documentation is very important and will likely be the final barrier to determine whether or not we can recruit many of the engineering folk.

Specifics First, I'll just list the primary issues that were brought up by the engineers. I'll then go into more detail.

1. Time queries take far too long.
2. Descale doesn't always work (it relies on node displacement, which doesn't always get through to VisIt).
3. Saving curve plots is overly complicated, and curve plots are missing some of the features they desire.
4. VisIt's interface complexity, in general, becomes a hindrance.
5. Outdated and missing information in the documentation will likely cause users to give up.

As I mentioned earlier, the Griz users tend to have a set of specific operations that they continually rely on in their workflow. While they acknowledge that Griz is not nearly as powerful as VisIt, it allows them direct and simple access to these specific operations, and this seems to be a key reason they return to Griz. Having said that, my impression was not that they use or even care about all of the features in Griz. There was really only a small subset that stood out.

First and foremost are time queries. We've known this for a while, but it was good to hear this directly from both users. It was very clear that one of the biggest barriers keeping them from using VisIt is the length of time it takes to perform these time queries. Because of some assumptions that Griz makes about time series meshes, it can perform time queries very quickly. VisIt, on the other hand, does not make these assumptions and ends up taking a very long time when performing these same queries.

Second, the descale operator is something they rely heavily on, and this doesn't seem to always be available in VisIt. The operator relies on the node displacement field, which, for whatever reason, isn't always available when loading mili data into VisIt.

Third, curve plots are a huge part of their workflow, and both users felt that the steps needed to interact with and, most importantly, save out curve plots were overly complicated. One user explained that he regularly would forget all of the steps needed to save out the current curve plot (select the active window, export database, choose the correct options, etc). They both expressed a desire to have some kind of simple "save plot" button, that would just perform the magic for them. There was also a desire for the curve plots to have greater functionality. Some of these functions actually do already exist in VisIt (like plotting two variables against each other over time), but it's just not apparent that this is an option. Other features probably do need to be implemented.

Fourth, VisIt's interface complexity, in general, becomes a barrier. These users have become accustomed to having a small set of very succinct commands that they can use for all of their primary needs. Having to wade through all of the various options within VisIt to find their way to this small set, especially if it's all they're using 90% of the time, becomes a dissuasion to use VisIt at all. Because of this, it does seem like GrizIt could be useful. However, I don't think they need (or even want) a fully featured Griz clone. In fact, there seems to be good reason to not create a Griz clone, not the least of which is that Griz itself might not be meeting the engineering needs (Griz is just the lesser of two evils in this case). For example, I spoke with one of the engineers about the explode operator in Griz and whether or not they ever use it. They said they've used it a few times, but they found that it was difficult to interact with and not really worth the trouble. That's not to say that they don't want this feature—they find CAD style explode operators very useful—they just don't want Griz's version of this feature.

Lastly, documentation appears to be a final barrier that will determine whether or not many of the engineers convert to VisIt. In particular, the more seasoned employees who have become accustomed to doing things themselves will likely give up and go back to Griz if they can't find what they're looking for in our documentation or, worse, if the documentation is just incorrect.

Moving Forward I'm planning on creating individual tickets for these issues. The computational issues (time queries and descale) should have fairly straight forward solutions. The issues regarding interface complexity are a bit more difficult to tackle and will probably require more discussion. Ideally, GrizIt will help ease most of these in the near future.

I'm planning on keeping in contact with both of the users I spoke with, and they are happy to participate in testing out new developments in these efforts. If possible, I'd also like to bring some more of their colleagues into the conversation in the future.

[markcmiller86]

@aowen87 ... this is terrifice information. Thanks for taking the initiative to go collect it. Except for item 2 (Descale...as an aside...I don't think I even know what that is), I think these are all relatively known issues with time-query being at the top of the list. We should tackle that, I think we have the know-how to do it such that it works fast for all VisIt I/O formats, not must mili.

Regarding interface complexity, I agree that even simple things are relatively complicated in VisIt. We should make the simple, common things easy. OTOH, I get concerned about taking precious development resources away from those aspects of VisIt that set it apart from any other tools out there; scalability and performance. I think we as a team need to think carefully about how we commit resources to address problems that are not in that part of our tool box. There are literaly hundreds of useful curve plotting tools out there. Do we want to compete with them? I don't think so. So, I think we be bee seeking ways to make it easy for VisIt to integrate with such tools so that users can then use those tool(s). I've made these arguments before with little traction.

So, saving or re-formatting curve data so another third-party tool can grab and use it seems to me like a great thing to spend resources on. However, investing heavily in improving VisIt's curve plotting features themselves does not. What do others think about that?

Regarding item 5, outdated/missing information and documentation...were either of these users aware of our updated docs on RTD? It would be good to know because I actually think we have resolved are in process of resolving this issue and that this situation should continue to improve with time.

So, my take-away from your notes are that items 1 and 2 and only part of 3 should get some resources soon.

[aowen87]

Yeah, I agree. I talked with Eric last week about this, and he suggested I create an extension of the time series database that allows access to a variable across a given chunk of time (as a vector or something like that). It could then be handled in the plugin without reading the entire mesh in for every cycle.

I also agree with the curve plot comments. Both users seemed most interested in saving the plot data, and one of them even said that they rely on other visualization tools that focus on curve plots to actually render and interact with them once they are saved. They wanted to be able to initially view the curve in VisIt just to make sure that it is sensible, but the rest is just exporting the data.

They did not seem to be aware of the documentation upgrades. I think this, along with Eddie's work on finding holes in the documentation, will help considerably.

[markcmiller86]

and he suggested I create an extension of the time series database that allows access to a variable across a given chunk of time

I think all the 4 database format types already support what we need. They all have a built-in notion of time (though there are huge complexities in how we deal with the set of time/cycle/index values). What is missing is the ability to pass into them a query for data across time. This relates to other topics with large architectural implications such as ensemble navigation as well as ability to parallelize operations across any dimension of an ensemble and, in particular, parallelization in time

A user can have all of their data over some of the time or some of their data or all of the time. But, they can't have all thair data over all time ;)