Gather strategic questions related to future VEDA UI after GHGC/EIC release

[j08lue]

We will very soon have made it to the release of the GHG Center and EIC supported by this library. 🎉

Soon after, we will be revisiting the strategy towards managing multiple instances - time frames, scope, resources.

What questions or architecture considerations are important for these decisions?

• https://github.com/NASA-IMPACT/veda-ui/issues/537

Acceptance criteria

• [x] Listed guiding technical / architecture considerations for the upcoming phase of strategizing for VEDA UI

[j08lue]

Challenges

There are several challenges we have with our current VEDA UI implementation, which can be considered as drivers for our longer-term strategy:

1. Limited and unmaintained design system / component library

Our design system is outdated and is lacking some basic components like nested nav or side nav. Adopting a design system / component library like the US Web Design System or Chakra UI could mean that we can spend less time on implementing these standard components while at the same time complying with the latest standards for accessibility etc.

2. Optimization to one data backend type, reliance on configuration

The diversity of data backends and formats we have encountered in the past year (STAC vs CMR, COG vs Zarr, raster vs vector) exposed the limitations of a frontend that is optimized for a single standard (STAC+COG/TiTiler) that it can be configured for. Projects like EODC are looking for ways to use well developed data exploration interfaces with different data providers and make the visualization interface more adaptable to the format (i.e. dynamic, data-driven). VEDA UI could benefit from that model, as our users expect to integrate data from various cloud-native geospatial providers. https://github.com/developmentseed/nasa-eodc/issues/83

3. Page layout and functional components cannot be used independently of each other - limits of overrides

Page layout, nav, footer, etc. are right now connected to the functional components like data management and visualization elements. To accommodate for the different needs that the projects have, we currently heavily rely on overrides, which works but seems to get more hacky as we scale to make more things configurable. Separating function from layout could mean that we can build custom sites more cheaply / with fewer constraints while re-using the core data handling and visualization functionality, which our engineering efforts can focus on.

4. Dynamically rendered application has challenges with routing and analytics

We have experienced challenges with our site being a client-side rendered application: routing is brittle and the dynamic page generation is limiting how we can gather analytics. Is there a case for our next iteration to use static page generation? e.g. https://github.com/NASA-IMPACT/veda-ui/issues/783

5. Content editing is engineering-heavy

We have not yet solved the challenge providing content for VEDA UI instances currently is quite engineering-heavy, which does not scale well if the websites should also support frequently changing content like events, announcements, etc., as for the new GHG Center announcements page. If VEDA UI as a project for "Scientific Data and Information Center" websites should fulfil also the content autonomy needs of these projects, should we consider to get this functionality out of the box from a stock CMS with admin interface?

Tech and tactics

1. We cannot halt development for a ground-up re-engineering - can we do the Ship of Theseus thing and rebuild while sailing, starting with some part we know that will both benefit existing instances and move the library forward in the direction we envision? Or does it make more sense to build some new patterns on other projects first, like EODC? I'd prefer the first, if we can pull it off.
2. We need to map out what consequences and complexity the suggested changes would have and which ones are related to each other. E.g. could we modernize our design system while keeping a possible change from MDX to a stock CMS for later?

[faustoperez]

Here's some info on the requirements for adopting the digital.gov framework: https://github.com/GSA/digitalgov.gov

[faustoperez]

This is the ticket about digital.gov design recommendations: https://github.com/NASA-IMPACT/veda-config-ghg/issues/197

If we decide to adopt this framework, I think it should apply to all VEDA instances and not just the GHG Center.

[hanbyul-here]

Thanks for this write-up. I agree with most of the challenges, and it is so helpful to see how another person would phrase the problems.

To piggyback a few things

2. I wonder if we had problems with different formats. Zarr layer does seem to require a special configuration maneuver, but adding a different type of layer has been smoother than I've expected. Is there any specific problem you recall?
3. I think your phrasing already has this content, but to more explicitly write it out: Currently, the new instance almost always requires a developer who knows the pipeline between veda-config and veda-ui. If we can make the relationship between the instance and UI modules less customized in a way that any front-end developer can understand right away, (ex. ui module is a dependency of the instance - related to the registry ticket) we would be able to be less involved in the instance development.
4. I'd like to add SEO as a benefit of statically generated sites.

[sandrahoang686]

Thanks for creating an issue for this and starting to document the discussion here @j08lue! I wanted to add a few comments and opinions on top of the challenges you have highlighted...

1. Limited and unmaintained design system / component library The design system is also very opinionated and its hard to understand how to override certain styles and create standardization across instances without touching core logic. Using a well documented and community adopted ui library like chakra would help with not having to worry about maintaining our current outdated ui library. But usually with these libraries, you can customize to a certain extent yet it is still highly opinionated. This shouldn't be a problem if we arn't looking to customize so much but want to put that out there.

Questions that arise for me are.. what level of customization do we want instances to have (stuff like rendering behavior and accessibility could probably be standardized across instances but what about other stylistic choices)? How granular do we want to get with our components? If components are re-used (e.g. slider, form controls, menus, etc) they make sense to live in their own single file as a module. Same pattern for even single use components because of “functionality”. Do we care to create the ability to override these granular components? (we should get rid of the idea of overriding whole pages / page container components..)

If we want to still allow some level of customization, an idea could be to have a default state or prop for higher level container components (pages). Based on the default state, we default to using said design library otherwise we allow the container to compose with components provided by the user (risk would be a clash with design on not adhering to general design standards within the rest of the app, this is at the instance level though and on the consumer to manage that risk)

2. Optimization to one data backend type, reliance on configuration I’m not too sure of the geospatial needs here but I think the issue remains the same and that is we need to support different data sources and have them easily consumed by our ui regardless of source or format? This can be done by abstracting all our data logic into a data layer and have it completely decoupled from our components. We can do this by implementing the Context API mechanism which is a provider and consumer pattern where our higher level containers (the page components or “smart” components) would deal with the data fetching by utilizing this data layer and then provide the data downstream. This allows us to scale the code to whatever data needs change in the future as long as the end format remains the same and allows us to decouple the functionality which removes complexity inside the code.

3. Page layout and functional components cannot be used independently of each other - limits of overrides I’ve addressed some of this in # 1 but to expand on this. Some of the questions that arise for me is how much flexibility do we want to allow our instances to have. How much do we compose within the core ui logic vs what does each instance have control over? What makes sense as well? Some functionality makes sense to be coupled together, like Hanbyul has mentioned to me that it makes sense for the components in the Exploration and Analysis logic/page to stay mostly coupled because it doesn’t make sense to export parts of this logic. But this page can be composed of granular components that can be reused. What about opinions on page layout? I personally think it makes sense to leave the page layout decisions to the instance level. All granular/presentation components should be able to be imported and consumed at the instance level as well. This allows us to add additional functionality or features without messing with core logic. There could be container components that have a default state but we can also allow flexibility through decomposition.

4. Single-page application has challenges with routing and analytics I dont think there is anything wrong with VEDA being a single page application. Routing should be managed at the instance level and the framework chosen would help manage routing. Routing in a single page application is just rendering different views in our application. These views just need to be well defined with what container component view to render. NextJs is a framework that helps build single page applications and its key benefit is how it manages routes within code structure. This can be something we can ideally start using in our instances but instances should have the autonomy to decide what framework it would like to use.

5. Content editing is engineering-heavy Agreed

6. Other problems

• Standardized error handling that is isolated to the part of the app that is erroring. If one component errors out, we should have an error state and still load the rest of the app
• Working with VEDA is not intuitive which leads to long ramp up times. We should stray away from very custom code and custom solutions and move to a more “react” way of building. Would we pass the lottery test!?

[faustoperez]

I've been investigating a bit the US web design system and it's pretty customisable out of the box while having core needs covered (open source, accessibility, US government standards, breadth of basic components, etc.) 👉 https://designsystem.digital.gov/documentation/settings/ https://designsystem.digital.gov/documentation/developers/#sass-and-theme-settings

My take would be to use its basic building blocks as our design language foundation and extend it to satisfy our customisation needs: we would create a visualization library, map components, embeds, widgets, E&A app, etc.

Would it make sense to create a test VEDA playground to evaluate?

[j08lue]

Btw, this was the framework that VEDA UI originally used took inspiration from, by NASA EarthData: https://cdn.earthdata.nasa.gov/eui/latest/docs/eui/index.html

While an excellent choice at the time for a NASA EarthData application, it seems now not well maintained and is very rudimentary in terms of components it includes. Furthermore, VEDA now provides applications that are not expected and not supposed to be affiliated with NASA EarthData.

[hanbyul-here]

Just to note, (please correct me if I am wrong), I believe veda-ui never used the framework as it is. ex. we took the main color schemes to our styles, but I don't recall I ever used CSS offerd by this system.