search

junjizhi / junji-blog

My own blog
MIT License
0 stars 0 forks source link

Building data apps with Vue #27

Open junjizhi opened 3 years ago

junjizhi commented 3 years ago

Building Data Analytics App with Vue

Introduction

Recently I've been using Vue to build a data analytics web app. The app fetches data from a private API and render them in different forms, e.g., data tables, charts, graphs, and other visualizations.

This is my first project using Vue. After playing with Vue for a few months, I understand why such a well-structured, light-weight web framework becomes popular.

Web frameworks debate aside, I want to talk about some lessons learned about building data analytics apps.

This post is for you if you are:

Server logs analytics app

Without revealing too much details, let's say I was building a server logs analytics app. A screenshot looks like below:

vue-data-app-screenshots

Users can:

Think of the app as a simplified version of Datadog.

Comparison: Vuetify vs. Bootstrap-Vue vs. other UI libraries

The project would need to render data tables. I found a good article with the following criteria for good data table libraries:

When I first started, choosing a UI library was a no-brainer. It's 2021 and rendering basic UI data analytics components, like tables or charts, should have been solved.

I narrowed the choices to BootstrapVue and Vuetify. These two libs check most items in the list above.

In the end I selected BootstrapVue because I worked with Bootstrap before and felt comfortable with the CSS styling naming. Also Bootstrap provides more freedom and space to tinker than Materials design.

Another consideration is, using Vuetify makes the app's look-and-feel too Google-like. I didn't want to get locked-in early on.

Using Bootstrap-Vue

Using a UI library like BootstrapVue sped things up a lot. I could quickly build a basic UI skeleton with stub data and asked for user feedback.

Especially, using b-table was convenient. Most of the controls are supported. It is matter of finding the right props / variants / events for customization or implementing behaviours.

And its documentation is well-organized and useful.

The difficult parts

I ran into some difficulties not long after: It requires work to integrate b-table with Vuex and the backend API.

Let me expand on this.

Most examples given in the b-table docs are using a data() which is assumed available in the component. To fetch data dynamically from the API, we need to use itemsProvider callbacks. I blogged about the trickiness to get itemsProvider worked well with Vuex.

I spent a lot of time to on planning, implementing, and refactoring the code for the following features:

For example, when implementing the global search bar, I assume the search results would apply globally. In other words, global search affects all tabs.

To translate into the Vue terms, the table component needs to watch the search input.

As mentioned, b-table manages data by itself, so to make it play nicely with global search, I had to force refresh the table while taking extra care to preserve all current table configurations.

Data live in the front or back end?

During the implementation, this question came up a lot.

In particular, since I used Vuex to share data among components, it is easy to treat Vuex store as a cache.

But with the Vue reactivity design, the UI soon became laggy even though the data array is not so large!

A solution to this problem is make the large data array (which comes from the backend) immutable. Namely, I used Object.freeze(...) before passing the long array to Vuex store.

There are also other best practices to organize the Vuex store logic:

Overall, it needs some care and planning to get reasonable performance when rendering the data.

Rendering data in charts and other visualization

I picked vue-echarts for rendering the time series data in line or bar charts. These two turned out to be simple and asked most often.

For topology graphs, I used vue-d3-network which is a thin wrapper around d3 in Vue. I picked d3 because it seemed the most mature and powerful graph visualization chart I found. It opens lots of customization opportunities and potentially can be applied to other projects.

Short notes about the back end API

As I mentioned above, I was using a private backend API, which takes care of all data ingestion, modelling, indexing and supporting querying data via API.

These are hard data engineering problems, especially when dealing with large volume of real-time graph data, together with the requirements to support historical data. There's no one size for all solution.

The complexity of dealing with all these problem is out of scope.

Summary

This post talks about my experience of implementing the front end for a data analytics application using Vue.

I used Bootstrap-Vue, Vuex, together with a private API.

During the implementation, I ran into tricky problems for data wrangling tasks like searching / filtering / sorting.

For data visualization, I used vue-echarts and vue-d3-network

Overall, Vue helped me spin up a quick front end scaffold and lays the foundation for future customization.

junjizhi commented 3 years ago

What's it's good for?

junjizhi commented 3 years ago

Using Vue to build a good data table

Regarding dashboards:

For the end user, the essence of a dashboard is essentially this: a collection of the filtered and carefully curated information on a particular topic, conveyed through visual/graphic instruments for quick ingestion. They don’t care about the subtleties of your data pipeline engineering, or how aesthetic your code is — all they want is a high-level view in 3 seconds. Therefore, our crude application displaying text data means nothing to them, and it’s high time we implement mechanisms to wrap the data with charts.

But Argos is not a dashboard. Below is from this site:

Adopting a detective mindset to study a users’ workflow and find unique opportunities to help them make data-informed decision.

When thinking about the product design, ask what their workflow is.

About the data table, how would they use it?

Good Data table must-have

Bootstrap vue tables evaluation

Idea: Build a open source data table with BootstrapVue

junjizhi commented 3 years ago

React vs. Vue

Reference: https://vuejs.org/v2/guide/comparison.html

React wins

Vue wins