vscode-data-preview

Data Preview 🈸 extension for importing 📤 viewing 🔎 slicing 🔪 dicing 🎲 charting 📊 & exporting 📥 large .json array .arrow .avro .parquet data files, .config .env .properties .ini .yml configurations files, .csv/.tsv & .xlsx/.xlsb Excel files and .md markdown tables with Perspective - streaming data analytics WebAssembly library.

![Data Preview](https://github.com/RandomFractals/vscode-data-preview/blob/master/images/vscode-data-preview.png?raw=true "Data Preview")

🈸 Features

Preview .json .arrow .avro .parquet .yml .csv/.tsv & .xlsx/.xlsb data files in a Data Grid w/Sorting & Filtering
Grid Data Summary display w/Aggregate Functions, Row & Column Pivots (a.k.a. Group By & Split By)
Basic Charts 📊 creation w/Aggregate Functions, Row & Column Pivots
Pluggable Charting 📊 libraries for bult-in Charts: d3fc || highcharts
Persistent Data Preview Settings (View, Sort, Filter, Pivots, etc.) for restore of open Data View panels on VSCode Reload
Mulptiple Spreadsheets Data Preview for Excel data files
Markdown tables data preview for .md documentation files
Data .schema.json generation for Arrow & Avro Data Schema Text Previews in JSON format
Binary Data files .json generation for Arrow, Avro & Excel formats for Text Data Preview
Property Grid display for .json .config .env .properties .ini & .yml configuration files
Open Data Preview on Side option for slim data || config files
Quick Launch new Data Preview input box for data files in open workspace
Save Filtered Data Grid || Chart 📊 Data in .arrow .csv .json(s) .yml & .properties formats
Save & Load Data View .config options
Dark, Light, Dense & High Contrast Blue Data Preview 🈸 Panel UI Themes

Next V.

Will include large text & binary data files loading & Apache Arrow data streaming.

Note: Data Preview 🈸 is already capable of loading a few 10+MB's large data files with 100+K records & extensive list of supported Data Formats you'll be hard pressed to find on VSCode marketplace in one extension.

See data/large/... data folder for sample large data files and Data View .configs you can try in Data Preview 🈸.

Installation

Install Data Preview 🈸 via vscode Extensions tab (Ctrl+Shift+X) by searching for data preview || via VSCode marketplace search results.

List of Data Preview 🈸 extension config Settings, data.preview command(s), keyboard shortcut(s), augmented vscode UI context menus, added Data Language mappings, supported Data Files list & configurable Theme & Charts 📊 Settings:

![Data Preview Contributions](https://github.com/RandomFractals/vscode-data-preview/blob/master/images/vscode-data-preview-contributions.png?raw=true "Data Preview Contributions")

Configuration

Create User or Workspace Settings in vscode to change default Data Preview 🈸 extension Settings:

Setting	Type	Default Value	Description
data.preview.theme	string	dark	Data Preview UI Theme: `dark`, `light`, `dense.light`, `dense.dark`, or `vaporwave` (hight contrast blue theme)
data.preview.charts.plugin	string	d3fc	Data Preview Charts 📊 library to use for built-in charts: d3fc or highcharts
data.preview.create.json.files	boolean	false	Creates `.json` data files for Arrow, Avro & Excel binary data formats
data.preview.create.json.schema	boolean	true	Creates `.schema.json` files for Arrow & Avro metadata binary data formats
data.preview.openSavedFileEditor	boolean	true	Opens created data file Content Editor on Data Save
data.preview.log.level	string	`info`	Data Preview run log level: `info` or `debug` for issues troubleshooting

Data Preview 🈸 example using dark UI theme with d3fc Charts 📊 Data View config, viewing superstore.arrow data file :)

![Data Preview Dark](https://github.com/RandomFractals/vscode-data-preview/blob/master/images/vscode-data-preview-dark.png?raw=true "Data Preview Dark")

Usage Tips

Data Preview 🈸 Launch Tips

Run View -> Command Palette...>Data: Preview Data command or Ctrl+Shift+D in an open .json .config .env .properties .ini .yml or .csv/.tsv text data file document to launch Data Preview panel.
File -> Save (Ctrl+S) your text data file for immediate updates in an open Data Preview 🈸 panel.
Right-click on a Binary .xlsx/.xlsb, .arrow or .avro data file in VSCode File Explorer to launch Data Preview panel.
Use exposed explorer/context, editor/title or editor/title/context Preview Data 🈸 or Preview Data on Side context menu options to preview your data files.
Click on the Data View Filename toolbar link to Load saved Data View Grid || Chart 📊 .config.
Click on the Data View 🈸 icon to Launch new Data Preview Panel for new view config changes.
Use Open Data File or URL 📤 option from Data View toolbar to launch new Data Preview 🈸.
Run View -> Command Palette...>Data: Preview Remote Data command or Ctrl+Shift+R to launch Data Preview for remote http(s) data files.

![Data Preview Open Data File](https://github.com/RandomFractals/vscode-data-preview/blob/master/images/vscode-data-preview-open-file.png?raw=true "Data Preview Open Data File")

Data Grid/Filter/Columns UX Tips

Double click on the Grid Column header to Sort data by that column.
Drag and drop a column from the left-side Columns control panel into Filter fields for data filtering (Group By, Split By, Sort, Filter).
Drag columns up and down in the left-side Columns control panel to reorder displayed columns in the Data Grid.
Uncheck a column in the Columns control panel to remove it from a Chart 📊 or Data Grid display.

Usage Scenarios

Use Data Preview 🈸 to:

Load large data files for sorting, filtering & charting 📊
Export displayed data in a compact binary .arrow data format, .ods .xlsb || .xlsx spreadsheet format, or .csv .json .yml .md or .properties text formats
Preview .properites and other key-value pairs configuration files to reformat them or find set config option values
Generate Arrow & Avro .shema.json for metadata text preview of those binary data files
Generate .json files for text data preview of binary Excel files
Preview Excel files and workbooks with multiple workseets without opening Excel
Convert .csv or .tsv data to .json or .yml format
Extract, sort, filter and save markdown tables from .md documentation files
Use built-in Charts 📊 for Exploratory Data Analysis

Supported JSON, Config, Binary & Excel Data File Formats

Tip: try sample data and Data View .config files from this repository data/... folders: data/arrow data/avro data/parquet data/config data/excel data/json data/yaml data/large

Vega datasets repository also has a broad collection of sample .csv & .json array data files you can try in Data Preview 🈸

Data Preview 🈸 Files Matching Rules

{
  "when": "resourceFilename =~ /.*\\.(json|jsonl|json5|hjson|ndjson|arrow|arr|avro|parquet|env|config|properties|ini|yml|md|csv|tsv|txt|tab|dif|ods|xls|xlsb|xlsx|xlsm|xml|html)/",
  "command": "data.preview",
  "group": "navigation"
}

Data Preview 🈸 Files Loading Details

Note: .json .config & .yml configuration files that don't contain array data are converted to flat properties key/value pairs Object and displayed in a Property Grid Data View mode. See json.utils.ts for more info.

Data File Extension(s)	File Type	Data Parsing Library/Method Used	Data Format Specification
`.json` `.config` `.jsonl` `.ndjson`	text	`JSON.parse()`	https://json.org/
`.json5`	text	json5/`JSON5.parse()`	https://json5.org/
`.hjson`	text	hjson-js/`Hjson.parse()`	https://hjson.org/
`.arrow` `.arr`	binary	apache-arrow/`Table.from()`	https://arrow.apache.org/
`.parquet`	binary	parquets/`ParquetReader.openFile()`	https://parquet.apache.org/documentation/latest
`.properties` `.env`	text	node-properties/`properties.parse()`	https://en.wikipedia.org/wiki/.properties
`.ini`	text	node-properties/`properties.parse()`	https://en.wikipedia.org/wiki/INI_file
`.md`	text	RegExp/`markdownToCsv()`	https://en.wikipedia.org/wiki/Markdown
`.yml` `.yaml`	text	js-yaml/`yaml.load()`	https://yaml.org/
`.csv` `.tsv` `.txt` `.tab`	text	perspective/`perspectiveViewer.load(text)`	https://en.wikipedia.org/wiki/Comma-separated_values https://en.wikipedia.org/wiki/Tab-separated_values
`.dif` `.ods` `.xls` `.xlsb` `.xlsx` `.xlsm` `.xml` `.html`	binary/text	js-xlsx/`XLSX.read()`	See https://github.com/SheetJS/js-xlsx#file-formats for more info on `Excel` file formats

See Data Manager API & src/data.providers folder for data loading and saving imlementation details.

Provided Chart 📊 Types

Area Chart
Bar Chart
Candlesick Chart (d3fc only))
Heatmap
Line Chart
OHLC Chart (d3fc only)
Scatter Chart
Sunburst
Tree Map

![Data Preview Chart Types](https://github.com/RandomFractals/vscode-data-preview/blob/master/images/vscode-data-preview-chart-types.png?raw=true "Data Preview Chart Types")

Supported Filter Functions

<, <=, ==, !=, >, >= for dates and number columns/fields
==, !=, contains, in, not in, begins with, ends with for string fields and dictionaries
&, |, and, or, ==, != for bolean fields

Supported Aggregate Functions

any
avg
count
distinct count
dominant
first by index
last by index
last
high
low
mean
mean by count
median
pct sum parent
pct sum grand total
sum
sum abs
sum not null
unique

Recommended VSCode Extensions

Other extensions Data Preview 🈸 replaces, enhances or supplements for working with supported data file formats in VSCode:

Extension	Description
Excel Viewer	View Excel spreadsheets and CSV files
Avro Viewer	`.avro` file viewer
avro-idl	Avro IDL Syntax Highlighter
DotENV	`.env` Syntax Highlighter
Ini for VSCode	Provides outline view and section folding for INI files
Hjson	Hjson language syntax support
JSON5 syntax	Adds syntax highlighting of JSON5 files
NDJSON Colorizer	Colorizes NDJSON (Newline Delimited JSON) files
YAML	YAML Language Support by Red Hat, with built-in Kubernetes and Kedge syntax support
docs-yaml	YAML schema validation and auto-completion for docs.microsoft.com authoring
YAML to JSON	Convert YAML from clipboard or current document/selection to JSON and vice versa
Properties To Yaml	Convert properties to yaml
Markdown Table Prettifier	Transforms markdown tables to be more readable

Dev Log

See #DataPreview 🈸 tag on Twitter for the latest and greatest updates on this vscode extension and what's in store next.

Dev Build

$ git clone https://github.com/RandomFractals/vscode-data-preview
$ cd vscode-data-preview
$ npm install
$ code .

F5 to launch Data Preview extension VSCode debug session.

vscode-data-preview>vsce package

to generate VSIX Data Preview extension package from our latest for local dev install in VSCode.

Note: Use Help -> Toggle Developer Tools vscode menu option to view Data Preview console log.

Contributions

Any and all test, code or feedback contributions are welcome.

Open an issue or create a pull request to make this Data Preview 🈸 extension work better for all.

Backers


SheetJS	Aman Mittal

RandomFractals / vscode-data-preview

readme