regular-table

A regular

library, for async and virtual data models.

finos/regular-tableregular-table

README

regular-table


NPM VersionNPM VersionBuild Status


#

A Javascript library for the browser, regular-table exports
a custom element
named ``,which renders a regular HTML `
` to a `sticky` position within a scollable
viewport.  Only visible cells are rendered and queried from a natively async
virtual data model, making regular-table ideal for enormous or remote data
sets.  Use it to build Data Grids, Spreadsheets, Pivot Tables, File Trees, or
anytime you need:

* Just a regular `
`.
Virtually rendered for high-performance.
async data model handles slow, remote, enormous, and/or distributed backends.
* Easy to style, works with any regular CSS for `
`.
Small bundle size, no dependencies.

Examples


||||
|:--|:--|:--|
|two_billion_rows|canvas_data_model|perspective_headers|
|two_billion_rows|canvas_data_model|perspective_headers|
|minesweeper|file_browser|spreadsheet|
|minesweeper|file_browser|spreadsheet|

Features


||||
|:--|:--|:--|
|row_mouse_selection|column_mouse_selection|area_mouse_selection|
|row_mouse_selection|column_mouse_selection|area_mouse_selection|
|row_stripes|
|row_stripes|

Documentation


What follows functions as a quick-start guide, and will explain the basics of
the Virtual Data Models, Styling and Interaction APIs.  Complete API docs
and documented examples
are also available.

- QuickStart
  - Installation
- [`` Custom Element](#regular-table-custom-element)
  - [.setDataListener() Virtual Data Model](#setdatalistener-virtual-data-model)
    - Column and Row Headers
    - Hierarchial/Group Headers
    - [async Data Models](#async-data-models)
  - [.addStyleListener() and getMeta() Styling](#addstylelistener-and-getmeta-styling)
    - [.invalidate()](#invalidate)
  - [.addEventListener() Interaction](#addeventlistener-interaction)
  - Scrolling
  - [Pivots, Filters, Sorts, and Column Expressions with perspective](#pivots-filters-sorts-and-column-expressions-with-perspective)
  - Development

- API Docs

- Annotated Examples
  - 2d_array.md
  - canvas_data_model.md
  - file_browser.md
  - minesweeper.md
  - perspective.md
  - react.md
  - spreadsheet.md
  - two_billion_rows.md

Installation


Include via a CDN like JSDelivr:

  1. ```html
  2. <script src="https://cdn.jsdelivr.net/npm/regular-table"></script>
  3. <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/regular-table/dist/css/material.css">
  4. ```

Or, add to your project via yarn:

  1. ```bash
  2. yarn add regular-table
  3. ```

... then import into your asset bundle.

  1. ```javascript
  2. import "regular-table";
  3. import "regular-table/dist/css/material.css";
  4. ```

## `` Custom Element

`regular-table` exports no symbols, only the `` Custom Element
which is registered as a module import side-effect.  Once loaded,
`` can be used just like any other `HTMLElement`, using regular
browser APIs:

  1. ```javascript
  2. const regularTable = document.createElement("regular-table");
  3. document.body.appendChild(regularTable);
  4. ```

... or from regular HTML:

  1. ```html
  2. <regular-table></regular-table>
  3. ```

... or from your library of choice, as long as it supports regular HTML! Here's
an example for React/JSX:

  1. ```javascript
  2. const App = () => <regular-table></regular-table>;
  3. ReactDOM.render(<App />, document.getElementById("root"));
  4. ```

.setDataListener() Virtual Data Model


Let's start with with a simple data model, a two dimensional Array.  This one
is very small at 3 columns x 6 rows, but even for very small data sets,
regular-table won't read your entire dataset at once.  Instead, we'll need
to write a simple _virtual_ data model to access DATA and COLUMN_NAMES
indirectly.

  1. ```javascript
  2. const DATA = [
  3.     [0, 1, 2, 3, 4, 5],
  4.     ["A", "B", "C", "D", "E", "F"],
  5.     [true, false, true, false, true, false],
  6. ];
  7. ```

When clipped by the scrollable viewport, you may end up with a `
` of just
a rectangular region of DATA, rather than the entire set.  A simple viewport
2x2 may yield this `
`:

0A
1B

  1. ```json
  2. {
  3.     "num_rows": 26,
  4.     "num_columns": 3,
  5.     "data": [
  6.         [0, 1],
  7.         ["A", "B"]
  8.     ]
  9. }
  10. ```

Here's a an implementation for this simple _virtual_ data model,
the function getDataSlice().  This function is called by your
`` whenever it needs more data, with coordinate arguments,
(x0, y0) to (x1, y1).  Only
this region is needed to render the viewport, so getDataSlice() returns
this rectangular slice of DATA.  For the window (0, 0) to (2, 2),
getDataSlice() would generate an Object as above, containing the data slice,
as well as the overall dimensions of
DATA itself ( num_rows, num_columns), for sizing the scroll area.  To
render this virtual data model to a regular HTML ``, register this data
model via the setDataListener() method:

  1. ```javascript
  2. function getDataSlice(x0, y0, x1, y1) {
  3.     return {
  4.         num_rows: (num_rows = DATA[0].length),
  5.         num_columns: DATA.length,
  6.         data: DATA.slice(x0, x1).map((col) => col.slice(y0, y1)),
  7.     };
  8. }

  10. regularTable.setDataListener(getDataSlice);
  11. ```

This will render your regular HTML `
` ! Your DOM will look something
like this, depending on the size of your viewport.  Notice there are fewer rows
and columns in the resulting HTML, e.g. the column Column 3 (boolean) - as you
scroll, more data will be fetched from getDataSlice(), and parts of the
`
` will redrawn or extended as needed.

  1. ```html
  2. <regular-table>
  4.     <table>
  5.         <tbody>
  6.             <tr>
  7.                 <td>0</td>
  8.                 <td>A</td>
  9.             </tr>
  10.             <tr>
  11.                 <td>1</td>
  12.                 <td>B</td>
  13.             </tr>
  14.         </tbody>
  15.     </table>
  17. </regular-table>
  18. ```


virtual_mode Option


regular-table supports four modes of virtual scrolling, which can be
configured via the virtual_mode optional argument.  Note that using a
virtual_mode other than the default "both" will render the _entire_
`
` along the non-virtual axis(es), and may cause rendering performance
degradation.

"both" (default) virtualizes scrolling on both axes.
"vertical" only virtualizes vertical (y) scrolling.
"horizontal" only virtualizes horizontal (x) scrolling.
"none" disable all scroll virtualization.

  1. ```javascript
  2. table.setDataListener(listener, {virtual_mode: "vertical"})
  3. ```

Column and Row Headers


regular-table can also generate Hierarchial Row and Column Headers, using
``), or Row Headers (the first
children of each tbody tr), via the column_headers and row_headers
properties (respectively) of your data model's Response object.  This can be
renderered with column_headers, a two dimensional Array which must be of
length x1 - x0, one Array for every column in your data window.


` elements which layout in a `fixed` position within the virtual table.It can generate Column Headers (within the `
Column 1 (number)Column 2 (string)
0A
1B

  1. ```json
  2. {
  3.     "num_rows": 26,
  4.     "num_columns": 3,
  5.     "data": [
  6.         [0, 1],
  7.         ["A", "B"]
  8.     ],
  9.     "column_headers": [
  10.         ["Column 1 (number)"],
  11.         ["Column 2 (string)"]
  12.     ]
  13. }
  14. ```

  Hierarchial/Group Headers


`regular-table` supports multiple `` of ``, and also uses `colspan` and
rowspan to merge simple consecutive names, which allows description of simple
Row and Column Group Hierarchies such as this:


Colgroup 1
Column 1Column 2
Rowgroup 1Row 10A
Row 21B

  1. ```json
  2. {
  3.     "num_rows": 26,
  4.     "num_columns": 3,
  5.     "data": [
  6.         [0, 1],
  7.         ["A", "B"]
  8.     ],
  9.     "row_headers": [
  10.         ["Rowgroup 1", "Row 1"],
  11.         ["Rowgroup 1", "Row 2"]
  12.     ],
  13.     "column_headers": [
  14.         ["Colgroup 1", "Column 1"],
  15.         ["Colgroup 1", "Column 2"]
  16.     ]
  17. }
  18. ```

Note that in the rendered HTML, for these Row and Column Array,
repeated elements in a sequence will be automatically merged via rowspan and
colspan attributes.  In this example, e.g. "Rowgroup 1" will only output
to one `` node in the resulting ``.

metadata Data-Aware Styling


A dataListener may also optionally provide a metadata field in its response,
a two dimensional Array of the same dimensions as data.  The values in this
field will accompany the metadata records returned by regular-table's
getMeta() method (as described in the next section).

  1. ```json
  2. {
  3.     "num_rows": 26,
  4.     "num_columns": 3,
  5.     "data": [
  6.         [-1, 1],
  7.         ["A", "B"]
  8.     ],
  9.     "metadata": [
  10.         ["pos", "neg"],
  11.         ["green", "red"]
  12.     ],
  13. }
  14. ```

async Data Models


With an async data model, it's easy to serve getDataSlice() remotely
from node.js or re-implement the JSON response protocol in any language.
Just return a Promise() from, or use an async function as an argument to,
`setDataListener()`. Your `` won't render until the
Promise is resolved, nor will it call your data model function again until
the current call is resolved or rejected.  The following async example uses a
Web Worker, but the same principle applies to Web Sockets, readFile() or any
other asynchronous source.  Returning a Promise blocks rendering until the Web
Worker replies:

  1. ```javascript
  2. // Browser

  4. let callback;

  6. worker.addEventListener("message", (event) => {
  7.     callback(event.data);
  8. });

  10. regularTable.setDataListener((...viewport) => {
  11.     return new Promise(function (resolve) {
  12.         callback = resolve;
  13.         worker.postMessage(viewport);
  14.     });
  15. });
  16. ```

  1. ```javascript
  2. // Web Worker

  4. self.addEventListener("message", async (event) => {
  5.     const response = await getDataSlice.apply(null, event.data);
  6.     self.postMessage(response);
  7. });
  8. ```

.addStyleListener() and getMeta() Styling


`regular-table` can be styled trivially with just regular CSS for `
`.

  1. ```css
  2. // Zebra striping!
  3. regular-table tr:
  4.     background: rgba(0,0,0,0.2);
  5. }
  6. ```
However, CSS alone cannot select on properties of your _data_ - if you scroll
this example, the 2nd row will always be the striped one.  Some other
data-reliant style examples include:

* Styling a specific column in the virtual data set, as `
` may represent
  a different column based on horizontal scroll position.
Styling cells by value, +/-, heatmaps, categories, etc.
Styling cells based on data within-or-outside of the virtual viewport,
  grouping depth, grouping categories, etc.

To make CSS that is virtual-data-model-aware, you'll need to use
`addStyleListener()`, which invokes a callback whenever the `` is
re-rendered, such as through API invocations of draw() and user-initiated
events such as scrolling.  Within this optionally async callback, you can
select `
`, ``, etc. elements via regular DOM API methods like
querySelectorAll().

  1. ```javascript
  2. // Only select row_headers!
  3. table.addStyleListener(() => {
  4.     for (const th of table.querySelectorAll("tbody th")) {
  5.         style_th(th);
  6.     }
  7. });
  8. ```

Once you've selected the `		` and `` you want to paint, `getMeta()`
will return a MetaData record of information about the HTMLElement's
virtual position.  This example uses meta.x, the position in data-space,
to make virtual-scroll-aware zebra striping.

  1. ```javascript
  2. function style_th(th) {
  3.     const meta = table.getMeta(th);
  4.     th.classList.toggle("zebra-striped", meta.x % 2 === 0);
  5. }
  6. ```

  1. ```css
  2. .zebra-striped {
  3.     background-color: rgba(0,0,0,0.2);
  4. }
  5. ```

.invalidate()


To prevent DOM renders, `` conserves DOM calls like `offsetWidth`to an internal cache. When a `		` or ``'s `width` is modified within a callback to `.addStyleListener()`, you must indicate to `` that
its dimensions have changed in order to invalidate this cache, or you may not
end up with enough rendered columns to fill the screen!

A call to invalidate() that does not need new columns only imparts a small
runtime overhead to re-calculate virtual width per async draw iteration, but
should be used conservatively if possible.  Calling invalidate() outside of
a callback to .addStyleListener() will throw an Error.

  1. ```javascript
  2. table.addStyleListener(() => {
  3.     for (const th of table.querySelectorAll("tbody th")) {
  4.         th.style.maxWidth = "20px";
  5.     }
  6.     table.invalidate();
  7. });
  8. ```

.addEventListener() Interaction


`` is a normal `HTMLElement`! Use the `regular-table` API
in concert with regular DOM API methods that work on other HTMLElement
to create advanced functionality, such as this example of virtual row
select:

  1. ```javascript
  2. const selected_rows = [];

  4. table.addEventListener("mousedown", (event) => {
  5.     const meta = table.getMeta(event.target);
  6.     if (meta && meta.y >= 0) {
  7.         selected_rows.push(meta.y);
  8.         table.draw();
  9.     }
  10. });

  12. table.addStyleListener(() => {
  13.     for (const td of table.querySelectorAll("td")) {
  14.         const meta = table.getMeta(td);
  15.         td.classList.toggle("row-selected", selected_rows.includes(meta.y));
  16.     }
  17. });
  18. ```

Advanced examples can be found in the [examples](https://github.com/finos/regular-table/tree/master/examples)
directory, and in the [bl.ocks example gallery](https://github.com/finos/regular-table#examples).

Scrolling


Because of the structure of the HTML `` element, `
` elements must bealigned with their respective row/column, which causes default ``
to only be able to scroll in increments of a cell, which can be irregular when
column data is of different lengths.  Optionally, you may implement
_sub-cell scrolling_ in CSS via `` slotted CSS variables.
The provided material.css theme does exactly this, or you can implement this
in any custom style by importing the sub_cell_scrollling.css stylesheet
explicitly:

  1. ```html
  2. <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/regular-table/dist/css/sub-cell-scrolling.css">
  3. ```

Pivots, Filters, Sorts, and Column Expressions with perspective


regular-table is natively compatible with [perspective](https://github.com/finos/perspective/),
a WebAssembly streaming visualization engine.  By using a perspective.Table as a
Virtual Data Nodel, it becomes simple to achieve user-driven row and
column pivots, filters, sorts, and column expressions, as well as charts
and persistent layouts, from high-frequency updating data.


Development


First install dev_dependencies:

  1. ```bash
  2. yarn
  3. ```

Build the library

  1. ```bash
  2. yarn build
  3. ```

Run the test suite

  1. ```bash
  2. yarn test
  3. ```

Start the example server at [http://localhost:8080/examples/](http://localhost:8080/examples/)

  1. ```bash
  2. yarn start
  3. ```
<!--

Stats

npm bundle size
-->