virtual-scroll: integrate with relevant existing components

[mmalerba]

Integrate virtual-scrolling as an optional add-on for relevant existing components, including:

• [ ] Accordion
• [ ] Autocomplete
• [ ] Datepicker
• [ ] Grid List
• [ ] List
• [ ] Select
• [ ] Table
• [ ] Tree
• [ ] Drag & Drop

Part of each integration should include adding docs examples of how to set it up

[ahsan]

Hi, thanks for the awesome virtual scroll feature. Any ETA for mat-tree?

[shlomiassaf]

I managed to implement virtual scroll with the table, working pretty amazing out of the box with both fixed and auto strategies.

I used the original CdkVirtualScrollViewport component and provided my "version" of CdkVirtualForOf (which should actually be the CdkTable + measuring capabilities)

I added 1 more strategy - NoStrategy and built my own directives put on the table to control which strategy to use.

I had some issues, most of them were easily solved, the most difficult thing to expect is the handling of meta rows, which are header and footer rows.

Handling of headers and footers require special treatment because they are not part of the datasource and thus mess up the entire flow, once you have multiple header/footer rows it get's messy quickly.

I solved all issues with size measurements when working with multi-header/footer table.

The only issues i'm facing now is sticky rows, which does not work because the virtual table now has a container that "offsets" the sticky row so position top 0 is no longer 0 after it was translated by the parent.

Working around this should also be easy if I had access to the function that sets the offset - I dont because it's private in CdkVirtualScrollViewport

[kiwikern]

I think integrating it with the Grid list would also be a great feature.

[IlCallo]

@shlomiassaf can you share the code you used or try a PR? Virtual scroll is a thing in many are waiting to see integrated into MatTable component, a workaround while we wait for official support can be useful

[ahsan]

I needed Virtual Scroll in MatTree component, I hacked the MatList component to emulate behavior of MatTree as it already has the virtual scrolling capability. I did have to manage the indentations of individual nodes myself though. Maybe a similar sort of approach can be used for MatTable component as well.

[shlomiassaf]

@IlCallo It would be difficult to share it right now, I need to clean some IP stuff from there.

The virtual scroll is also a part of a table component (on top of CdkTable) so it has some things that will not work as they are part of that table eco-system, things like global configuration, plugin integration etc...

I can confirm that it works, and works quite fast! for both Auto and Fixed size strategies. I will try to extract it somehow, but I can't commit to a timeframe.

For now, I will try to describe how I did it:

This is my way of implementing it, there might be other ways!

First, the general layout we will use:

<cdk-virtual-scroll-viewport>
    <cdk-table></cdk-table>
</cdk-virtual-scroll-viewport>

The virtual scroll is external to the table.

Now we need to take care of 3 topics:

• Adjusting CdkVirtualScrollViewport to work with the table
• Providing our own version of CdkVirtualFor
• Adjusting strategies to work with the table

Adjusting CdkVirtualScrollViewport to work with the table

The general idea is to create a custom viewport component that inherits from CdkVirtualScrollViewport and apply minor adjustments so the table will work.

Our CdkVirtualScrollViewport will also control which strategy we use, replacing to our own table-targeted strategy when needed.

Providing our own version of CdkVirtualFor

We need to connect CdkVirtualScrollViewport with CdkTable, but CdkVirtualScrollViewport requires a CdkVirtualFor, which is a structural directive...

In general, CdkVirtualFor will render a subset (range) of rows from a DataSource and act upon changes in the datasource or range so it will always render the right subset.

CdkTable already does all of that, and some more...

We will use a simple class that mimics CdkVirtualFor while bridging the two components.

---

On a side-note, @mmalerba: CdkVirtualScrollViewport.attach(forOf: CdkVirtualForOf<any>): void; is probably narrow, forOf should proably be:

interface CdkVirtualScrollAdapter {
  dataStream: Observable<T[] | ReadonlyArray<T>>
  measureRangeSize(range: ListRange, orientation: 'horizontal' | 'vertical'): number;
}

---

Our CdkVirtualFor does 2 things:

Make sure sticky rows works

Because the table position is not static (transformed all over in the virtual viewport) the sticky rows will not stick... we need to compensate for the transformation so their top root following the virtual top root offset. We do that by listening to offset changes (from CdkVirtualScrollViewport) and update the sticky rows with the offset.

Adjust the range to compensate for header/footer rows

Because the table might contain Header or Footer rows, we need to adjust the range accordingly. The virtual viewport is wrapping both header/footer rows and the actual table rows...

Our CdkVirtualFor adapter listens to renderedRangeStream changes from the viewport, and pass it on the the table (via CdkTable.viewChange.next(range)).

The viewport calculates the range using the strategy and it will return how many rows to render. This number is then used to extract rows from the data source. If we have a header row in view we need to reduce the range.

Adjusting strategies to work with the table

FixedSizeVirtualScrollStrategy works fine without changes, it's AutoSizeVirtualScrollStrategy that we need to amend.

The problem is not really in AutoSizeVirtualScrollStrategy but in CdkVirtualScrollViewport and how it emits the initial range, this is a known issue - See this comment by @mmalerba

To fix it I use a custom strategy that wraps AutoSizeVirtualScrollStrategy and ItemSizeAverager . Basically, I use a TableItemSizeAverager that has access to the how many actual rows are rendered. When ItemSizeAverager.addSample() is called - if no rows are rendered it will use the default row height otherwise will work as is.

This could probably get solved differently, but because CdkVirtualScrollViewport has most of its logic methods private I had to go this way...

If this fix is not applied the average size will get very small values because it will get a large range of "rows" before rows are rendered... so the total height/rows will be small.

Hope it helps!!!

[shlomiassaf]

I managed to upload a small demo app I have for the table....

https://shlomiassaf.github.io/table-demo

Look at the "Demo" link on the left, it shows a large list with a virtual scroll (AUTO). You can set it to 100,000 rows about 23-24 columns.

The "All In One" link shows a virtual scroll with FIXED strategy.

It's a POC for all of you that want to use it.

Note that this is a quick and dirty demo app, expect bugs :)

[shlomiassaf]

OK, also managed to implement Drag and Drop using CdkDrag and my own version of CdkDropList.

See demo:

https://shlomiassaf.github.io/table-demo

It does both column and row d&d.

There is no "real" need to create a custom CdkDropList component, the material team just need to refactor it a bit so people can extend it (it's CDK after all)

Most of it is private and some functions are just big, if they port most to protected and split some functions (mainly _sortItem) I would be able to reuse it...

[CuriousDolphin]

it is possible to use virtual scroll with a grid list of element?.

[vrady]

OK, also managed to implement Drag and Drop using CdkDrag and my own version of CdkDropList.

See demo:

https://shlomiassaf.github.io/table-demo

It does both column and row d&d.

There is no "real" need to create a custom CdkDropList component, the material team just need to refactor it a bit so people can extend it (it's CDK after all)

Most of it is private and some functions are just big, if they port most to protected and split some functions (mainly _sortItem) I would be able to reuse it...

Hello @shlomiassaf , can you provide source code of your demo application? We need to figure out how to implement features that you show in demo. Thanks

[ghost]

I managed to upload a small demo app I have for the table....

https://shlomiassaf.github.io/table-demo

Look at the "Demo" link on the left, it shows a large list with a virtual scroll (AUTO). You can set it to 100,000 rows about 23-24 columns.

The "All In One" link shows a virtual scroll with FIXED strategy.

It's a POC for all of you that want to use it.

Note that this is a quick and dirty demo app, expect bugs :)

Hi,

It would be really great if you can share the code,

Thanks

[mmalerba]

@shlomiassaf Thanks for the detailed summary! That will be a great starting point for exploring integration with the table

[codestitch]

@shlomiassaf is there a way to check neg-table in your demo to experiment?

[nahgrin]

I got a basic version of virtual scroll working with the grid. I'll give a brief overview of what I did and try to come back and post a working example in a bit. I tried to follow what @shlomiassaf did and I ended up with a slightly different approach.

For the following code, I "borrowed" heavily from the material table examples. I'll try to describe what I did here and then just leave the code below to hopefully help answer any questions that my explanation leaves.

For the HTML, I wrapped the table element in the cdk-virtual-scroll-viewport as was suggested. However, I also had to modify the outlet for the row data so that it combined the cdkVirtualFor with the matRowDef. Instead of using the structural directive for the row, I expanded it out and kind of merged it with the cdkVirtualFor. Another important thing was that the datasource for the cdkVirtualFor is not the same one that is feeding the table. The rows observable is basically the true observable of the data in the grid while the dataSource observable is a filtered version of the rows for the table.

I created my own strategy for dealing with the virtual scroll in the table and it's mostly just an exceedingly simplified version of the FixedSizeVirtualScrollStrategy from @angular/cdk/scrolling. The reason I did this was that the FixedSizeVirtualScrollStrategy was producing some really weird rendering errors where the table would routinely display elements in the table off by a certain index. I think that it was causing the cdkVirtualFor and the mat-table to fight each other for rendering or something, but I'm not informed enough to say for sure. Other than that problem, the FixedSizeVirtualScrollStrategy can just be dropped in and will work without concern.

The component stitches the data in the table and the strategy together and creates the separate dataSource observable for the table. Every time that the index of the scroll is updated it modifies the slice of the array so that the table only renders the piece of the table that should be in view.

That's basically what I've done and it is working pretty well for me. If anyone has any insight into getting the FixedSizeVirtualScrollStrategy, that would be wonderful.

table.component.html

<cdk-virtual-scroll-viewport [style.height.px]="gridHeight">
  <table mat-table [dataSource]="dataSource" class="mat-elevation-z8">

    <ng-container matColumnDef="position">
      <th mat-header-cell *matHeaderCellDef> No. </th>
      <td mat-cell *matCellDef="let element"> {{element.position}} </td>
    </ng-container>

    <ng-container matColumnDef="name">
      <th mat-header-cell *matHeaderCellDef> Name </th>
      <td mat-cell *matCellDef="let element"> {{element.name}} </td>
    </ng-container>

    <ng-container matColumnDef="weight">
      <th mat-header-cell *matHeaderCellDef> Weight </th>
      <td mat-cell *matCellDef="let element"> {{element.weight}} </td>
    </ng-container>

    <ng-container matColumnDef="symbol">
      <th mat-header-cell *matHeaderCellDef> Symbol </th>
      <td mat-cell *matCellDef="let element"> {{element.symbol}} </td>
    </ng-container>

    <tr mat-header-row *matHeaderRowDef="displayedColumns"></tr>
    <ng-template let-row matRowDef cdkVirtualFor [matRowDefColumns]="displayedColumns"[cdkVirtualForOf]="rows">
      <tr mat-row></tr>
    </ng-template>
  </table>
</cdk-virtual-scroll-viewport>

table-vs-strategy.service.ts

@Injectable()
export class TableVirtualScrollStrategy implements VirtualScrollStrategy {

  private scrollHeight!: number;
  private scrollHeader!: number;
  private readonly indexChange = new Subject<number>();

  private viewport: CdkVirtualScrollViewport;

  public scrolledIndexChange: Observable<number>;

  constructor() {
    this.scrolledIndexChange = this.indexChange.asObservable().pipe(distinctUntilChanged());
  }

  public attach(viewport: CdkVirtualScrollViewport): void {
    this.viewport = viewport;
    this.onDataLengthChanged();
    this.updateContent(viewport);
  }

  public detach(): void {
    // no-op
  }

  public onContentScrolled(): void {
    this.updateContent(this.viewport);
  }

  public onDataLengthChanged(): void {
    this.viewport.setTotalContentSize(this.viewport.getDataLength() * this.scrollHeight);
  }

  public onContentRendered(): void {
    // no-op
  }

  public onRenderedOffsetChanged(): void {
    // no-op
  }

  public scrollToIndex(index: number, behavior: ScrollBehavior): void {
    // no-op
  }

  public setScrollHeight(rowHeight: number, headerHeight: number) {
    this.scrollHeight = rowHeight;
    this.scrollHeader = headerHeight;
    this.updateContent(this.viewport);
  }

  private updateContent(viewport: CdkVirtualScrollViewport) {
    const newIndex = Math.max(0, Math.round((viewport.measureScrollOffset() - this.scrollHeader) / this.scrollHeight) - 2);
    viewport.setRenderedContentOffset(this.scrollHeight * newIndex);
    this.indexChange.next(
      Math.round((viewport.measureScrollOffset() - this.scrollHeader) / this.scrollHeight) + 1
    );
  }
}

table.component.ts

@Component({
  providers: [{
    provide: VIRTUAL_SCROLL_STRATEGY,
    useClass: TableVirtualScrollStrategy
  }],
  ...
})
export class TableComponent implements OnInit {

  // Manually set the amount of buffer and the height of the table elements
  static BUFFER_SIZE = 3;
  rowHeight = 48;
  headerHeight = 56;

  rows: Observable<Array<any>> = of(new Array(1000).fill({position: 1, name: 'Hydrogen', weight: 1.0079, symbol: 'H'}));

  displayedColumns: string[] = ['position', 'name', 'weight', 'symbol'];

  dataSource: Array<any>;

  gridHeight = 400;

  constructor(@Inject(VIRTUAL_SCROLL_STRATEGY) private readonly scrollStrategy: TableVirtualScrollStrategy) {}

  public ngOnInit() {
    const range = Math.ceil(this.gridHeight / this.rowHeight) + TableComponent.BUFFER_SIZE;
    this.scrollStrategy.setScrollHeight(this.rowHeight, this.headerHeight);

    this.dataSource = combineLatest([this.rows, this.scrollStrategy.scrolledIndexChange]).pipe(
      map((value: any) => {

        // Determine the start and end rendered range
        const start = Math.max(0, value[1] - GridComponent.BUFFER_SIZE);
        const end = Math.min(value[0].length, value[1] + range);

        // Update the datasource for the rendered range of data
        return value[0].slice(start, end);
      })
    );
  }
}

[garrettld]

@nahgrin I hope you don't mind, I've thrown your code into StackBlitz so that I could play around with it.

https://stackblitz.com/edit/nahgrin-virtual-scroll-table

There was a tiny bit of cleanup, but it pretty much just worked. Thanks so much!

[nahgrin]

@garrettld Awesome! Thanks a bunch for doing that!

[shlomiassaf]

@nahgrin Nice work!

The problem is in the header/footer rows, you need to take them into account when you calculate the range from the data source.

For example, if I have 5 header rows and 1000 items. Let's say I can fit 10 rows in my viewport and for simplicity, there is no buffer.

When i'm in 0 scroll offset I can't return 10 rows because 5 are used by headers... same goes for footer rows.

You need to calculate the header/footer rows and their visible height in the view, remove that and use the height left to calculate what is the actual range.

Here's an example with 5 header rows that breaks it:

https://stackblitz.com/edit/nahgrin-virtual-scroll-table-wdlqlm?file=src%2Fapp%2Ftable%2Ftable.component.ts

Of course there are more things to take care of, sticky rows and AutoSize strategy...

You might want to refactor TableVirtualScrollStrategy into a directive. You can put that directive on the table and then inject the table to it. You will have access to the table instance and the CdkVirtualScrollViewPort instance.

On that directive you can define all the heights... and you can also use a special input for the datasource, the directive will take that datasource and assign it to the table so it's a real plugin.

[ghost]

@nahgrin Tremendous work. Thanks for your efforts.

[nahgrin]

@shlomiassaf Good point about the headers. I tooled around a little more and moved some of the logic in towards the strategy (and removed a little bit of the hardcoding) and got a solution for headers:

https://stackblitz.com/edit/nahgrin-virtual-scroll-table-cvxa7v

I'm starting to get a distinct feeling that I'm reinventing the wheel, though, with the strategy. The FixedSizeVirtualScrollStrategy does basically everything the table needs, it's just off by a little bit off because of the headers. @mmalerba Would it be possible to add an offset field to the FixedSizeVirtualScrollStrategy for these sorts of use cases?

I'm not sure about sticky headers, but after tooling around a little and looking at the earlier example table, I'd imagine the table needs to apply a reverse transform to the headers to keep them pinned to the top. @shlomiassaf Did you use the display: flex table to specifically fix any of these problems?

[shlomiassaf]

@nahgrin For sticky you just need to compensate for the transformation done on cdk-virtual-scroll-content-wrapper.

For header rows, you just add the CSS property top with the negative value of the transform.
So if the transform is translateY(5196.05px) you need to set the css top: -5196.05px on each header.

For footer rows, you just add the CSS property bottom with the value of the transform.
So if the transform is translateY(5196.05px) you need to set the css bottom: 5196.05px on each footer.

[shlomiassaf]

@nahgrin I have to say that i'm still quite confused.

I believe that there are 2 rendering "engines" running.

1) The CdkTable 2) The CdkVirtualFor

    <ng-template let-row matRowDef cdkVirtualFor [matRowDefColumns]="displayedColumns" [cdkVirtualForOf]="rows">
      <tr mat-row></tr>
    </ng-template>

In this section of your template, you are passing a row template to CdkTable which will render it for every row in range.

But, you are also passing a template to CdkVirtualFor which will render it for every row in range.

The rendering of CdkTable will also add cells, the CdkVirtualFor rendering will just render an empty <tr>...

For every change in range the view port will notify both of them, causing a redundant diff operation in CdkVirtualFor.ngDoCheck() which is followed by a DOM update.

You can put breaking points in CdkVirtualFor and see how it works when you scroll and/or load a data source.

I'm not sure this design will scale...

This is why I built my own version of CdkVirtualFor so it won't double the work.

[danzrou]

Anyone found any workaround for using it inside <select> ?

[nahgrin]

@shlomiassaf I took a look at things and pulled the CdkVirtualFor out of the implementation. The only thing that I was using it for was to get the length of the data set, so it doesn't really do a lot for the virtual scrolling. It cleaned up the implementation in the table a little bit (and removed the need for the table component to know anything about the virtual scrolling), but I'm not seeing a lot of changes to the performance.

I created a separate stackblitz for comparison: https://stackblitz.com/edit/nahgrin-virtual-scroll-table-hampgc

[shlomiassaf]

You won't see it in this table...

If you have a huge one, with a lot of columns and rich content it might appear, anyway it wasn't supposed to be used like that so it's good you removed it.

The new implementation is much better, but you end up with something that will cause pain in the future.

You created a custom implementation for the scroll strategy that is completely different from the core strategies (fixed and auto-size).

When auto-size lands it will be difficult to use because you will have to again, rebuild it from scratch based on the cdk source code instead of just inheriting it and fixing here and there.

The TableVirtualScrollStrategy you built is actually a mix of the cdkVirtualFor and the FixedSizeVirtualScrollStrategy take code from both classes and mixing it up into one class.

Again, this is limiting because there are other strategies and you don't want to couple the cdkVirtualFor with it. Moreover, you don't want to re-write logical code written by others, if the logic changes you will need to follow.

Another thing I noticed is that the view-port is not used as intended, I didn't see any call to attach on the view port. It works but probably because you did things done in viewport.attach internally in the service...

[BenLayet]

@shlomiassaf I aggree with you, we need to

1. reuse viewport.attach
2. and not be tied to cdkVirtualFor

and the only clean solution for this is what you suggested earlier to @mmalerba:

CdkVirtualScrollViewport.attach(forOf: CdkVirtualScrollAdapter <any>):

interface CdkVirtualScrollAdapter {
  dataStream: Observable<T[] | ReadonlyArray<T>>
  measureRangeSize(range: ListRange, orientation: 'horizontal' | 'vertical'): number;
}

I will send a pull request for this.

[BenLayet]

@shlomiassaf @nahgrin I have sent the PR to add the scroll adapter: #14287, but the PR checks fail for an unrelated reason...

[jacob-8]

I managed to upload a small demo app I have for the table.... https://shlomiassaf.github.io/table-demo

@shlomiassaf If it's not too much trouble for you, I'd like to chime in with the others that it'd be so neat to see the code running your table-demo. Thanks!

[sebastien-savalle]

@ben-henoida Do you have a working example using the scroll adapter ? @shlomiassaf : Can you share the code of your table demo ?

[iSerganov]

Hello gentlemen. Could you please let me know whether there are plans to make virtual scroll with server side pagination (dataSource) in nearest future? If no, could you please advise on possible workarounds? Thank you in advance.

[shlomiassaf]

Guys, extracting the code is a pain! sorry! I can't commit to a timeframe here, it's just a hell lot of work.

@mmalerba @andrewseguin

It seems that sticky positioning with virtual scroll is a HUGE pain! I'v managed to position it correctly but on fast scrolling (whee, touchmove) it will go out of bounds and return once the viewport hit's a new update... making the sticky rows flicker...

It's not that simple to tame...

[IlCallo]

@ben-henoida Any advancements on the PR?

[IlCallo]

@nahgrin I played a bit with your last stackblitz and probably will do more in some weeks. There seem to be some problems with sticky headers after a certain scroll threshold and they start moving to the bottom randomly.

[vytautas-pranskunas-]

Any progress on this?

[temaisgod]

Hi!

first thanks for this POST. Im create my Virtual Scroll Table with yours code and it's works!

But i have a problem:

My table has a filter by id input text. Im using MatTableDataSource Object in my 'row' Observable . The filter change the value of MatTableDataSource but only refresh data in the table when I move scroll bar.

my question is:

¿How could I bind DATA REFRESH to filter KEY UP ?

Thanks!

[temaisgod]

Hi!

first thanks for this POST. Im create my Virtual Scroll Table with yours code and it's works!

But i have a problem:

My table has a filter by id input text. Im using MatTableDataSource Object in my 'row' Observable . The filter change the value of MatTableDataSource but only refresh data in the table when I move scroll bar.

my question is:

¿How could I bind DATA REFRESH to filter KEY UP ?

Thanks!

Im resolve this problem using BehaviorSubject Object instead of Obervable in my variable data.

When apply filter, I call to .next() method with the data for this object emmit changes to table.

setData(){
     this.misDatos = new BehaviorSubject<Array<any>>(this.dataSource.filteredData);
     this.rows = this.misDatos.asObservable();
    this.misDatosObservable = combineLatest([this.rows, this.scrollStrategy.scrolledIndexChange]).pipe(
      map((value: any) => {

        // Determina el rango a extraer del array
        const start = Math.max(0, value[1] - TableComponentVS.BUFFER_SIZE);
        const end = Math.min(value[0].length, value[1] + this.range);

        // Extrae el rango
        return value[0].slice(start, end);
      })
    );
}

applyFilter(value: string) {
    this.dataSource.filter = value;
    this.misDatos.next(this.dataSource.filteredData);
    this.scrollStrategy.onContentScrolled();
}

[CRACKISH]

When is feature will be in box?

[Nevaan]

Hi, Are there any plans of integration with mat-autocomplete, or anyone found any working solution for issue #13958 ?

[rabelloo]

I just spent a couple days hacking at this, after reading through all the amazingly helpful code and information here, and I just feel like I've reached an impasse.

I was able to successfully integrate mat-table and cdk-virtual-scroll-viewport, building off some of my old code and some of @nahgrin's. However, as @shlomiassaf pointed out, sticky headers are definitely an issue.

Here's my stackblitz for anyone interested: https://stackblitz.com/edit/mat-table-virtual-scroll

My implementation supports column filters, sorting, pagination and virtual-scroll.

I did rewrite the whole DataSource instead of extending from MatTableDataSource because I felt limited with the filters at one point, and my CoreTableVirtualScrollStrategy, like @nahgrin's, could also extend FixedSizeVirtualScrollStrategy, but as others said there's no point in doing so if they could all potentially change when getting the requested feature.

There's also one dirty instance of component composition by extending a CoreTable class, which I'm sure could be made better by a crazy complex component with TemplateRefs, structural directives and all that fun stuff.

Anyway, here's my problem with sticky headers: One can definitely calculate the offset position of the viewport and compensate with transform: translateY() on the <th> elements. However, flickering still happens and I'm not sure it can be avoided at all due to how the browser renders.

You can check my approach in example-table.component.ts, but essentially you subscribe to Viewport changes and do the math (I used a fixed magic number just for the purposes of testing this one out). It works well until the point where the container transform starts to mismatch the <th>'s, which is when flickering starts. Mind you, the header stays positioned correctly after scrolling (or it would, if you actually coded the math), but only after some delay.

I've seen other approaches to VirtualScrolling in tables like ag-grid and @shlomiassaf's table demo, but they all forego the <table> elements in favor of easily controllable <div>s which can render the header outside of the viewport - if only we could have tbody > viewport > tr...

Which finally brings me to my impasse: it seems like we can either care for HTML semantics and struggle with presentation, or just go for the easy and smooth implementation instead.

I even tried a different approach without cdk-virtual-scroll-viewport where the height of the <table> element would stay dynamically constant - if that makes sense - by using :after and :before pseudo elements that would resize according to scroll position, which involved about the same amount of math as the previously mentioned approach, but to no avail, flicker all the same - even worse I'd say.

Hopefully someone out there will be able to work this out, either with CSS or JS, maybe I'll even try again in the future. For now I'm sticking with the slightly less user-friendly approach of paginating. Sticky headers with filters really are a must at my current project.

Anyway, I just felt like sharing my experience and my code in hopes it further helps people that come across this issue, like how other comments before mine helped me.

tl;dr: Could make mat-table virtually scrollable, but sticky headers flicker :/

[desdmit]

Hi, I was able to implement sticky header with virtualization and infinite-scroll: https://stackblitz.com/edit/nahgrin-virtual-scroll-table-3tx3mt

Any thoughts?

Thanks to @nahgrin

[rabelloo]

@desdmit I don't mean to rain on your parade, but this helps illustrate that we'd have to forego HTML semantics if we wanted an easy way of implementing sticky headers. Not using the <table> elements also means column width is determined by flex and therefore they are all equally sized instead of adjusting to content, in which case I'd rather use a CSS grid instead.

Also, there's no need to use [cdkVirtualFor], [cdkTable] should be enough as a rendering engine.

I have forked my own code to incorporate your changes, as a proof of concept. https://stackblitz.com/edit/mat-table-virtual-scroll-div

Still hoping that we get official support on this one.

[ThisIsIvan]

@rabelloo Thanks a lot for the implementation! It works great.

There are some TypeScript errors, which are easily fixable. Apart from that there is a mistake in the select toggling of items. When the table is initialised with a large data set not all are selected after hitting the select all checkbox. That's fixable by replacing this.data with this.allData in data-source.ts::toggleAll() and data-source.ts::selectedAll().

The initialisation of the table with a large data set still takes a while unfortunately.

[rabelloo]

@ThisIsIvan That was actually a deliberate decision to select only visible items. It's rather controversial which behavior is expected by users, but we found that to be more intuitive, at least when paginating.

Still this stackblitz version has some flaws, like when toggling all on a page, then trying to toggle all on a diferent page, but I wasn't trying to reach for completion.

Anyway, glad you could easily alter it to your liking.

[shlomiassaf]

Hi guys,

Yes, sticky headers are an issue and probably will be due to the way they are implemented.

I did a lot of things to get around it and I was able to reduce it to minimal using some tricks and I thinks its ok now...

I also think most users will want the headers outside of the scrolling area so I support both mods as shown here: https://shlomiassaf.github.io/table-demo/table-demo#/features/sticky

As for <table> vs <div> I chose <div> because table might limit in complex scenarios / features.

By default all cells do get the same % but that's because mat-table doesn't have any size mechanism to control that.

I added that, and its possible to specify minWidth, maxWidth in absolute pixels and width in % or px and if not set the width will behave like in mat-table. This actually works great!

For example, with 10 columns, if minWidth: 150 is set for each we get 1500px minimum. If the table's width is 1000 the user will get 500 px HZ scrolling...

The user can choose his strategy here which I find great when I use the table...

In the demo site, there are actual demos (dedicated) with an action-bar that allow changing the width strategy

The menus just call table API's to make it work..

So <div> works OK I guess :)

[lujian98]

@rabelloo Thank you for the great implementation. One question, is possible to use MatTableDataSource instead of use your custom CoreTableDataSource?

[rabelloo]

@lujian98 Glad you liked it! Well, technically yes, but:

• For virtual scrolling you'd have to provide MatTableDataSource a MatPaginator instance and keep feeding it page index and size values (size can be constant);
• For this kind of custom filtering you'd have to provide your own filterPredicate and trigger emission by changing the filter property. I used to simply invert the value e.g. filter = `${filter === 'false'}`

Overall not too complex, just a little bit hacky which is why I chose to write my own, especially since I'm going to be working on more advanced filters in the future like Date, "contains in list of possible values", boolean and enumerable icons, etc.

You'd still want some kind of abstraction, probably, which I have in the form of CoreTable. That's the part of my code that I like the least, with how you extend from it and some things are just magic, but it definitely makes it pretty easy to reuse. Feel free to write a component, directive, or whatever you think best to replace it.

[lujian98]

Thank you @rabelloo The MatTableDataSource cannot be extends since the connect() { return this._renderData; } is different from the CoreTableDataSource connect() { return this.visibleData; }, which give the error message. Not sure why cannot override the parent function.

I may have to create my own TableDataSource. It's bad the Angular Material datatable not support virtual-scroll.

[rabelloo]

@lujian98 Oh I thought you meant to use a MatTableDataSource instance instead of a CoreTableDataSource instance, which you can definitely do.

If you mean to alter CoreTableDataSource and extend MatTableDataSource, you can remove most of the code - like filtering and pagination, rename conflicting properties that are left over and then implement the filter and MatPaginator overhaul that I mentioned.

[lujian98]

Thank you, @rabelloo Based on your code, I made a simplify code for virtual scroll only with Angular Material data table. So main issue here will be custom tableDataSource. Like what you did, build-in sort, filter will need to be added. For the reference: https://github.com/lujian98/Angular-Material-Virtual-Scroll

[ThisIsIvan]

@rabelloo Do you see a way to improve the initialisation speed of the table given a large initial data set? I have a list with multiple thousand entries. Your virtual scroll helps immensely in improving the scrolling speed, but initialising the table still takes a while.

[rabelloo]

@ThisIsIvan I think it comes down to memory allocation, so other than incrementally loading the data instead of all of it at once, I don't know if much can be done.

It shouldn't be too hard to subscribe to the viewport's observables like renderedRangeStream and fetch the next batch when getting close to the list's bottom.

If you want to get quick and dirty though, you can just setup an interval and load your batches that way, e.g. https://stackblitz.com/edit/mat-table-virtual-scroll-dirty-load?file=src/app/app.component.ts

I would recommend fetching on request tho, i.e. when the user scrolls to the bottom.