Make the Tabulator functionality of the Frictionless Framework a separate library on which Framework depends

[mcarans]

Overview

I have seen that the functionality of this library is being moved into the Frictionless Framework. Why couldn't this library be kept as a separate library on which the Frictionless Framework depends?

---

Please preserve this line to notify @roll (lead of this repository)

[roll]

Hi @mcarans,

Thanks for your question. It's a kinda long story but I'll try to keep it short.

There are different aspects of why we decided to change the way we develop our libraries:

• the main thing is the administration costs. Having limited resources we ended up maintaining around 10-20 Python libraries which basically do one thing (read/write/validate/etc data) but we had to manage all of them separately. It was really ineffective for us
• technically such a layered system wasn't really performant. It didn't affect tabulator but our other libraries were suboptimal performance-wise just because we had to maintain all the logical layers we had. It had too many abstraction layers. It was also a maintenance problem

For example, currently frictionless.Table uses around 80% of Frictionless Framework concepts I mean it's like tabulator.Table but also supports schema inferring and data casting using tableschema.Schema, it supports unified error messages from goodtables, it supports Table Dialect from datapackage etc. Basically, it's not possible to extract this functionality from Frictionless just because it will be almost the whole library :smiley: It's a design decision to make things more coupled as I think it models the reality better and it's easier to use and work on.

So what are your doubts regarding using frictionless as a basically drop-in replacement of tabulator? The new library is event more lightweight dependencies wise because it uses a plugin system. Yes, it has some additional functionality comparing to tabulator but it will not affect the Table class users.

[mcarans]

Hi @roll. Thanks for your reply. If I was only interested in its Tabulator functionality, what would be the minimum set of dependencies that I could bring in with Frictionless Framework?

[roll]

@mcarans By default, Frictionless comes with support for CSV, Excel, and JSON formats. All the plugins are for supporting other formats or integrations e.g. like SQL. You can check it out here - https://github.com/frictionlessdata/frictionless-py/blob/master/setup.py#L42-L78

I would be happy to drop a few minor dependencies though additionally.

[mcarans]

That sounds good, thx @roll.

[roll]

Cool, thanks. :+1:

Really looking forward to your feedback and please let me know if something missing in Frictionless you need comparing to tabulator.