timvink / mkdocs-table-reader-plugin

MkDocs plugin that enables a markdown tag like {{ read_csv('table.csv') }} to directly insert various table formats into a page
https://timvink.github.io/mkdocs-table-reader-plugin/
MIT License
124 stars 20 forks source link

Add support for multiple CSVs #40

Closed LukeEvansTech closed 1 year ago

LukeEvansTech commented 1 year ago

I'm looking to add support for multiple CSVs for this plugin.

I don't have a great deal of experience in Python so I have just taken the existing CSV function and extended it!

Would love some feedback if this approach works

timvink commented 1 year ago

Hi Luke,

So, you're joining all the tables together into a single table (df = pd.concat(df_list)) and then displaying it. This approach has some downsides:

I learned than whenever the implementation becomes awkard, it's often better to re-design it.

In this case, I would argue that we are going into table preprocessing rather than reading. Mkdocs 1.4 adds the hooks feature, which makes it easy to run any python script at mkdocs serve or mkdocs build.

For this specific use-case, I would write a hook that reads the tables of interest, combines them, writes them back (f..e into a .gitignoreed folder).

I will update the documentation of table-reader adding an example of how to use hooks to do data preprocessing.

As per contributing guidelines, I will close this PR as it won't be merged. Hope you didn't put in too much work. For next time, it's better to open an issue and discuss if you want to avoid 'wasted' work.

LukeEvansTech commented 1 year ago

No worries and thank you for the response.

I certainly take onboard the point about how we would do it for the multiple readers as I was thinking the same when submitting the PR.

Appreciate the input on the hooks feature, I had no idea that this was available. If you were able to provide an example, then I'm sure I could take it forward after that 😃

timvink commented 1 year ago

FYI, the docs now have a new 'how to' section that includes a bit on 'preprocessing tables': https://timvink.github.io/mkdocs-table-reader-plugin/howto/preprocess_tables/