`from_pandas_dataframe` is very slow

[kerim371]

Hi,

Thank you for the library, it is very easy to use!

I'm trying to write pandas.Dataframe with n*1000 rows using the following code:

table = TableUtil.from_pandas_dataframe(df, flexible_column_width=False, font_size=Decimal(8), round_to_n_digits=2)

but it is incredibly slow.

I added here a print(i) and I noticed that it starts pretty fast but it becomes slower and slower and after 100 iterations it becomes unbearably slow.

Is there a way to increase the speed of table creation?

[jorisschellekens]

You can try:

• profiling the code to figure out where time is being consumed
• splitting the dataframe (for instance per 100 rows) and then simply adding a separate Table for each group of 100

I'm curious about the first option though. I wonder why the time complexity would not behave linearly.

Kind regards, Joris Schellekens

[kerim371]

@jorisschellekens hi,

Thank you for the response.

Splitting the dataframe works and that is the current solution that I use.

About the profiling, I have fihured out that the slowest part is t.add(p). Haven't done any further investigations.

Is it possible to make a list of cells and then add those cells at once? Like:

p_list = [p1, p2, p3,...]
t.add(p_list)

[jorisschellekens]

You can certainly create a list of LayoutElement objects ahead of time, but there is no method to add them all in bulk.

[kerim371]

Thank you for response. Ok, splitting dataframe works pretty good. That is enough for the now.