[Python][Documentation] Add column limit recommendations Parquet page

[asfimport]

Users would be well advised to not write columns with large numbers (> 1000) of columns

Reporter: Wes McKinney / @wesm

_{Note: This issue was originally created as ARROW-8746. Please see the migration documentation for further details.}

[asfimport]

Maarten Ballintijn / @maartenb: @wesm, you've mentioned this before and as this is a not uncommon use-case can you maybe expand a bit on the following related questions. (use-cases include daily or minute data for a few 10's of thousands items like stocks or other financial instruments, IoT sensors, etc).

• Parquet Standard - Is the issue intrinsic to the Parquet standard you think? The ability to read a sub-set of the columns and or row-groups, compact storage through the use of RLE, categoricals etc, all seem to point to the format being well suited for these use-cases?
• Parquet-C++ implementation - Is the issue with current Parquet-C++ implementation, or any of the dependencies? Is it something which could be fixed? Would a specialized implementation help? Is the problem related to going from Parquet -> Arrow -> Python/Pandas? E.g. would a Parquet -> numpy reader work better?

• Alternatives - What would you recommend as a superior solution? Store this data tall i.s.o wide? Use another storage format?

  Appreciate your (and others) insights.

  Cheers, Maarten.

[asfimport]

Wes McKinney / @wesm: I appreciate the interest but perhaps let's discuss on the Parquet dev mailing list where more people can offer their advice and recommendations.

[asfimport]

Maarten Ballintijn / @maartenb: Redirected as requested.