search

microsoft / azuredatastudio

Azure Data Studio is a data management and development tool with connectivity to popular cloud and on-premises databases. Azure Data Studio supports Windows, macOS, and Linux, with immediate capability to connect to Azure SQL and SQL Server. Browse the extension library for more database support options including MySQL, PostgreSQL, and MongoDB.
https://learn.microsoft.com/sql/azure-data-studio
MIT License
7.59k stars 908 forks source link

[Feature Request] For flat file import, allow to change the default number of rows to use for data type inference #19829

Open rrmistry opened 2 years ago

rrmistry commented 2 years ago

Currently, the data types are automatically inferred based on the first 50 rows:

image

It would be great if the same import wizard would let you choose how many rows to use for data type inference.

At least to allow us to change to a different value (with warning if choosing too many rows that it would cause slowdown)

This is much needed feature on what is already a great product.

Thank you team for all your hard work and efforts 🙏🙏🙏

Galaeci commented 1 year ago

As it stands the feature is pretty much useless for any real work. My team deals on a regular basis with the need to load csv files with hundreds of thousands of rows and this wizard, has never given us a acceptable result upon first run. We ended up using a combination of PowerQuery in Excel to profile the rows (which lacks information about the max length of a string) and importing the data directly with Import-DbaCSV -AutoCreateTable which defines every row as VARCHAR(MAX) but then we can use a custom defined ProfileTable SP to figure out the length and types of the different columns. So, yes, adding this parameter to the wizard would greatly increase productivity for my team and I'm sure others.