search

microsoft / sql-server-samples

Azure Data SQL Samples - Official Microsoft GitHub Repository containing code samples for SQL Server, Azure SQL, Azure Synapse, and Azure SQL Edge
Other
10k stars 8.86k forks source link

WWI Generate data takes days #840

Open petrheckoHims opened 4 years ago

petrheckoHims commented 4 years ago

I've been trying to generate the WWI data as per the documentation. The docs says it takes around 10 minutes to generate 1 year of data, however, even with 5 orders per day, it took ~25 hours to generate the data from June 1, 2016 till today. It was generated from SSMS running on Azure VM and the target DB was running on Azure SQL Server. I was trying also 1000 orders per day and that would be taking around 4 days to load all the data.

Example screenshot - data loaded from Jun 2016 till Nov 2018 after 16 hours of query running:

image

Is this expected? Any way we can load the data faster, or any way we can only load 2020 year?

dmitriysolovev commented 3 years ago

Seems it is necessary to enforce delayed durability... Forcing delayed durability on the database may improve data generation speed, particularly when the database transaction log is on a high latency storage subsystem. Be aware of potential data loss implications when using delayed durability, and consider only enabling delayed durability for the duration of data generation.