[BUG] Workflows Scanner using SP Modernization Scanner Tool doesn't scan all sites from CSV

[VimmiRawat]

SharePoint Modernization Scanner tool doesn't scan all sites but doesn't throw any error either.

Issue : We provided a CSV with approx. 1,15,000 SharePoint sites and csv doesn't have any header. We started running the scanner , it completed without any issues and folder also got created ; but it didn't scan all the sites ; It just scanned 10,000 out if it and even if we check error file ; it shows another 2,000 records. This has been tried twice and we see the same behavior all the times

[jansenbe]

Please make your batches smaller + check the performance best practices: https://docs.microsoft.com/en-us/sharepoint/dev/transform/modernize-scanner-faq#whats-the-impact-of-running-a-scan-and-how-do-i-get-the-best-performance. Also consider turning on the "Don't use search" option on the last page of the scan wizard to further increase performance.

[VimmiRawat]

What data will I miss on to if we turn on "turning on the "Don't use search" option" ? Also what we should ideally mention for "No of Threads" ?

[jansenbe]

You'll miss Site views and site unique visitor data. Threads: 5 x processors you have on the machine...max 30

[jansenbe]

@VimmiRawat : were you able to scan your tenant using the given guidance? If so then please close this issue

[jansenbe]

@VimmiRawat : did you manage to get the scan working with above tips?

[VimmiRawat]

Hello,

I am working on it . Scanner actually takes a lot of time even for 25k sites . It’s approx. 1 day . Is it expected ?

Regards, Vimmi

From: Bert Jansen notifications@github.com Sent: Friday, December 11, 2020 9:07 AM To: pnp/sp-dev-modernization sp-dev-modernization@noreply.github.com Cc: Rawat, Vimmi vimmi.rawat@accenture.com; Mention mention@noreply.github.com Subject: [External] Re: [pnp/sp-dev-modernization] [BUG] Workflows Scanner using SP Modernization Scanner Tool doesn't scan all sites from CSV (#558)

This message is from an EXTERNAL SENDER - be CAUTIOUS, particularly with links and attachments.

---

@VimmiRawathttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_VimmiRawat&d=DwMCaQ&c=eIGjsITfXP_y-DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU&r=RxdAoBPFsyo_ysmJ6Zhb1B-Y7gjfjYc0tMXS6bZlNCM&m=LinKthmSD2nqZ2GaQVtdEr-YJcOENoZWbh2qUAGJSEc&s=LFtzWMTKFZk2h3SJl_51iDQF7ct4cb0hBg1psFnyomU&e= : did you manage to get the scan working with above tips?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_pnp_sp-2Ddev-2Dmodernization_issues_558-23issuecomment-2D742945531&d=DwMCaQ&c=eIGjsITfXP_y-DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU&r=RxdAoBPFsyo_ysmJ6Zhb1B-Y7gjfjYc0tMXS6bZlNCM&m=LinKthmSD2nqZ2GaQVtdEr-YJcOENoZWbh2qUAGJSEc&s=Lm7p_LVV8HzeCU6LcL6Q9uTmQdLk6JPjfl3cv06oXnQ&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AO54P62M3IFFRLKJ4SO6TJTSUGHVZANCNFSM4UJGY4JA&d=DwMCaQ&c=eIGjsITfXP_y-DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU&r=RxdAoBPFsyo_ysmJ6Zhb1B-Y7gjfjYc0tMXS6bZlNCM&m=LinKthmSD2nqZ2GaQVtdEr-YJcOENoZWbh2qUAGJSEc&s=ELu8vQ_ofGIdbNKVZ4Omt6QfkBxflfPVUF0lRt2Wpag&e=.

---

This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. Your privacy is important to us. Accenture uses your personal data only in compliance with data protection laws. For further information on how Accenture processes your personal data, please see our privacy statement at https://www.accenture.com/us-en/privacy-policy.

---

www.accenture.com

[jansenbe]

Yes, that's normal. It's however better to not run too large batches...25K sites is quite large.

[VimmiRawat]

But , with company of our size ( approx, 4,80,000 SPO sites ) ; it's difficult to run the scanner with just 10 k sites as a batch as it will take more than a month to complete. Should we run the batch of 10k on 5 servers simultaneously and change the thread count ? If yes, what should we keep the thread count if we are running it on 5 servers at the same time ? If not , then ; what is the best and fastest way to run the scanner for some 5,00,000 sites ? Note : We can't wait for a month's time for this.

[jansenbe]

First step is assuring you follow the earlier mentioned performance guidance:

Please make your batches smaller + check the performance best practices: https://docs.microsoft.com/en-us/sharepoint/dev/transform/modernize-scanner-faq#whats-the-impact-of-running-a-scan-and-how-do-i-get-the-best-performance. Also consider turning on the "Don't use search" option on the last page of the scan wizard to further increase performance.

Import is to only check the workflow scan option, definitely do not do a full scan. Also I assume you're not scanning the each user's OneDrive for Business site, makes no sense to do that with regard to workflow.

The scanner uses multi threading in a way that it splits the given set of sites over the available threads, if you've sites with a ton of sub sites these will take a very long time as they're processed by a single thread. How many sites to scan in one go really depends on your environment, but if you follow above rules you should be able to run the scan for fairly large batches of sites. You can consider running a number of scans in parallel from multiple machines, but let's first get one scan run to successfully complete.

[jansenbe]

Closing per no feedback anymore