search

dotnet / Nerdbank.GitVersioning

Stamp your assemblies, packages and more with a unique version generated from a single, simple version.json file and include git commit IDs for non-official builds.
https://www.nuget.org/packages/Nerdbank.GitVersioning
MIT License
1.36k stars 167 forks source link

IOException when building from mapping pack files into memory #584

Closed ssa3512 closed 3 years ago

ssa3512 commented 3 years ago

Several developers reporting and at least one CI build failed. Rolled back to 3.3.37 for now.

System.IO.IOException: Not enough memory resources are available to process this command.

   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.MemoryMappedFiles.MemoryMappedView.CreateView(SafeMemoryMappedFileHandle memMappedFileHandle, MemoryMappedFileAccess access, Int64 offset, Int64 size)
   at Nerdbank.GitVersioning.ManagedGit.GitPack..ctor(GetObjectFromRepositoryDelegate getObjectFromRepositoryDelegate, String indexPath, String packPath, GitPackCache cache)
   at Nerdbank.GitVersioning.ManagedGit.GitPack..ctor(GitRepository repository, String name)
   at Nerdbank.GitVersioning.ManagedGit.GitRepository.LoadPacks()
   at System.Lazy`1.CreateValue()
   at System.Lazy`1.LazyInitValue()
   at System.Lazy`1.get_Value()
   at Nerdbank.GitVersioning.ManagedGit.GitRepository.TryGetObjectBySha(GitObjectId sha, String objectType, Stream& value)
   at Nerdbank.GitVersioning.ManagedGit.GitRepository.GetObjectBySha(GitObjectId sha, String objectType)
   at Nerdbank.GitVersioning.ManagedGit.GitRepository.GetCommit(GitObjectId sha, Boolean readAuthor)
   at Nerdbank.GitVersioning.ManagedGit.GitRepository.GetHeadCommit(Boolean readAuthor)
   at Nerdbank.GitVersioning.Managed.ManagedGitContext..ctor(String workingDirectory, String dotGitPath, String committish)
   at Nerdbank.GitVersioning.GitContext.Create(String path, String committish, Boolean writable)
   at Nerdbank.GitVersioning.Tasks.GetBuildVersion.ExecuteInner()
   at MSBuildExtensionTask.ContextAwareTask.Execute()
   at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
   at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__26.MoveNext()
AArnott commented 3 years ago

@qmfrederik Could this be because of trying to map a single, very large pack file? Or is this due to a memory leak in the object cache you build up?

qmfrederik commented 3 years ago

Thanks, for reporting, @ssa3512 . Could you share the size of your pack files (i.e. .git\objects\pack\*.pack)?

Do you know whether MSBuild is running as a 32-bit or 64-bit process?

ssa3512 commented 3 years ago

@qmfrederik here is from the CI server that failed. Working on getting from the users that also reported issues.

The CI build was set to MSBuild x86 - I am assuming this controls what it runs with. As for the users, they are using Visual Studio 2019 16.8 or newer. Is there a setting somewhere that would control 32 vs 64 bit for building?

image

Directory of F:\agent\ADO-1\_work\2\s\.git\objects\pack

04/09/2021  08:00 AM    <DIR>          .
04/09/2021  08:00 AM    <DIR>          ..
02/19/2021  09:00 AM            34,616 pack-030521c82632a3cbca431a40f4201c8763c91cee.idx
02/19/2021  09:00 AM         3,275,714 pack-030521c82632a3cbca431a40f4201c8763c91cee.pack
03/18/2021  08:00 AM            34,224 pack-09713bb4d25f255eefdd25ee53a9965acb71e5ce.idx
03/18/2021  08:00 AM         2,966,240 pack-09713bb4d25f255eefdd25ee53a9965acb71e5ce.pack
02/18/2021  09:00 AM            19,020 pack-0dccb0a4290da5559f7cd6e706accf4867bdd15b.idx
02/18/2021  09:00 AM         1,065,820 pack-0dccb0a4290da5559f7cd6e706accf4867bdd15b.pack
02/01/2021  09:00 AM            45,816 pack-1291fa4212f89bb3bb809d47d6951af9fd6b3baf.idx
02/01/2021  09:00 AM         3,334,810 pack-1291fa4212f89bb3bb809d47d6951af9fd6b3baf.pack
03/25/2021  08:00 AM            44,836 pack-1b9c5580e5b8661519ddf6885a10b22a049be859.idx
03/25/2021  08:00 AM         3,419,913 pack-1b9c5580e5b8661519ddf6885a10b22a049be859.pack
04/08/2021  08:00 AM            47,832 pack-251aef105561f2856da25e4577018d30b459b769.idx
04/08/2021  08:00 AM         4,338,411 pack-251aef105561f2856da25e4577018d30b459b769.pack
03/24/2021  08:00 AM            41,896 pack-25d345c594df14fca6188fdab584ab594e154bae.idx
03/24/2021  08:00 AM         3,151,989 pack-25d345c594df14fca6188fdab584ab594e154bae.pack
02/15/2021  09:00 AM            19,692 pack-2707cb44f800b30aa9ce30e727a3b759c1d6d308.idx
02/15/2021  09:00 AM         1,512,454 pack-2707cb44f800b30aa9ce30e727a3b759c1d6d308.pack
03/15/2021  08:00 AM            33,244 pack-2e6350ad2dbf91e7ed7e2a6a06c05a7e86f8c8f0.idx
03/15/2021  08:00 AM         2,766,538 pack-2e6350ad2dbf91e7ed7e2a6a06c05a7e86f8c8f0.pack
01/26/2021  09:00 AM            37,892 pack-3050b1273d645f82f3c9a08e4783e976bde9eaba.idx
01/26/2021  09:00 AM         3,044,022 pack-3050b1273d645f82f3c9a08e4783e976bde9eaba.pack
02/23/2021  09:00 AM            41,476 pack-342ec42a0677c9f8c9a42058d4f827f1b312dc02.idx
02/23/2021  09:00 AM         2,906,868 pack-342ec42a0677c9f8c9a42058d4f827f1b312dc02.pack
01/25/2021  09:02 AM        17,888,648 pack-381bb6ec83ab07db872d3c63d9836885fe3db4c6.idx
01/25/2021  09:02 AM       363,090,944 pack-381bb6ec83ab07db872d3c63d9836885fe3db4c6.pack
03/11/2021  09:00 AM            47,468 pack-3902fc024943f78be73623dd925b518381a06499.idx
03/11/2021  09:00 AM         2,721,750 pack-3902fc024943f78be73623dd925b518381a06499.pack
03/17/2021  08:00 AM            38,984 pack-3de378f9f4558260c8577ee15535a601fb0c3047.idx
03/17/2021  08:00 AM         2,839,087 pack-3de378f9f4558260c8577ee15535a601fb0c3047.pack
04/05/2021  08:00 AM            16,472 pack-3f69c5f9fbb834a62506b565903a156671e895e7.idx
04/05/2021  08:00 AM         1,333,196 pack-3f69c5f9fbb834a62506b565903a156671e895e7.pack
03/01/2021  09:00 AM            30,500 pack-4034dcc023628cd8c6da65cac72753e4d9d995bb.idx
03/01/2021  09:00 AM         2,528,528 pack-4034dcc023628cd8c6da65cac72753e4d9d995bb.pack
03/02/2021  09:00 AM            26,048 pack-40ee7da667b36f6d3311fe9e1197e52b5db742cd.idx
03/02/2021  09:00 AM         1,675,770 pack-40ee7da667b36f6d3311fe9e1197e52b5db742cd.pack
03/23/2021  08:00 AM            35,288 pack-450fb1522a8548238ee13147b8bece661d896ebe.idx
03/23/2021  08:00 AM         2,307,081 pack-450fb1522a8548238ee13147b8bece661d896ebe.pack
02/12/2021  09:00 AM            29,520 pack-46e3988115bca08baf3a5aa0edb59788b3cea39b.idx
02/12/2021  09:00 AM         2,991,780 pack-46e3988115bca08baf3a5aa0edb59788b3cea39b.pack
03/19/2021  08:00 AM            35,148 pack-4794bbf4e8dc4128fa7770ae50cc4d1d716d45cf.idx
03/19/2021  08:00 AM         2,823,373 pack-4794bbf4e8dc4128fa7770ae50cc4d1d716d45cf.pack
03/29/2021  08:00 AM            18,404 pack-4dbfcfbd7858bc2f9e9aabb989feb82ec8c2195b.idx
03/29/2021  08:00 AM         1,571,299 pack-4dbfcfbd7858bc2f9e9aabb989feb82ec8c2195b.pack
03/22/2021  08:00 AM            30,248 pack-5102ca857646f4a824b65cbe96ae2334ef2a13b7.idx
03/22/2021  08:00 AM         3,021,428 pack-5102ca857646f4a824b65cbe96ae2334ef2a13b7.pack
04/09/2021  08:00 AM            24,984 pack-5202dbd373cda37ee88b95edca9909f8d939d820.idx
04/09/2021  08:00 AM         2,102,816 pack-5202dbd373cda37ee88b95edca9909f8d939d820.pack
02/25/2021  09:00 AM            38,592 pack-5c735cc1bd0c219384b8aae19b4230b89c9f0527.idx
02/25/2021  09:00 AM         3,310,911 pack-5c735cc1bd0c219384b8aae19b4230b89c9f0527.pack
02/10/2021  09:00 AM            29,912 pack-63dcac36c9bd94b534394f3353806787d2116b19.idx
02/10/2021  09:00 AM         2,254,670 pack-63dcac36c9bd94b534394f3353806787d2116b19.pack
02/09/2021  09:00 AM            38,228 pack-64ab946a4f3e94b9bd811316be99aea2d2c52562.idx
02/09/2021  09:00 AM         2,918,755 pack-64ab946a4f3e94b9bd811316be99aea2d2c52562.pack
02/03/2021  09:00 AM            29,268 pack-6699d0b55579b22a9614f1a3437d56504fda8cb7.idx
02/03/2021  09:00 AM         2,380,234 pack-6699d0b55579b22a9614f1a3437d56504fda8cb7.pack
02/11/2021  09:00 AM            26,636 pack-6953e37bb4ce0fdb21502a195331a3c2bf713da0.idx
02/11/2021  09:00 AM         2,359,121 pack-6953e37bb4ce0fdb21502a195331a3c2bf713da0.pack
02/05/2021  09:00 AM            51,052 pack-69ddea155fda22c995d50304429106178eb53d18.idx
02/05/2021  09:00 AM         3,972,743 pack-69ddea155fda22c995d50304429106178eb53d18.pack
03/04/2021  09:00 AM            13,812 pack-6b2510e40508fe7aaabff790182d8dc3b0b96a0a.idx
03/04/2021  09:00 AM         1,391,647 pack-6b2510e40508fe7aaabff790182d8dc3b0b96a0a.pack
01/27/2021  09:00 AM            24,340 pack-719dceeb3b6668f90ea180164e7e75454b42a4e8.idx
01/27/2021  09:00 AM         2,211,789 pack-719dceeb3b6668f90ea180164e7e75454b42a4e8.pack
02/26/2021  09:00 AM            24,844 pack-8449c5b8b7bffde1a2c7a52e0c49b93541879380.idx
02/26/2021  09:00 AM         1,851,742 pack-8449c5b8b7bffde1a2c7a52e0c49b93541879380.pack
03/16/2021  08:01 AM            27,952 pack-887613fee2cf1844843fd7ec29a2c51faea4c5e7.idx
03/16/2021  08:01 AM         2,055,706 pack-887613fee2cf1844843fd7ec29a2c51faea4c5e7.pack
03/03/2021  09:00 AM            33,468 pack-8c01a5d1eb2f3b3c24cde8254d9c1656316f0615.idx
03/03/2021  09:00 AM         2,648,884 pack-8c01a5d1eb2f3b3c24cde8254d9c1656316f0615.pack
03/08/2021  09:00 AM            30,668 pack-917c73c3025a3b5e03b58c2bd5b7fdcedb613747.idx
03/08/2021  09:00 AM         2,778,181 pack-917c73c3025a3b5e03b58c2bd5b7fdcedb613747.pack
02/08/2021  09:00 AM            36,436 pack-97b842e62c6ab7b6128d0e8efab2a62d4ef710f7.idx
02/08/2021  09:00 AM         2,809,333 pack-97b842e62c6ab7b6128d0e8efab2a62d4ef710f7.pack
02/22/2021  09:00 AM            28,008 pack-9ae6d83a3ce8fb35a4ccd6e4629811d2e0aa317f.idx
02/22/2021  09:00 AM         2,272,214 pack-9ae6d83a3ce8fb35a4ccd6e4629811d2e0aa317f.pack
01/29/2021  09:00 AM           103,776 pack-a0ad6559e73a36d87315ff5ccea38bfbea3aef6f.idx
01/29/2021  09:00 AM         4,836,028 pack-a0ad6559e73a36d87315ff5ccea38bfbea3aef6f.pack
04/01/2021  08:00 AM            24,340 pack-ae500605699950842a0162b339faaa11fc729adc.idx
04/01/2021  08:00 AM         2,605,064 pack-ae500605699950842a0162b339faaa11fc729adc.pack
02/17/2021  09:00 AM            34,868 pack-b0af955c1bbae503d4e98f3c3483254b1db06465.idx
02/17/2021  09:00 AM       129,425,975 pack-b0af955c1bbae503d4e98f3c3483254b1db06465.pack
02/24/2021  09:00 AM            52,424 pack-b63177bcce972f68047f1d6a88ab9311551fd836.idx
02/24/2021  09:00 AM         3,128,224 pack-b63177bcce972f68047f1d6a88ab9311551fd836.pack
04/02/2021  08:01 AM            22,548 pack-b7b5ca6774b6903dfb54ab0378b0460258ef4009.idx
04/02/2021  08:01 AM         2,262,420 pack-b7b5ca6774b6903dfb54ab0378b0460258ef4009.pack
03/30/2021  08:00 AM            36,576 pack-ba01731671b772cc2aa8d829c8f3e404e5ede74b.idx
03/30/2021  08:00 AM         2,195,260 pack-ba01731671b772cc2aa8d829c8f3e404e5ede74b.pack
01/28/2021  09:00 AM            23,976 pack-c4bd5865a2e6315c62a7cc877ce7b977025fc179.idx
01/28/2021  09:00 AM         1,826,227 pack-c4bd5865a2e6315c62a7cc877ce7b977025fc179.pack
03/05/2021  09:00 AM            38,032 pack-cc70e5922ae2f4240f23f7c12d8fadd03c8bbf96.idx
03/05/2021  09:00 AM         2,080,619 pack-cc70e5922ae2f4240f23f7c12d8fadd03c8bbf96.pack
03/09/2021  09:00 AM            33,300 pack-cc9added0012a5c61061898fdb3d18900a455df4.idx
03/09/2021  09:00 AM         3,061,010 pack-cc9added0012a5c61061898fdb3d18900a455df4.pack
02/02/2021  09:00 AM            19,552 pack-db3eeb8229b4316dcfd97dc5729d139fd72f722e.idx
02/02/2021  09:00 AM         2,158,313 pack-db3eeb8229b4316dcfd97dc5729d139fd72f722e.pack
02/16/2021  09:00 AM            21,708 pack-df766599352e2992655a60e07c93123de5e61a0e.idx
02/16/2021  09:00 AM         1,797,674 pack-df766599352e2992655a60e07c93123de5e61a0e.pack
03/12/2021  09:00 AM            23,052 pack-e80d3505be4f0024b6efb1d80b7dc1295f039250.idx
03/12/2021  09:00 AM         1,872,492 pack-e80d3505be4f0024b6efb1d80b7dc1295f039250.pack
03/26/2021  08:00 AM            24,760 pack-e831d3bd0d3a06b91124b06c5c6a49a9e0df24cb.idx
03/26/2021  08:00 AM         2,014,086 pack-e831d3bd0d3a06b91124b06c5c6a49a9e0df24cb.pack
03/31/2021  08:00 AM            38,480 pack-eafdee9892e7660f52f4c14863c34fe6cbdd6644.idx
03/31/2021  08:00 AM         2,497,242 pack-eafdee9892e7660f52f4c14863c34fe6cbdd6644.pack
02/04/2021  09:00 AM            27,308 pack-ec92ef7230771854230a970de1d6ca6e47c6e8ba.idx
02/04/2021  09:00 AM         1,925,410 pack-ec92ef7230771854230a970de1d6ca6e47c6e8ba.pack
03/10/2021  09:00 AM            37,192 pack-f6bb304d7cdf2a49c6cfc077c73111e1fbfffa06.idx
03/10/2021  09:00 AM         3,415,778 pack-f6bb304d7cdf2a49c6cfc077c73111e1fbfffa06.pack
04/06/2021  08:00 AM            20,700 pack-f8f4eb0eb2282c49c32f5f95aceb8b55dfdd7f78.idx
04/06/2021  08:00 AM         2,190,479 pack-f8f4eb0eb2282c49c32f5f95aceb8b55dfdd7f78.pack
             108 File(s)    644,936,126 bytes
               2 Dir(s)  511,256,244,224 bytes free
ssa3512 commented 3 years ago

Here's one from a user that was getting the error in Visual Studio.

Another giant directory listing 
04/09/2021  07:59 AM              .
04/09/2021  07:59 AM              ..
03/30/2021  09:14 AM            40,412 pack-01803f62f88707186aa464f19881e52d01df35a2.idx
03/30/2021  09:14 AM         2,392,047 pack-01803f62f88707186aa464f19881e52d01df35a2.pack
03/19/2021  11:22 AM            23,024 pack-05dfd19c5ae32c399376322121f6ba1a69e04756.idx
03/19/2021  12:51 PM         1,686,001 pack-05dfd19c5ae32c399376322121f6ba1a69e04756.pack
08/11/2020  06:49 AM            35,960 pack-076e7bac8432438b685027efd843bccf98c4d9d4.idx
08/11/2020  06:49 AM         3,012,101 pack-076e7bac8432438b685027efd843bccf98c4d9d4.pack
03/17/2021  08:08 AM            29,380 pack-0abf19dc303bcd17da481e5c198eb65ad0dbb41c.idx
03/17/2021  08:28 AM         2,522,447 pack-0abf19dc303bcd17da481e5c198eb65ad0dbb41c.pack
09/21/2020  01:56 PM            13,896 pack-0c1b205912662991bcde09e2300893ab8810badc.idx
09/21/2020  01:56 PM         1,257,200 pack-0c1b205912662991bcde09e2300893ab8810badc.pack
11/20/2020  01:15 PM            58,332 pack-0d90ec3556098845284b7458c998947ee4676025.idx
03/17/2021  08:28 AM         4,614,980 pack-0d90ec3556098845284b7458c998947ee4676025.pack
02/25/2021  06:53 PM            40,384 pack-0fdefdd3a326fb65a942f72658b3a08f28fdf30c.idx
02/25/2021  06:53 PM         3,148,512 pack-0fdefdd3a326fb65a942f72658b3a08f28fdf30c.pack
03/16/2021  07:58 AM            16,220 pack-109342b8d9e9d0c48e7fc6960583a7236267720b.idx
03/17/2021  08:28 AM           804,283 pack-109342b8d9e9d0c48e7fc6960583a7236267720b.pack
07/28/2020  10:35 AM            20,420 pack-1122f515b6a2dd53a6565a3a53ed1dc1cb356c96.idx
07/28/2020  10:35 AM         1,828,842 pack-1122f515b6a2dd53a6565a3a53ed1dc1cb356c96.pack
01/06/2021  02:40 PM            49,792 pack-13ec7aa5dc8a8d4155438947b418348ba87acff6.idx
01/06/2021  02:40 PM         3,674,454 pack-13ec7aa5dc8a8d4155438947b418348ba87acff6.pack
07/24/2020  08:01 AM            25,880 pack-14cec300e3548035694c1d3fa0168139624ca18a.idx
07/24/2020  04:57 PM         1,314,327 pack-14cec300e3548035694c1d3fa0168139624ca18a.pack
10/13/2020  03:50 PM            21,904 pack-164e975e1b10219758c275a2e7e8af2b798a6ae5.idx
10/13/2020  03:50 PM         1,086,486 pack-164e975e1b10219758c275a2e7e8af2b798a6ae5.pack
09/21/2020  05:03 PM             7,736 pack-1c24214daa4519574a6bf0ea3942c7e3518d50ab.idx
09/21/2020  05:03 PM           713,043 pack-1c24214daa4519574a6bf0ea3942c7e3518d50ab.pack
12/18/2020  12:02 PM           168,316 pack-1c3108ca7afaa538e9d8f1509a031b4f2a2236dc.idx
12/18/2020  12:02 PM        13,118,570 pack-1c3108ca7afaa538e9d8f1509a031b4f2a2236dc.pack
02/19/2021  03:25 PM            54,020 pack-1e3e4b2d62d48be2331610b4557def59b03f9216.idx
02/19/2021  03:38 PM         5,013,214 pack-1e3e4b2d62d48be2331610b4557def59b03f9216.pack
09/17/2020  11:55 AM            26,944 pack-1f9d771e1168344f013a87b034d49308c00b6be0.idx
09/17/2020  11:55 AM         2,493,598 pack-1f9d771e1168344f013a87b034d49308c00b6be0.pack
08/13/2020  07:28 AM            17,900 pack-218e337f36c1807768bdab850e336363b24ae7df.idx
08/13/2020  07:28 AM         1,733,500 pack-218e337f36c1807768bdab850e336363b24ae7df.pack
10/21/2020  06:50 AM            25,992 pack-21ccce6e248e7be17f21a3f0c9e3cf6b6eb2629f.idx
10/21/2020  06:50 AM         1,842,945 pack-21ccce6e248e7be17f21a3f0c9e3cf6b6eb2629f.pack
08/18/2020  05:55 PM            23,444 pack-21e2b8cf2396ee2eefa4bb841c6d1535f6a2c7d4.idx
08/18/2020  05:55 PM         1,153,609 pack-21e2b8cf2396ee2eefa4bb841c6d1535f6a2c7d4.pack
12/01/2020  08:10 AM            90,448 pack-21f6633d78dd8717c8101d17160281fb978772f5.idx
12/01/2020  10:07 AM         5,981,766 pack-21f6633d78dd8717c8101d17160281fb978772f5.pack
03/26/2021  04:29 PM             7,176 pack-24220745fb0f29005b3160f926864689308204f1.idx
03/26/2021  04:29 PM           653,001 pack-24220745fb0f29005b3160f926864689308204f1.pack
08/03/2020  04:12 PM            54,860 pack-24737f6dacfb2612980e2dd7b01d697551c98303.idx
08/03/2020  04:12 PM         1,707,387 pack-24737f6dacfb2612980e2dd7b01d697551c98303.pack
08/25/2020  10:13 AM            19,216 pack-267ec64f7309ce5b08f1ff62630ca48520605935.idx
08/25/2020  10:13 AM         1,507,367 pack-267ec64f7309ce5b08f1ff62630ca48520605935.pack
03/26/2021  12:42 PM             6,112 pack-28bc0d26f4d8867c7978c2d83eb908dcd1883f60.idx
03/26/2021  01:18 PM           268,585 pack-28bc0d26f4d8867c7978c2d83eb908dcd1883f60.pack
12/07/2020  03:53 PM            86,892 pack-2ab8a046e1d7acdb8373ff8de43ed4af8be2d7cd.idx
12/07/2020  03:53 PM         5,194,831 pack-2ab8a046e1d7acdb8373ff8de43ed4af8be2d7cd.pack
10/14/2020  08:36 AM            12,132 pack-35aebaad2b76d2562b0e3bb07172aa8c5899ea2b.idx
10/14/2020  08:36 AM           739,242 pack-35aebaad2b76d2562b0e3bb07172aa8c5899ea2b.pack
03/08/2021  06:07 PM            14,764 pack-35d8ecf49e89fc7c73550e52032301ce2c68bed9.idx
03/09/2021  01:27 PM         1,415,446 pack-35d8ecf49e89fc7c73550e52032301ce2c68bed9.pack
08/28/2020  12:02 PM            22,408 pack-36783a86394a4531edce1a78745af85369abef33.idx
08/28/2020  12:02 PM         1,571,347 pack-36783a86394a4531edce1a78745af85369abef33.pack
03/25/2021  07:02 PM            60,936 pack-36da8d8aa8e0a16056405c45467d970db26fac4e.idx
03/26/2021  01:18 PM         4,452,828 pack-36da8d8aa8e0a16056405c45467d970db26fac4e.pack
07/20/2020  04:37 PM            15,184 pack-390b3b99b4011f328c57977b074429465add87e5.idx
07/20/2020  04:38 PM         1,450,479 pack-390b3b99b4011f328c57977b074429465add87e5.pack
01/13/2021  10:56 AM            40,468 pack-3a4e2d94e54bc992526d0896deb09222b9d7b720.idx
01/13/2021  10:56 AM         2,749,804 pack-3a4e2d94e54bc992526d0896deb09222b9d7b720.pack
01/07/2021  01:31 PM            22,744 pack-3c832e155a33357d3a2ad8671e1c6964c3756596.idx
01/07/2021  01:31 PM         1,918,367 pack-3c832e155a33357d3a2ad8671e1c6964c3756596.pack
03/18/2021  03:33 PM            36,184 pack-418807ef74e23f53ae29f0c956f1ea3a122b7b1b.idx
03/23/2021  12:11 PM         3,052,910 pack-418807ef74e23f53ae29f0c956f1ea3a122b7b1b.pack
07/31/2020  05:53 PM            14,092 pack-437886dd2ef76e91bcec305a9c231f0e50587f92.idx
07/31/2020  06:00 PM         1,034,333 pack-437886dd2ef76e91bcec305a9c231f0e50587f92.pack
01/26/2021  01:09 PM            62,672 pack-45daa4a5a4cbb1369a174882651343ffab02f00a.idx
01/26/2021  01:09 PM         4,918,244 pack-45daa4a5a4cbb1369a174882651343ffab02f00a.pack
09/29/2020  07:31 AM            37,724 pack-46ce1df128cf0eeccb07f3348ddd38ab3e66981b.idx
10/01/2020  12:09 PM         3,347,366 pack-46ce1df128cf0eeccb07f3348ddd38ab3e66981b.pack
07/24/2020  04:57 PM             7,708 pack-47dcfc1e725a2c46a588f49f320ea84d02bf8443.idx
07/25/2020  08:02 AM           875,562 pack-47dcfc1e725a2c46a588f49f320ea84d02bf8443.pack
10/15/2020  04:09 PM            46,572 pack-48caba21f21774b73054046c91998f543a622083.idx
10/15/2020  04:09 PM         2,976,098 pack-48caba21f21774b73054046c91998f543a622083.pack
08/27/2020  04:57 PM            26,076 pack-4a767ebf0b4a0066a168843cd4ab14f3293ae234.idx
08/27/2020  04:57 PM         1,851,859 pack-4a767ebf0b4a0066a168843cd4ab14f3293ae234.pack
09/25/2020  07:32 AM            21,204 pack-4aa2168b1c12c168a5c44585406e4efb94cb1c74.idx
03/17/2021  08:28 AM         1,960,662 pack-4aa2168b1c12c168a5c44585406e4efb94cb1c74.pack
01/27/2021  02:51 PM            27,028 pack-4b60fd8a92adb273242d16398a526990568c6987.idx
01/27/2021  02:51 PM         2,242,478 pack-4b60fd8a92adb273242d16398a526990568c6987.pack
02/26/2021  12:19 PM            17,452 pack-4c759da68ca85c069b67e1c7610ff0c382bc72cc.idx
02/26/2021  12:19 PM         1,515,288 pack-4c759da68ca85c069b67e1c7610ff0c382bc72cc.pack
03/12/2021  12:21 PM            10,452 pack-4d07282a0a2cff2b5e3019299f1e4cb364f42203.idx
03/12/2021  12:21 PM           885,366 pack-4d07282a0a2cff2b5e3019299f1e4cb364f42203.pack
03/24/2021  07:50 AM            56,064 pack-4e568b945cc191bd5690bb0f7211e30a48cc87da.idx
03/24/2021  07:50 AM         4,036,885 pack-4e568b945cc191bd5690bb0f7211e30a48cc87da.pack
10/08/2020  02:24 PM            44,332 pack-4f079f217bbdbf45341805afb6c219c15ba04fc3.idx
10/08/2020  02:24 PM         2,945,031 pack-4f079f217bbdbf45341805afb6c219c15ba04fc3.pack
07/21/2020  01:58 PM            12,440 pack-517e3ece0b8f92a68dea7c8c5a5db3e523563c3f.idx
07/23/2020  11:16 AM           984,830 pack-517e3ece0b8f92a68dea7c8c5a5db3e523563c3f.pack
09/24/2020  11:30 AM            16,444 pack-531210c9d65b90dcdf6c48441d2723210ea736fe.idx
09/24/2020  11:30 AM         1,417,287 pack-531210c9d65b90dcdf6c48441d2723210ea736fe.pack
11/18/2020  08:54 AM            64,632 pack-551690ea206190e54f0dea63c3a5d7b2d9647b6b.idx
11/23/2020  11:56 AM         4,234,456 pack-551690ea206190e54f0dea63c3a5d7b2d9647b6b.pack
02/24/2021  12:10 PM            33,048 pack-562b62fa31283c077c960a7e874ba17d3b7d8d93.idx
03/04/2021  08:46 PM         2,498,164 pack-562b62fa31283c077c960a7e874ba17d3b7d8d93.pack
03/10/2021  12:46 PM            47,664 pack-580e0c1bcad7919135d92f28150a2620db7bc049.idx
03/11/2021  12:44 PM         3,107,270 pack-580e0c1bcad7919135d92f28150a2620db7bc049.pack
11/03/2020  02:45 PM           150,480 pack-586f0c4434105004e263ce20c0ef9a675f92248c.idx
11/03/2020  03:05 PM        10,636,384 pack-586f0c4434105004e263ce20c0ef9a675f92248c.pack
08/07/2020  11:21 AM            14,316 pack-5b21f0ead4c6581723eba3e73bd672b31b4e5c4f.idx
08/07/2020  11:21 AM         1,040,564 pack-5b21f0ead4c6581723eba3e73bd672b31b4e5c4f.pack
12/08/2020  05:55 PM            28,120 pack-5bffe1875f15dda3b0697e2af35cb18431df9ebe.idx
12/08/2020  05:55 PM         2,325,476 pack-5bffe1875f15dda3b0697e2af35cb18431df9ebe.pack
01/15/2021  02:22 PM            94,956 pack-5c6279cc90de80f404381a2ce75a7996f7a317dc.idx
01/15/2021  02:22 PM         4,867,561 pack-5c6279cc90de80f404381a2ce75a7996f7a317dc.pack
09/22/2020  06:59 AM             5,636 pack-5d2ec3c137099f689d6efe9fb918b28c725c3a15.idx
09/22/2020  06:59 AM           655,868 pack-5d2ec3c137099f689d6efe9fb918b28c725c3a15.pack
09/10/2020  12:13 PM            37,192 pack-61177358fceeaa89d911edf5e2f89b940ad1ffd7.idx
09/10/2020  12:13 PM         2,720,069 pack-61177358fceeaa89d911edf5e2f89b940ad1ffd7.pack
07/22/2020  03:20 PM            19,664 pack-641bffd8a3217ad9d4a505ff0fdb41c9ae633c5a.idx
07/24/2020  04:57 PM         1,950,678 pack-641bffd8a3217ad9d4a505ff0fdb41c9ae633c5a.pack
03/04/2021  07:20 PM            26,524 pack-64e20846183cb0e83c10962bd0f57130a575f225.idx
03/04/2021  08:46 PM         1,366,912 pack-64e20846183cb0e83c10962bd0f57130a575f225.pack
11/06/2020  12:37 PM            21,064 pack-659818b2b9c3f2e8c64c8eadb528f209972a4cce.idx
11/06/2020  12:37 PM         1,868,674 pack-659818b2b9c3f2e8c64c8eadb528f209972a4cce.pack
08/12/2020  11:11 AM            31,116 pack-6601e52cd2b879367aa40e2fef34b84e2e897ef0.idx
08/12/2020  11:11 AM         2,314,262 pack-6601e52cd2b879367aa40e2fef34b84e2e897ef0.pack
10/13/2020  11:06 AM            27,756 pack-692029b8f164eea0c3610d46da511a6bb17cfccf.idx
10/13/2020  11:06 AM         1,873,168 pack-692029b8f164eea0c3610d46da511a6bb17cfccf.pack
09/03/2020  02:01 PM            11,572 pack-6db1165d5872604a09bb65868ec22175d516b33e.idx
09/03/2020  02:01 PM           742,263 pack-6db1165d5872604a09bb65868ec22175d516b33e.pack
01/18/2021  12:12 PM            45,172 pack-6e3a0762b861db289a8a858c6490217fc3f50965.idx
01/18/2021  12:12 PM         4,055,257 pack-6e3a0762b861db289a8a858c6490217fc3f50965.pack
01/22/2021  09:12 AM            53,852 pack-6ee9be9eb1658896a45aa1dfcedc0ce04e6359eb.idx
01/22/2021  09:12 AM         5,039,255 pack-6ee9be9eb1658896a45aa1dfcedc0ce04e6359eb.pack
07/23/2020  11:07 AM            12,272 pack-6ff510f6b0d88cfb96e5cd63161f10d5e0589a39.idx
07/23/2020  11:16 AM           420,320 pack-6ff510f6b0d88cfb96e5cd63161f10d5e0589a39.pack
07/21/2020  09:22 AM            11,908 pack-74700b7b4d0ec598d32a8aa1f020ce6907f7b5a8.idx
07/23/2020  11:16 AM           931,899 pack-74700b7b4d0ec598d32a8aa1f020ce6907f7b5a8.pack
03/26/2021  09:13 AM             7,624 pack-74b77c568b3a464d7287c9bdb30fb6e0dd25029e.idx
03/26/2021  09:13 AM           738,358 pack-74b77c568b3a464d7287c9bdb30fb6e0dd25029e.pack
10/07/2020  07:27 AM            29,856 pack-77542c7411b49580eb5df85fe384f2939e87103c.idx
10/07/2020  07:27 AM         2,478,628 pack-77542c7411b49580eb5df85fe384f2939e87103c.pack
02/03/2021  06:20 PM            39,768 pack-7787890a91d4c39781a1aeb1f573cef1e0532d24.idx
02/03/2021  06:20 PM         3,021,186 pack-7787890a91d4c39781a1aeb1f573cef1e0532d24.pack
07/28/2020  01:29 PM             6,224 pack-77d2593ae87de649a9ad3ceb176e88cf24e8e666.idx
07/28/2020  01:29 PM           585,497 pack-77d2593ae87de649a9ad3ceb176e88cf24e8e666.pack
02/17/2021  08:13 AM            88,852 pack-7a78e417c1346d9b49903bf9a65d84d66a659fac.idx
03/23/2021  12:11 PM       133,721,058 pack-7a78e417c1346d9b49903bf9a65d84d66a659fac.pack
07/29/2020  07:17 AM            19,104 pack-7b6491d55f3170016dcafb64efc5f3bbc8d4d3ea.idx
07/30/2020  06:43 PM         1,721,476 pack-7b6491d55f3170016dcafb64efc5f3bbc8d4d3ea.pack
04/08/2021  10:25 AM            30,192 pack-7d3abe2ca8d4354bd45269c6da459360ba752724.idx
04/08/2021  10:25 AM         2,721,669 pack-7d3abe2ca8d4354bd45269c6da459360ba752724.pack
04/05/2021  06:21 PM            15,968 pack-7dc59c2888a89b051965c34542087ae39a7851b9.idx
04/05/2021  06:21 PM         1,507,448 pack-7dc59c2888a89b051965c34542087ae39a7851b9.pack
02/02/2021  08:29 AM           150,648 pack-7e6e62288276d60aa8c219fc536c9dbf720af914.idx
02/02/2021  08:29 AM         8,412,479 pack-7e6e62288276d60aa8c219fc536c9dbf720af914.pack
09/21/2020  07:41 AM            49,316 pack-825f5352dd97d6e0809bf1902d4b7b7bba74eb33.idx
09/21/2020  07:41 AM         3,724,871 pack-825f5352dd97d6e0809bf1902d4b7b7bba74eb33.pack
07/22/2020  05:04 PM            10,956 pack-85c1d972a5e3dbc0661cd10ef31b96d5eb23c4bb.idx
07/24/2020  04:57 PM           625,519 pack-85c1d972a5e3dbc0661cd10ef31b96d5eb23c4bb.pack
10/09/2020  06:31 PM            28,260 pack-888ea11200f695e0bbd9ef6ad6a4bf527e83a686.idx
10/09/2020  06:31 PM         3,019,069 pack-888ea11200f695e0bbd9ef6ad6a4bf527e83a686.pack
03/04/2021  11:56 AM            12,944 pack-8a3bc4ce8c28943e414a4266ec3f0e8e8dd8d46d.idx
03/04/2021  08:46 PM         1,292,359 pack-8a3bc4ce8c28943e414a4266ec3f0e8e8dd8d46d.pack
07/30/2020  02:58 PM            25,320 pack-8b3866d7edf0412a13305acd26d5599f5259318c.idx
07/31/2020  06:00 PM         1,781,236 pack-8b3866d7edf0412a13305acd26d5599f5259318c.pack
04/07/2021  09:46 AM            38,396 pack-900eb27c5f6c94f766285a567f4c182f62ed3b1f.idx
04/07/2021  09:46 AM         3,926,534 pack-900eb27c5f6c94f766285a567f4c182f62ed3b1f.pack
02/23/2021  09:05 AM            41,980 pack-912ed8a45a2b3cc2b17b0423313dde04e95ff4ac.idx
02/23/2021  09:05 AM         2,869,274 pack-912ed8a45a2b3cc2b17b0423313dde04e95ff4ac.pack
03/15/2021  02:44 PM            45,564 pack-91bb1ce2d00f464cbd3469655f4075e0810f2f18.idx
03/17/2021  08:28 AM         3,723,818 pack-91bb1ce2d00f464cbd3469655f4075e0810f2f18.pack
08/04/2020  02:46 PM            14,120 pack-933a0ff3e3e0a0eb02108ba0d8da7df949c82dae.idx
08/06/2020  09:43 AM           910,392 pack-933a0ff3e3e0a0eb02108ba0d8da7df949c82dae.pack
08/24/2020  08:27 AM            52,256 pack-9670f0b2e160e4a50c195363e7340e7c47db8bd2.idx
08/24/2020  08:27 AM         2,588,198 pack-9670f0b2e160e4a50c195363e7340e7c47db8bd2.pack
09/16/2020  10:42 AM            32,012 pack-97240f9262d0980ff06f44598d17a5913d2cfc06.idx
09/16/2020  10:42 AM         2,501,190 pack-97240f9262d0980ff06f44598d17a5913d2cfc06.pack
11/12/2020  10:26 AM            42,596 pack-985af2f2a323508abc732971a0dcd2bbad33515d.idx
11/12/2020  10:26 AM         3,305,479 pack-985af2f2a323508abc732971a0dcd2bbad33515d.pack
04/09/2021  07:59 AM            19,272 pack-9b01f79352c78cc1ad817da71c47986653f0ad78.idx
04/09/2021  07:59 AM         1,834,097 pack-9b01f79352c78cc1ad817da71c47986653f0ad78.pack
09/23/2020  02:20 PM            47,832 pack-9b9915c71fbc5048ca00ec7567bb076c22db475d.idx
09/23/2020  02:20 PM         3,634,444 pack-9b9915c71fbc5048ca00ec7567bb076c22db475d.pack
09/08/2020  08:31 AM            48,112 pack-9d22bbd40535618666243393e8ecbc87caf53451.idx
09/08/2020  08:31 AM         3,376,049 pack-9d22bbd40535618666243393e8ecbc87caf53451.pack
11/10/2020  03:10 PM            30,416 pack-9dc893a2377bbd973e41cb25dfb17be3a4ad3ba1.idx
11/10/2020  04:55 PM         2,270,019 pack-9dc893a2377bbd973e41cb25dfb17be3a4ad3ba1.pack
03/20/2021  09:09 AM            17,648 pack-9f5c1a840b8850e338de646888bc95eca8274cea.idx
03/22/2021  03:47 PM         2,249,700 pack-9f5c1a840b8850e338de646888bc95eca8274cea.pack
12/02/2020  08:17 AM            32,964 pack-a0770e99c3def71ac624a11ccddf177418b98808.idx
12/02/2020  08:17 AM         2,263,076 pack-a0770e99c3def71ac624a11ccddf177418b98808.pack
08/20/2020  07:42 AM            23,080 pack-a158cfc23069a5dff70b61028d1f687c146e9b50.idx
08/20/2020  07:42 AM         1,425,144 pack-a158cfc23069a5dff70b61028d1f687c146e9b50.pack
10/06/2020  07:16 AM            49,568 pack-a20e5472a2b233cfef31d95af853ed887e6ef26a.idx
10/06/2020  07:16 AM         5,221,047 pack-a20e5472a2b233cfef31d95af853ed887e6ef26a.pack
12/29/2020  09:37 AM            82,776 pack-a571e412546fe7d9bae11702f19fa68e85da7d18.idx
12/29/2020  09:37 AM         6,614,377 pack-a571e412546fe7d9bae11702f19fa68e85da7d18.pack
12/03/2020  01:42 PM            32,460 pack-ab85e3618aa2e107fa9c0b26ea4c1952032efa81.idx
12/03/2020  01:42 PM         1,890,888 pack-ab85e3618aa2e107fa9c0b26ea4c1952032efa81.pack
08/06/2020  09:37 AM            17,032 pack-b07a34ded940420dad43dc214f78297d8c79e103.idx
08/06/2020  09:52 AM         1,093,783 pack-b07a34ded940420dad43dc214f78297d8c79e103.pack
02/04/2021  12:52 PM            27,112 pack-b2ab9c8c33a206737dd0d2d327709551e628a043.idx
02/04/2021  12:52 PM         2,036,715 pack-b2ab9c8c33a206737dd0d2d327709551e628a043.pack
03/02/2021  07:49 AM            26,412 pack-b34328050a7094f42128ffdc79812e0d1a4782fb.idx
03/02/2021  07:49 AM         1,679,994 pack-b34328050a7094f42128ffdc79812e0d1a4782fb.pack
09/29/2020  10:10 AM             8,688 pack-b5d43c2d2b662936c84d34072e6248049be47e60.idx
09/29/2020  10:10 AM           884,969 pack-b5d43c2d2b662936c84d34072e6248049be47e60.pack
09/23/2020  03:30 PM             6,280 pack-b8bde221b1bec40696656009f50bf0239adc9ef7.idx
09/23/2020  03:30 PM           566,013 pack-b8bde221b1bec40696656009f50bf0239adc9ef7.pack
02/23/2021  11:24 AM            25,152 pack-b8f344ce951f82d568eaa59583527187fd10f999.idx
02/23/2021  11:24 AM         1,114,020 pack-b8f344ce951f82d568eaa59583527187fd10f999.pack
09/14/2020  11:40 AM            31,536 pack-b8f64a18dc81a02f797d20f3d5820e15b75bd4ad.idx
09/14/2020  11:40 AM         3,000,974 pack-b8f64a18dc81a02f797d20f3d5820e15b75bd4ad.pack
08/04/2020  08:42 AM             9,248 pack-ba3a186876fe4979a8f124c67e68afc97e07935d.idx
08/04/2020  08:42 AM         1,027,919 pack-ba3a186876fe4979a8f124c67e68afc97e07935d.pack
03/11/2021  12:12 PM            46,544 pack-bab71f246ac53517686fca7aa5001a695336de6d.idx
03/11/2021  12:44 PM         2,535,125 pack-bab71f246ac53517686fca7aa5001a695336de6d.pack
03/02/2021  10:43 AM             5,860 pack-bb54e4b947f771a64e5caebab8265af69d126bd0.idx
03/02/2021  10:43 AM           438,355 pack-bb54e4b947f771a64e5caebab8265af69d126bd0.pack
03/31/2021  10:02 AM            33,356 pack-bf8a81dd4bfd80e06ebb31c2376c8e7883e17e84.idx
03/31/2021  10:02 AM         2,302,249 pack-bf8a81dd4bfd80e06ebb31c2376c8e7883e17e84.pack
09/01/2020  06:53 AM            27,588 pack-bfb19de17acf70480f157d7238e3cc7d182f150a.idx
09/01/2020  06:53 AM         2,065,733 pack-bfb19de17acf70480f157d7238e3cc7d182f150a.pack
01/12/2021  09:10 AM            94,172 pack-c3c0d7bb4630ff422f9a36864eb53a63cc53cbd7.idx
01/12/2021  09:10 AM         4,912,214 pack-c3c0d7bb4630ff422f9a36864eb53a63cc53cbd7.pack
01/05/2021  08:47 AM            45,032 pack-c5a21a8b6d524177ee78b688650ae61e7ec4166e.idx
01/05/2021  08:47 AM         3,643,010 pack-c5a21a8b6d524177ee78b688650ae61e7ec4166e.pack
10/01/2020  11:53 AM            42,680 pack-c9406de06d0a0cc435c5e58bc8cb8cfd2549fd73.idx
10/01/2020  12:09 PM         2,661,417 pack-c9406de06d0a0cc435c5e58bc8cb8cfd2549fd73.pack
03/17/2021  01:49 PM            11,376 pack-c9cee6cf48d0d9c2a7ca1b9949ea0cec0284a266.idx
03/17/2021  01:49 PM         1,166,344 pack-c9cee6cf48d0d9c2a7ca1b9949ea0cec0284a266.pack
10/19/2020  07:39 AM            24,536 pack-cb61bfbb0abcf5e0dbdad5c76a9db4d305ef547a.idx
10/19/2020  07:39 AM         2,157,755 pack-cb61bfbb0abcf5e0dbdad5c76a9db4d305ef547a.pack
07/20/2020  10:20 AM        16,062,124 pack-ce3d47e72afdf1590f202e297e12ca5d8b68ec44.idx
03/17/2021  08:28 AM       248,384,478 pack-ce3d47e72afdf1590f202e297e12ca5d8b68ec44.pack
03/08/2021  04:00 PM            43,436 pack-ce60697498609309686c89907c7e998155b70a15.idx
03/09/2021  01:27 PM         3,849,985 pack-ce60697498609309686c89907c7e998155b70a15.pack
11/23/2020  10:39 AM            58,136 pack-d01b2869dd9a3e024d38d690999fa370b4e17693.idx
11/23/2020  11:56 AM         4,149,900 pack-d01b2869dd9a3e024d38d690999fa370b4e17693.pack
12/22/2020  09:59 AM            94,788 pack-d0bec3c16cca20b65265efe3b09143c1051feb24.idx
12/22/2020  09:59 AM         7,137,702 pack-d0bec3c16cca20b65265efe3b09143c1051feb24.pack
02/27/2021  09:45 AM            25,936 pack-d3ce40bedf9c60e7b9433f7ae06c64fdbeeadba7.idx
02/27/2021  12:57 PM         2,133,322 pack-d3ce40bedf9c60e7b9433f7ae06c64fdbeeadba7.pack
03/16/2021  12:18 PM             8,856 pack-d43f6a2eb7b53bde41c02c200c68b7e616e55f94.idx
03/16/2021  12:18 PM           730,887 pack-d43f6a2eb7b53bde41c02c200c68b7e616e55f94.pack
02/05/2021  12:17 PM            39,628 pack-d45be1f6024f503dc0c34f5a53d6d45507d5edb0.idx
02/05/2021  12:17 PM         3,518,972 pack-d45be1f6024f503dc0c34f5a53d6d45507d5edb0.pack
12/22/2020  03:24 PM            12,944 pack-d52783d990ed4fa0626cf62a0782023d181845f3.idx
12/22/2020  03:25 PM           659,511 pack-d52783d990ed4fa0626cf62a0782023d181845f3.pack
03/11/2021  07:01 PM            15,268 pack-d8d692c6daa2025d9b8078742b8067e1b355a09c.idx
03/11/2021  07:01 PM         1,511,879 pack-d8d692c6daa2025d9b8078742b8067e1b355a09c.pack
08/18/2020  08:43 AM            45,116 pack-da81ca8c46fc64b054fe571435df15502697064c.idx
08/18/2020  08:43 AM         3,712,332 pack-da81ca8c46fc64b054fe571435df15502697064c.pack
09/03/2020  08:45 AM            55,252 pack-dd23585a5d27bfec93827bfaeb2dcc48a739c99f.idx
09/03/2020  08:45 AM         3,952,032 pack-dd23585a5d27bfec93827bfaeb2dcc48a739c99f.pack
08/05/2020  12:48 PM            19,524 pack-decd70896c329b3a265cba079203cc337dca7c28.idx
08/06/2020  09:43 AM         1,766,307 pack-decd70896c329b3a265cba079203cc337dca7c28.pack
02/08/2021  12:39 PM            43,660 pack-dfdc8db6552813b23daf274338babd3b37693f2f.idx
02/08/2021  12:39 PM         3,039,969 pack-dfdc8db6552813b23daf274338babd3b37693f2f.pack
10/01/2020  01:40 PM            10,872 pack-e5e4725c7b8f3a29bcd81f0e18ff68c64ff33c62.idx
10/01/2020  01:40 PM           966,259 pack-e5e4725c7b8f3a29bcd81f0e18ff68c64ff33c62.pack
09/30/2020  07:37 AM            26,524 pack-e60e339a34ffa21487c8c593fbd500011a98b285.idx
10/01/2020  12:09 PM         1,771,974 pack-e60e339a34ffa21487c8c593fbd500011a98b285.pack
09/09/2020  08:56 AM            49,652 pack-e90c16f9ecd313235b5b4268a01cfd1806a48216.idx
09/09/2020  08:56 AM         3,213,624 pack-e90c16f9ecd313235b5b4268a01cfd1806a48216.pack
11/05/2020  10:35 AM            25,292 pack-e96e488f0be40d600cdb73ddda7ad3c38aad477c.idx
11/05/2020  10:35 AM         2,168,434 pack-e96e488f0be40d600cdb73ddda7ad3c38aad477c.pack
11/18/2020  01:27 PM            12,104 pack-eaddf47738cd4ead0e532c85a69ec99cb938c2a6.idx
11/18/2020  01:27 PM         1,360,602 pack-eaddf47738cd4ead0e532c85a69ec99cb938c2a6.pack
07/24/2020  04:49 PM            26,244 pack-eddf2fee745d93349babc0ab04545192a141c873.idx
07/24/2020  04:57 PM         1,977,277 pack-eddf2fee745d93349babc0ab04545192a141c873.pack
03/03/2021  11:58 AM            30,836 pack-ee3e3f0aee2cb605597e1a85b75b533a473d1316.idx
03/04/2021  08:46 PM         2,727,769 pack-ee3e3f0aee2cb605597e1a85b75b533a473d1316.pack
04/05/2021  08:07 AM            43,436 pack-eeaf040c76681eddac82bfafef27850dd3c913a2.idx
04/05/2021  08:07 AM         4,103,828 pack-eeaf040c76681eddac82bfafef27850dd3c913a2.pack
02/10/2021  09:43 AM            43,184 pack-f1e8072bfde37a422a70b0e58fd5990ff1900e54.idx
02/10/2021  09:43 AM         3,182,217 pack-f1e8072bfde37a422a70b0e58fd5990ff1900e54.pack
03/09/2021  12:50 PM            23,080 pack-f426c6ec21410346ae23d15a26a23a64db2cbdbd.idx
03/09/2021  01:27 PM         1,978,187 pack-f426c6ec21410346ae23d15a26a23a64db2cbdbd.pack
08/26/2020  02:56 PM            27,532 pack-f4b7d648ca91eddf2f601e631d3b831a941ecee5.idx
08/26/2020  02:56 PM         2,522,700 pack-f4b7d648ca91eddf2f601e631d3b831a941ecee5.pack
01/19/2021  09:11 AM            37,052 pack-f4f7bc0f18322f4c0196b9b806b84202c9fe7243.idx
01/19/2021  09:11 AM         3,581,798 pack-f4f7bc0f18322f4c0196b9b806b84202c9fe7243.pack
09/15/2020  10:36 AM            17,172 pack-fb748fe5a0238cff7a6c30b1dafc4580d4a19e10.idx
09/15/2020  10:36 AM         1,035,589 pack-fb748fe5a0238cff7a6c30b1dafc4580d4a19e10.pack
08/14/2020  08:56 AM            12,888 pack-fc9f855b62f2e2c5f77fffac02aecf1b4b32af03.idx
08/14/2020  10:45 AM         1,043,188 pack-fc9f855b62f2e2c5f77fffac02aecf1b4b32af03.pack
             282 File(s)    756,021,106 bytes
               2 Dir(s)  89,487,589,376 bytes free
AArnott commented 3 years ago

None of these files look very large, but I suppose perhaps the 129MB+ files might be hard to map into contiguous memory that large. @qmfrederik I believe when memory mapping files that it's possible to map only portions of files at a time. I wonder if a fix for this would be to set a max "window" size to map into memory and move it as necessary?

filipnavara commented 3 years ago

Here's one more data point:

filipnavara@172-4-1-17 emclient % ls -l .git/objects/pack
total 8302608
-r--r--r--  1 filipnavara  staff    14635208 Jun 11 10:00 pack-80c7ee391de74c1da6faf79c8b49c68d8c2dacf6.idx
-r--r--r--  1 filipnavara  staff  4236296121 Jun 11 10:00 pack-80c7ee391de74c1da6faf79c8b49c68d8c2dacf6.pack

Mapping the index files is fine but mapping the whole pack files is definitely problematic.

filipnavara commented 3 years ago

Here's an experiment with mapping only small chunks of the pack file: https://github.com/dotnet/Nerdbank.GitVersioning/compare/master...filipnavara:big-pack?expand=1

The approach is not perfect but it highlights few things:

In the end I have a feeling that it may be more reasonable to map only the index files and not the pack files. If it's a problem performance wise it's probably fine to do that only as a fallback when certain parameters are exceeded (file size threshold, 32-bit process, the mapping fails, etc.). .NET 6 will have better APIs to handle this kind of random reads (https://github.com/dotnet/runtime/pull/53669) but unfortunately that's still some months away.

AArnott commented 3 years ago

@qmfrederik Do you have time to investigate @filipnavara's data and proposal?

filipnavara commented 3 years ago

I'd be happy to do some more testing and try out other approaches or even run benchmarks. However, with the window for .NET 6 runtime changes slowly closing in I need to focus on completing some work there before moving on to other projects.

qmfrederik commented 3 years ago

All help is welcome, so I'd be more than happy if @filipnavara wants to invest some time in this 😄 .

Not sure what is the motivation for MemoryMappedStream compared to the built-in MemoryMappedViewStream

I think the main reason was .Dispose behavior. Pooling streams had a significant impact on performance. There was an attempt to use the built-in version (0c5a573) which was reverted (61e877a, https://github.com/dotnet/Nerdbank.GitVersioning/pull/521#issuecomment-719987210).

In the end I have a feeling that it may be more reasonable to map only the index files and not the pack files.

Yes, that may be the case. If I remember correctly, the stream pooling and usage of memory mapped files was introduced at about the same time. There's probably a real benefit for index files, since we do a binary search there.

It is possible that the memory mapping doesn't have much of a perf impact on pack files, and the real gain comes from stream pooling.

There are a couple of perf tests which run as part of CI, so it should be fairly easy to tell whether changes regressed performance or not 😃

AArnott commented 3 years ago

Keep in mind the workaround we have of setting the NBGV_GitEngine environment variable to LibGit2 so you don't have to stick with the 3.3 version, folks.

benmccallum commented 3 years ago

I'm now starting to run into this in local dev too in VS. I guess VS 2022 can't come fast enough :P

Searching for a way to configure the use of 64-bit msbuild from VS but that doesn't seem to be possible. Guessing that's a limitation of a 32-bit process not being able to spawn a 64-bit one, but this certainly isn't my area of expertise. https://docs.microsoft.com/en-us/visualstudio/msbuild/msbuild?view=vs-2019

So rather than need to run msbuild from cmdline, is there a way to perhaps toggle off ngvb in local dev? Which would probably be something we'd want in any case to avoid the compile-time overhead...

Is toggling it out of the csproj if not Release configuration a viable alternative? Can't say I've ever toggled off a package ref by Configuration. Feels a bit wrong; but trying to avoid getting the whole team to set an env var.

<PackageReference Include="Nerdbank.GitVersioning" Condition=" '$(Configuration)' == 'Release' ">
  <Version>3.4.190</Version>
  <PrivateAssets>all</PrivateAssets>
</PackageReference>
qmfrederik commented 3 years ago

@benmccallum If you have the time to dig into this, could you give https://github.com/dotnet/Nerdbank.GitVersioning/pull/626 a try and let us know if this fixes the problem for you? There may be a perf regression n macOS, but since you mention VS, I guess you're on Windows so you should be good.

Alternatively, set the NBGV_GitEngine environment variable to LibGit2 to sidestep the managed implementation altogether.

AArnott commented 3 years ago

Guessing that's a limitation of a 32-bit process not being able to spawn a 64-bit one, but this certainly isn't my area of expertise.

There's never a problem with 32-bit processes spawning 64-bit ones. It's just that that's not what msbuild in VS was designed to do.

benmccallum commented 3 years ago

@qmfrederik, I can look to give #626 a go at some point, is it available as a nuget preview release somewhere?

AArnott commented 3 years ago

@benmccallum It used to be, but it got GC'd by Azure Pipelines. I've queued a new build here: https://dev.azure.com/andrewarnott/OSS/_build/results?buildId=4966&view=results You should be able to download the nupkg from the pipeline artifacts when it's done.

AArnott commented 3 years ago

This is now available in https://github.com/dotnet/Nerdbank.GitVersioning/releases/tag/v3.4.228

benmccallum commented 3 years ago

Thanks @AArnott , sorry I didn't get around to give feedback in time. Will get on the new version tomorrow and give it a whirl.

benmccallum commented 3 years ago

Seems to be working well for us on all platforms, 👍🏻 Thanks again