search

drewnoakes / string-theory

Identify and reduce memory used by duplicate .NET strings
Other
297 stars 10 forks source link

Create a 32-bit and a 64-bit exe #11

Open 0xd4d opened 5 years ago

0xd4d commented 5 years ago

CLRMD can only attach to processes with the same bitness. StringTheory is currently prefer32bit AnyCpu so it won't be able to attch to 64-bit processes. If you try to attach to it, CLRMD throws Microsoft.Diagnostics.Runtime.ClrDiagnosticsException but the exception is ignored by the app and it just keeps showing the indeterminate progressbar.

drewnoakes commented 5 years ago

Thanks, and good point. So far I've just been looking at devenv.exe. I'll set up a build to produce to executables.

glenn-slayden commented 5 years ago

Based on my testing, it's a simple matter of removing the 'Prefer32Bit' option in the string-theory .csproj and then it was able to attach to my 64-bit WPF app (It's a great app you have here, by the way).

This result is using a build of string-theory from your current github 'master' sources, although I should mention that I had to make quite a few changes to get it to build, mostly adding a new definition for struct ClrObjectReference into clrmd, and also replicating EnumerateObjectReferences to EnumerateObjectReferencesWithFields throughout that project. I'm a github novice, so perhaps I didn't correctly understand how to clone your repo (with its clrmd branch).

[edit: also, I just found out about string-theory not detaching from the target app, which causes an ugly crash if you close the former first, but I think you already have an open issue on this]

0xd4d commented 5 years ago

@glenn-slayden I had no problem building it without updating any source code. You can try this command since it will also checkout the clrmd submodule: git clone --recursive https://github.com/drewnoakes/string-theory.git

glenn-slayden commented 5 years ago

@0xd4d Really confused. If you go to the provided branch "clrmd @ 0ae3d41" of drewnoakes/clrmd -- or the master of that fork -- or even in fact the Microsoft/clrmd master -- on github, and type ClrObjectReference into the search box at the top left for any of these, you will get zero results. What am I doing wrong then?

drewnoakes commented 5 years ago

it's a simple matter of removing the 'Prefer32Bit' option

IIRC this stops it working for 32 bit executables. I'll double check, but I suspect we do actually need both 32 and 64 bit executables.

I had to make quite a few changes to get it to build

@glenn-slayden I think you didn't check out the submodule correctly. ClrObjectReference exists on the commit pointed to by this repo:

https://github.com/drewnoakes/clrmd/blob/0ae3d415b4b465d8a5ab4c8bef14e79352d52af5/src/Microsoft.Diagnostics.Runtime/src/Common/ClrHeap.cs#L374-L388

To sync up you can try:

git submodule init
git submodule update

not detaching from the target app

See #9. Would be great if you wanted to investigate the cause and provide a PR.

glenn-slayden commented 5 years ago

Here's the cloning problem I encounter when I try to clone string-theory in Visual Studio with the "Recursively Clone Submodules" option selected:

Could not read from remote repository. Please make sure you have the correct access rights and the repository exists. clone of 'git@github.com:drewnoakes/clrmd.git' into submodule path 'D:/github/string-theory/clrmd' failed Failed to clone 'clrmd'. Retry scheduled Cloning into 'D:/github/string-theory/clrmd'... git@github.com: Permission denied (publickey). Could not read from remote repository.

It gets the string-theory project itself just fine, but crashes out on the submodule, which unfortunately seems to leave the whole shebang unusable. Could the following line be incorrect somehow?

https://github.com/drewnoakes/string-theory/blob/be5a1db710147a72ffffdcf6494970e7ced1dcb2/.gitmodules#L3

Meanwhile, I'll try to do the submodule manually as you suggested. When 'clone submodules' isn't selected, it does successfully do the parent project.

[update:] same error is given by the git console when issuing the command you provided git submodule update.

D:\github\string-theory>git submodule update
Cloning into 'D:/github/string-theory/clrmd'...
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:drewnoakes/clrmd.git' into submodule path 'D:/github/string-theory/clrmd' failed
Failed to clone 'clrmd'. Retry scheduled
Cloning into 'D:/github/string-theory/clrmd'...
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:drewnoakes/clrmd.git' into submodule path 'D:/github/string-theory/clrmd' failed
Failed to clone 'clrmd' a second time, aborting

[edit] I did manage to get it working manually, but have no additional clues to provide regarding the error message mentioned above. Thanks for your patience with me.

drewnoakes commented 5 years ago

@glenn-slayden I suspect you don't have an SSH key registered with GitHub. I've updated the .gitmodules file to use HTTPS, so if you clone again it should work. Alternatively you can update the supermodule and type git submodule sync then git submodule update (if I remember correctly).

drewnoakes commented 5 years ago

Reopening this as I'd like to automate production of the release archive, meaning a single script produces both builds.

glenn-slayden commented 5 years ago

I suspect you don't have an SSH key registered with GitHub.

Surely.

aolszowka commented 5 years ago

FWIW if it really only is removing that flag you can use CorFlags (https://docs.microsoft.com/en-us/dotnet/framework/tools/corflags-exe-corflags-conversion-tool) to remove it from the binary. That is what we're doing now to work around this issue.

Use like this (modifies the binary):

corflags.exe StringTheory.exe /32BITPREF-

We are/were seeing weird behavior where if we attempted to open up a dump from a x64 context that String-Theory would spin and spin without coming back but after applying the fix to remove the prefer 32bit flag it comes back instantly. Very helpful for us debugging dumps from 64bit MSBuild. Would you like me to open up another bug for that?

drewnoakes commented 5 years ago

@aolszowka thanks for the insight here. It'll be useful when revisiting this.

If it's spinning indefinitely that's a bug. If you have a way to repro it then yes please, open a bug. If you run StringTheory in a debugger, is there an exception that's being swallowed? Can you see what it's doing when spinning?

aolszowka commented 5 years ago

@drewnoakes So apparently the issue is only in the released binary; after git-ing and building from source it appears you recently change this to throw the exception up and put it in the clipboard:

System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: Mismatched architecture between this process and the dac.
   at Microsoft.Diagnostics.Runtime.DataTarget.CreateRuntime(ClrInfo clrInfo) in S:\GitHub\string-theory\clrmd\src\Microsoft.Diagnostics.Runtime\src\DataTargets\DataTarget.cs:line 280
   at Microsoft.Diagnostics.Runtime.ClrInfo.CreateRuntime() in S:\GitHub\string-theory\clrmd\src\Microsoft.Diagnostics.Runtime\src\DataTargets\ClrInfo.cs:line 96
   at StringTheory.Analysis.HeapAnalyzer..ctor(DataTarget dataTarget) in S:\GitHub\string-theory\StringTheory\Analysis\HeapAnalyzer.cs:line 35
   at StringTheory.Analysis.HeapAnalyzer..ctor(String dumpFilePath) in S:\GitHub\string-theory\StringTheory\Analysis\HeapAnalyzer.cs:line 17
   at StringTheory.UI.HomePage.<>c__DisplayClass19_1.<.ctor>b__4(Action`1 progressCallback, CancellationToken token) in S:\GitHub\string-theory\StringTheory\UI\HomePage.cs:line 52
   at StringTheory.UI.LoadingOperation.<Start>b__10_0() in S:\GitHub\string-theory\StringTheory\UI\LoadingOperation.cs:line 34
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
   --- End of inner exception stack trace ---
---> (Inner Exception #0) System.InvalidOperationException: Mismatched architecture between this process and the dac.
   at Microsoft.Diagnostics.Runtime.DataTarget.CreateRuntime(ClrInfo clrInfo) in S:\GitHub\string-theory\clrmd\src\Microsoft.Diagnostics.Runtime\src\DataTargets\DataTarget.cs:line 280
   at Microsoft.Diagnostics.Runtime.ClrInfo.CreateRuntime() in S:\GitHub\string-theory\clrmd\src\Microsoft.Diagnostics.Runtime\src\DataTargets\ClrInfo.cs:line 96
   at StringTheory.Analysis.HeapAnalyzer..ctor(DataTarget dataTarget) in S:\GitHub\string-theory\StringTheory\Analysis\HeapAnalyzer.cs:line 35
   at StringTheory.Analysis.HeapAnalyzer..ctor(String dumpFilePath) in S:\GitHub\string-theory\StringTheory\Analysis\HeapAnalyzer.cs:line 17
   at StringTheory.UI.HomePage.<>c__DisplayClass19_1.<.ctor>b__4(Action`1 progressCallback, CancellationToken token) in S:\GitHub\string-theory\StringTheory\UI\HomePage.cs:line 52
   at StringTheory.UI.LoadingOperation.<Start>b__10_0() in S:\GitHub\string-theory\StringTheory\UI\LoadingOperation.cs:line 34
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()<---

Might be worth updating the binary drop. Its my bad because its the first place I go unless I think I found an issue (I should have cloned and repo'ed it on the latest; sorry!).

drewnoakes commented 4 years ago

Another idea is creating both 32/64 bit sub processes so that the user doesn't have to know ahead of time which version to launch.