Closed steve02081504 closed 3 months ago
How can I use this project to generate anycpu exe?
AnyCPU binaries are simply i386 + PE32 binaries that have the Bit32Required
flag unset in their respective .NET data directory.
Note that the whole reason TinySharp only produces 64-bit specific apps is to avoid the creation of the import directories (something which is a requirement for 32-bit .NET PEs). An AnyCPU program will therefore be larger than the 64-bit equivalent.
How can I use this project to read ico files and append icons to the exe?
You will need to add an Icon and GroupIcon win32 resource to your PE image. Add AsmResolver.PE.Win32Resources
, and make use of the IconResource
class. Unfortunately the win32 resources package is still very limited (and also quite undocumented) so a lot of things need to be done manually. Here is an example snippet:
Is there any example that makes it clear how to add this resources in exe?
This would be similar to adding an icon, but instead adding a VersionInfoResource
.
Is there a way to remove the extra NUL at the end when generating the exe, rather than relying on some other logic to do that later?
If you're referring to the stripping of zeroes as described in the blog post, then no, as this actually invalidates the PE file. If you still want to do it, you'd have to do this as a post-processing step yourself.
Thank you for your kind help! I would like to ask some further questions.
sorry to say that I consulted some code, but I didn't find out how to set IsBit32Required
or Bit32Required
, I tried dozen times but I couldn't find it. Can you give more details on exactly where to set this value?
thanks for the friendly reminder, but my project itself is user-definable to generate X86 or X64, and ANY CPU exe, so I have to make sure that the generated program doesn't differ from what the user wants.
I realized that I don't have to add NUL at the end of the string, in practice its always nul terminated in memory, can we always assume that to reduce the output by one byte?
var segment = new DataSegment(Encoding.ASCII.GetBytes(outputValue+"\0"));
var segment = new DataSegment(Encoding.ASCII.GetBytes(outputValue));
is there any possibility of more syntactic sugar and helper functions to help users like me who don't know much about the program to get work done with it?
how we can improve this program to support UTF 8 if the output values are not ascii characters?
tired by using _putws
with Encoding.Unicode.GetBytes(outputValue)
but seems not working on "你好"
I tried replacing puts
with MessageBoxA
in this program, and the dependency dll file became User32
, but I don't know how to modify the call section to make it generate a GUI application
sorry to say that I consulted some code, but I didn't find out how to set IsBit32Required or Bit32Required, I tried dozen times but I couldn't find it. Can you give more details on exactly where to set this value?
This flag can be found in the .NET directory of the PE image. By default it is already unset for most binaries (and also when you create a new PE image from scratch using AsmResolver), but if you want to make sure it is unset, you can use something like the following:
using AsmResolver.PE;
using AsmResolver.PE.DotNet;
PEImage image = ...;
image.DotNetDirectory.Flags &= ~DotNetDirectoryFlags.Bit32Required;
I realized that I don't have to add NUL at the end of the string, in practice its always nul terminated in memory, can we always assume that to reduce the output by one byte?
That entirely depends on where in the binary you put the string. Place it in the middle of a section filled with zeroes, then you won't have any problems. However, placing it somewhere at random in e.g., a .text
or .data
section may not guarantee it will have a zero byte after it.
is there any possibility of more syntactic sugar and helper functions to help users like me who don't know much about the program to get work done with it?
What kind of syntax sugar or helper functions are you looking for?
how we can improve this program to support UTF 8 if the output values are not ascii characters? tired by using
_putws
withEncoding.Unicode.GetBytes(outputValue)
but seems not working on "你好"
I don't immediately see a problem with this approach. What problems are you facing?
I tried replacing
puts
withMessageBoxA
in this program, and the dependency dll file became User32, but I don't know how to modify the call section
Whether it is puts
or MessageBoxA
should not matter. Keep in mind that MessageBoxA
expects more than just one string parameter (see msdn). You will have to adjust your code accordingly such that it also passes along arguments for those as well.
... generate a GUI application
I am not entirely sure what you mean by this, but if you don't want a console window to pop up, you should change the SubSystem
of the PE to WindowsGui
:
using AsmResolver.PE;
using AsmResolver.PE.File.Headers;
PEImage image = ...
image.SubSystem = SubSystem.WindowsGui;
Thank you for your patience and help!
What kind of syntax sugar or helper functions are you looking for?
I think that providing a way for people to express the content of programs in a more abstract and efficient language without having to actually manipulate tokens might make coding more efficient and easier for beginners. For example, if I could just do something like this with my program
image.VersionInfoHelper.FileVersion = "xxx"
without having to manipulate a bunch of hard-to-remember tokens, that will be nice.
others...
Here's the thing, my PowerShell module uses a speculative method: It uses a simpler code to generate the exe, provided that the input script has a defined output and there are no side effects. this way the file is smaller. https://github.com/steve02081504/ps12exe/blob/master/src/programFrames/constexpr.cs However, the size of the generated file is still not as small as the one generated by directly manipulating the PE structure, which is why I want to port TinySharp to my project. For the moment, I gave up after a day of fiddling with it. I'm a complete novice when it comes to PE structures and even C#. There is absolutely no way I can re-implement the existing functionality all over again with your module. I might consider a different, simpler and more generalized approach to try to cut down on the size of the exe files outputs one day. Have a nice day!
By going with the TinySharp project as a starting point, you have chosen a very difficult path for a beginner, as it only makes use of the very low level APIs of AsmResolver and makes a ton of assumptions and compromises to squeeze the most out of it. It is an experiment that is not meant to be generalized to applications more complicated than a hello world program.
For a more simple, higher level manipulation of .NET binaries that doesn't require you to do any of the manual token manipulation etc., you should check out the AsmResolver.DotNet
package instead. This is much easier to work with as it abstracts away many of these low level concepts (no manual offset or token management required at all), has many of these properties as you mentioned, and is also well documented (See basics and member tree).
Hi I'm back after a day! After I got my head around it a bit I probably tried integrating TinySharp into my module and simply extended it with some parameter for controlling the exit value of the program and whether or not there is an output, and it looks pretty good. I've given up thinking about using TinySharp for generating GUI program for the time being, because my constexpr GUI program does a little more than simply popping up message boxes, and implementing it in TinySharp is probably too much for me at this time (and will predictably require long piece of code).
#if noConsole
// load assembly:AssemblyTitle
AssemblyTitleAttribute titleAttribute = (AssemblyTitleAttribute) Attribute.GetCustomAttribute(Assembly.GetExecutingAssembly(), typeof(AssemblyTitleAttribute));
string title;
if (titleAttribute != null)
title = titleAttribute.Title;
else
title = System.AppDomain.CurrentDomain.FriendlyName;
MessageBox.Show("$ConstResult", title, MessageBoxButtons.OK);
#else
Just one more question, I tried replacing puts
with _putws
but it looks like it doesn't output the multibyte characters correctly, it just outputs all the ASCII characters up to the first multibyte character and then exits.
Source code here: https://github.com/steve02081504/ps12exe/blob/master/src/programFrames/TinySharp.cs#L36
Basically I just replaced puts
with _putws
and replaced ASCII
with Unicode
in the output data conversion.
Here is a test case for output value:
Hello 世界!👾
Any possible fixes for this? Thanks!
https://github.com/ayaka14732/TinyPE-on-Win10 Here's the less than 1kb runnable program if helps
Just one more question, I tried replacing puts with _putws but it looks like it doesn't output the multibyte characters correctly, it just outputs all the ASCII characters up to the first multibyte character and then exits.
This is a locale/codepage issue. puts
and _putws
do not automatically adjust for the appropriate active code pages active on your current terminal. You will need to use a function such as WriteConsoleW
instead, and/or potentially also change the current active codepage with SetConsoleOutputCP
to CP_UTF8
.
https://github.com/ayaka14732/TinyPE-on-Win10 Here's the less than 1kb runnable program if helps
I am familiar with this project, but this is not going to bring down the file size for a .NET application, as the .NET runtime is hardwired to require at least one PE section containing .NET directories, something TinyPE does not need as it does not depend on .NET.
It just occurred to me, is it possible to provide a function to convert .net IL to assembly-like text and back? That might make it a lot easier for us to do these hacks and not have to deal with tokens.
What you're asking for is a text-based IL disassembler and assembler like the tools ilasm
and ildasm
. This is a no-go for AsmResolver for a variety of reasons:
Instead, as said before, use the AsmResolver.DotNet
package instead of the lower level APIs of AsmResolver.PE
for a higher-level DOM-like API where you do not have to deal with raw tokens and/or file offsets. The only reason TinySharp doesn't do that is because it requires very specific lowlevel hacks that are normally not required for any (normal) .NET PE.
A simple hello world app can be constructed from scratch with AsmResolver.DotNet using the following:
public static void CreateHello()
{
// Create new assembly.
var assembly = new AssemblyDefinition("MyAssembly", new Version(1,0,0,0));
// Add a default module.
var module = new ModuleDefinition("MyAssembly.exe");
assembly.Modules.Add(module);
// Import System.Console.WriteLine(string)
var writeLine = module.CorLibTypeFactory.CorLibScope
.CreateTypeReference("System", "Console")
.CreateMemberReference("WriteLine", MethodSignature.CreateStatic(
module.CorLibTypeFactory.Void,
module.CorLibTypeFactory.String))
.ImportWith(module.DefaultImporter);
// Create a main method that calls it with "Hello, world!" as string (can be unicode).
var main = new MethodDefinition("Main", MethodAttributes.Static, MethodSignature.CreateStatic(module.CorLibTypeFactory.Void));
main.CilMethodBody = new CilMethodBody(main)
{
Instructions = {
{CilOpCodes.Ldstr, "Hello, world!"},
{CilOpCodes.Call, writeLine},
{CilOpCodes.Ret}
}
};
// Add main method to <Module> type.
var type = module.GetOrCreateModuleType();
type.Methods.Add(main);
// Define the main method as the entrypoint.
module.ManagedEntryPoint = main;
// Save
module.Write("helloworld.exe");
}
Thank you for your help!
I've been trying for a while longer and now TinySharp can handle non-ASCII characters.
https://github.com/steve02081504/ps12exe/commit/71006a47b057e43d95235b468a2fb63c940b9182
@Washi1337 A few days later I tried to write this based on the code you gave:
public void SetAssemblyInfo(string description, string company, string title, string product, string copyright, string trademark, string version) {
// Create new version resource.
var versionResource = new VersionInfoResource();
if (string.IsNullOrEmpty(version))
version = "0.0.0.0";
var TypedVersion = new System.Version(version);
version = TypedVersion.ToString();
// Add info.
var fixedVersionInfo = new FixedVersionInfo {
FileVersion = TypedVersion,
ProductVersion = TypedVersion,
FileDate = DateTime.Now,
FileType = FileType.App,
FileOS = FileOS.Windows32,
FileSubType = FileSubType.DriverInstallable,
};
versionResource.FixedVersionInfo = fixedVersionInfo;
// Add strings.
var stringFileInfo = new StringFileInfo();
var stringTable = new StringTable(0, 0x4b0){
[StringTable.ProductNameKey] = product;
[StringTable.FileVersionKey] = version;
[StringTable.ProductVersionKey] = version;
[StringTable.FileDescriptionKey] = title;
[StringTable.CommentsKey] = description;
[StringTable.LegalCopyrightKey] = copyright;
};
stringFileInfo.Tables.Add(stringTable);
versionResource.AddEntry(stringFileInfo);
// Register translation.
var varFileInfo = new VarFileInfo();
var varTable = new VarTable();
varTable.Values.Add(0x4b0);
varFileInfo.Tables.Add(varTable);
versionResource.AddEntry(varFileInfo);
// Add to resources.
versionResource.WriteToDirectory(this.Image.Resources);
}
The above code is reporting an error and I can't figure out what the cause is
There is also a code for adding Icon:
public void SetWin32Icon(string IconFile) {
// Create a new icon group with 1 icon.
var newGroup = new IconGroupDirectory { Type = 1, Count = 1 };
// Add the icon to the group (header stripped).
byte[] header = File.ReadAllBytes(IconFile);
var iconBytes = BitConverter.ToUInt32(header, 14);
// cut head and tail
byte[] icon = header.Skip(0x16).Take((int)iconBytes).ToArray();
newGroup.AddEntry(
new IconGroupDirectoryEntry {
Id = (ushort) 1, // Icon ID (must be unique)
BytesInRes = iconBytes,
PixelBitCount = BitConverter.ToUInt16(header, 12),
Width = header[6],
Height = header[7],
ColorCount = 0,
ColorPlanes = 1
},
new IconEntry { RawIcon = icon }
);
// Add icon to icon resource group.
var newResource = new IconResource();
newResource.AddEntry(0x7f00, newGroup);
// Add resource to PE.
if(this.Image.Resources!=null) this.Image.Resources = new ResourceDirectory(0u);
this.Image.Resources.Entries.Insert(0, new ResourceDirectory(ResourceType.Icon));
this.Image.Resources.Entries.Insert(1, new ResourceDirectory(ResourceType.GroupIcon));
newResource.WriteToDirectory(this.Image.Resources);
}
This code passes compilation, but gives an error at runtime:
PS C:\Users\steve02081504> '23' | ps12exe -iconFile "D:\Doki Doki Literature Club MAS\CustomIconWindows.ico"
ps12exe : 使用“1”个参数调用“SetWin32Icon”时发生异常:“未将对象引用设置到对象的实例。”
所在位置 行:1 字符: 8
+ '23' | ps12exe -iconFile "D:\Doki Doki Literature Club MAS\CustomIcon ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
+ FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,ps12exe
I guess I missed something somewhere?
I don't speak Chinese, and I am not super familiar with Powershell. However the highlighted lines indicates you are (or powershell is) using an old C# version that does not support dictionary initializers via indexers (see https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/how-to-initialize-a-dictionary-with-a-collection-initializer). You will have to either use a newer C# version or use the equivalent instead. As for the runtime error, that is hard to say without more context and/or precise lines where it goes wrong.
Thank you for your help! Now both methods pass compilation and exhibit the same issue...
PS C:\Users\steve02081504> '23' | ps12exe -iconFile "D:\Doki Doki Literature Club MAS\CustomIconWindows.ico"
ps12exe : 使用“1”个参数调用“SetWin32Icon”时发生异常:“未将对象引用设置到对象的实例。”
所在位置 行:1 字符: 8
+ '23' | ps12exe -iconFile "D:\Doki Doki Literature Club MAS\CustomIcon ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
+ FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,ps12exe
Compiled file written -> 372736 bytes
PS C:\Users\steve02081504> '23' | ps12exe -title 'lll' -description 'xxx'
ps12exe : 使用“7”个参数调用“SetAssemblyInfo”时发生异常:“未将对象引用设置到对象的实例。”
所在位置 行:1 字符: 8
+ '23' | ps12exe -title 'lll' -description 'xxx'
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
+ FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,ps12exe
Compiled file written -> 3584 bytes
PS C:\Users\steve02081504>
It's looks like NullReferenceException
but idk why...
I don't really understand c#, do I need to save the resource instances's lifetime in the function until File.write is called?
By the way I noticed that the following lines are not necessary in TinySharp, looks the program can run without them
// Define assembly manifest.
tablesStream.GetTable<AssemblyDefinitionRow>().Add(new AssemblyDefinitionRow(
0,
1, 0, 0, 0,
0,
0,
stringsStreamBuffer.GetStringIndex("puts"), // The CLR does not allow for assemblies with a null name. Reuse the name "puts" to safe space.
0
));
It's looks like
NullReferenceException
but idk why...
This means a variable was null
but the program is trying to use it as a non-null value. Again, without context, line numbers, stack trace, etc. it is hard to say why this happens, whether it is a bug in your program or a bug in AsmResolver. If you think what you're experiencing is a bug in AsmResolver, please provide a (minimal) reproducible test and/or stack traces.
I noticed that the following lines are not necessary in TinySharp, looks the program can run without them...
Let's keep issues on this issue board related to AsmResolver only. For comments on TinySharp specifically, either add comments to the gist or on the blog post itself.
please provide a (minimal) reproducible test and/or stack traces.
I'm not much of a C# developer, so I've put the code here: https://github.com/steve02081504/ps12exe/blob/TinySharpResource/src/programFrames/TinySharp.cs The call way can be seen here: https://github.com/steve02081504/ps12exe/blob/TinySharpResource/src/TinySharpCompiler.ps1#L22L30
Please take the time to check it if it's convenient
Your resources directory is null
. You made the mistake in the lowering of the C# version here:
if(this.Image.Resources!=null) this.Image.Resources = new ResourceDirectory(0u);
Invert the if statement like so:
if(this.Image.Resources == null) this.Image.Resources = new ResourceDirectory(0u);
Thank you for helping me find this stupid bug!
After fixing the bug and replacing varTable.Values.Add(0x4b0);
with varTable.Values.Add(0x4b00000);
, I made the string resource part work.
Now the question comes to the icon resource section:
I've tried all 3 of these codes, but none of them generate the correct icon resources, and I can't find the problem myself
var iconBytes = BitConverter.ToUInt32(header, 14);//from wiki
byte[] icon = header.Skip(0x16).Take((int)iconBytes).ToArray();
byte[] icon = header.Skip(0x16).ToArray();
byte[] icon = header;
You will have to make sure that the resource data is in the right format. That is, BMP pixel data (sequence of colors, no bmp header) or a PNG (in its entirety). Currently, AsmResolver cannot do this automatically for you out-of-the-box.
Ok Ill just leave it in todo... Thx for help!✨
Description
had the pleasure of reading your article on how to generate the smallest exe and tried to add it to my project (just for fun) https://github.com/steve02081504/ps12exe/commit/d9e89216ac1fde275c39d0ac48cfa6a88ad67959 Ran into some problems on the way to adding the relevant code: