dotnet / sdk

Core functionality needed to create .NET Core projects, that is shared between Visual Studio and CLI
https://dot.net/core
MIT License
2.67k stars 1.06k forks source link

Provide a way for easily including localized text resources into a build #26337

Open tcrass opened 2 years ago

tcrass commented 2 years ago

Is your feature request related to a problem? Please describe.

For like two days I've tried to figure out how to include localized restext files (like in /Resources/Strings.en_US.restext) into a dotnet build in a way that would provide me with corresponding strongly typed resource files, embedded resources and the corresponding satelite assemblies, but this seems to be currently impossible due to the fact that the current .NET Core MSBuild implementation does not support the required funtionality. At least that's what I deduce from

    <ItemGroup>
        <TextResource Include="Resources\Strings\*.restext"/>
    </ItemGroup>

    <Target Name="GenerateResourceBinaries" BeforeTargets="Build">
        <Message Text="Generating resource binaries..."/>
        <GenerateResource Sources="@(TextResource)" OutputResources="Resources/Strings->'$(IntermediateOutputPath)%(Filename).resources')">
            <Output TaskParameter="OutputResources" ItemName="ResourceBinary" />
        </GenerateResource>
    </Target>

resulting in

error : ResGen.exe not supported on .NET Core MSBuild

when running "dotnet.exe build" on my project.

Of course, I could try to locate resgen.exe and manually run it on my project before the dotnet build, but for instance in a CI environment, a dotnet SDK project should be self-contained and be fully buildable by dotnet.exe on its own -- at least as long as such fundamental things are concerned as localizing an app.

Describe the solution you'd like

I think .NET Core MSBuild should really be able to transform restext files into the usual target formats also supported by resgen.exe.

dotnet-issue-labeler[bot] commented 2 years ago

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

KalleOlaviNiemitalo commented 2 years ago

Add ExecuteAsTool="false" to the GenerateResource task to avoid the ResGen.exe error.

Also, OutputResources="Resources/Strings->'$(IntermediateOutputPath)%(Filename).resources')" looks malformed.

tcrass commented 2 years ago

Add ExecuteAsTool="false" to the GenerateResource task to avoid the ResGen.exe error.

Ah, right, that indeed allowed me to generate *.resources from restext files. Never would have guessed that from the documentation, though:

Optional Boolean parameter.

If true, runs tlbimp.exe and aximp.exe from the appropriate target framework out-of-proc to generate the necessary wrapper assemblies. This parameter allows multi-targeting of ResolveComReferences.

Regarding this:

Also, OutputResources="Resources/Strings->'$(IntermediateOutputPath)%(Filename).resources')" looks malformed.

You are, of course, right -- I accidently copied & pasted some intermediate state of my experiments.

When I now try to generate resx files from my restext (as, I understand, a prerequisite for the generation of strongly typed resources aka .Designer.cs files) using something like

    <ItemGroup>
        <TextResource Include="Resources\**\*.restext"/>
    </ItemGroup>

    <Target Name="GenerateTextXmlResources" BeforeTargets="Build">
        <Message Text="Generating text xml resources" />
        <GenerateResource ExecuteAsTool="false" Sources="@(TextResource)" OutputResources="@(TextResource->'$(MSBuildProjectDirectory)\Properties\%(Filename).resx')">
            <Output TaskParameter="OutputResources" ItemName="TextXmlResources" />
        </GenerateResource>
    </Target>

the build complains

error : XML not supported on .NET Core MSBuild

Am I on the wrong track? Is there in dotnet sdk another recommended way to create strongly typed resources from restext files?

KalleOlaviNiemitalo commented 2 years ago

OutputResources="@(TextResource->'$(MSBuildProjectDirectory)\Properties\%(Filename).resx')"

Why would you generate resx files from restext files at build time? That is not necessary for strongly-typed resource classes.

KalleOlaviNiemitalo commented 2 years ago

Have you tried the StronglyTypedLanguage parameter of the GenerateResource task?

KalleOlaviNiemitalo commented 2 years ago

If this is a C# project rather than custom MSBuild project, then you can do the following instead of adding a custom target.

StrongRes.csproj:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>

  <ItemGroup>
    <None Remove="Demo.restext" />
  </ItemGroup>

  <ItemGroup>
    <EmbeddedResource Include="Demo.restext">
      <StronglyTypedLanguage>C#</StronglyTypedLanguage>
      <StronglyTypedNamespace>$(RootNamespace)</StronglyTypedNamespace>
      <StronglyTypedClassName>%(Filename)</StronglyTypedClassName>
    </EmbeddedResource>
  </ItemGroup>

</Project>

Demo.restext:

Key=Value

This generates obj/Debug/net6.0/StrongRes.Demo.cs, which defines internal class Demo in namespace StrongRes, and obj/Debug/net6.0/StrongRes.Demo.resources. Then compiles the generated source and embeds the resource into the assembly.

tcrass commented 2 years ago

Why would you generate resx files from restext files at build time? That is not necessary for strongly-typed resource classes.

OK, seems I got that wrong in the first place.

Have you tried the StronglyTypedLanguage parameter of the GenerateResource task?

I tried now -- and it kinda works. However, apparently strongly typed resource generation can't cope with file collections, so I can't use @(TextResource), but I'd have to call the GenerateResource task for each language version separately. But anyway, this...

If this is a C# project rather than custom MSBuild project, then you can do the following instead of adding a custom target.

...appears to be more like the "canonical" way. (Yes, it's about a C# project, which is supposed to be both developed in VS and built with dotnet.exe on our CI server.)

This solution also kind of works -- I indeed do get both binary and strongly typed resources bearing their respective language code in their filenames as well as satellite assemblies, but

Am I missing something, or am I demanding the impossible?

KalleOlaviNiemitalo commented 2 years ago

the generated .cs files contain classes whose names still contain the restexts' language codes (like "class strings_en_US {...}")

Rename the restext (or resx) file to remove the language code. Place it in the NeutralLanguage property instead. If you translate the same resources to other languages, keep the language codes in the names of those files but don't generate C# from them.

"var text = Some.Root.Namespace.strings_en_US.Foo" gives me a "type does not exist" error.

Is that a build-time error or something from a code editor?

tcrass commented 2 years ago

Rename the restext (or resx) file to remove the language code. Place it in the NeutralLanguage property instead. If you translate the same resources to other languages, keep the language codes in the names of those files but don't generate C# from them.

OK, did that, seems to work. But...

Is that a build-time error or something from a code editor?

...this VS editor message still remains. The project compiles fine on the command-line, though.

KalleOlaviNiemitalo commented 2 years ago

Do you need to generate strongly-typed classes at build time, or could you do it at design time?

tcrass commented 2 years ago

Do you need to generate strongly-typed classes at build time, or could you do it at design time?

Design time, I guess -- for specifying labels and captions in UIs, for instance.

KalleOlaviNiemitalo commented 2 years ago

In Visual Studio 2017, I selected a .NET SDK project, chose Project → Add new item… (Ctrl+Shift+A), selected "Resources File", changed the name from "Resource1.resx" to "Resource1.restext", and clicked Add. Visual Studio created "Resource1.restext" with XML (not plain text) content and immediately generated a "Resource1.Designer.cs" file from that. It added the following to the csproj file:

  <ItemGroup>
    <Compile Update="Resource1.Designer.cs">
      <DesignTime>True</DesignTime>
      <AutoGen>True</AutoGen>
      <DependentUpon>Resource1.restext</DependentUpon>
    </Compile>
  </ItemGroup>

  <ItemGroup>
    <None Update="Resource1.restext">
      <Generator>ResXFileCodeGenerator</Generator>
      <LastGenOutput>Resource1.Designer.cs</LastGenOutput>
    </None>
  </ItemGroup>

I then opened the "Resource1.restext" file in the editor, replaced all the XML with just one line of text "Key=Value", and saved that. Visual Studio did not generate a "Key" property in the "Resource1.Designer.cs" file. I right-clicked "Resource1.restext" in Solution Explorer and chose "Run Custom Tool". That had no apparent effect.

Because the item has <Generator>ResXFileCodeGenerator</Generator>, I think this generator requires the resx format and does not support the restext format. However, I don't know where ResXFileCodeGenerator is defined, or whether a similar generator exists for restext. I think ResXFileCodeGenerator is part of Visual Studio rather than .NET SDK. Perhaps https://developercommunity.visualstudio.com/ would be a better forum for design-time code generation.

tcrass commented 2 years ago

OK, almost there:

<Project Sdk="Microsoft.NET.Sdk">

    <PropertyGroup>
        <TargetFrameworks>net48</TargetFrameworks>
        <OutputType>WinExe</OutputType>
        <UseWPF>true</UseWPF>
                ...
        <NeutralLanguage>en-US</NeutralLanguage>
    </PropertyGroup>

    <!-- Include output folder for generated strongly typed text resources -->
    <ItemGroup>
        <Folder Include="Properties\" />
    </ItemGroup>

    <!-- Prevent default handling of text resource files -->
    <ItemGroup>
        <None Remove="Resources\Strings\*.restext" />
    </ItemGroup>

    <!-- Use a copy of en-US-specific text resource file for neutral-language resource generation -->
    <Target Name="CreateNeturalLanguageTextResource" BeforeTargets="BeforeBuild">
        <Message Text="Creating neutral language text resource" />
        <Copy SourceFiles="Resources\Strings\Strings.en-US.restext" DestinationFiles="Resources\Strings\Strings.restext" />
    </Target>

    <!-- Text resources handling -->
    <ItemGroup>
        <!-- Generate binary und strongly typed resource from neutral language text resource file -->
        <EmbeddedResource Include="Resources\Strings\Strings.restext">
            <DesignTime>True</DesignTime>
            <StronglyTypedLanguage>C#</StronglyTypedLanguage>
            <StronglyTypedClassName>%(Filename)</StronglyTypedClassName>
            <StronglyTypedNamespace>$(RootNamespace).Properties</StronglyTypedNamespace>
            <StronglyTypedFilename>$(MSBuildProjectDirectory)\Properties\%(Filename).cs</StronglyTypedFilename>
            <PublicClass>true</PublicClass>
        </EmbeddedResource>
        <!-- Generate language-specific resources (binary only) -->
        <EmbeddedResource Include="Resources\Strings\Strings.en-US.restext"/>
        <EmbeddedResource Include="Resources\Strings\Strings.de-DE.restext"/>
    </ItemGroup>

        ...

</Project>

This gives me a single Strings.cs file at a defined location which also gets recognized by VS, and the mentioned "unknow type" error goes away. I additionally included some logic that would allow me to use restext files explicitly featuring their corresponding language in their file name, even for the neutral language, so that it's obvious which .restext file is dedicated to which language.

But when trying to run the compiled application it crashes saying that no resource is found, neither for the current nor the neutral lanuage (which should be en-US in both cases, as specified by the NaturalLanguage tag>. The satellite assemblies, hoverer, are definitely sitting in their expected locations ({project root}\bin\Debug\net48\ {language code}\ {main assembly name}.resources.dll). How come they're not picked up when running the application?

KalleOlaviNiemitalo commented 2 years ago

Use ildasm.exe or Assembly.GetManifestResourceNames to check whether the logical names of the resources in the assemblies match the string that the generated code passes to the ResourceManager constructor, plus the language code and the ".resources" suffix.

tcrass commented 2 years ago

Use ildasm.exe or Assembly.GetManifestResourceNames to check whether the logical names of the resources in the assemblies match the string that the generated code passes to the ResourceManager constructor, plus the language code and the ".resources" suffix.

I opened the satellites with ildasm; here's the German language version as an example:

.assembly Some.Root.Namespace.resources
{
  .ver 1:0:0:0
  .locale = (64 00 65 00 2D 00 44 00 45 00 00 00 )             // d.e.-.D.E...
}

Here's the manifest content (with some sensitive information removed):

// Metadata version: v4.0.30319
.assembly extern mscorlib
{
  .publickeytoken = (B7 7A 5C 56 19 34 E0 89 )                         // .z\V.4..
  .hash = (4A CA 27 B1 F0 34 A9 72 97 C1 91 2D DE 4F 70 A8   // J.'..4.r...-.Op.
           4B 5F 60 C3 )                                     // K_`.
  .ver 4:0:0:0
}
.assembly Some.Root.Namespace.resources
{
  .custom instance void [mscorlib]System.Reflection.AssemblyTitleAttribute::.ctor(string) = ( REMOVED )
  .custom instance void [mscorlib]System.Reflection.AssemblyCompanyAttribute::.ctor(string) = ( REMOVED )
  .custom instance void [mscorlib]System.Reflection.AssemblyProductAttribute::.ctor(string) = ( REMOVED ) 
  .custom instance void [mscorlib]System.Reflection.AssemblyInformationalVersionAttribute::.ctor(string) = ( 01 00 05 31 2E 30 2E 30 00 00 )                   // ...1.0.0..
  .custom instance void [mscorlib]System.Reflection.AssemblyFileVersionAttribute::.ctor(string) = ( 01 00 07 31 2E 30 2E 30 2E 30 00 00 )             // ...1.0.0.0..
  .hash algorithm 0x00008004
  .ver 1:0:0:0
  .locale = (64 00 65 00 2D 00 44 00 45 00 00 00 )             // d.e.-.D.E...
}
.mresource public 'Some.Root.Namespace.Resources.Strings.Strings.de-DE.resources'
{
  // Offset: 0x00000000 Length: 0x000000CD
}
.module Some.Root.Namespace.resources.dll
// MVID: {70CE1301-858D-4FE9-9DFA-8D37807E7BB8}
.imagebase 0x10000000
.file alignment 0x00000200
.stackreserve 0x00100000
.subsystem 0x0003       // WINDOWS_CUI
.corflags 0x00000001    //  ILONLY
// Image base: 0x0B680000

Now when comparing the .resource declaration with the ResourceManager's constructor arguments in the generated Strings.cs file...

        /// <summary>
        ///   Returns the cached ResourceManager instance used by this class.
        /// </summary>
        [global::System.ComponentModel.EditorBrowsableAttribute(global::System.ComponentModel.EditorBrowsableState.Advanced)]
        public static global::System.Resources.ResourceManager ResourceManager {
            get {
                if (object.ReferenceEquals(resourceMan, null)) {
                    global::System.Resources.ResourceManager temp = new global::System.Resources.ResourceManager("Some.Root.Namespace.Properties.Strings", typeof(Strings).Assembly);
                    resourceMan = temp;
                }
                return resourceMan;
            }
        }

...I'm not quite sure what to make of it -- the resource's name is "Some.Root.Namespace.Resources.Strings.Strings.de-DE.resources", but the constructor argument is "Some.Root.Namespace.Properties.Strings". So with...

the string that the generated code passes to the ResourceManager constructor, plus the language code and the ".resources" suffix

...I'd expect the resource to be called "Some.Root.Namespace.Resources.Strings.de-DE.resources" instead of "Some.Root.Namespace.Resources.Strings.Strings.de-DE.resources".

tcrass commented 2 years ago

FYI: Raised a corresponding issue at https://stackoverflow.com/questions/72773242/generate-localized-string-resources-when-building-with-dotnet-sdk

KalleOlaviNiemitalo commented 2 years ago

It seems you can make the names match in any of these ways:

<DependentUpon>..\..\Properties\Strings.cs</DependentUpon> metadata might also work, but it requires that the C# file already exist and define a class in the correct namespace.

Or move the restext files to a directory that matches the namespace, I suppose.

Sdk26337.csproj ```XML net48 Exe en-US $(DefaultItemExcludes);Properties\Strings.cs True C# %(Filename) $(RootNamespace).Properties $(MSBuildProjectDirectory)\Properties\%(Filename).cs true $(RootNamespace).Resources.Strings ```
Program.cs ```C# using System; using System.Globalization; namespace Sdk26337 { class Program { static void Main() { CultureInfo.CurrentUICulture = CultureInfo.GetCultureInfo("de-DE"); Console.WriteLine(Properties.Strings.Heart); } } } ```
Resources/Strings/Strings.de-DE.restext ``` Heart=Herz ```
Resources/Strings/Strings.en-US.restext ``` Heart=heart ```
tcrass commented 2 years ago

Nice! I tried out your first suggestion (which required just on an additional tag, as opposed to the other two choices), and it worked like a charme!

So my résumé is that in SDK-style projects it is already possible to generate strongly-typed, localized string resources from restext files, but it is far from obvious how to actualy do this. To this end I'd like to re-phrase my feature request:

Or, even better, do both! :)

marcpopMSFT commented 2 years ago

@tdykstra Per the last comment, there's an ask to improve our localization documentation on how to do this. Seems like the customer found a way but it took some time. Moving the request to make this easier in the SDK onto the backlog for future consideration.

CyberSinh commented 1 year ago

Not having <Generator>ResTextFileCodeGenerator</Generator> provided by default is a real miss. The ResX format seems to be the only first-class citizen resource file format.