search

dotnet / fsharp

The F# compiler, F# core library, F# language service, and F# tooling integration for Visual Studio
https://dotnet.microsoft.com/languages/fsharp
MIT License
3.9k stars 783 forks source link

F# XML Documentation is not outputting diacritics / accented characters #4160

Closed enricosada closed 6 years ago

enricosada commented 6 years ago

moved from https://github.com/fsharp/fsharp/issues/800 by @TonyHenrique

by @TonyHenrique:

I am developing a F# .Net Standard 2 Class Library. F# XML Documentation is not outputting diacritics / accented characters, for example: ç ã õ á outputs as � � � �

Repro steps

Provide the steps required to reproduce the problem

  1. Put some XML comments with letters ç ã o on a F# type:
///<summary>Descrição do tipo Caderno</summary>

or

/// tipo Assunto çá ã õ
  1. Inspect the generated XML

Expected behavior

<?xml version="1.0" encoding="UTF-8"?>
<doc>
<assembly>
<name>EntidadesFS</name>
</assembly>
<members>
<member name="T:EntidadesFS.DO.Assunto">
<summary> tipo Assunto </summary>
</member>
<member name="T:EntidadesFS.DO.Caderno">
<summary>Descricao do tipo Caderno</summary>
</member>
<member name="T:EntidadesFS.DO.DiarioOficial">
<summary>Descrição do tipo Diário Oficial</summary>
</member>
</members>
</doc>

Actual behavior

<?xml version="1.0" encoding="UTF-8"?>
<doc>
<assembly>
<name>EntidadesFS</name>
</assembly>
<members>
<member name="T:EntidadesFS.DO.Assunto">
<summary> tipo Assunto </summary>
</member>
<member name="T:EntidadesFS.DO.Caderno">
<summary>Descricao do tipo Caderno</summary>
</member>
<member name="T:EntidadesFS.DO.DiarioOficial">
<summary>Descri��o do tipo Di�rio Oficial</summary>
</member>
</members>
</doc>

Known workarounds

Avoid use ç ã õ á

Related information

Microsoft Visual Studio Community 2017 Preview Version 15.5.0 Preview 5.0 VisualStudio.15.Preview/15.5.0-pre.5.0+27128.1 Microsoft .NET Framework Version 4.7.02556

Visual F# 4.1 00369-60000-00001-AA447 Microsoft Visual F# 4.1

Click to expand for more environment info Microsoft Visual Studio Community 2017 Preview Version 15.5.0 Preview 5.0 VisualStudio.15.Preview/15.5.0-pre.5.0+27128.1 Microsoft .NET Framework Version 4.7.02556 Installed Version: Community Visual Basic 2017 00369-60000-00001-AA447 Microsoft Visual Basic 2017 Visual C# 2017 00369-60000-00001-AA447 Microsoft Visual C# 2017 Visual F# 4.1 00369-60000-00001-AA447 Microsoft Visual F# 4.1 Application Insights Tools for Visual Studio Package 8.10.01106.1 Application Insights Tools for Visual Studio ASP.NET and Web Tools 2017 15.0.31120.0 ASP.NET and Web Tools 2017 ASP.NET Core Razor Language Services 1.0 Provides languages services for ASP.NET Core Razor. ASP.NET Web Frameworks and Tools 2017 5.2.51007.0 For additional information, visit https://www.asp.net/ Azure App Service Tools v3.0.0 15.0.31106.0 Azure App Service Tools v3.0.0 Azure Data Lake Node 1.0 This package contains the Data Lake integration nodes for Server Explorer. Azure Data Lake Tools for Visual Studio 2.3.1000.1 Microsoft Azure Data Lake Tools for Visual Studio Azure Data Lake Tools for Visual Studio 2.3.1000.1 Microsoft Azure Data Lake Tools for Visual Studio Common Azure Tools 1.10 Provides common services for use by Azure Mobile Services and Microsoft Azure Tools. DotNetForHtml5.VisualStudioExtension.ShellIntegration 1.0 DotNetForHtml5.VisualStudioExtension.ShellIntegration Fabric.DiagnosticEvents 1.0 Fabric Diagnostic Events JavaScript Language Service 2.0 JavaScript Language Service JavaScript Project System 2.0 JavaScript Project System JavaScript UWP Project System 2.0 JavaScript UWP Project System Merq 1.1.17-rc (cba4571) Command Bus, Event Stream and Async Manager for Visual Studio extensions. Microsoft Azure HDInsight Azure Node 2.3.1000.1 HDInsight Node under Azure Node Microsoft Azure Hive Query Language Service 2.3.1000.1 Language service for Hive query Microsoft Azure Service Fabric Tools for Visual Studio 1.8 Microsoft Azure Service Fabric Tools for Visual Studio Microsoft Azure Stream Analytics Language Service 2.3.1000.1 Language service for Azure Stream Analytics Microsoft Azure Stream Analytics Node 1.0 Azure Stream Analytics Node under Azure Node Microsoft Azure Tools 2.9 Microsoft Azure Tools for Microsoft Visual Studio 2017 - v2.9.51120.3 Microsoft Continuous Delivery Tools for Visual Studio 0.3 Simplifying the configuration of continuous build integration and continuous build delivery from within the Visual Studio IDE. Microsoft JVM Debugger 1.0 Provides support for connecting the Visual Studio debugger to JDWP compatible Java Virtual Machines Microsoft MI-Based Debugger 1.0 Provides support for connecting Visual Studio to MI compatible debuggers Microsoft Visual Studio Tools for Containers 1.1 Develop, run, validate your ASP.NET Core applications in the target environment. F5 your application directly into a container with debugging, or CTRL + F5 to edit & refresh your app without having to rebuild the container. Microsoft Visual Studio VC Package 1.0 Microsoft Visual Studio VC Package Mono Debugging for Visual Studio 4.8.4-pre (3fe64e3) Support for debugging Mono processes with Visual Studio. NuGet Package Manager 4.5.0 NuGet Package Manager in Visual Studio. For more information about NuGet, visit http://docs.nuget.org/. SQL Server Data Tools 15.1.61710.120 Microsoft SQL Server Data Tools ToolWindowHostedEditor 1.0 Hosting json editor into a tool window TypeScript Tools 15.5.11025.1 TypeScript Tools for Microsoft Visual Studio Visual Studio Code Debug Adapter Host Package 1.0 Interop layer for hosting Visual Studio Code debug adapters in Visual Studio Visual Studio Tools for Unity 3.5.0.2 Visual Studio Tools for Unity Visual Studio Tools for Universal Windows Apps 15.0.27128.01 The Visual Studio Tools for Universal Windows apps allow you to build a single universal app experience that can reach every device running Windows 10: phone, tablet, PC, and more. It includes the Microsoft Windows 10 Software Development Kit. VisualStudio.Mac 1.0 Mac Extension for Visual Studio WebJobs Tools v1.0.0 15.0.31114.0 WebJobs Tools v1.0.0 Xamarin 4.8.0.752 (4a1f0ea) Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android. Xamarin Designer 4.8.182 (76aa1723e) Visual Studio extension to enable Xamarin Designer tools in Visual Studio. Xamarin.Android SDK 8.1.0.23 (HEAD/f65f87747) Xamarin.Android Reference Assemblies and MSBuild support. Xamarin.iOS and Xamarin.Mac SDK 11.4.0.214 (c4240f3) Xamarin.iOS and Xamarin.Mac Reference Assemblies and MSBuild support. XAML Editor (C#/XAML for HTML5) 1.0 (...)
eriawan commented 6 years ago

@enricosada To use diacritics successfully on summary in XML comments, I usually use HTML entity codes for that.

For example, instead of this on your sample:

<summary>Descrição do tipo Diário Oficial</summary>

I use this HTML Entity char codes:

<summary>Descri&ccedil;&atilde;o do tipo Di&atilde;rio Oficial</summary>

I have used this because basically the XML comments will be yielded after parsed as XML, therefore it must comply to HTML/XML Character Entity spec.

Some sample informal references: https://www.w3schools.com/charsets/ref_utf_diacritical.asp and this: http://www.starr.net/is/type/htmlcodes.html

Please let me know if it works or doesn't work 🙂

dsyme commented 6 years ago

Is this a bug vis.a.vis C# behaviour?

enricosada commented 6 years ago

@dsyme F# bug. i can repro. c# is ok

to repro:

same xml comment

    /// <summary>Descrição do tipo Diário Oficial</summary>

C#:

dotnet new lib
// add the xml comment
dotnet build  /p:GenerateDocumentationFile=true

output is:

<?xml version="1.0"?>
<doc>
    <assembly>
        <name>dia</name>
    </assembly>
    <members>
        <member name="T:dia.Class1">
            <summary>Descrição do tipo Diário Oficial</summary>
        </member>
    </members>
</doc>

F

dotnet new lib -lang F#
// add the xml comment
dotnet build  /p:GenerateDocumentationFile=true

output is:

<?xml version="1.0" encoding="utf-8"?>
<doc>
<assembly><name>diafs</name></assembly>
<members>
<member name="M:diafs.Say.hello(System.String)">
 <summary>Descri��o do tipo Di�rio Oficial</summary>
</member>
</members>
</doc>
enricosada commented 6 years ago

@dsyme checking the repro, the issue is that by default, F# source file Library.fs is ANSI, while ClassLibrary1.cs is UTF-8

saving Library.fs as UTF-8 fix the issue.

i'll check why default encoding of template differ

as a note, locally the c# doesnt generate the same issue, if i use ANSI for ClassLibrary1.cs, but generate same valid xml

enricosada commented 6 years ago

as additional note, the xml doc should also indent on save, like C#. atm it doesnt do that

eriawan commented 6 years ago

@enricosada

Saving to UTF-8 solve this? Thanks for the solution! I usually have to use HTML char entity for XML comment. Silly me!

dsyme commented 6 years ago

Closing as saving as UTF-8 solved the issue