[3.0.0] invalid utf8 characters in comments

[timsnyder]

Using the tarball downloaded from https://github.com/accellera-official/systemc/archive/refs/tags/3.0.0_pub_rev_20231129.tar.gz

Latest versions of clang have a -Winvalid-utf8 warning that complains if comments have invalid utf8 characters in them. When I build on macos using the following versions of things:

Build system Details

``` Agent name: 'Azure Pipelines 21' Agent machine name: 'Mac-1703939497278' Current agent version: '3.232.0' Operating System macOS 11.7.10 20G1427 Runner Image Image: macos-11 Version: 20231216.1 Included Software: https://github.com/actions/runner-images/blob/macOS-11/20231216.1/images/macos/macos-11-Readme.md Image Release: https://github.com/actions/runner-images/releases/tag/macOS-11%2F20231216.1 ## Package Plan ## environment location: /Users/runner/miniforge3/conda-bld/systemc-split_1703954718054/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place The following NEW packages will be INSTALLED: libcxx: 16.0.6-hd57cbcb_0 conda-forge Preparing transaction: ...working... done Verifying transaction: ...working... done Executing transaction: ...working... done Reloading output folder: /Users/runner/miniforge3/conda-bld Reloading output folder: /Users/runner/miniforge3/conda-bld ## Package Plan ## environment location: /Users/runner/miniforge3/conda-bld/systemc-split_1703954718054/_build_env The following NEW packages will be INSTALLED: autoconf: 2.71-pl5321hed12c24_1 conda-forge automake: 1.16.5-pl5321h694c41f_0 conda-forge bzip2: 1.0.8-h10d778d_5 conda-forge c-ares: 1.24.0-h10d778d_0 conda-forge ca-certificates: 2023.11.17-h8857fd0_0 conda-forge cctools_osx-64: 973.0.1-ha1c5b94_15 conda-forge clang: 16.0.6-hac416ee_3 conda-forge clang-16: 16.0.6-default_h6b1ee41_3 conda-forge clang_impl_osx-64: 16.0.6-h8787910_7 conda-forge clang_osx-64: 16.0.6-hb91bd55_7 conda-forge clangxx: 16.0.6-default_h6b1ee41_3 conda-forge clangxx_impl_osx-64: 16.0.6-h6d92fbe_7 conda-forge clangxx_osx-64: 16.0.6-hb91bd55_7 conda-forge cmake: 3.28.1-h7c85d92_0 conda-forge compiler-rt: 16.0.6-ha38d28d_2 conda-forge compiler-rt_osx-64: 16.0.6-ha38d28d_2 conda-forge icu: 73.2-hf5e326d_0 conda-forge krb5: 1.21.2-hb884880_0 conda-forge ld64_osx-64: 609-ha20a434_15 conda-forge libclang-cpp16: 16.0.6-default_h6b1ee41_3 conda-forge libcurl: 8.5.0-h726d00d_0 conda-forge libcxx: 16.0.6-hd57cbcb_0 conda-forge libedit: 3.1.20191231-h0678c8f_2 conda-forge libev: 4.33-h10d778d_2 conda-forge libexpat: 2.5.0-hf0c8a7f_1 conda-forge libiconv: 1.17-hd75f5a5_2 conda-forge libllvm16: 16.0.6-hbedff68_3 conda-forge libnghttp2: 1.58.0-h64cf6d3_1 conda-forge libssh2: 1.11.0-hd019ec5_0 conda-forge libtool: 2.4.7-hf0c8a7f_0 conda-forge libuv: 1.46.0-h0c2f820_0 conda-forge libxml2: 2.12.3-hc0ae0f7_0 conda-forge libzlib: 1.2.13-h8a1eda9_5 conda-forge llvm-tools: 16.0.6-hbedff68_3 conda-forge m4: 1.4.18-haf1e3a3_1001 conda-forge make: 4.3-h22f3db7_1 conda-forge ncurses: 6.4-h93d8f39_2 conda-forge openssl: 3.2.0-hd75f5a5_1 conda-forge perl: 5.32.1-7_h10d778d_perl5 conda-forge rhash: 1.4.4-h0dc2134_0 conda-forge sigtool: 0.1.3-h88f4db0_0 conda-forge tapi: 1100.0.11-h9ce4665_0 conda-forge xz: 5.2.6-h775f41a_0 conda-forge zstd: 1.5.5-h829000d_0 conda-forge Preparing transaction: ...working... done Verifying transaction: ...working... done Executing transaction: ...working... done ``` [The full output of the build will be available for 30 days on Azure.](https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=850858&view=logs&jobId=58ac6aab-c4bc-5de2-2894-98e408cc8ec9&j=58ac6aab-c4bc-5de2-2894-98e408cc8ec9&t=933f325c-924e-533d-4d95-e93b5843ce8b)

I run into the following:

x86_64-apple-darwin13.4.0-clang++ -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -stdlib=libc++ -fvisibility-inlines-hidden -fmessage-length=0 -isystem $PREFIX/include -fdebug-prefix-map=$SRC_DIR=/usr/local/src/conda/systemc-dev-3.0.0dev20231129 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem $PREFIX/include -fdebug-prefix-map=$SRC_DIR=/usr/local/src/conda/systemc-dev-3.0.0dev20231129 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -g -Wall -pedantic -Wno-long-long -Werror -I../include -I../../common/include -I. -I.. -I$PREFIX/include  -c ../../common/src/traffic_generator.cpp -o traffic_generator.o
../../common/src/traffic_generator.cpp:251:30: error: invalid UTF-8 in comment [-Werror,-Winvalid-utf8]
      // The address for the <93>gp<94> is used as expected data.  The address filed of 
                             ^
../../common/src/traffic_generator.cpp:251:33: error: invalid UTF-8 in comment [-Werror,-Winvalid-utf8]
      // The address for the <93>gp<94> is used as expected data.  The address filed of 
                                   ^
../../common/src/traffic_generator.cpp:252:15: error: invalid UTF-8 in comment [-Werror,-Winvalid-utf8]
      //  the <93>gp<94> is a mutable field and is changed by the SimpleBus interconnect. 
              ^
../../common/src/traffic_generator.cpp:252:18: error: invalid UTF-8 in comment [-Werror,-Winvalid-utf8]
      //  the <93>gp<94> is a mutable field and is changed by the SimpleBus interconnect. 
                    ^

NOTE: this is not an exhaustive list of the failures, there are other files that fail after this first one.

I was able to successfully work around this by adding -Wno-error=invalid-utf8. I was also able to clean individual files by asking Vim to :set fileencoding=utf8 and then writing the files. If a PR would be valuable, just let me know and I'll open one.

It would probably be nice to clean this up before the official 3.0 release as I'm sure more folks will run into clang being annoying as we move into the future.

[timsnyder]

I suppose I should try building on master, only to confirm that the issue isn't already resolved there...

[timsnyder]

I also see this on master as of bd3b79756597f9debc5d255d3ee0fa6c70ff73d2. I see that #65 recently added GHA CI for the examples but only on some Linux targets. Might be good to also add macos? I don't mind filing issues if/when I see issues on macos or windows in my conda-forge builds downstream.

[lmailletcontoz]

Thanks for the input. Would it be possible for you to create the Pull Request for us (following the contributor guidelines), please?

[barnasc]

Should be fixed in https://github.com/accellera-official/systemc/commit/7e36d43296e81b48d907d00eb08f07436971316a. I will close the ticket, feel free to reopen if the issue remains