Valid Base64 can be treated as Invalid by Base64.DecodeFromUtf8 in net9.0

[keegan-caruso]

Description

While upgrading to net9, our tests picked up on a case where valid base64 data was rejected as invalid data by System.Buffers.Text.Base64.DecodeFromUtf8.

Reproduction Steps

This passes on net8 but fails on net9.

        [Fact]
        public void SmokeTest_Base64()
        {
            var data = "G/i6gi4rnMda3wgPSuSt1AsXnxnINzql1ptdoq3yV0IPGDXsxrmqL+YiWpIXcU0MQEZ/1EZqZOxdbg7CCx6347BtMxoSM6SOQzZZxXTh4WQjT9/WYxiqxDHfTRaHVs3k4mwaLqaQugF9ZURR0tGJvsqKcX1SyIYWlH5n+eNTjJFBvOcW6HVNMh1chWEHmqwkYCTrHiDnSwIqN/+/cLrxO8wonFnf/bkQuAq/nxVNMoTpG08QQ2Mlc7J2/oFWZCgP8KxpHFbo6axMNzgj54RBxj0+n5ctDq1ncvLoZjhydEr4mSxogJ6rvuhZElmy3H7H9qWq2rw87MvZmnl1XqO4/Aa=";

            var resultFromConvert = Convert.FromBase64String(data);

            var buffer = new byte[System.Buffers.Text.Base64.GetMaxEncodedToUtf8Length(data.Length)];
            var result = System.Buffers.Text.Base64.DecodeFromUtf8(Encoding.UTF8.GetBytes(data), buffer, out int bytesConsumed, out int bytesWritten);

            var resultFromDecode = buffer.AsMemory(0, bytesWritten);
            Assert.Equal(resultFromConvert.AsMemory(), resultFromDecode);
            Assert.Equal(System.Buffers.OperationStatus.Done, result);
        }

Convert.FromBase64String behaves the same on both net8 and net9.

Expected behavior

Valid base64 is parsed.

Actual behavior

When valid base64 is parsed, OperationStatus.InvalidData is returned in some cases.

Regression?

Yes, this worked in net8.

Known Workarounds

At a minimum, Convert.FromBase64String, parses this string as valid base64 data.

Configuration

.NET SDK: Version: 9.0.100 Commit: 59db016f11 Workload version: 9.0.100-manifests.c6f19616 MSBuild version: 17.12.7+5b8665660

Runtime Environment: OS Name: Windows OS Version: 10.0.26100 OS Platform: Windows RID: win-x64 Base Path: C:\Program Files\dotnet\sdk\9.0.100\

Host: Version: 9.0.0 Architecture: x64 Commit: 9d5a6a9aa4

.NET SDKs installed: 6.0.428 [C:\Program Files\dotnet\sdk] 8.0.307 [C:\Program Files\dotnet\sdk] 9.0.100-rc.2.24474.11 [C:\Program Files\dotnet\sdk] 9.0.100 [C:\Program Files\dotnet\sdk]

Other information

No response

[pmaytak]

Also System.Buffers.Text.Base64.IsValid(data); returns true on .NET 8 and false on .NET 9.

[dotnet-policy-service[bot]]

Tagging subscribers to this area: @dotnet/area-system-buffers See info in area-owners.md if you want to be subscribed.

[MihaZupan]

This was an intentional change made in #105262 to treat inputs with non-zero trailing bits as invalid. "/Aa=" in your input was decoded as [0xFC, 0x06], but the proper way to encode those two bytes is /AY=.

The behavior not matching between Convert.FromBase64String and Base64.Decode feels like an oversight wasn't implemented yet - #105264.