Flaky test for phx.gen.schema_test.exs when using different command line arguments

[Exadra37]

Environment

• Elixir version (elixir -v):

  
  $ elixir -v
  Erlang/OTP 27 [erts-15.0] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit:ns]

Elixir 1.17.0 (compiled with Erlang/OTP 27)

* Phoenix version (mix deps):

{:phoenix, ">= 1.7.0"},

* Operating system:

$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu DISTRIB_RELEASE=22.04 DISTRIB_CODENAME=jammy DISTRIB_DESCRIPTION="Ubuntu 22.04.4 LTS"

### Actual behavior

If we modify the test at `test/mix/tasks/phx.gen.schema_test.exs:19` to use `~w(Blog.Post posts title:string desc:string` as the command line arguments, and then adjust the expected values we get it to pass the majority of the times but we can expect to experience some random failures.

#### How to reproduce

Add the flaky test to `test/mix/tasks/phx.gen.schema_test.exs` :

```elixir
  test "build (flaky)" do
    in_tmp_project "build", fn ->
      schema = Gen.Schema.build(~w(Blog.Post posts title:string desc:string), [])

      assert %Schema{
        alias: Post,
        module: Phoenix.Blog.Post,
        repo: Phoenix.Repo,
        migration?: true,
        migration_defaults: %{title: ""},
        plural: "posts",
        singular: "post",
        human_plural: "Posts",
        human_singular: "Post",
        attrs: [title: :string, desc: :string],
        types: %{title: :string, desc: :string},
        optionals: [],
        route_helper: "post",
        defaults: %{title: ""},
        params: %{
          create: %{title: "some title", desc: "some desc"},
          default_key: :title,
          update: %{title: "some updated title", desc: "some updated desc"}
        },
      } = schema
      assert String.ends_with?(schema.file, "lib/phoenix/blog/post.ex")
    end
  end

Then run from the terminal:

while mix test test/mix/tasks/phx.gen.schema_test.exs; do :; done

Then wait for it to fail with:

Expected behavior

This command should run for minutes/hours without experiencing a random failure:

while mix test test/mix/tasks/phx.gen.schema_test.exs; do :; done

[SteffenDE]

Is there something flaky about the generated files as well, or is it really just the tests?

[Exadra37]

It's just the tests. At a glance I cannot see any issue in the schema file.

[Gazler]

The test is likely failing due to a map ordering issue. The attributes are converted to a map at:

https://github.com/phoenixframework/phoenix/blob/293791fcd8570efc5375638267648459d9b125b3/lib/mix/phoenix/schema.ex#L495

For reference, inspecting these values in the tests:

Failure:

.[title: :string, desc: :string]
%{desc: :string, title: :string}

Success:

[title: :string, desc: :string]
%{title: :string, desc: :string}

Presumably these are used as a map to prevent duplicates? If they were kept as a keyword list, then the ordering would be deterministic.

[josevalim]

@Gazler I don't know see why we use maps. +1 for converting to a keyword list.