Archiver failed to make tar

hisea commented 8 years ago

I'm trying to make a prod release but it failed to make the tar:

{{case_clause,
     {'EXIT',
         {function_clause,
             [{filename,join,[[]],[{file,"filename.erl"},{line,395}]},
              {erl_tar,split_filename,4,[{file,"erl_tar.erl"},{line,471}]},
              {erl_tar,create_header,3,[{file,"erl_tar.erl"},{line,400}]},
              {erl_tar,add1,4,[{file,"erl_tar.erl"},{line,323}]},
              {erl_tar,foreach_while_ok,2,[{file,"erl_tar.erl"},{line,1000}]},
              {systools_make,add_to_tar,3,
                  [{file,"systools_make.erl"},{line,1902}]},
              {systools_make,add_appl,7,
                  [{file,"systools_make.erl"},{line,1773}]},
              {systools_make,'-add_applications/5-fun-0-',6,
                  [{file,"systools_make.erl"},{line,1592}]}]}}},
 [{systools_make,'-add_applications/5-fun-0-',6,
      [{file,"systools_make.erl"},{line,1592}]},
  {lists,foldl,3,[{file,"lists.erl"},{line,1263}]},
  {systools_make,add_applications,5,[{file,"systools_make.erl"},{line,1591}]},
  {systools_make,mk_tar,6,[{file,"systools_make.erl"},{line,1585}]},
  {systools_make,mk_tar,5,[{file,"systools_make.erl"},{line,1561}]},
  {systools_make,make_tar,2,[{file,"systools_make.erl"},{line,336}]},
  {'Elixir.Mix.Releases.Archiver',make_tar,1,
      [{file,"lib/mix/lib/releases/archiver.ex"},{line,45}]},
  {'Elixir.Mix.Releases.Archiver',archive,1,
      [{file,"lib/mix/lib/releases/archiver.ex"},{line,17}]}]}
==> Problem generating release tarball:
    {:tar_generation_error, :unknown}

I actually dont need the tar file but didn't find anything in the doc to turn this off.

hisea commented 8 years ago

I tried on Mac El Capitan but same result in a ubuntu docker env. maybe I'm missing some system libs?

bitwalker commented 8 years ago

You can skip the archival step with mix release --no-tar. This error is caused due to filenames exceeding the length limit that systools allows.

bitwalker commented 8 years ago

I still have it on my list of things to look into, but this may be a hard limit imposed by erl_tar, at least until the Erlang/OTP team addresses it, if it's indeed something which can be addressed. Unfortunately I'm not sure that it's something I can work around, but like I said, I still have to look into how it might be fixed.

hisea commented 8 years ago

Thanks @bitwalker for your fast response.

Release builds fine now I got another rookie question though, It looks like the deps are not built in with the release.

if i run the release in console or foreground, I get an module not found error: function Guardian.Plug.VerifySession.call/2 is undefined (module Guardian.Plug.VerifySession is not available)

it fix this error if I include guardian in my applications in mix.exs.

however, another one shows up for one of the lib I use.

here's my rel/config.exs

use Mix.Releases.Config,
    # This sets the default release built by `mix release`
    default_release: :default,
    # This sets the default environment used by `mix release`
    default_environment: :dev

# For a full list of config options for both releases
# and environments, visit https://hexdocs.pm/distillery/configuration.html

# You may define one or more environments in this file,
# an environment's settings will override those of a release
# when building in that environment, this combination of release
# and environment configuration is called a profile

environment :dev do
  set dev_mode: true
  set include_erts: false
end

environment :prod do
  set include_erts: true
  set include_src: false
end

# You may define one or more releases in this file.
# If you have not set a default release, or selected one
# when running `mix release`, the first release in the file
# will be used by default

release :myapp do
  set version: current_version(:boston)
end

bitwalker commented 8 years ago

Could you also post your mix.exs? This may be related to another bug which is open, and if so I'll close this one and update the other issue. Distillery should be handling adding your deps to the release, whether they are in applications or not, but it's possible something is getting missed.

hisea commented 8 years ago

sure, here it is:

defmodule Myapp.Mixfile do
  use Mix.Project

  def project do
    [app: :myapp,
     version: "0.0.1",
     elixir: "~> 1.0",
     elixirc_paths: elixirc_paths(Mix.env),
     compilers: [:phoenix, :gettext] ++ Mix.compilers,
     build_embedded: Mix.env == :prod,
     start_permanent: Mix.env == :prod,
     aliases: aliases,
     deps: deps]
  end

  # Configuration for the OTP application.
  #
  # Type `mix help compile.app` for more information.
  def application do
    [mod: {Myapp, []},
     applications: [:phoenix, :phoenix_html, :phoenix_pubsub, :icalendar, :cowboy, :logger, :gettext,
                    :phoenix_ecto, :ex_admin, :postgrex, :bamboo, :ueberauth, :ueberauth_cas, :timex]]
  end

  # Specifies which paths to compile per environment.
  defp elixirc_paths(:test), do: ["lib", "web", "test/support"]
  defp elixirc_paths(_),     do: ["lib", "web"]

  # Specifies your project dependencies.
  #
  # Type `mix help deps` for examples and options.
  defp deps do
    [{:phoenix, "~> 1.2.0"},
     {:phoenix_pubsub, "~> 1.0"},
     {:postgrex, ">= 0.0.0"},
     {:phoenix_ecto, "~> 3.0"},
     {:phoenix_html, "~> 2.4"},
     {:phoenix_live_reload, "~> 1.0", only: :dev},
     {:bamboo, git: "https://github.com/hisea/bamboo.git", branch: "feature/attachments"},
     {:icalendar, git: "https://github.com/praveenperera/icalendar.git"},
     {:ex_admin, github: "smpallen99/ex_admin"},
     {:gettext, "~> 0.9"},
     {:timex, "~> 3.0"},
     {:ueberauth, "~> 0.2"},
     {:guardian, "~> 0.12.0"},
     {:timex, "~> 3.0.4"},
     {:distillery, github: "bitwalker/distillery"},
     {:ueberauth_cas, git: "https://github.com/EdevMosaic/ueberauth-cas.git" },
     {:exrm, "1.0.4"},
     {:scrivener_html, "~> 1.1"},
     {:mosaic_statux, git: "https://mosaicbot:7fa53a7db52a9b24a7ba337425c7c1ec2511ce17@github.com/EdevMosaic/mosaic_statux.git"},
     {:cowboy, "~> 1.0"}]
  end

  # Aliases are shortcut or tasks specific to the current project.
  # For example, to create, migrate and run the seeds file at once:
  #
  #     $ mix ecto.setup
  #
  # See the documentation for `Mix` for more info on aliases.
  defp aliases do
    ["ecto.setup": ["ecto.create", "ecto.migrate", "run priv/repo/seeds.exs"],
     "ecto.reset": ["ecto.drop", "ecto.setup"],
    "test": ["ecto.create --quiet", "ecto.migrate", "test"]]
  end
end

bitwalker commented 8 years ago

Indeed it does look like this is an issue. I'll close this for now, and update the other ticket. In the meantime, either add applications to your applications list, or:

release :myapp do
  set applications: [
    :some_dep_a, # to load and start the application
    some_dep_b: :load # to only load the application, not start it
  ]

I'll be pushing a fix for this today.

hisea commented 8 years ago

Thanks @bitwalker

slashdotdash commented 7 years ago

I've stumbled into the tar_generation_error bug when attempting to build a release for an umbrella app.

==> Writing tarball to rel/example_app/releases/0.1.0/example_app.tar.gz
{{case_clause,
     {'EXIT',
         {function_clause,
             [{filename,join,[[]],[{file,"filename.erl"},{line,395}]},
              {erl_tar,split_filename,4,[{file,"erl_tar.erl"},{line,471}]},
              {erl_tar,create_header,3,[{file,"erl_tar.erl"},{line,400}]},
              {erl_tar,add1,4,[{file,"erl_tar.erl"},{line,323}]},
              {systools_make,add_to_tar,3,
                  [{file,"systools_make.erl"},{line,1903}]},
              {lists,foreach,2,[{file,"lists.erl"},{line,1338}]},
              {systools_make,'-add_applications/5-fun-0-',6,
                  [{file,"systools_make.erl"},{line,1593}]},
              {lists,foldl,3,[{file,"lists.erl"},{line,1263}]}]}}},
 [{systools_make,'-add_applications/5-fun-0-',6,
      [{file,"systools_make.erl"},{line,1593}]},
  {lists,foldl,3,[{file,"lists.erl"},{line,1263}]},
  {systools_make,add_applications,5,[{file,"systools_make.erl"},{line,1592}]},
  {systools_make,mk_tar,6,[{file,"systools_make.erl"},{line,1586}]},
  {systools_make,mk_tar,5,[{file,"systools_make.erl"},{line,1562}]},
  {systools_make,make_tar,2,[{file,"systools_make.erl"},{line,336}]},
  {'Elixir.Mix.Releases.Archiver',make_tar,1,
      [{file,"lib/mix/lib/releases/archiver.ex"},{line,46}]},
  {'Elixir.Mix.Releases.Archiver',archive,1,
      [{file,"lib/mix/lib/releases/archiver.ex"},{line,17}]}]}
==> Problem generating release tarball:
    {:tar_generation_error, :unknown}

I added a log statement to capture the opts passed to :systools.make_tar:

[path: ['rel/example_app/lib/*/ebin'], dirs: [:include], outdir: 'rel/example_app/releases/0.1.0', erts: '/usr/lib/erlang']

Any suggestions on how to workaround this issue? I am attempting to use distillery with edeliver. This requires the tar file to be created on the build host so that it can copied to the production host. So I can't use the --no-tar workaround.

slashdotdash commented 7 years ago

@bitwalker ~~Where would I find out more about the filename length limit restriction in systools?~~

I've been reading about the 99 character filename limit in erl_tar.

That is problematic for Elixir apps that implement third party library protocols due to the length of the generated filename format: Elixir.Some.External.Library.Example.Protocol.MyApp.Foo.Bar.Baz.Qux.Example.Protocol.Implementation.beam. I have many instances of filenames exceeding 99 characters in the Elixir app I've built for this reason.

Attempting to create and extract a tar archive containing the release lib directory using tar works fine: tar cvf app.tar rel/app/lib/*/ebin.

slashdotdash commented 7 years ago

I've tracked down the source of the 100 character limit. It's the th_name_len constant configured in erl_tar. This is called by split_filename when writing out the header for each file added to the archive in create_header.

slashdotdash commented 7 years ago

Here's a forked version of distillery containing a branch with an additional module in the standard app test fixture with a name exceeding 100 characters.

https://github.com/slashdotdash/distillery/commits/bug/tar-filename-length

This exhibits the tar generation failure when running the unit tests.

bitwalker commented 7 years ago

After some research while attempting to put together a PR, I've found the following:

Due to the way GNU tar is implemented, we have a hard limit on the filename component of the path being 100 bytes long. See here for more information, I'll touch on the gist below.
The total path (which is relative) can be 255 bytes long, but because the path must be split on a path boundary, where the prefix component can be up to 155 bytes, and the filename component can be up to 100 bytes, in practice this means that you likely have less than 255 bytes to work with.
erl_tar doesn't properly handle errors when a path component is too long, so it's very difficult to understand what's going on
erl_tar doesn't properly handle the case where the total filename length is over 255

I've got a PR built up to solve the last two, but ultimately, the maximum filename length is limited to 100 bytes (this is the maximum size of the name field of the tar header, even with in ustar-formatted archives). This limitation is due to the fact that GNU tar assumes the prefix and the name parts of the header are split at a path boundary, the maximum length of the path holding a file is 155 bytes, and the maximum length of the filename portion of the path is 100 bytes. In an experiment, I made it possible to have filenames up to 255 bytes long, depending on the length of the path it's under, by simply splitting the path at 100 bytes, then joining the prefix and the name portions of the header when extracting. This worked fine, but unfortunately breaks when extracting the release tarball with GNU tar.

I'm starting to think it may be more desirable to switch to a packaging format which allows us to work with less restrictions. Zip allows up to 260 bytes for a name I believe, not counting the path, which is likely good enough for our purposes - however, the problem is that we cannot tell systools to pack/unpack releases using anything other than erl_tar. Making that configurable is a more significant task I believe, but one that is probably worth discussing with the OTP team - assuming they are concerned about problems Elixir is surfacing which don't really apply to Erlang.

/cc @josevalim @martin-langhoff Any thoughts you two may have on the above, I'd appreciate. The PR I have makes the errors much better, but we still ultimately face the same problem, which is that we are very constrained on the filename length, it will be very easy for people to hit this limit.

bitwalker commented 7 years ago

PR for the error handling fixes here

josevalim commented 7 years ago

@bitwalker I attempted to fix erl_tar and then all of the points you described above were pointed out to me: https://github.com/erlang/otp/pull/1268 :D

I believe the OTP team will welcome better error messages on erl_tar. They will probably welcome using something like zlib on systools as well. I assume we can detact which lib to use based on the file extension? If you need help with figuring stuff out in erlang/otp just ping me on IRC. :)

martin-langhoff commented 7 years ago

Perhaps erl_tar can be taught to write posix tar format instead of ustar? See https://www.gnu.org/software/tar/manual/html_section/tar_68.html

bitwalker commented 7 years ago

@josevalim I'm planning to investigate @martin-langhoff's suggestion of supporting the newer (and now standard) posix format, as it didn't exist at the time erl_tar was written/last updated, but I also have a PR (mostly complete) adding support for zipped releases, but I hit a snag there, and am still deciding whether to press on there, or just make sure erl_tar is updated. I kind of favor the latter to be honest, for simplicity (and I really think erl_tar should support the newer format anyway). Thanks for offering to help! Hopefully I won't need to take any of your time, but it's much appreciated :)

bitwalker commented 7 years ago

I have a PR open with OTP to solve this here for those interested in following along.

bitwalker commented 7 years ago

PR has been merged!

bitwalker / distillery

Archiver failed to make tar #39