search

akka / alpakka

Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka.
https://doc.akka.io/docs/alpakka/current/
Other
1.26k stars 647 forks source link

TarArchiveEntry has some issues with padding #2768

Open marcadella opened 2 years ago

marcadella commented 2 years ago

Versions used

akka-stream-alpakka-file version: 3.0.3

Expected Behavior

I use the following code to create a tar.gz archive from a bunch of files:

val streams = paths.map { p =>
      TarArchiveMetadata(
        "",
        workdir.resolve(j.filesPath).relativize(p).toString,
        Files.size(p),
        Files.getLastModifiedTime(p).toInstant
      ) -> FileIO.fromPath(p)
    }

    Source(streams)
      .log("Archiving")
      .via(Archive.tar().via(akka.stream.scaladsl.Compression.gzip))

and it worked fine overall....

Actual Behavior

... But when the path list contains one specific file (16GB) I receive the following error from TarArchiveEntry: requirement failed: the padded data is 13 bytes, which does not fit into 12 bytes (cf log below). So it looks like there is a bug in TarArchiveEntry and padding with large files...

Relevant logs

requirement failed: the padded data is 13 bytes, which does not fit into 12 bytes
at scala.Predef$.require(Predef.scala:337)
at akka.stream.alpakka.file.impl.archive.TarArchiveEntry$.akka$stream$alpakka$file$impl$archive$TarArchiveEntry$$padded(TarArchiveEntry.scala:107)
at akka.stream.alpakka.file.impl.archive.TarArchiveEntry.headerBytesWithoutChecksum(TarArchiveEntry.scala:178)
at akka.stream.alpakka.file.impl.archive.TarArchiveEntry.headerBytes(TarArchiveEntry.scala:163)
at akka.stream.alpakka.file.impl.archive.TarArchiveManager$.$anonfun$tarFlow$1(TarArchiveManager.scala:24)
at akka.stream.impl.fusing.Map$$anon$1.onPush(Ops.scala:52)
at akka.stream.impl.fusing.GraphInterpreter.processPush(GraphInterpreter.scala:541)
at akka.stream.impl.fusing.GraphInterpreter.execute(GraphInterpreter.scala:423)
at akka.stream.impl.fusing.GraphInterpreterShell.runBatch(ActorGraphInterpreter.scala:625)
at akka.stream.impl.fusing.GraphInterpreterShell$AsyncInput.execute(ActorGraphInterpreter.scala:502)
at akka.stream.impl.fusing.GraphInterpreterShell.processEvent(ActorGraphInterpreter.scala:600)
at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:775)
at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$shortCircuitBatch(ActorGraphInterpreter.scala:762)
at akka.stream.impl.fusing.ActorGraphInterpreter$$anonfun$receive$1.applyOrElse(ActorGraphInterpreter.scala:791)
at akka.actor.Actor.aroundReceive(Actor.scala:537)
at akka.actor.Actor.aroundReceive$(Actor.scala:535)
at akka.stream.impl.fusing.ActorGraphInterpreter.aroundReceive(ActorGraphInterpreter.scala:691)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579)
at akka.actor.ActorCell.invoke(ActorCell.scala:547)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270)
at akka.dispatch.Mailbox.run(Mailbox.scala:231)
at akka.dispatch.Mailbox.exec(Mailbox.scala:243)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
ennru commented 2 years ago

Thank you for reporting. Can somebody dig up how tar stores sizes that don't fit into the 12 bytes numbers / > 16GB?