search

flatpak / flatpak-builder

Tool to build flatpaks from source
GNU Lesser General Public License v2.1
142 stars 93 forks source link

Unzip failures related to strip-components is unclear #35

Open AsciiWolf opened 7 years ago

AsciiWolf commented 7 years ago

I am getting this error when trying to unpack zip archive using flatpak-builder:

Error: module tremulous-data: Error moving file /home/asciiwolf/Code/com.grangerhub.Tremulous/.flatpak-builder/build/tremulous-data-3/.uncompressLT904Y/gpp_11/vm: File exists

For more information see this PR.

darkdragon-001 commented 4 years ago

Same problem here with https://github.com/cake-build/cake/releases/download/v0.38.4/Cake-bin-net461-v0.38.4.zip.

  inflating: Autofac.dll             
  inflating: cake-medium.png         
  inflating: Cake.Common.dll         
  inflating: Cake.Common.pdb         
  inflating: Cake.Common.xml         
  inflating: Cake.Core.dll           
  inflating: Cake.Core.pdb           
  inflating: Cake.Core.xml           
  inflating: Cake.exe                
  inflating: Cake.exe.config         
  inflating: Cake.NuGet.dll          
  inflating: Cake.NuGet.pdb          
  inflating: Cake.NuGet.xml          
  inflating: Cake.pdb                
  inflating: Cake.xml                
  inflating: LICENSE                 
  inflating: Microsoft.CodeAnalysis.CSharp.dll  
  inflating: Microsoft.CodeAnalysis.CSharp.Scripting.dll  
  inflating: Microsoft.CodeAnalysis.dll  
  inflating: Microsoft.CodeAnalysis.Scripting.dll  
  inflating: Microsoft.DotNet.PlatformAbstractions.dll  
  inflating: Microsoft.Win32.Primitives.dll  
  inflating: netstandard.dll         
  inflating: Newtonsoft.Json.dll     
  inflating: NuGet.Common.dll        
  inflating: NuGet.Configuration.dll  
  inflating: NuGet.Frameworks.dll    
  inflating: NuGet.Packaging.dll     
  inflating: NuGet.Protocol.dll      
  inflating: NuGet.Resolver.dll      
  inflating: NuGet.Versioning.dll    
  inflating: System.AppContext.dll   
  inflating: System.Buffers.dll      
  inflating: System.Collections.Concurrent.dll  
  inflating: System.Collections.dll  
  inflating: System.Collections.Immutable.dll  
  inflating: System.Collections.NonGeneric.dll  
  inflating: System.Collections.Specialized.dll  
  inflating: System.ComponentModel.dll  
  inflating: System.ComponentModel.EventBasedAsync.dll  
  inflating: System.ComponentModel.Primitives.dll  
  inflating: System.ComponentModel.TypeConverter.dll  
  inflating: System.Console.dll      
  inflating: System.Data.Common.dll  
  inflating: System.Diagnostics.Contracts.dll  
  inflating: System.Diagnostics.Debug.dll  
  inflating: System.Diagnostics.FileVersionInfo.dll  
  inflating: System.Diagnostics.Process.dll  
  inflating: System.Diagnostics.StackTrace.dll  
  inflating: System.Diagnostics.TextWriterTraceListener.dll  
  inflating: System.Diagnostics.Tools.dll  
  inflating: System.Diagnostics.TraceSource.dll  
  inflating: System.Diagnostics.Tracing.dll  
  inflating: System.Drawing.Primitives.dll  
  inflating: System.Dynamic.Runtime.dll  
  inflating: System.Globalization.Calendars.dll  
  inflating: System.Globalization.dll  
  inflating: System.Globalization.Extensions.dll  
  inflating: System.IO.Compression.dll  
  inflating: System.IO.Compression.ZipFile.dll  
  inflating: System.IO.dll           
  inflating: System.IO.FileSystem.dll  
  inflating: System.IO.FileSystem.DriveInfo.dll  
  inflating: System.IO.FileSystem.Primitives.dll  
  inflating: System.IO.FileSystem.Watcher.dll  
  inflating: System.IO.IsolatedStorage.dll  
  inflating: System.IO.MemoryMappedFiles.dll  
  inflating: System.IO.Pipes.dll     
  inflating: System.IO.UnmanagedMemoryStream.dll  
  inflating: System.Linq.dll         
  inflating: System.Linq.Expressions.dll  
  inflating: System.Linq.Parallel.dll  
  inflating: System.Linq.Queryable.dll  
  inflating: System.Memory.dll       
  inflating: System.Net.Http.dll     
  inflating: System.Net.NameResolution.dll  
  inflating: System.Net.NetworkInformation.dll  
  inflating: System.Net.Ping.dll     
  inflating: System.Net.Primitives.dll  
  inflating: System.Net.Requests.dll  
  inflating: System.Net.Security.dll  
  inflating: System.Net.Sockets.dll  
  inflating: System.Net.WebHeaderCollection.dll  
  inflating: System.Net.WebSockets.Client.dll  
  inflating: System.Net.WebSockets.dll  
  inflating: System.Numerics.Vectors.dll  
  inflating: System.ObjectModel.dll  
  inflating: System.Reflection.dll   
  inflating: System.Reflection.Extensions.dll  
  inflating: System.Reflection.Metadata.dll  
  inflating: System.Reflection.Primitives.dll  
  inflating: System.Resources.Reader.dll  
  inflating: System.Resources.ResourceManager.dll  
  inflating: System.Resources.Writer.dll  
  inflating: System.Runtime.CompilerServices.Unsafe.dll  
  inflating: System.Runtime.CompilerServices.VisualC.dll  
  inflating: System.Runtime.dll      
  inflating: System.Runtime.Extensions.dll  
  inflating: System.Runtime.Handles.dll  
  inflating: System.Runtime.InteropServices.dll  
  inflating: System.Runtime.InteropServices.RuntimeInformation.dll  
  inflating: System.Runtime.Numerics.dll  
  inflating: System.Runtime.Serialization.Formatters.dll  
  inflating: System.Runtime.Serialization.Json.dll  
  inflating: System.Runtime.Serialization.Primitives.dll  
  inflating: System.Runtime.Serialization.Xml.dll  
  inflating: System.Security.Claims.dll  
  inflating: System.Security.Cryptography.Algorithms.dll  
  inflating: System.Security.Cryptography.Csp.dll  
  inflating: System.Security.Cryptography.Encoding.dll  
  inflating: System.Security.Cryptography.Primitives.dll  
  inflating: System.Security.Cryptography.ProtectedData.dll  
  inflating: System.Security.Cryptography.X509Certificates.dll  
  inflating: System.Security.Principal.dll  
  inflating: System.Security.SecureString.dll  
  inflating: System.Text.Encoding.CodePages.dll  
  inflating: System.Text.Encoding.dll  
  inflating: System.Text.Encoding.Extensions.dll  
  inflating: System.Text.RegularExpressions.dll  
  inflating: System.Threading.dll    
  inflating: System.Threading.Overlapped.dll  
  inflating: System.Threading.Tasks.dll  
  inflating: System.Threading.Tasks.Extensions.dll  
  inflating: System.Threading.Tasks.Parallel.dll  
  inflating: System.Threading.Thread.dll  
  inflating: System.Threading.ThreadPool.dll  
  inflating: System.Threading.Timer.dll  
  inflating: System.ValueTuple.dll   
  inflating: System.Xml.ReaderWriter.dll  
  inflating: System.Xml.XDocument.dll  
  inflating: System.Xml.XmlDocument.dll  
  inflating: System.Xml.XmlSerializer.dll  
  inflating: System.Xml.XPath.dll    
  inflating: System.Xml.XPath.XDocument.dll  
  inflating: cs/Microsoft.CodeAnalysis.CSharp.resources.dll  
  inflating: cs/Microsoft.CodeAnalysis.CSharp.Scripting.resources.dll  
  inflating: cs/Microsoft.CodeAnalysis.resources.dll  
  inflating: cs/Microsoft.CodeAnalysis.Scripting.resources.dll  
  inflating: de/Microsoft.CodeAnalysis.CSharp.resources.dll  
  inflating: de/Microsoft.CodeAnalysis.CSharp.Scripting.resources.dll  
  inflating: de/Microsoft.CodeAnalysis.resources.dll  
  inflating: de/Microsoft.CodeAnalysis.Scripting.resources.dll  
  inflating: es/Microsoft.CodeAnalysis.CSharp.resources.dll  
  inflating: es/Microsoft.CodeAnalysis.CSharp.Scripting.resources.dll  
  inflating: es/Microsoft.CodeAnalysis.resources.dll  
  inflating: es/Microsoft.CodeAnalysis.Scripting.resources.dll  
  inflating: fr/Microsoft.CodeAnalysis.CSharp.resources.dll  
  inflating: fr/Microsoft.CodeAnalysis.CSharp.Scripting.resources.dll  
  inflating: fr/Microsoft.CodeAnalysis.resources.dll  
  inflating: fr/Microsoft.CodeAnalysis.Scripting.resources.dll  
  inflating: it/Microsoft.CodeAnalysis.CSharp.resources.dll  
  inflating: it/Microsoft.CodeAnalysis.CSharp.Scripting.resources.dll  
  inflating: it/Microsoft.CodeAnalysis.resources.dll  
  inflating: it/Microsoft.CodeAnalysis.Scripting.resources.dll  
  inflating: ja/Microsoft.CodeAnalysis.CSharp.resources.dll  
  inflating: ja/Microsoft.CodeAnalysis.CSharp.Scripting.resources.dll  
  inflating: ja/Microsoft.CodeAnalysis.resources.dll  
  inflating: ja/Microsoft.CodeAnalysis.Scripting.resources.dll  
  inflating: ko/Microsoft.CodeAnalysis.CSharp.resources.dll  
  inflating: ko/Microsoft.CodeAnalysis.CSharp.Scripting.resources.dll  
  inflating: ko/Microsoft.CodeAnalysis.resources.dll  
  inflating: ko/Microsoft.CodeAnalysis.Scripting.resources.dll  
  inflating: pl/Microsoft.CodeAnalysis.CSharp.resources.dll  
  inflating: pl/Microsoft.CodeAnalysis.CSharp.Scripting.resources.dll  
  inflating: pl/Microsoft.CodeAnalysis.resources.dll  
  inflating: pl/Microsoft.CodeAnalysis.Scripting.resources.dll  
  inflating: pt-BR/Microsoft.CodeAnalysis.CSharp.resources.dll  
  inflating: pt-BR/Microsoft.CodeAnalysis.CSharp.Scripting.resources.dll  
  inflating: pt-BR/Microsoft.CodeAnalysis.resources.dll  
  inflating: pt-BR/Microsoft.CodeAnalysis.Scripting.resources.dll  
  inflating: ru/Microsoft.CodeAnalysis.CSharp.resources.dll  
  inflating: ru/Microsoft.CodeAnalysis.CSharp.Scripting.resources.dll  
  inflating: ru/Microsoft.CodeAnalysis.resources.dll  
  inflating: ru/Microsoft.CodeAnalysis.Scripting.resources.dll  
  inflating: tr/Microsoft.CodeAnalysis.CSharp.resources.dll  
  inflating: tr/Microsoft.CodeAnalysis.CSharp.Scripting.resources.dll  
  inflating: tr/Microsoft.CodeAnalysis.resources.dll  
  inflating: tr/Microsoft.CodeAnalysis.Scripting.resources.dll  
  inflating: zh-Hans/Microsoft.CodeAnalysis.CSharp.resources.dll  
  inflating: zh-Hans/Microsoft.CodeAnalysis.CSharp.Scripting.resources.dll  
  inflating: zh-Hans/Microsoft.CodeAnalysis.resources.dll  
  inflating: zh-Hans/Microsoft.CodeAnalysis.Scripting.resources.dll  
  inflating: zh-Hant/Microsoft.CodeAnalysis.CSharp.resources.dll  
  inflating: zh-Hant/Microsoft.CodeAnalysis.CSharp.Scripting.resources.dll  
  inflating: zh-Hant/Microsoft.CodeAnalysis.resources.dll  
  inflating: zh-Hant/Microsoft.CodeAnalysis.Scripting.resources.dll

Error: module cake: Error moving file [...]/.flatpak-builder/build/cake-13/asdf/fff/.uncompressCU7IP0/de/Microsoft.CodeAnalysis.CSharp.resources.dll: File exists

It looks like the problem occurs when the same file name appears in multiple sub-directories.

@barthalion Can you have a look?

Roboron3042 commented 2 years ago

This is solved by adding strip-components: 0 to "sources".

However, it should not happen in the first place...

ThubLives commented 3 months ago

This is solved by adding strip-components: 0 to "sources".

I'm glad I found this workaround here, this was driving me crazy!

However, it should not happen in the first place...

Definitely. How has this gone since 2017 without being addressed?! Surely a lot of packagers have run into this.

TingPing commented 2 months ago

strip-components is a pretty standard concept and the default works for nearly every package on flathub.

Maybe it could have better logic to autodetect failures or print more helpful errors, patches welcomed.

ThubLives commented 2 months ago

strip-components is a pretty standard concept and the default works for nearly every package on flathub.

That sounds like selection bias to me. If it's in flathub, then someone has successfully packaged it. It's possible that the minority of packages which use other values are only in flathub at all because those who packaged them happened to be familiar with the strip-components concept and that it would be applied by default. Or maybe they stumbled on this issue like I did.

It's also possible that I'm just choosing apps that are particularly difficult to build on Linux in the first place and I was always going to be beating my head against a wall to make them work.

Maybe it could have better logic to autodetect failures or print more helpful errors, patches welcomed.

If I understood why strip-components: 1 was the default I might have a chance at approaching this myself. My best web searching skills have failed me here.

To be clear, in my case I was never getting the error reported in this issue. I was simply having the directory structure of my .zip file contents silently flattened and struggled to figure out why my app wasn't able to find the files it was looking for. Better failure detection and error messages would never have lead me to this solution.

I think the bigger problem is that the documentation is not clear about what is happening during extraction. As someone new to , I would think that it's fair to assume that the default behaviour wouldn't be doing anything that I didn't put into the manifest. That is, if I have to specify a command-line argument to apply the behaviour (e.g. using tar or unzip), then I would also have to specify some related option in the manifest and vice-versa.

Worse, when I read through the section carefully, the fact that the default value was 1 didn't stand out to me as necessarily a change from the default command-line behaviour since I wasn't familiar with this tar argument; it just seemed like an odd off-by-one type quirk. I had to find this issue to realise the real implications and it still seemed like bizarre default behaviour from a newcomer's perspective.

tl;dr: The documentation should be clearer about defaults that differ from what someone might expect from the relevant default command-line.