xetus-oss / docker-archiva

A docker image for Apache Archiva
Apache License 2.0
55 stars 34 forks source link

Increase upload size #30

Closed CatsNipYummy closed 3 years ago

CatsNipYummy commented 3 years ago

Do we have a way to increase the maximum upload size of an artifact? I see refernce to WEB-INF/classes/structs.properties in this link https://archiva.apache.org/docs/1.3.6/userguide/deploy.html#:~:text=Note%3A%20The%20maximum%20size%20of,uploaded%20is%2010M%20by%20default

But that does not seem to exist.

Is there a key I can provide in the nginx configuration?

tkent commented 3 years ago

👋 @CatsNipYummy - This is the first time we've heard a need to change the max upload size. The docs you're referencing are from an older version of Archiva (1.3.6). The version of the image here is currently 2.2.5.

Did you run into an upload size constraint? If so, was this through a browser-based upload or through the mvn API? Also, two things I should note:

  1. There is no nginx server in the archiva image we provide (though it can be used with one!).
  2. Anecdotally, we worked with fairly large artifacts (100MB+) without any problem. We don't happen to use the UI for that though.
CatsNipYummy commented 3 years ago

👋 @tkent I am using the mvn API to upload the artifact. We have files close to 4GB in our system. Probably not the best use case for Archiva. but its been working well for us. The way I got it to work is by editing the following on docker creation:

JVM_EXTRA_OPTS="-Dorg.eclipse.jetty.server.Request.maxFormContentSize=2147483647" (JAVA's INT_MAX value)

Seems to have fixed the issue. Not sure if that's the best way forward

tkent commented 3 years ago

Ah, good to know! If setting maxFormContentSize resolves the issue, we may want to want to just default the value going forward. That way other people with similar needs don't bump into it.

Just confirm once you know for sure and I'll see about adding that into the default image going forward.

CatsNipYummy commented 3 years ago

I can confirm this fixes things for us. Been uploading artifacts close to 4GB without a hitch using mvn.

tkent commented 3 years ago

Hmmm.... I went to update the image to include org.eclipse.jetty.server.Request.maxFormContentSize as 2147483647 by default. Before making that change, I setup a test that uploaded a 6GB file to confirm the issue (so I could verify it was fixed by the change).

However, I was able to upload a 6GB zip file in the current v2.2.5 image version without a problem using a regular gradle publishing block (which uses the mvn API).

Did you perhaps change other things around the same time you set the org.eclipse.jetty.server.Request.maxFormContentSize value? Maybe a proxy in front of your archive container?

CatsNipYummy commented 3 years ago

Yes. we did. that was handled by another team so I'm not aware of all the changes that went into that side. But I do remember them setting a large upload size and still facing this issue with Archiva. but I cant confirm if both the changes happened at the same time resulting in the successful upload.

tkent commented 3 years ago

Hmmm. Since I can can successfully upload very large files, can you do one of two things:

  1. Remove the org.eclipse.jetty.server.Request.maxFormContentSize settings from your archiva instance and see if things still work. I suspect they will.
  2. See if you can reproduce the issue with a local v2.2.5 instance? Since I'm able to upload a 6GB file using a test gradle project to a local container, there is a slim chance the problem only shows up with some different tooling.

Here is thebuild.gradle.kts I used for testing:

plugins {
  `maven-publish`
}

publishing {
  repositories {
    maven {
      url = uri("http://localhost:8080/repository/internal/")
      isAllowInsecureProtocol = true
      credentials {
        username = "admin"
        password = "abc123!!"
      }
    }
  }

  publications {
    create<MavenPublication>("maven") {
      groupId = "org.gradle.sample"
      artifactId = "library"
      version = "1.1"
      artifact("bigol.zip")
    }
  }
}

It assumes:

CatsNipYummy commented 3 years ago

I'll take a look at this over the weekend and let you know how it goes

tkent commented 3 years ago

/bump - Just checking to see if you were able to confirm on your side? If not, no big deal! I'll just close this issue in 10 days.