search

MicrosoftDocs / azure-devops-docs

This repo is the home of the official Azure DevOps documentation for Microsoft. GitHub Issues filed in this repository should be for problems with the documentation.
https://docs.microsoft.com/azure/devops/
Creative Commons Attribution 4.0 International
830 stars 2.45k forks source link

#10426 Add hard size limit (384000 bytes) of Maven pom.xml files of Azure Artifacts to the documentation #10427

Closed j-tim closed 3 years ago

j-tim commented 3 years ago

Add hard size limit (384000 bytes) of Maven pom.xml files of Azure Artifacts to the documentation.

PRMerger13 commented 3 years ago

@j-tim : Thanks for your contribution! The author(s) have been notified to review your proposed change.

ramiMSFT commented 3 years ago

@j-tim Thank you for your contribution, however your suggested change does not reflect the full picture of how Artifacts interact with POM files. We now take the uploaded POM, transform it, then apply the size limit to the transformed object. The transformed object is typically much smaller than the input.

j-tim commented 3 years ago

@ramiMSFT apparently there is a hard limit on pom.xml files in Azure Artifacts. I and many other people ran into this issue: 

400 Bad Request - pom.xml file is too large; the limit is 384000 bytes

See also: https://developercommunity.visualstudio.com/t/remove-or-increase-the-hard-limit-of-384000-bytes/809379

jmyersmsft commented 3 years ago

@j-tim I didn't see before that this PR actually predates the change ramiMSFT mentioned, which we made last month. Are you still experiencing this problem?

j-tim commented 3 years ago

Hi @jmyersmsft yes, I'm still experiencing this problem.

jmyersmsft commented 3 years ago

@j-tim is this with a private package or one from Maven Central? If the latter, can you tell me the exact group/artifact/version you're having trouble with?

jmyersmsft commented 3 years ago

If it's a private package, can you put the pom.xml in a zip file and tell me the size of the original POM and of the zip file?

j-tim commented 3 years ago

@jmyersmsft it's not a private package. It's the grand parent pom of Lucene (used by Elasticsearch) for an example of the pom.xml file see: https://repo1.maven.org/maven2/org/apache/lucene/lucene-solr-grandparent/8.8.1/

jmyersmsft commented 3 years ago

hmm... that should only be ~20KB after transformation, way under the limit. I'll have to check some things on my end.

jmyersmsft commented 3 years ago

Well, this is embarrassing -- we never actually turned that feature on after we deployed the code. I have begun that process. Should reach your org within a few hours to a few days from now.

j-tim commented 3 years ago

Thanks for the update @jmyersmsft. I'll try to run my project in the upcoming days and see whether this will fix the issue. Just curious does this mean the hard limit of 384000 bytes for Maven pom.xml files will be gone?

jmyersmsft commented 3 years ago

We hit a snag while enabling it last night, so it may be a few days before it's actually on. I'll try to remember to keep this thread updated.

The limit is technically still there (it's ultimately caused by a hard limit that we don't control in a particular part of our underlying infrastructure), but with the feature enabled, the internal compressed representation is what counts, so it should allow much, much larger actual pom.xml files. For example, org.apache.lucene:lucene-solr-grandparent:8.8.1's POM shrinks from a little under 400KB to ~20KB

j-tim commented 3 years ago

That's great @jmyersmsft.