Rewrote pom with dependency metadata inclusion

[durganmcbroom]

Hey, sent you a message on discord already, but here's some more info if you didn't get it:

Features

Pom resolution:

1. Fetch mod metadata from the CurseForge API
2. Fetch file metadata from the CurseForge API
3. For each declared dependency that the mod file metadata contains, filter out only dependencies that are either flagged as required, to include, or tools. Then resolve that dependency (as we are only given a project ID, not file ID)
4. Write pom and return

Child dependency resolution (given a project ID (no file), given the publication date of the parent mod, given a Game ID, and given gameVersion information [which includes modloader/minecraft version]):

1. Fetch metadata for that mod
2. Fetch all files that mod has listed
3. First filter files by availability (no use to return a mod file that can't be downloaded)
4. Filter by publication date, only selecting files that have been published BEFORE the publication date of the mod depending on them. I choose to do it this way because only files published before the release time of the calling mod could have been used/tested when developing it.
5. Sort files from latest to oldest publication
6. Find the first file that has the same gameID (is the same game...), has an intersection between mod loaders, and has an intersection between game versions (basically, are their mod loaders and gameVersions compatible)
7. If all of the previous fails, return the file which the mods metadata declares as the mainFile.

Checksums: Added SHA1 and MD5 checksums to both the '.pom' and '.jar' routes. To create a checksum for the pom, I had to add the node library 'crypto' and digest the output of whatever the pom route produces. For the '.jar' route, we can just fetch the mod metadata and return its checksum.

Mod Metadata Included 3 functions for fetching mod metadata, either of an entire project, specific file, or all files of a given project. Under 'modmetadata.ts' are a collection of interfaces representing these data types.

Issues/predicted problems

1. The pom route is considerably more intensive and slower than it was (as it just returned a string with almost zero computation). Now it has to fetch information multiple times from CurseForge which slows the whole process down.
2. Im not entirely sure how loom will treat transitive dependencies now being part of the metadata for curse maven mods. However, 1: I think the having curse maven be a fully featured maven repository that generates accurate data about the dependencies of mods is important for consumers. 2: If this does cause issues with loom (which in my limited testing it has not) we can either include a check for user agent, and then return an old style pom, or (the less good option) consumers can mark what were once non-transitive dependencies, as non-transitive so gradle doesn't follow the tree.

[Wyn-Price]

Also I'm unsure if you can see, but the tests failed:

build.log

``` [13:53:23.040] Running build in Washington, D.C., USA (East) – iad1 [13:53:23.141] Cloning github.com/Wyn-Price/CurseMaven (Branch: pom-rewrite, Commit: 8200234) [13:53:23.469] Cloning completed: 327.567ms [13:53:26.897] Restored build cache [13:53:26.976] Running "vercel build" [13:53:27.459] Vercel CLI 33.2.0 [13:53:27.609] WARN! Due to `builds` existing in your configuration file, the Build and Development Settings defined in your Project Settings will not apply. Learn More: https://vercel.link/unused-build-settings [13:53:28.166] Installing dependencies... [13:53:28.575] yarn install v1.22.17 [13:53:28.666] [1/4] Resolving packages... [13:53:28.767] warning crypto@1.0.1: This package is no longer supported. It's now a built-in Node module. If you've depended on crypto, you should switch to the one that's built-in. [13:53:29.117] [2/4] Fetching packages... [13:53:46.659] [3/4] Linking dependencies... [13:53:52.192] [4/4] Building fresh packages... [13:53:52.913] success Saved lockfile. [13:53:52.917] Done in 24.35s. [13:53:52.944] Running "yarn run vercel-build" [13:53:53.143] yarn run v1.22.17 [13:53:53.182] $ yarn test && mkdir -p /vercel/output/functions/src/app.ts.func && cp -r public /vercel/output/functions/src/app.ts.func [13:53:53.414] $ cross-env NODE_ENV=test jest --testTimeout=30000 [13:53:58.213] PASS tests/static.test.ts [13:53:58.585] FAIL tests/download.test.ts [13:53:58.586] ● POM Generation › POM should be generated correctly [13:53:58.586] [13:53:58.586] expect(received).toStrictEqual(expected) // deep equality [13:53:58.586] [13:53:58.586] - Expected - 2 [13:53:58.586] + Received + 2 [13:53:58.586] [13:53:58.586] @@ -11,9 +11,9 @@ [13:53:58.586] https://cursemaven.com [13:53:58.586] [13:53:58.586] [13:53:58.586] [13:53:58.586] curse.maven [13:53:58.586] - 531761 [13:53:58.587] + balm-531761 [13:53:58.587] - balm-3474695 [13:53:58.587] + 4939245 [13:53:58.587] [13:53:58.587] [13:53:58.587] [13:53:58.587] 73 | const res = await requestWithSupertest.get(downloadUrl('waystones', '245755', '4946115', '.pom')) [13:53:58.587] 74 | expect(res.status).toStrictEqual(200) [13:53:58.587] > 75 | expect(res.text).toStrictEqual(` [13:53:58.587] | ^ [13:53:58.587] 76 | [13:53:58.587] 78 | 4.0.0 [13:53:58.587] [13:53:58.587] at tests/download.test.ts:75:22 [13:53:58.587] at fulfilled (tests/download.test.ts:5:58) [13:53:58.587] [13:53:58.619] Test Suites: 1 failed, 1 passed, 2 total [13:53:58.619] Tests: 1 failed, 13 passed, 14 total [13:53:58.619] Snapshots: 0 total [13:53:58.619] Time: 4.528 s [13:53:58.619] Ran all test suites. [13:53:58.689] error Command failed with exit code 1. [13:53:58.689] info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command. [13:53:58.714] error Command failed with exit code 1. [13:53:58.714] info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command. [13:53:58.730] Error: Command "yarn run vercel-build" exited with 1 [13:53:59.073] ```

[durganmcbroom]

Checksums removed (will branch off once this gets merged) and headers added.

[Wyn-Price]

Sorry for the delay,

One thing I'm wondering is how useful this would be inside cursemaven. You have your own key, and the ability to write the code, so is it worth you separating this out into your own project? The dependencies inside the POM are a bit of a "best guess", and it's very complicated to even obtain this information.

There's not many upsides of trying to automatically resolve dependencies, as that's not really how playing the mods work. You always have to add the mods yourself, or if you're making a modpack you have to define the dependencies yourself. Either way, it's usually left up to the user/developer to figure it out, and thus there's not great support for it.

A worry I have with this is that it's possible with this that it resolved to a dependency that simply doesn't work with the mods, and as the POM only returns one working version id will be difficult to override. Ideally, mods on curseforge would statically define their dependencies, and mods would follow correct semver versioning and everything would work nicely, but in reality that is not the case.

Let me know what you think, and if you need help with setting up / hosting a new project I'm happy to help.

[durganmcbroom]

Based on your/the communities needs for this project are I would agree that this POM information is probably not inline with that. In terms of the current workflow for users figuring out mod dependency information themselves, thats what i've been looking to write a better solution for; as this isn't how any modding frameworks/tools already work though I think your suggestion of branching out into a separate project might be the best bet as to not muddy up your codebase here.

I really appreciate all your help and time spent on this! If semi(🥲)-accurate dependency information ever becomes a needed-feature for other use-cases I'd be happy to reopen and work on it further with you!