search

ido50 / morgan

PyPI Mirror for Restricted/Offline Environments
Apache License 2.0
105 stars 7 forks source link

multiple morgan.ini files? #18

Open dimasahmad opened 8 months ago

dimasahmad commented 8 months ago

I am utilizing this tool in my company for my team to centrally pull many packages into our locked down environment, Since every project has their own set of python version, environment, and packages, a single morgan.ini isn't enough. My current workaround is to overwrite morgan.ini content with morgan-[project].ini before running morgan mirror and loop for every .ini files.

Do you have any plan to support --config or --outputDir argument so it can build a library of packages for many projects instead of just one, or am I missing the docs?

alex-orange commented 7 months ago

I'm looking to do similar to overcome a similar situation. I understand the ask for --config, what's the reasoning behind --outputDir? I haven't tried your solution, but you don't actually need separate output directories do you? Just copy over the morgan.ini and re-run, right? Also, is --index-path not the same as outputDir?

dimasahmad commented 7 months ago

Reading the source code, morgan.ini is hardcoded to load from --index-path or cwd if undefined.

My goal was similar to how goproxy (the package, not protocol) works. They have the option to change where the storage is located by using an environment variable.

While --index-path might works to set the storage path, I think it would be nice to allow loading different named config with --config option.

alex-orange commented 7 months ago

Right, and what I'm saying is that I'm working on a patch to support what you're asking for wrt --config, but I don't think there's a need for --outputDir since that's already handled by --index-path which from what I can tell you don't disagree with.

ido50 commented 7 months ago

By now I realize that I also need to support multiple projects with different requirements through the same mirror. This can either be implemented with multiple ini files or still with just the one, it doesn't really matter. I'm not sure about --outputDir though, if you want different directories for different projects, then you can simply run morgan separately for each one.

@alex-orange, you're saying you're already working on a patch, can you elaborate on your current plan?

dimasahmad commented 7 months ago

I agree, the --config argument is probably the best solution to this as the storage path option is already covered by --index-path.

alex-orange commented 7 months ago

I just added a pull request #22 to address the config option. It just allows the config to be specified overriding the fixed morgan.ini in index-path location. So one can use say morgan-tf.ini, morgan-numpy.ini, etc, etc.

There's a few reasons I'm doing this:

  1. I don't have to have the morgan.ini in the index-path. I could for instance have a location (perhaps in a git repo) that has the scripts that I use to run morgan, and the ini files, but that references a different index-path without having to copy the ini file in.
  2. At the moment there's a bit of trouble with the way requirements and fulfillments of those requirements are handled so for some packages I can't have both 3.11 environment and 3.12 environment in the same file because morgan can't find a single package that will satisfy both. This is hopefully a short-term problem, but it's particularly easy to just split these kind of problems into multiple ini files that will each run on their own and write a script to run morgan multiple times.