[Magic CLI]: support pypi optional package args

[guidorice]

Issue description

magic add --pypi httpx[http2] versus magic add --pypi httpx

this seems to result in the exact same magic.lock file, making me think the package options are being ignored. For example these options are often used to install different sets of dependencies, for example the installation instructions here: https://www.python-httpx.org/

Install with pip: $ pip install httpx Or, to include the optional HTTP/2 support, use: $ pip install httpx[http2] To include the optional brotli decoder support, use: $ pip install httpx[brotli]

Steps to reproduce

$ magic add --pypi httpx
✔ Added httpx >=0.27.0, <0.28
Added these as pypi-dependencies.
[moxarray] $ md5 magic.lock 
MD5 (magic.lock) = 7a5cb806bb7086220e941a3f80bd361e
[moxarray] $ magic add --pypi httpx[http2,brotli]
✔ Added httpx[http2,brotli]
Added these as pypi-dependencies.
[moxarray] $ md5 magic.lock 
MD5 (magic.lock) = 7a5cb806bb7086220e941a3f80bd361e

Version Info

- magic 0.1.1
- osx-arm64

[ehsanmok]

Similar to pip, the extras should be wrapped in quotes so magic add --pypi "httpx[http2]" works and diff shows things like hyperframe which are transitive dependencies were added.

[guidorice]

@ehsanmok thanks for your prompt followup on this. However, I believe this should be reopened.

• Quoting is not actually required in the pip cli options, but that's kind of beside the point.
• Here is a script to reproduce from the python side whether the packages were installed correctly, or not. Magic does not correctly install them.

def check_httpx_http2():
    import httpx
    _ = httpx.Client(http2=True)

def check_httpx_brotli():
    import brotli

def main():
    try:
        check_httpx_brotli()
    except Exception as e:
        print(e)

    try:
        check_httpx_http2()
    except Exception as e:
        print(e)

if __name__=="__main__":
    main()

Reproduce steps (magic- prints errors shown)

$ magic add --pypi "httpx[brotli,http2]"
✔ Added httpx[brotli,http2]
Added these as pypi-dependencies.
$ which python
/Users/guidorice/mojo/moxarray/.magic/envs/default/bin/python
$ python check_httpx_installation.py 
No module named 'brotli'
Using http2=True, but the 'h2' package is not installed. Make sure to install httpx using `pip install httpx[http2]`.

Reproduce steps (pip- works)

python3.11 -m venv try-httpx
source try-httpx/bin/activate
pip install "httpx[http2,brotli]"
python check_httpx_installation.py 

[ehsanmok]

Thanks! I've actually made two separate projects magic init --pyproject test1/2 and without quote magic fails to install for me with

no matches found: httpx[brotli]

and diffing their lockfiles shows difference and one has more. I'll reopen it to get confirmation.

[guidorice]

Thanks! Also, seems to be fine as a workaround: magic add --pypi brotli h2

[ehsanmok]

Adjusting the pyproject.toml works dependencies = ["httpx[brotli,http2]>=0.27.0,<0.28"] but it's an issue that should be fixed.

[guidorice]

In the case of mojoproject.toml it seems to work with this format (manually edited):

[pypi-dependencies]
httpx = {version = ">=0.27.0, <0.28", extras = ["brotli", "http2"]}

But yeah, it seems like the magic add tool is not parsing the extras.

[ehsanmok]

Sure! we should see the exact same behaviour as indicated in these docs.

[iamtimdavis]

@ehsanmok and @zbowling - is this resolved? can we close this out?

[ehsanmok]

@iamtimdavis I don't think so. Waiting to here from Zach.

[zbowling]

Yeah now with the new magic 0.2.1 release it should work like that doc link. That should go out with our doc update.

[guidorice]

confirmed with magic 0.2.1- thanks all!