pip install records[pandas]==0.5.3 doesn't install pandas

[atugushev]

The records[pandas]==0.5.3 depends on tablib>=0.11.4(via install_requirements) and tablib[pandas](via extras_require=pandas). The tablib[pandas] depends on pandas(via extras_require=pandas). So pip install records[pandas]==0.5.3 should install pandas, but it doesn't install it:

$ pip --version
pip 19.2.2 from /Users/albert/Projects/pip-tools/.venv/lib/python3.7/site-packages/pip (python 3.7)

$ pip install records[pandas]==0.5.3
Collecting records[pandas]
  Using cached https://files.pythonhosted.org/packages/ef/93/2467c761ea3729713ab97842a46cc125ad09d14a0a174cb637bee4983911/records-0.5.3-py2.py3-none-any.whl
Collecting SQLAlchemy; python_version >= "3.0" (from records[pandas])
  Downloading https://files.pythonhosted.org/packages/c2/75/6217c626fa22ad56ae5ccb1a36e7c4f17f5ca31543887e00179468d10464/SQLAlchemy-1.3.7.tar.gz (5.9MB)
     |████████████████████████████████| 5.9MB 4.1MB/s
Collecting docopt (from records[pandas])
  Using cached https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz
Collecting tablib>=0.11.4 (from records[pandas])
  Downloading https://files.pythonhosted.org/packages/7b/c7/cb74031b330cd94f3580926dc707d148b4ba9138449fc9f433cb79e640d8/tablib-0.13.0-py3-none-any.whl (75kB)
     |████████████████████████████████| 81kB 4.5MB/s
Collecting openpyxl<2.5.0 (from records[pandas])
  Using cached https://files.pythonhosted.org/packages/77/26/0bd1a39776f53b4f28e5bb1d26b3fcd99068584a7e1ddca4e09c0d5fd592/openpyxl-2.4.11.tar.gz
Collecting backports.csv (from tablib>=0.11.4->records[pandas])
  Using cached https://files.pythonhosted.org/packages/8e/26/a6bd68f13e0f38fbb643d6e497fc3462be83a0b6c4d43425c78bb51a7291/backports.csv-1.0.7-py2.py3-none-any.whl
Collecting xlrd (from tablib>=0.11.4->records[pandas])
  Using cached https://files.pythonhosted.org/packages/b0/16/63576a1a001752e34bf8ea62e367997530dc553b689356b9879339cf45a4/xlrd-1.2.0-py2.py3-none-any.whl
Collecting xlwt (from tablib>=0.11.4->records[pandas])
  Using cached https://files.pythonhosted.org/packages/44/48/def306413b25c3d01753603b1a222a011b8621aed27cd7f89cbc27e6b0f4/xlwt-1.3.0-py2.py3-none-any.whl
Collecting pyyaml (from tablib>=0.11.4->records[pandas])
Collecting odfpy (from tablib>=0.11.4->records[pandas])
  Using cached https://files.pythonhosted.org/packages/85/7d/8f6d1f2a4683be362b101c00232b4c3839e4e4a90e0945d8d43ec6aa671d/odfpy-1.4.0.tar.gz
Collecting jdcal (from openpyxl<2.5.0->records[pandas])
  Using cached https://files.pythonhosted.org/packages/f0/da/572cbc0bc582390480bbd7c4e93d14dc46079778ed915b505dc494b37c57/jdcal-1.4.1-py2.py3-none-any.whl
Collecting et_xmlfile (from openpyxl<2.5.0->records[pandas])
  Using cached https://files.pythonhosted.org/packages/22/28/a99c42aea746e18382ad9fb36f64c1c1f04216f41797f2f0fa567da11388/et_xmlfile-1.0.1.tar.gz
Collecting defusedxml (from odfpy->tablib>=0.11.4->records[pandas])
  Using cached https://files.pythonhosted.org/packages/06/74/9b387472866358ebc08732de3da6dc48e44b0aacd2ddaa5cb85ab7e986a2/defusedxml-0.6.0-py2.py3-none-any.whl
Building wheels for collected packages: SQLAlchemy, docopt, openpyxl, odfpy, et-xmlfile
  Building wheel for SQLAlchemy (setup.py) ... done
  Created wheel for SQLAlchemy: filename=SQLAlchemy-1.3.7-cp37-cp37m-macosx_10_14_x86_64.whl size=1145604 sha256=5397c866669ee8461c23ce93e38c11eaa3783f08e835c961d1663bfb374cef4f
  Stored in directory: /Users/albert/Library/Caches/pip/wheels/80/e0/68/3d11cc7209b5bd2c7d55cbb56c6bda843cc82f77c8387468ea
  Building wheel for docopt (setup.py) ... done
  Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13704 sha256=97b55e96cd59a303ecd1b5c514f1c9953c4d92085e57d8feeaa7382f955c0da2
  Stored in directory: /Users/albert/Library/Caches/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e
  Building wheel for openpyxl (setup.py) ... done
  Created wheel for openpyxl: filename=openpyxl-2.4.11-py2.py3-none-any.whl size=222820 sha256=98c8b31114291ed8ab8fc6e2f95173fec10993b41733296a19aed0ef76dce57d
  Stored in directory: /Users/albert/Library/Caches/pip/wheels/59/44/27/63b211425501ad51d197ff8ed00e9e469e38b9e516cb69b1c2
  Building wheel for odfpy (setup.py) ... done
  Created wheel for odfpy: filename=odfpy-1.4.0-py2.py3-none-any.whl size=137219 sha256=a423bce5e06e8dca85dfb13d4f3ec22dbb02004f4e2cf438d661a9c026b18f5c
  Stored in directory: /Users/albert/Library/Caches/pip/wheels/06/2d/19/f5a4eed468fecff295ff8ac49e5dd5fb22d7ffc7ff072deabf
  Building wheel for et-xmlfile (setup.py) ... done
  Created wheel for et-xmlfile: filename=et_xmlfile-1.0.1-cp37-none-any.whl size=8914 sha256=9308fef4af95b86f18f79ac5ddf4e859bb9193aee7048373ec345f0374772e6b
  Stored in directory: /Users/albert/Library/Caches/pip/wheels/2a/77/35/0da0965a057698121fc7d8c5a7a9955cdbfb3cc4e2423cad39
Successfully built SQLAlchemy docopt openpyxl odfpy et-xmlfile
Installing collected packages: SQLAlchemy, docopt, backports.csv, xlrd, jdcal, et-xmlfile, openpyxl, xlwt, pyyaml, defusedxml, odfpy, tablib, records
Successfully installed SQLAlchemy-1.3.7 backports.csv-1.0.7 defusedxml-0.6.0 docopt-0.6.2 et-xmlfile-1.0.1 jdcal-1.4.1 odfpy-1.4.0 openpyxl-2.4.11 pyyaml-5.1.2 records-0.5.3 tablib-0.13.0 xlrd-1.2.0 xlwt-1.3.0

Hope it helps someday.

[pradyunsg]

Thanks for filing this @atugushev! IIUC, records[pandas] should depend on tablib[pandas], instead of the plain tablib.

[ddelange]

@pradyunsg shouldn't pip actually install pandas when records[pandas] has tablib in install_requires and tablib[pandas] in extras_require (i.e. union of all extras provided by install_requires and any chosen extras_require)?

this combo, of tablib with and without [pandas] should imo correctly result in pandas being installed:

$ pipgrip --tree  --lock records[pandas]==0.5.3

records[pandas]==0.5.3 (0.5.3)
├── docopt (0.6.2)
├── openpyxl<2.5.0 (2.4.11)
│   ├── et-xmlfile (1.0.1)
│   └── jdcal (1.4.1)
├── sqlalchemy (1.3.16)
├── tablib>=0.11.4 (1.1.0)
└── tablib[pandas] (1.1.0)
    └── pandas (1.0.3)
        ├── numpy>=1.13.3 (1.18.2)
        ├── python-dateutil>=2.6.1 (2.8.1)
        │   └── six>=1.5 (1.14.0)
        └── pytz>=2017.2 (2019.3)

$ cat ./pipgrip.lock

records==0.5.3
docopt==0.6.2
openpyxl==2.4.11
et-xmlfile==1.0.1
jdcal==1.4.1
sqlalchemy==1.3.16
tablib==1.1.0
pandas==1.0.3
numpy==1.18.2
python-dateutil==2.8.1
six==1.14.0
pytz==2019.3

[pradyunsg]

It should, but it doesn't, because... the existing pip resolver doesn't actually do the right thing. :)

pip's resolver is seeing the tablib requirement before the tablib[pandas] requirement, and moving forward with the first one (because the name matches an already-made choice). That's how the old resolver works (it's a bug, yes), it's a really stupid bug but it's fairly tricky to solve correctly in the current approach of the resolver.

We'd fix this in the next-gen resolver for pip though, which is what we're collecting test cases like this here for!

[atugushev]

Fixed in the new resolver 👍🏼