conda-forge / conda-forge.github.io

The conda-forge website.
https://conda-forge.org
BSD 3-Clause "New" or "Revised" License
129 stars 274 forks source link

List of feedstocks that are out-of-date #1034

Closed synapticarbors closed 4 years ago

synapticarbors commented 7 years ago

I pulled a list of all of the feedstocks on conda-forge and ran them through feedstockrot. Here is a list of all feedstocks that are out-of-date as of 2017-June-27:

I'm going to try to go through and at a minimum submit issues for each one to notify maintainers that the feedstock is out-of-date. I'll put in updating PRs when possible. Any help with this would of course be appreciated.

jakirkham commented 7 years ago

So we do have a script that is a WIP that will update the version. However it is not quite sufficient as one needs to check the dependencies by hand. Would be nice if it could get the dependencies from the setup.py at least.

ref: https://github.com/conda-forge/conda-forge.github.io/blob/d2cd7bc2c8bbfc83e7cbc11a32379fb9b77c0883/scripts/tick_my_feedstocks.py

scopatz commented 7 years ago

Also, there are definitely bugs with the above. It seems that it assumes that the name on conda-forge is the same name on pypi. This is not true, for example, for xo (conda-forge) and exofrills (pypi). xo on pypi is a totally separate project.

ocefpaf commented 7 years ago

It seems that it assumes that the name on conda-forge is the same name on pypi.

Same for iris. feedstockrot is designed to work with PyPI only. So we need to skip those that are not a 1-1 match to PyPI.

goanpeca commented 7 years ago

@ocefpaf, @scopatz we could might as well just add this info to the meta.yaml extra

extra:
  pypi_name: 'name'

or the "official" name in the package?

package:
  name: "xo"
  version: "1.1.4"
  pypi: "exofrills"
  # official_name: "exofrills"  # ??? 

Maintaining a "manual" list of packages that do not have the same name seems error prone and time consuming?

@kalefranz, @msarahan thougths?

ocefpaf commented 7 years ago

I like

extra:
  pypi_name: 'name'

it is simple and we can make that an optional info that would solve many problems.

goanpeca commented 7 years ago

So we do have a script that is a WIP that will update the version. However it is not quite sufficient as one needs to check the dependencies by hand. Would be nice if it could get the dependencies from the setup.py at least.

@jakirkham do we have dev_url on all recipes at this point?

jakirkham commented 7 years ago

Not necessarily. Also not everything that is on PyPI has source code that is easily accessible. I do like the idea of extracting it from the PyPI sdist though.

goanpeca commented 7 years ago

PyPI sdist

Good point

goanpeca commented 7 years ago

@ocefpaf, @msarahan, actually I do not like

extra:
  pypi_name: 'name'

that much. Although it was my initial suggestion because it feels too python centric... whereas we can have conda skeleton for R and other packages that have the same "problem", something like:

package:
  name: "xo"
  version: "X.X.X"
  src_name: 'name'

is more generic and probably useful in other contexts besides pypi. I don't think we want to end with

extra:
  pypi_name: 'name'

and also

extra:
  cran_name: 'name'

and also

extra:
  cpan_name: 'name'

and also

extra:
  luarocks_name: 'name'
scopatz commented 7 years ago

Why not? The latter seems more correct and specific. The name could be different for every different package manager

goanpeca commented 7 years ago

The name could be different for every different package manager

Why? Why is specific better for an agnostic tool, the idea is not to generate N keys as needed, we can be more thoughtful and have a single one

ocefpaf commented 7 years ago

Also, src_name is not always the PyPI (or any other host service name), in fact some confusions comes from when the name == src_name != pypi_name.

goanpeca commented 7 years ago

Also, src_name is not always the PyPI (or any other host service name), in fact some confusions comes from when the name == src_name != pypi_name.

The key name is up for debate, it can be whatever we agree is a good thing

ocefpaf commented 7 years ago

@goanpeca there is no way we can be agnostic there b/c the differences in the name are not. The best solution is to be as literal as we can.

goanpeca commented 7 years ago

@goanpeca there is no way we can be agnostic there b/c the differences in the name are not. The best solution is to be as literal as we can.

Yes we can, because this key conveys the same message/meaning, than 4 (or more) different keys would.

The "official" package ecosystem has a name for that package and "this" specific conda package uses something different...

ocefpaf commented 7 years ago

I guess that you are missing the point where the original source name is in conflict with the multiple host services like PyPI. That is more common that what we would like and we do need to be literal.

Anyways, I am OK adding an extra metadata that has a package host service to make it easier to write tools that take advantage of that name, as long as it is not mandatory I really don't want o burden maintainers and new contributors with that, specially b/c they are special cases.

goanpeca commented 7 years ago

I guess that you are missing the point where the original source name is in conflict with the multiple host services like PyPI. That is more common that what we would like and we do need to be literal.

Please explain


Anyways, I am OK adding an extra metadata that has a package host service to make it easier to write tools that take advantage of that name, as long as it is not mandatory I really don't want o burden maintainers and new contributors with that, specially b/c they are special case

Hey I agree with that, and that is what I suggested in the first place, I just want to understand why one single key does not work

ocefpaf commented 7 years ago

Please explain

Your second proposal, and the one you seems to prefer, is:

package:
  name: "xo"
  version: "X.X.X"
  src_name: 'name'

Here name is the conda package name and src_name implies the name of the source package. That would be confusing for cases like iris where both the conda package and the original source are names iris while the PyPI package "iris" is another package unrelated to the iris package we want. (There will be a PyPI package soon with yet another name :unamused:)

By using a pypi_name we are conveying an extra info that does not imply the original source name, but the name used in the host service. So the multiple *_name is more informative, useful, and unambiguous that your second option. I also prefer that information in the extra section rather than by the conda package name at the top.


PS: in the iris case we could even use that to inform that it doe not exist on PyPI with:

extra:
  pypi_name: 'none'
goanpeca commented 7 years ago

@ocefpaf you seem to reiterate on the key name src_name even when I just clarified that the name is up for debate. We can call it whatever we want, to convey the right message without the need of 4 or more extra keys as conda offers skeletons for more packaging systems.

package:
  name: "xo"
  version: "X.X.X"
  <name that conveys that this is the name from the "official" package manager>: 'name'

or, since actually this information belongs to the source of the package, not the (conda) package itself, this would be better

package:
  name: "xo"
  version: "X.X.X"

source:
  <name that conveys that this is the name from the "official" package manager>: 'name'

Moving forward it would be better to have a key that denoted the package manager from which this conda package was generated, than prepending this to every extra key, so we could do something like

package:
  name: "xo"
  version: "X.X.X"

source:
  package_manager: 'pypi'
  package_name: 'name'
ocefpaf commented 7 years ago

@ocefpaf you seem to reiterate on the key name src_name even when I just clarified that the name is up for debate.

B/c my point is to not have a key name :wink:

source:
  package_manager: 'pypi'
  package_name: 'name'

My point is that a single pypi_name covers both new metadata lines you introduced there.

goanpeca commented 7 years ago

@ocefpaf, well no, you are suggesting we have 4 news keys (pypi_name cran_name cpan_name luarocks_name), I am suggesting we have 2 more and have this 2 more potentially be used to hold some information on source that we are manually using right now (pypi.io etc etc...)

ocefpaf commented 7 years ago

no, you are suggesting we have 4 news keys

You are thinking like a parser machine :wink:

Your original proposal is better for humans writing, reading, and maintaining the recipe b/c it is just a single, very descriptive, line. The 4 keys point of view are up to the parser.

scopatz commented 7 years ago

I agree with @ocefpaf

mingwandroid commented 7 years ago

Since we have a lot of different package managers in play I would prefer the following to be emitted to every recipe generated by conda skeleton:

At the top:

{% set pkg_mgr='pypi' %}
{% set pkg_mgr_name='iris' %}

and at the bottom:

extra:
  skeleton:
    generated_date: '20172806'
    regenerator: 'conda skeleton {{ pkg_manager }} {{ pkg_mgr_name }}'
    checker: 'conda skeleton --check {{ pkg_manager }} {{ pkg_mgr_name }}'

Here conda skeleton --check would need to be written but would return 0 or 1 if the package was up to date or out of date respectively.

isuruf commented 7 years ago

How about getting the name of the pypi package from the url?

souravsingh commented 7 years ago

@synapticarbors I am facing an issue with updating the smart_open package here- https://github.com/conda-forge/smart_open-feedstock/pull/3

The tests for Windows don't pass.

gwerbin commented 7 years ago

I'm not sure it counts, but Airflow has been renamed "apache-airflow" on Pypi, and the Airflow feedstock is out of date: https://pypi.python.org/pypi/apache-airflow/

stuertz commented 6 years ago

pypi may not be the only source for python packages. Some packages are not published by there maintainers via pypi. e.g the apsw package. Its source comes directly from github.

CJ-Wright commented 6 years ago

Personally I like the suggestion here: https://github.com/conda-forge/feedstocks/issues/12#issuecomment-311690103

package:
  name: "xo"
  version: "1.1.4"
  pypi: "exofrills"

Although I would appreciate it maybe if we were a bit more verbose about things eg

package:
  name: "xo"
  version: "1.1.4"
  github: 
    name: "exofrills"
    org: "scopatz"

It allows us to have special keys for certain places (on github a name alone is not enough). Edit: I'm ok with this being in extra though.

CJ-Wright commented 6 years ago

On a related note I'd love to see some closure to this since either methods would be very helpful for an auto-ticking system.

gwerbin commented 6 years ago

Why does it have to be specific to Github? I know it's the most popular open source platform nowadays, but it won't be forever. I think generic options for Git and FTP/HTTP download might be more useful to more people.

jakirkham commented 6 years ago

For one arbitrary targets don’t necessarily have a standard way to get feeds unlike GitHub and PyPI.

isuruf commented 6 years ago

Homebrew has a command that does the same thing, https://github.com/Homebrew/homebrew-livecheck. We can take some inspiration from it

jakirkham commented 6 years ago

So @CJ-Wright et al have since created the autoupdate bot. Also I'm guessing this list of outdated repos is a bit out-of-date. Given all of this, should we close out this issue?