search

OSGeo / gdal

GDAL is an open source MIT licensed translator library for raster and vector geospatial data formats.
https://gdal.org
Other
4.82k stars 2.53k forks source link

Bring console_scripts to python bindings as well as gdal-utils #5559

Closed maphew closed 5 months ago

maphew commented 2 years ago

The next stage of #5296 would be to bring _consolescripts to swig/python setup,py as well as _gdalutils.

It should be a relatively straightforward, replacing bare scripts glob with short define entry points function. I'll create a branch for this shortly and we'll see what CI has to say about said 'straightforwardness'. ;-)

Another approach, perhaps cleaner, might be to remove gdal-utils from the python bindings entirely and instruct people to install gdal-utils package independently. I haven't spent much time yet thinking about this path. I don't know what the impact might be on the many downstream packagers.

cc @idanmiara

idanmiara commented 2 years ago

Hi Matt,

When we discussed the gdal-utils RFC https://gdal.org/development/rfc/rfc78_gdal_utils_package.html The were PSC members who preferred to keep gdal-utils also inside the gdal package for full backwards compatibility.

I'm not sure if they feel that the situation now is different as perhaps gdal-utils is more mature and that it's worth discussing it again, as this split is a breaking change.

It would require people to run pip install gdal gdal-utils Instead of just pip install gdal

For what it's worth I am for this split as I think it wouldn't be too much trouble to instruct people to install both, and it might also bring to their awareness that these are not independent scripts anymore and that they can be reused as modules.

Idan

On Fri, 1 Apr 2022, 19:08 matt wilkie, @.***> wrote:

The next stage of #5296 https://github.com/OSGeo/gdal/pull/5296 would be to bring console_scripts to swig/python setup,py as well as gdal_utils.

It should be a relatively straightforward, replacing bare scripts glob with short define entry points function. I'll create a branch for this shortly and we'll see what CI has to say about said 'straightforwardness'. ;-)

Another approach, perhaps cleaner, might be to remove gdal-utils from the python bindings entirely and instruct people to install gdal-utils package independently. I haven't spent much time yet thinking about this path. I don't know what the impact might be on the many downstream packagers.

cc @idanmiara https://github.com/idanmiara

— Reply to this email directly, view it on GitHub https://github.com/OSGeo/gdal/issues/5559, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGJBBLLEC5KTSR5DT3T6DYLVC4NN3ANCNFSM5SJEOZAA . You are receiving this because you were mentioned.Message ID: @.***>

maphew commented 2 years ago

Thanks @idanmiara for reminding me to refresh from the history. Now that I've been working on this for awhile more of it is meaningful.

The branch I'm exploring this in is "py-conscripts", https://github.com/maphew/gdal/tree/py-conscripts/swig/python.

So far I'm following the first path, nested and not split. Currently install works but running the scripts breaks, I think because of mismatched installed binary gdal libs and source code versions where I'm testing on linux (Gitpod). This is as far as I have time for next couple of days.

... ImportError: /home/gitpod/.pyenv/versions/3.8.12/lib/python3.8/site-packages/osgeo/_gdal.cpython-38-x86_64-linux-gnu.so: undefined symbol: VSIClearCredentials

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "gdal-utils/osgeo_utils/gdal_edit.py", line 33, in from osgeo import gdal ... ModuleNotFoundError: No module named '_gdal'

maphew commented 2 years ago

meta: is it better to discuss via mailing list or in issues? I'm consistently fuzzy on where and when the best place to talk about X is when what to do/not do is yet to be discovered.

rouault commented 2 years ago

is it better to discuss via mailing list or in issues?

things that have backward compatibility challenges and affect a presumable significant portion of the users are best discussed on the mailing list, and a RFC might be appropriate

maphew commented 2 years ago

So far I'm following the first path, nested and not split. Currently install works but running the scripts breaks, I think because of mismatched installed binary gdal libs and source code versions where I'm testing on linux (Gitpod). This is as far as I have time for next couple of days.

The Ubuntu 20.04 CI build is happy, https://github.com/maphew/gdal/runs/5792052371?check_suite_focus=true, which I think is the closest corollary to the env I was testing in. With most of the other builds happy too it looks to me like a successful proof of concept for the nested approach.

things that have backward compatibility challenges and affect a presumable significant portion of the users are best discussed on the mailing list, and a RFC might be appropriate

Idan, you've championed gdal-utils as an independent package to this point and have the best overall understanding. I'll follow your lead; let me know how I can help!

idanmiara commented 2 years ago

Idan, you've championed gdal-utils as an independent package to this point and have the best overall understanding. I'll follow your lead; let me know how I can help!

Hi @maphew Thanks, but unfortunately currently I don't have the capacity to lead such a move but if you want to lead it I'm willing to help editing. I think first we need to present the mains pros/cons of this split in the mailing list (You could write here your draft) and if you have a general positive feeling about this then you could draft RFC as discussed above.

rouault commented 5 months ago

looks like this was implemented per 45380dd94adb763dd9a69400f76b74f5bc8b43a6 / #8811