option to not follow symlinks when globbing

[c1b2d23c-2f4d-4d32-b768-4bdf89431bb7]

BPO	29475
Nosy	@ethanfurman, @o11c, @wimglenn
Files	python-glob-symlink.diff: Patch to add symlinks= argument to glob.glob and glob.iglob

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = None created_at = labels = ['type-feature', 'library'] title = 'option to not follow symlinks when globbing' updated_at = user = 'https://github.com/o11c' ``` bugs.python.org fields: ```python activity = actor = 'wim.glenn' assignee = 'none' closed = False closed_date = None closer = None components = ['Library (Lib)'] creation = creator = 'o11c' dependencies = [] files = ['46566'] hgrepos = [] issue_num = 29475 keywords = ['patch'] message_count = 3.0 messages = ['287256', '287279', '302002'] nosy_count = 3.0 nosy_names = ['ethan.furman', 'o11c', 'wim.glenn'] pr_nums = [] priority = 'normal' resolution = None stage = None status = 'open' superseder = None type = 'enhancement' url = 'https://bugs.python.org/issue29475' versions = ['Python 3.6'] ```

[c1b2d23c-2f4d-4d32-b768-4bdf89431bb7]

Background: I have a data hierarchy with a lot of "sibling" symlinked directories/files. I want to glob only the non-symlink files, because it's a *huge* performance increase.

Before os.scandir, I was using a local copy of glob.py and calling os.path.islink every time, which was faster for my use case, but unacceptable for upstreaming. With os.scandir, my new patch should be acceptable.

The patch includes tests.

Current discussion points:

• Am I making the right decision to still accept symlinks for fully-literal components (in glob0)? It doesn't apply to my use-case, and I imagine some people might want to handle that case separately.
• Are my tests sufficient? I just copied and modified the existing symlink tests.
• Should my flags TODO be implemented before this patch? IMO it would be clearer after, even if it makes the diffs longer.

Future discussion points (don't derail):

• Should my flags TODO be implemented internally (this would significantly shrink any future patches)? (I can work on this)
• Should flags also be exposed externally?
• What additional flags might be useful? (my list: GLOB_ERR, GLOB_MARK, ~GLOB_NOSORT, ~GLOB_NOESCAPE, GLOB_PERIOD, GLOB_BRACE, GLOB_TILDE_CHECK, GLOB_ONLYDIR (+ equivalent for files - also, why doesn't os.scandir have accessors for the other types without doing an unnecessary stat?))
• Is there a bitwise enum (or equivalently, enum set) in the standard library so flags can get sane reprs? (I've implemented this before, but imagine it would be overwhelmed with bikeshedding if it doesn't exist yet)
• Should pathlib really be implementing globbing on its own? That makes it hard to ensure feature parity. Perhaps the glob module needs some additional APIs? (I don't want to work on pathlib itself)

[ethanfurman]

Before talking about the patch, have you signed the Contributer License Agreement yet? The issue tracker isn't showing that you have. Check out https://www.python.org/psf/contrib/contrib-form/ to do so.

---

The symlinks flag: can you give some glob examples showing when, and when not, symlinks will be matched when the symlinks param is False?

A bit flags enum: There is now an IntFlag Enum type which implements bit flags.

[310dd13a-adf5-42f8-b3ef-a4a4f2d763e3]

+1, would like to use this feature too, and I would like it also in pathlib.PosixPath.glob

[gpshead]

The upcoming 3.13 release gains a follow_symlinks= argument to the pathlib.Path.glob API: https://docs.python.org/3.13/library/pathlib.html#pathlib.Path.glob via https://github.com/python/cpython/issues/77609.

[barneygale]

I plan to work on this once #77609 and #116380 are done.