Put *.py{,c} in /usr/share for FHS compliance

[doko42]

BPO	588756
Nosy	@loewis, @warsaw, @jackjansen, @rhettinger, @doko42, @facundobatista, @merwok

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = created_at = labels = ['type-feature', 'invalid'] title = 'Put *.py{,c} in /usr/share for FHS compliance' updated_at = user = 'https://github.com/doko42' ``` bugs.python.org fields: ```python activity = actor = 'eric.araujo' assignee = 'none' closed = True closed_date = closer = 'facundobatista' components = ['None'] creation = creator = 'doko' dependencies = [] files = [] hgrepos = [] issue_num = 588756 keywords = [] message_count = 23.0 messages = ['53586', '53587', '53588', '53589', '53590', '53591', '53592', '53593', '53594', '53595', '53596', '53597', '53598', '53599', '53600', '53601', '53602', '53603', '53604', '53605', '53606', '56709', '113929'] nosy_count = 8.0 nosy_names = ['loewis', 'barry', 'jackjansen', 'rhettinger', 'doko', 'facundobatista', 'cantanker', 'eric.araujo'] pr_nums = [] priority = 'low' resolution = 'not a bug' stage = 'resolved' status = 'closed' superseder = None type = 'enhancement' url = 'https://bugs.python.org/issue588756' versions = [] ```

[doko42]

[From: http://bugs.debian.org/134762]

FHS Compliance - .py{,c} are architecture independant thus belong in /usr/share

The Python manual makes it clear that byte compiled python files are platform independant, and thus belong in arch-independant packages and stored in /usr/share, as per the FHS recommendations for such things.

So the request is to store them in \<prefix>/share/pythonX.Y.

[doko42]

Logged In: YES user_id=60903

FHS: Filesystem Hierarchy Standard

http://www.pathname.com/fhs/

[61337411-43fc-4a9c-b8d5-4060aede66d0]

Logged In: YES user_id=21627

I think this requires a PEP. A Python package can consist of byte code modules and extension modules; arranging the package loader to find those in different directories is a challenge.

[8a54bb72-6b25-473e-a449-295c9db8c6ab]

Logged In: YES user_id=163326

Python runs on dozens of platforms besides Debian Linux. Thus the Linux FHS shouldn't concern Python at all. I'd propose to close this bug as "Invalid".

[doko42]

Logged In: YES user_id=60903

The reason given to close the report seems to be invalid. The FHS has nothing to with Debian (except that we follow the FHS). The FHS is targeted at Unix distributions, so it's neither limited to a single distribution nor to Linux distributions in general.

[jackjansen]

Logged In: YES user_id=45365

I'm confused. If you configure with --exec-prefix=/foo --prefix=/foo/share/pythonX.Y isn't that good enough?

If it's good enough (i.e. if it allows you to build a Python that adheres to the FHS if you are so inclined) that I agree with ghaering: there's no reason to force people to adhere to the Filesystem Hierarchy Standard, so let's close the bug.

If it is impossible to make an FHS-compliant distribution with the current setup: please explain.

[doko42]

Logged In: YES user_id=60903

Not yet.

--prefix=/foo/share/pythonX.Y would lead to /foo/share/pythonX.Y/lib/pythonX.Y.

The SCRIPTDIR is somewhat hardcoded in getpath.c. So it's not possible to install into /foo/share/pythonX.Y, only /foo/share/lib/pythonX.Y is supported.

The FHS doesn't specify where to put files inside /usr/share, but most distributions put application specific files directly in /usr/share.

[doko42]

Logged In: YES user_id=60903

when configured with --prefix=/usr/share and --exec-prefix=/usr, python installs the header files into /usr/share/include/pythonX.Y, which is at least unusual. According to the FHS these files should go into /usr/include/pythonX.Y

[jackjansen]

Logged In: YES user_id=45365

Well... the usual way in which people implemented sharing between architectures was that the /usr/share hierarchy duplicated the other hierarchies (i.e. with bin, lib, etc), and it was simply mounted cross- platform from an nfs server. That's the architecture the Python install was created for. I have no idea why FHS modified this test-and-tried layout that's been in use for at least 15 years.

But: if you really want the other layout, why not submit a fix to configure.in and Makefile.pre.in? Simply add, say, --fhs-prefix=/usr/ share and if that option is present override the Makefile.pre.in declarations for SCRIPTDIR, LIBDEST and INCLUDEDIR?

(Hmm, coming to think of it: it seems rather silly that the FHS puts include files into /usr/include, where they aren't shared... If there's one thing that can be crossplatform it's source code....)

[jackjansen]

Logged In: YES user_id=45365

Well... the usual way in which people implemented sharing between architectures was that the /usr/share hierarchy duplicated the other hierarchies (i.e. with bin, lib, etc), and it was simply mounted cross- platform from an nfs server. That's the architecture the Python install was created for. I have no idea why FHS modified this test-and-tried layout that's been in use for at least 15 years.

But: if you really want the other layout, why not submit a fix to configure.in and Makefile.pre.in? Simply add, say, --fhs-prefix=/usr/ share and if that option is present override the Makefile.pre.in declarations for SCRIPTDIR, LIBDEST and INCLUDEDIR?

(Hmm, coming to think of it: it seems rather silly that the FHS puts include files into /usr/include, where they aren't shared... If there's one thing that can be crossplatform it's source code....)

[8a54bb72-6b25-473e-a449-295c9db8c6ab]

Logged In: YES user_id=163326

I assume that the Python directory layout is the same on all currently supported platforms by Python. I really don't know enough to be sure - the less that's true, the less my following argument will be valid:

There are really two concerns: 1) make Python conform to the FHS 2) make Python behave the same cross-platform (Windows, Unix, Mac, BeOS, OS/2, VMS, AS/400, ...)

You can't have both unless you force the FHS directory layout onto all other platforms. I personally think that 2) is a good thing.

I welcome the proposal of a configuration option to make Python FHS-compliant, but I think it should not be the default. Btw. you'd probably have to patch distutils, too.

Jack said that Pyhon include files should be cross-platform. AFAIK they are, with one exceptions: pyconfig.h.

[8a54bb72-6b25-473e-a449-295c9db8c6ab]

Logged In: YES user_id=163326

Sorry, Matthias. I was confusing the FHS with the Linux Standard Base.

[jackjansen]

Logged In: YES user_id=45365

The layouts are similar, but not the same. MacPython-OS9 uses the source tree layout, and Windows-Python too, IIRC. MacPython-OSX in a framework build uses a normal layout, but in a very funny place, with lots of symlinks to make it behave like a framework. I would be very surprised if, say WinCE python or AS/400 python had anything resembling a unix layout. But all these layouts are similar enough that I've never come across a Python where I felt completely lost.

I think there's nothing wrong with enabling people to use their preferred layout, if they have a good reason for it, but I would be against enforcing it unless the advantages are clear and universal.

And someone has to do the work:-)

[doko42]

Logged In: YES user_id=60903

Ok, I'll give --fhs-prefix a try. some questions:

• the lib_python in getpath.c hardcodes lib/pythonX.Y to search for the libraries. Is it enouogh to set PYTHONPATH to pythonX.Y?

• who to ask for distutils? are there concerns if a module/library is splitted over two directories? Or should there symlinks from /usr/lib/pythonX.Y to /usr/share/pythonX.Y?

• currently there is only one site-packages directory. how should two site-packages be supported (lib and share)?

• speaking of etc: this is a configuration file and should belong into the etc hierarchy. should etc be searched before or after the library directories?

Python's include files: not all packages separate platform specific headers from generic headers, probably therefore the FHS puts them in /usr/include.

[brettcannon]

Logged In: YES user_id=357491

Will PEP-304 solve your problem?

[doko42]

Logged In: YES user_id=60903

PEP-304 addresses one part: the location of the generated .py[co] files. I fail to see, how it adds support to put .py files in /usr/share.

So it partly solves the problem.

[brettcannon]

Logged In: YES user_id=357491

It won't help with that request. Doing that would require changing install paths as suggested below.

As for your questions about implementing --fhs-prefix, I can answer a few.
For Distutils questions you can just email python-dev to get the attention of Distutils developers. For adding a second site-packages directory I am against (use PYTHONPATH if that is needed).

[2ab43153-d677-4159-951b-5b7e34e5446d]

Logged In: YES user_id=209092

Did anyone bother *reading* the FHS?

http://www.pathname.com/fhs/2.2/fhs-4.7.html

/usr/lib includes object files, libraries, and internal binaries

http://www.pathname.com/fhs/2.2/fhs-4.11.html

The /usr/share hierarchy is for all read-only architecture independent data files.

.py{,c} files are *libraries, not *data files.

Thankyou, move along.

[brettcannon]

Logged In: YES user_id=357491

I think PYC files can be considered either libraries or data files.
Either way I am making this a feature request instead of a bug.

[rhettinger]

Logged In: YES user_id=80475

It seems probable that no changes are forthcoming. So, can this be closed?

[61337411-43fc-4a9c-b8d5-4060aede66d0]

Logged In: YES user_id=21627

As a feature request, this should be only closed if implemented, definitely rejected (for all times, e.g. because it is undesirable), or after being added to PEP-42.

[facundobatista]

Added to the PEP-42, rev 58638.

[merwok]

doko has closed the downstream bug on Thu, 19 Nov 2009, with the reason that .py{,c} are not data files in the FHS sense. (Note that even Lisp where code is data (and the reverse) is installed both under /usr/lib and /usr/share on Debian, so I don’t really know what to think.)

About the more general FHS problem, Tarek (the distutils and distutils2 maintainer) has a draft PEP about fine-grained control over every installation directory, so FHS layout for Python projects will be easily possible in the future. See http://bitbucket.org/tarek/distutils2/src/tip/docs/design/wiki.rst (product of Pycon 2010, done with OS packaging experts, doko and other people). As for Python itself, we’ll see if there are other bugs about FHS compliance, but it seems good right now.