what if cc changes?

[p5pRT]

Migrated from rt.perl.org#4417 (status was 'rejected')

Searchable as RT4417$

[p5pRT]

From @jhi

It was pointed out to me that we have a nasty problem now that vendors are shipping Perl with their OSes. This Perl is most naturally compiled with the vendor's compiler kit. Now\, if the compiler kit is 'unbundled' building any additional modules or trying to update existing ones will fail spectacularly because one or more of the compiler/linker name\, compiler/linker flags will mismatch. In some cases it could be possible and even feasible to try to probe for alternative compilers\, such as gcc. (*)

I can see some ways how Configure could help in this (by trying to encapsulate some of the wisdom currently buried in the hints files for easier reuse later) but I think the right place to do such runtime heuristic fixing would be MakeMaker. Basically\, try out $Config{cc} and if that fails\, try to be smart and find out a replacement. Most often this would mean probing for gcc\, but I can see also the opposite happening if somebody purchases vendor's compiler kit\, and wants to start using that. (So maybe there needs also be a way for the Perl maintainer to sub-re-Configure these parts of the Config.pm?) Of course\, sometimes mixing and matching libraries such as libperl\, between different compilers simply cannot be done.

(*) I will ruthlessly shoot down any attempt to discuss how evil vendors are by 'unbundling' their compiler kits. That is not the problem we are trying to address here. This issue comes up always when the compiler used to build Perl is not available.

[p5pRT]

From [Unknown Contact. See original ticket]

On Mon\, 9 Oct 2000\, Jarkko Hietaniemi wrote​:

It was pointed out to me that we have a nasty problem now that vendors are shipping Perl with their OSes. This Perl is most naturally compiled with the vendor's compiler kit. Now\, if the compiler kit is 'unbundled' building any additional modules or trying to update existing ones will fail spectacularly because one or more of the compiler/linker name\, compiler/linker flags will mismatch. In some cases it could be possible and even feasible to try to probe for alternative compilers\, such as gcc. (*)

I can see some ways how Configure could help in this (by trying to encapsulate some of the wisdom currently buried in the hints files for easier reuse later) but I think the right place to do such runtime heuristic fixing would be MakeMaker. Basically\, try out $Config{cc} and if that fails\, try to be smart and find out a replacement. Most often this would mean probing for gcc\, but I can see also the opposite happening if somebody purchases vendor's compiler kit\, and wants to start using that. (So maybe there needs also be a way for the Perl maintainer to sub-re-Configure these parts of the Config.pm?) Of course\, sometimes mixing and matching libraries such as libperl\, between different compilers simply cannot be done.

(*) I will ruthlessly shoot down any attempt to discuss how evil vendors are by 'unbundling' their compiler kits. That is not the problem we are trying to address here. This issue comes up always when the compiler used to build Perl is not available.

Might I mention yet another approach? How about taking a lesson from the microcomputer ports (specifically MacPerl and Windows perl) and provide a convenient MakeMaker based way to easily distribute an extension from one machine (that presumably has the expensive fully licensed compiler needed to build perl extensions) to another less well equipped machine.
Activestate's ppm and MacPerl's droplets could serve as templates for such an activity(?).

I guess that I have longed for a 'bindist' target to be in MM produced Makefile's such that the resultant tarball when unpacked into the correct location will give the target machine the same XS based capability as the machine where the `make bindist` was run. Actually I'd like it too if perl's Configure provided this (a bindist target in the Makefile). That is\, the thing that is currently missing from $installprefix is an ability to use the resultant perl to XS-tend itself with a few choice modules to produce a tar ball that is not all tangled up in the mess that I already have in /usr/local.

My guess is that adding 'bindist' to MakeMaker could proceed without harming an effort to produce a more dynamic Config.pm than what is current practice.

Peter Prymmer

[p5pRT]

From @gsar

On Mon\, 09 Oct 2000 17​:18​:40 CDT\, Jarkko Hietaniemi wrote​:

It was pointed out to me that we have a nasty problem now that vendors are shipping Perl with their OSes. This Perl is most naturally compiled with the vendor's compiler kit. Now\, if the compiler kit is 'unbundled' building any additional modules or trying to update existing ones will fail spectacularly because one or more of the compiler/linker name\, compiler/linker flags will mismatch. In some cases it could be possible and even feasible to try to probe for alternative compilers\, such as gcc. (*)

Messing around in Config.pm is the only viable approach I've found for this. The problem with doing that​: you lose information about how Perl was built\, since there's only one $Config{cc}. Ideally there'd be some way to keep the old one around. ($Config{cc_when_built}\, $Config{cc_current} and so on.)

The "messing around in Config.pm" part is also non-trivial\, since it needs parts of the Configure logic to do it right.

Sarathy gsar@​ActiveState.com

[p5pRT]

From @AlanBurlison

Peter Prymmer wrote​:

I guess that I have longed for a 'bindist' target to be in MM produced Makefile's such that the resultant tarball when unpacked into the correct location will give the target machine the same XS based capability as the machine where the `make bindist` was run. Actually I'd like it too if perl's Configure provided this (a bindist target in the Makefile). That is\, the thing that is currently missing from $installprefix is an ability to use the resultant perl to XS-tend itself with a few choice modules to produce a tar ball that is not all tangled up in the mess that I already have in /usr/local.

Wot\, you mean like this?

---

$ mktarpkg
This will build tar archives for Perl and any selected modules. You can opt either to just build individual tar archives for Perl and selected modules\, or you can choose to build a combined archive that also includes an install shell script. The shell script will allow selection of the modules to be installed. Do you wish to build a combined Perl + module archive? [y] n Finding all installed modules... Do you want to package the Perl core? [y] n

Found module Archive​::Tar Version 0.22 Do you want to package Archive​::Tar? [y]

Found module C​::Scan Version 0.74 Do you want to package C​::Scan? [y] n

...

Packaging Archive​::Tar Creating module tar archives   Executing "tar -cf Archive​::Tar.tar -I /tmp/mktarpkg.104423" Finished. The tar files for the selected modules have been created in your current directory

$mower# tar tvf Archive​::Tar.tar -r-xr-xr-x 0/1 1622 May 23 21​:14 2000 /usr/local/bin/ptar -r--r--r-- 0/1 31965 Apr 27 23​:50 2000 /usr/local/lib/perl5/site_perl/5.6.0/Archive/Tar.pm -rw-r--r-- 0/1 107 May 23 21​:14 2000 /usr/local/lib/perl5/site_perl/5.6.0/sun4-solaris-64int/auto/Archive/Tar/.packlist -r--r--r-- 0/1 15578 May 23 21​:14 2000 /usr/local/man/man3/Archive​::Tar.3

---

All the information needed to do this is there already in the form of the .packlist files\, along with the ExtUtils​::Packlist and ExtUtils​::Installed modules. The only thing missing is module dependency information. I've offered this up for consideration several times but it has been met with waves of indifference.

[p5pRT]

From [Unknown Contact. See original ticket]

From​: Peter Prymmer [mailto​:pvhp@​forte.com]

My guess is that adding 'bindist' to MakeMaker could proceed without harming an effort to produce a more dynamic Config.pm than what is current practice.

AIUI\, the "PPM" binary format is usable on any platform - just do "make ppd" to make the .ppd file\, and then tar/gz the "blib" directory from the build. Take the two of these to the target machine\, and use PPM to install them.

The only problem(s) are (1) PPM has dependencies (like XML​::Parder) which may not be present - chicken & egg problem\, and (2) as far as I know\, PPM is not available from CPAN\, or indeed anywhere independently of ActiveState's distributions. I don't believe that there is any licensing reason why it shouldn't be available\, maybe someone in ActiveState could put PPM onto CPAN\, so that it could be used.

Again\, I don't know how much interest there would be in this\, but it is there already.

Paul.

[p5pRT]

From @gsar

On Tue\, 10 Oct 2000 09​:39​:55 BST\, "Moore\, Paul" wrote​:

The only problem(s) are (1) PPM has dependencies (like XML​::Parder) which may not be present - chicken & egg problem\, and (2) as far as I know\, PPM is not available from CPAN\, or indeed anywhere independently of ActiveState's distributions.

Actually\, it is. Try http​://public.ActiveState.com/murray/.

I don't believe that there is any licensing reason why it shouldn't be available\, maybe someone in ActiveState could put PPM onto CPAN\, so that it could be used.

Murray was supposed to get a CPAN id\, but I dunno if he managed to. Cc-ed.

Sarathy gsar@​ActiveState.com

[p5pRT]

From [Unknown Contact. See original ticket]

On Tue\, 10 Oct 2000\, Moore\, Paul wrote​:

From​: Peter Prymmer [mailto​:pvhp@​forte.com]

My guess is that adding 'bindist' to MakeMaker could proceed without harming an effort to produce a more dynamic Config.pm than what is current practice.

AIUI\, the "PPM" binary format is usable on any platform - just do "make ppd" to make the .ppd file\, and then tar/gz the "blib" directory from the build. Take the two of these to the target machine\, and use PPM to install them.

The only problem(s) are (1) PPM has dependencies (like XML​::Parder) which may not be present - chicken & egg problem\, and (2) as far as I know\, PPM is not available from CPAN\, or indeed anywhere independently of ActiveState's distributions. I don't believe that there is any licensing reason why it shouldn't be available\, maybe someone in ActiveState could put PPM onto CPAN\, so that it could be used.

Again\, I don't know how much interest there would be in this\, but it is there already.

Yes there is interest in this -- judging by the number of posts in other forums that I have seen concerning folks' trials with .rpm\, .deb\, package\, etc. distributions of perl. I was aware of the ppd target for ppm\, but until you mentioned it I did not know that ppm itself was not widely available. That seems a shame.

Peter Prymmer

[p5pRT]

From [Unknown Contact. See original ticket]

On Tue\, 10 Oct 2000\, Alan Burlison wrote​:

Wot\, you mean like this?

-------------------- $ mktarpkg
This will build tar archives for Perl and any selected modules. You can opt either to just build individual tar archives for Perl and selected modules\, or you can choose to build a combined archive that also includes an install shell script. The shell script will allow selection of the modules to be installed. Do you wish to build a combined Perl + module archive? [y] n Finding all installed modules... Do you want to package the Perl core? [y] n

[snip]

All the information needed to do this is there already in the form of the .packlist files\, along with the ExtUtils​::Packlist and ExtUtils​::Installed modules. The only thing missing is module dependency information. I've offered this up for consideration several times but it has been met with waves of indifference.

Yes that is the functionality that I desire. It appears to use Archive​::Tar(?) I suppose there will be some who prefer the flexibility of "native" archive/compress but tar is widely implemented (IIRC Chris Nandor's Mac port of CPAN uses Archive​::Tar to unpackage tar balls on Mac OS).

I'll make a note to check out the package on a few different platforms. Thanks for the pointer.

Peter Prymmer

[p5pRT]

From @AlanBurlison

Peter Prymmer wrote​:

Yes that is the functionality that I desire. It appears to use Archive​::Tar(?) I suppose there will be some who prefer the flexibility of "native" archive/compress but tar is widely implemented (IIRC Chris Nandor's Mac port of CPAN uses Archive​::Tar to unpackage tar balls on Mac OS).

I'll make a note to check out the package on a few different platforms. Thanks for the pointer.

Actually it is just a script that its on top of ExtUtils​::Packlist et al. You can hack it to use whatever archiver you wish. I also have a version that makes SVR4 pkgadd format archives.

The script is under my authors directory on CPAN.

Alan Burlison

[p5pRT]

From [Unknown Contact. See original ticket]

On Tuesday\, 2000-10-10 at 18​:51​:46 +0100\, Alan Burlison wrote​:

Peter Prymmer wrote​:

Yes that is the functionality that I desire. It appears to use Archive​::Tar(?) I suppose there will be some who prefer the flexibility of "native" archive/compress but tar is widely implemented (IIRC Chris Nandor's Mac port of CPAN uses Archive​::Tar to unpackage tar balls on Mac OS).

I'll make a note to check out the package on a few different platforms. Thanks for the pointer.

Actually it is just a script that its on top of ExtUtils​::Packlist et al. You can hack it to use whatever archiver you wish. I also have a version that makes SVR4 pkgadd format archives.

Seems we have three ways here​: 1) Use a tarball\, like Alan does.

2) Use "native" formats like .rpm\, .deb\, SVR4 pkgadd format.

3) Use a Perl-specific format.

I believe whatever is used should maintain the packlist on the target system.

It should also be available on every platform perl currently supports\, and be supportable on any new platform.

My limited experience with .deb\, .rpm\, and SVR4 packages suggests we should avoid number 2 as a general solution. The variations are too great\, and some platforms have no such format.

A pure number 1 does not deliver the functionality. We need to update the packlist on the target system\, possibly create a customized Config.pm for the package\, etc.

I think number 3 is the only sensible route\, if it is a cross-platform PPM or not. We need an installation tool like pkgadd\, rpm\, or so. The format should support {pre\,post}{install\,remove} scripts.

Beneath this format we can use tar\, cpio\, or zip\, or whatever is convenient. It should just have a supporting perl module.

I have the most experience with SVR4 packages\, and I like their structure - fixed and relocatable contents\, pkginfo and pkgmap\, copyright\, etc. files\, and scripts.

One thing to keep in mind is that people might try to add a package multiple times\, maybe using downrev packages. They might also run into fixincludes-like problems.

The package format and tool must detect this\, and package dependencies. Think about a sharable library required by the perl module\, that might not be there because that OS package was not yet installed.

Doing the package generation right requires also a complex tool that can detect most such dependencies automatically (running ldd\, etc.)

Lupe Christoph

[p5pRT]

From @AlanBurlison

Lupe Christoph wrote​:

Seems we have three ways here​: 1) Use a tarball\, like Alan does.

Actually I did this just as a hopefully useful script\, I don't use it myself.

2) Use "native" formats like .rpm\, .deb\, SVR4 pkgadd format.

I use this one - using the 'make a SVR4' variant of the same script.

3) Use a Perl-specific format.

The last thing the world needs is YAPF (Yet Another Packaging Format).

I believe whatever is used should maintain the packlist on the target system.

Errm... mine already does - see the original mail I sent on the subject.

It should also be available on every platform perl currently supports\, and be supportable on any new platform.

.packlists are already produced as standard for the perl core and all add-on modules.

My limited experience with .deb\, .rpm\, and SVR4 packages suggests we should avoid number 2 as a general solution. The variations are too great\, and some platforms have no such format.

Yes\, but if the wherewithall already exists for creating such packages\, I don't see the problem with supporing them if people want them. Sysadmins will usually be familiar with the native packaging format - why make their lives more difficult?

A pure number 1 does not deliver the functionality. We need to update the packlist on the target system\, possibly create a customized Config.pm for the package\, etc.

a) The packlist is already updated (see above). b) Packages like CPAN\, Net et al either have a configure script or will autoconfigure when they are first run. Because the resulting Config.pm is not listed in the packlist\, when the package is bundled up and installed on another system the config process is rerun. Trust me\, it works.

I think number 3 is the only sensible route\, if it is a cross-platform PPM or not. We need an installation tool like pkgadd\, rpm\, or so. The format should support {pre\,post}{install\,remove} scripts.

Sigh. Go search the archives. I had a heated discussion with Dick Hardt over this very subject when it was first raised. He was insistent that PPM was the One True Way and that he would provide it for perl as a cross-platform standard. Well\, several years on and I'm still waiting. I gave up working on the .packlist stuff at that point because it seemed to be a waste of time\, and because it worked well enough for my purposes.

Beneath this format we can use tar\, cpio\, or zip\, or whatever is convenient. It should just have a supporting perl module.

Or RPM\, or SVR4 pkgadd\, or PPM...

One thing to keep in mind is that people might try to add a package multiple times\, maybe using downrev packages. They might also run into fixincludes-like problems.

The package format and tool must detect this\, and package dependencies. Think about a sharable library required by the perl module\, that might not be there because that OS package was not yet installed.

Doing the package generation right requires also a complex tool that can detect most such dependencies automatically (running ldd\, etc.)

All these issues are already known or detected at the time the module is originally built - all that it is necessary to do is to remember them. What is needed are some slight modifications to the .packlist format to add some missing attributes\, namely module name(s)\, module version(s) and dependencies. One wrinkle is when a module is installed that is already in the core the files are listed in two places - the core .packlist and the newly installed module's .packlist. For this reason I think modules included with the core should have their own seperate .packlists rather than being all stuffed into one.

I actually think that .packlists should be restructured as XML rather than the current format.

[p5pRT]

From [Unknown Contact. See original ticket]

On Wednesday\, 2000-10-11 at 09​:33​:57 +0100\, Alan Burlison wrote​:

Lupe Christoph wrote​:

Seems we have three ways here​: 1) Use a tarball\, like Alan does.

Actually I did this just as a hopefully useful script\, I don't use it myself.

Sorry about the phrasage. I mean "like Alan's script does".

3) Use a Perl-specific format.

The last thing the world needs is YAPF (Yet Another Packaging Format).

There seems to be no useful cross-platform format right now. It might be necessary to do a new one.

I believe whatever is used should maintain the packlist on the target system.

Errm... mine already does - see the original mail I sent on the subject.

I guess I should not only rereadthat but also look at you script. Right now I'm alittle too busy to do that.

It should also be available on every platform perl currently supports\, and be supportable on any new platform.

.packlists are already produced as standard for the perl core and all add-on modules.

... in the tarball/package yourscript generates? Fine!

My limited experience with .deb\, .rpm\, and SVR4 packages suggests we should avoid number 2 as a general solution. The variations are too great\, and some platforms have no such format.

Yes\, but if the wherewithall already exists for creating such packages\, I don't see the problem with supporing them if people want them. Sysadmins will usually be familiar with the native packaging format - why make their lives more difficult?

When you try to make their life easier you make life more difficult for the module and tool maintainers. I found .rpm and .deb sufficiently similar to SVR4 packages to sope with them. The tools are a different story\, though.

A pure number 1 does not deliver the functionality. We need to update the packlist on the target system\, possibly create a customized Config.pm for the package\, etc.

a) The packlist is already updated (see above). b) Packages like CPAN\, Net et al either have a configure script or will autoconfigure when they are first run. Because the resulting Config.pm is not listed in the packlist\, when the package is bundled up and installed on another system the config process is rerun. Trust me\, it works.

What about creating the new Config.pm? That needs support from the package itself.

I think number 3 is the only sensible route\, if it is a cross-platform PPM or not. We need an installation tool like pkgadd\, rpm\, or so. The format should support {pre\,post}{install\,remove} scripts.

Sigh. Go search the archives. I had a heated discussion with Dick Hardt over this very subject when it was first raised. He was insistent that PPM was the One True Way and that he would provide it for perl as a cross-platform standard. Well\, several years on and I'm still waiting. I gave up working on the .packlist stuff at that point because it seemed to be a waste of time\, and because it worked well enough for my purposes.

Maybe we should forward the discussion on this to him and see how/if he reacts?

One thing to keep in mind is that people might try to add a package multiple times\, maybe using downrev packages. They might also run into fixincludes-like problems.

The package format and tool must detect this\, and package dependencies. Think about a sharable library required by the perl module\, that might not be there because that OS package was not yet installed.

Doing the package generation right requires also a complex tool that can detect most such dependencies automatically (running ldd\, etc.)

All these issues are already known or detected at the time the module is originally built - all that it is necessary to do is to remember them. What is needed are some slight modifications to the .packlist format to add some missing attributes\, namely module name(s)\, module version(s) and dependencies. One wrinkle is when a module is installed that is already in the core the files are listed in two places - the core .packlist and the newly installed module's .packlist. For this reason I think modules included with the core should have their own seperate .packlists rather than being all stuffed into one.

I'd prefer that\, too. The one main gigantic evergrowing packlist bothers me. And whateverthis will be\, it will need support from MakeMaker. I just fear people who do not properly list their PREREQ_PMs.Doing cpan-tests I found just too many of those. Libraries are worse.

I actually think that .packlists should be restructured as XML rather than the current format.

XML seem to be becoming the problem to end all problems. ;-)

Lupe Christoph

[p5pRT]

From @schwern

Can we say that PAR has/will solved this problem? At least the "how do we create portable\, compiler-independent Perl dists" part.

The "look around and try to find a C compiler if $Config{cc} doesn't work" part I will defer to Module​::Build or someone who is not MakeMaker.

[p5pRT]

From @jkeenan

On Mon Oct 09 08​:18​:54 2000\, jhi wrote​:

It was pointed out to me that we have a nasty problem now that vendors are shipping Perl with their OSes. This Perl is most naturally compiled with the vendor's compiler kit. Now\, if the compiler kit is 'unbundled' building any additional modules or trying to update existing ones will fail spectacularly because one or more of the compiler/linker name\, compiler/linker flags will mismatch. In some cases it could be possible and even feasible to try to probe for alternative compilers\, such as gcc. (*)

I can see some ways how Configure could help in this (by trying to encapsulate some of the wisdom currently buried in the hints files for easier reuse later) but I think the right place to do such runtime heuristic fixing would be MakeMaker. Basically\, try out $Config{cc} and if that fails\, try to be smart and find out a replacement. Most often this would mean probing for gcc\, but I can see also the opposite happening if somebody purchases vendor's compiler kit\, and wants to start using that. (So maybe there needs also be a way for the Perl maintainer to sub-re-Configure these parts of the Config.pm?) Of course\, sometimes mixing and matching libraries such as libperl\, between different compilers simply cannot be done.

(*) I will ruthlessly shoot down any attempt to discuss how evil vendors are by 'unbundling' their compiler kits. That is not the problem we are trying to address here. This issue comes up always when the compiler used to build Perl is not available.

This ticket was originally filed nearly 13 years ago. There have been no postings in more than 10 years. In my reading\, Schwern ruled out using ExtUtils​::MakeMaker as a solution to the problem. After that\, discussion lapsed.

Cross-compilation is inherently difficult and does not admit of broad\, general solutions. Since the original post does not contain specific examples\, the ticket doesn't get us anywhere with respect to specific cross-compilation problems.

I recommend we close this ticket.

Thank you very much. Jim Keenan

[p5pRT]

From @jhi

On Saturday-201308-03 13​:01\, James E Keenan via RT wrote​:

On Mon Oct 09 08​:18​:54 2000\, jhi wrote​:

It was pointed out to me that we have a nasty problem now that vendors are shipping Perl with their OSes. This Perl is most naturally compiled with the vendor's compiler kit. Now\, if the compiler kit is 'unbundled' building any additional modules or trying to update existing ones will fail spectacularly because one or more of the compiler/linker name\, compiler/linker flags will mismatch. In some cases it could be possible and even feasible to try to probe for alternative compilers\, such as gcc. (*)

I can see some ways how Configure could help in this (by trying to encapsulate some of the wisdom currently buried in the hints files for easier reuse later) but I think the right place to do such runtime heuristic fixing would be MakeMaker. Basically\, try out $Config{cc} and if that fails\, try to be smart and find out a replacement. Most often this would mean probing for gcc\, but I can see also the opposite happening if somebody purchases vendor's compiler kit\, and wants to start using that. (So maybe there needs also be a way for the Perl maintainer to sub-re-Configure these parts of the Config.pm?) Of course\, sometimes mixing and matching libraries such as libperl\, between different compilers simply cannot be done.

(*) I will ruthlessly shoot down any attempt to discuss how evil vendors are by 'unbundling' their compiler kits. That is not the problem we are trying to address here. This issue comes up always when the compiler used to build Perl is not available.

This ticket was originally filed nearly 13 years ago. There have been no postings in more than 10 years. In my reading\, Schwern ruled out using ExtUtils​::MakeMaker as a solution to the problem. After that\, discussion lapsed.

Cross-compilation is inherently difficult and does not admit of broad\, general solutions. Since the original post does not contain specific examples\, the ticket doesn't get us anywhere with respect to specific cross-compilation problems.

I recommend we close this ticket.

Go ahead.

Thank you very much. Jim Keenan

[p5pRT]

From @jkeenan

On Sat Aug 03 17​:57​:22 2013\, jhi wrote​:

On Saturday-201308-03 13​:01\, James E Keenan via RT wrote​:

This ticket was originally filed nearly 13 years ago. There have been no postings in more than 10 years. In my reading\, Schwern ruled out using ExtUtils​::MakeMaker as a solution to the problem. After that\, discussion lapsed.

Cross-compilation is inherently difficult and does not admit of broad\, general solutions. Since the original post does not contain specific examples\, the ticket doesn't get us anywhere with respect to specific cross-compilation problems.

I recommend we close this ticket.

Go ahead.

Thanks for getting back on this. Closing.

[p5pRT]

@jkeenan - Status changed from 'open' to 'rejected'