Support for basic situational tests?

[Leont]

Should EU::HC support basic situational tests, such as

• What is the value of sizeof(int) ?
• Is header <foo.h> available?
• Can I link to libbar?

[ribasushi]

Actually there are two distinct use-cases listed here, which can roughly be separated as:

1. Is an OS-optional, but XS-building-essential part available ( is a .h present, etc )
2. What is the value of a variable that is always there in principle (various sizeof, etc)

Additions of the the type 2. are extremely undesirable as they facilitate (if not encourage) dual-implementation dists (XS with PP fallback within one tarball). Considering the long history of such distributions causing pain to both downstream packagers and fatpackers, they are de-facto "deprecated". Hence to be both maximally useful and maximally "nudging in the most utilitarian direction", EU::HC should concentrate on being able to reliably answer the following single question:

Should a CPAN client add <Something::Something>::XS to runtime_requires

Anything beyond this is decidedly out of scope (my 2c, not speaking for the PTG, yadayada :)

[ribasushi]

1. Is an OS-optional, but XS-building-essential part available ( is a .h present, etc )

I didn't mention what I think of the other case (1.) - EU::HC should try to answer questions like the above, as long as it can be done cheaply enough to not preclude bundling and/or add a large amount of moving parts. Naturally both cheaply and large amount are open to interpretation...

[Leont]

I didn't mention what I think of the other case (1.) - EU::HC should try to answer questions like the above, as long as it can be done cheaply enough to not preclude bundling and/or add a large amount of moving parts. Naturally both cheaply and large amount are open to interpretation...

If we'd implement header checking simply as adding an «#include <foo.h» in the source, it can be done in a handful of lines. Checking for libraries may be trickier though: I'm not sure how much of liblist we'd need there.

[ribasushi]

@Leont In this case it seems like a logical choice to allow the user to explicitly ask for "this list of headers", and leave the other part of the question for a future iteration...?

[ribasushi]

This ticket has seen suspiciously little activity :( I would really like others to chime in on the classification/reasoning as listed in https://github.com/Leont/extutils-hascompiler/issues/3#issuecomment-115696713

Pinging specifically: @shadowcat-mst, @dagolden, @bingos, @haarg, @ambs, @tsee, @rafl, @ap, @miyagawa, @tux

[Tux]

@Leont contacted me this week to test on AIX, and the tests PASSed

[dagolden]

@ribasushi, you said:

Additions of the the type 2. are extremely undesirable as they facilitate (if not encourage) dual-implementation dists (XS with PP fallback within one tarball)

Could you elaborate, please?

More generally, I am neutral on whether this should be a unix-philsophy single-purpose thing ("can I safely require an XS module?") or a better compilation environment inspector ("I am an XS module, but do you support the things I need?")

For example, in my work code, I'm using Config::AutoConf to emulate autotools in a limited way to generate a header file from a template. So questions like "does this particular snippet of code compile with this compiler?" or "does this compiler/library have strtoll or not?" still need to be answered eventually by some tool or another.

[ribasushi]

questions like "does this particular snippet of code compile with this compiler?" or "does this compiler/library have strtoll or not?" still need to be answered eventually by some tool or another.

Absolutely! But this is (imo) out of scope for EU::HC (and in fact C::AC already exists to answer this and you are using it)

I am focusing so strongly on "unix-philsophy single-purpose thing" in this instance, due to the multiple problems we keep re-encountering that are rooted in CPAN's love affair with dynamic configuration. In the case when XS is optional* I believe CPAN can be made much simpler, resilient and friendly from the PoV of the end-user If we have a single generally accepted and debugged-to-boot tool that does only one very simple thing by design: answering the question):

Should a CPAN client add Something::Something::XS to runtime_requires **

This would encourage authors to split their dependency chain early into a "low magic"(PP) and "high magic"(XS) paths, delivering a more "hands-free" experience. From prior conversations with @Leont, I think he had exactly the above in mind when designing EU::HC, but never articulated it in prose, only in code.

@Leont: can you please correct/clarify whatever I missed above?

* Your case of perl code that needs to link to a C-typed library is out of scope of this discussion - it can't work without compilation in the first place ** Or in the distant future: should it add it to runtime_recommends, so that a failure in the XS-branch of the depchain is invisible to the user

[dagolden]

@ribasushi, I think you're focusing too much on the PP vs XS split for a single logical distribution by a single author.

Consider this (slightly contrived) example: "If XS exists and libyaml is installed, then I want YAML::LibYAML, but otherwise if XS exists, then I want YAML::Syck, and otherwise I want YAML::Tiny."

For such a case I do need to know about a library (or headers at least) being available before being able to specify dependencies.

If EU::HC only checks if XS compilation/linkage is possible, then I still need another tool (Devel::CheckLib) or whatever to do the library work. If this isn't the tool for that, then I hope someone will tell me what the tool de jour is.

[shadowcat-mst]

@dagolden but that split is the primary use case for EUHC - the XS dist should then be using ... whatever.

riba already said "EU::HC should try to answer questions like the above, as long as it can be done cheaply enough to not preclude bundling and/or add a large amount of moving parts" wrt the library question so I'm not sure how your contrived example disagrees with him.

[shadowcat-mst]

Basically, EUHC's primary mission so far as I see it is "in the Makefile.PL for Foo, test to see whether we should be able to add Foo::XS to requires" - so adding a few minor extra features to support that is fine, but letting it creep into trying to be a checklib/autoconf as well just to try and improve the percentage value of 'should' isn't going to be a net win.

[dagolden]

@shadowcat-mst that doesn't preclude my comment that he's too focused on the "split PP/XS case" which affects a view of tradeoffs like "cheaply enough" when it's a single author who can control what's in both the PP and the XS parts.

When the XS dist (possibly by a different author) doesn't do good inspection of the environment, should we make it easy for the dependent to front-run those checks? I think so. I don't know if it's this tool that has to do it, but it seems like it's not a big step from what this tool does to more generalized probes and it would avoid having to bundle one more thing into inc if it could do both.

I'm still neutral on the outcome -- just wanted to raise attention to a different use case.

[ribasushi]

When (... a dependency .. ) doesn't do good inspection of the environment, should we make it easy for the dependent to front-run those checks

Errr... we should... fix the dependency? (as opposed to spreading the fixups around the depchain, or at least this is how I read your preferred resolution of this case)

[dagolden]

Errr... we should... fix the dependency?

In theory we should, but we all know how theory and practice diverge. :-)

[shadowcat-mst]

@dagolden it does not preclude your comment. It does however make it obvious why you suggestion isn't a good idea.

That is, as you say, a different use case. One that EUHC was not intended for.

If we let EUHC grow enough tentacles to fit that use case rather than either fixing or forking the dist as is normally done, then eventually we'll just need a new EUHC-like thing because we'll have destroyed its fitness for its primary purpose.

If "it seems like it's not a big step" to you ... then I think you're undersestimating just much how theory and practice diverge in terms of build systems :)

[shadowcat-mst]

From IRC, it appears @dagolden was using a different definition of generic to me -

16:33 <@mst> your comment about 'generic probes' was the scary part
16:33 <@xdg> "generic" in the most broad "does this piece of code run", not API 
             to extract arbitrary values
16:34 <@xdg> I don't think it needs to be autotools. :-)

[shadowcat-mst]

The other thing is - I really really dislike the "try and figure out if a downstream dist will build" beyond the very basics, for the reason of - the downstream dist's build system could get improved. We shouldn't be disabling things because of a build system bug that might not be there anymore.

[dagolden]

11:36AM @ribasushi | xdg: so in essence you'd be happy with ->can_compile_loadable_objects( custom_source =>
                   | <<'END_OF_C_SNIPPET_WHOSE_MAIN_MUST_EXIT_0' ) ?
11:36AM       @xdg | yes.  But the XS dist needs to have EU::HC, too, right?  So as long as it's there, if
                   | the *only* thing I need to do is check if one particular function links, making me go
                   | find/write yet another tool would make me sad.
11:36AM       @xdg | ribasushi, Yes
11:37AM @ribasushi | I think that's a reasonable "escape hatch"
11:37AM @ribasushi | mst: ^^
11:37AM       @mst | fine by me
11:37AM       @xdg | I suspect everything I currently use Config::AutoConf for could be replaced with
                   | 'custom_source' as you show.

[Leont]

11:36AM @ribasushi | xdg: so in essence you'd be happy with ->can_compile_loadable_objects( custom_source => <<'END_OF_C_SNIPPET_WHOSE_MAIN_MUST_EXIT_0' ) ?

Except that's subtly not what EUHC currently does. It compiles a loadable object, not an executable. I'm not entirely sure if that's a good or a bad thing in this case.

[dagolden]

I'm not concerned about the return value of main. Most two cases:

• source doesn't compile
• compiled object doesn't link (e.g. WinXP MSVCRT.dll has no strerror_s)

For the second case, I don't know if it's sufficient to generate a loadable object and see that dynaloader can load it. From the failure case I've seen, I suspect yes. (E.g. this report I believe is due to this missing strerror_s symbol and it compiled but failed during dynaload.)

[ribasushi]

@dagolden I suppose that running a subprocess witch tries to load the compiled .so/.dll under PERL_DL_NONLAZY is likely to be sufficient yes...