search

BrianGladman / mpir

Multiple Precision Integers and Rationals
GNU General Public License v3.0
75 stars 36 forks source link

deal with zero_p and perfpow in autoconf, correct includes #7

Closed dimpase closed 6 years ago

dimpase commented 6 years ago

I didn't expect concrete files to be dealt with in configure.ac, but here you go.

dimpase commented 6 years ago

too early. Now it works on OSX, but fails on Linux...

dimpase commented 6 years ago

It seems to be a bug in mpn/generic/perfpow.c: it calls TMP_ALLOC_LIMBS_3(), but there is no such function in MPIR as far as I can tell. This is only observed on Linux corei7-avx, see https://travis-ci.org/dimpase/mpir/jobs/347267406

BrianGladman commented 6 years ago

I am now wondering why this didn't fail in the MSVC build. I am just testing with the missing macro and will commit it in a few minutes.

BrianGladman commented 6 years ago

My apologies for the wild goose chase but it turns out that mpn_perfect_power_p from GMP is not as easy to move into MPIR as I thought since it uses a number of division functions that we don't have. So I am afraid I will have to remove it (adjusting it to use our division functions is an option but this will take some time and probably isn't worth it given that we have an mpz equivalent).

BrianGladman commented 6 years ago

I have taken mpn/generic/perfpow.c out of MPIR - do you need to remove it on the Linux/GCC side Dima?

BrianGladman commented 6 years ago

I took the reference to perfpow out of configure.ac and I think the library builds now. But it looks like the build fails in docs so I may have more work to do here.

BrianGladman commented 6 years ago

Travis is now showing that the current version passes. We need to bump the MPIR version number - what should it be? I would appreciate it if someone could build the pdf version of the MPIR documentation so that I can double check it in its final form.

dimpase commented 6 years ago

here is the pdf I got from your branch.

BrianGladman commented 6 years ago

Thanks Dima, I was able to find a few small errors that I will correct and commit soon.

However, before we can finalise the documentation, we need to decide what, if anything, we are going to do about the GIT sub-module issue since this might change what we say about the Visual Studio build system.

There are also other things that we can do to simplify the Windows build files within the distribution. First we could remove the pre-installed 32-bit builds, leaving users who need them to install them as necessary. We might also consider only pre-installing the three pure C/C++ library builds, leaving users to install the optimised assembler based builds that they need. We could also drop support for Visual Studio 2012 leaving the current and previous versions (Visual Studio 2017 and 2015).

I am quite relaxed about these possibilities since they do not have a big impact on the usability of the native build system on Windows (maybe removing support for Visual Studio 2012 will hit more users than I expect).

dimpase commented 6 years ago

I'm not sure what you mean by "builds". IMHO we first of all need a source distribution. It can be enhanced with pre-built binaries for for particular platforms.

BrianGladman commented 6 years ago

Each target on Windows requires a build file to build that particular target (i.e a *.vcxproj file that is loaded into Visual Studio). There are about 32 target architectures for each of 3 versions of Visual Studio and for static or dynamic libraries, making around 200 Visual Studio project files in total. I am referring to these files.

Most of these files will be generated by end users when they are needed but historically, to make things easier, we have pre-installed a number of these files in each MPIR distribution.

dimpase commented 6 years ago

How does one make such a build file? Why would one want a pre-made one, instead of making one on the spot?

On 3 Mar 2018 10:28, "Brian Gladman" notifications@github.com wrote:

Each target on Windows requires a build file to build that particular target (i.e a *.vcxproj file that is loaded into Visual Studio). There are about 32 target architectures for each of 3 versions of Visual Studio and for static or dynamic libraries, making around 200 Visual Studio project files in total. I am referring to these files.

Most of these files will be generated by end users when they are needed but historically, to make things easier, we have pre-installed a number of these files in each MPIR distribution.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/BrianGladman/mpir/pull/7#issuecomment-370137108, or mute the thread https://github.com/notifications/unsubscribe-auth/ABN8HAIyBrpgL-8_-LAa-DYYaDWHmGiFks5tanBAgaJpZM4SWpW- .

BrianGladman commented 6 years ago

Because users on Windows will expect MPIR to build in Visual Studio 'out of the box' and this requires a few popular build options to be pre-installed

For previous releases we have provided the pure C/C++ version, one optimised win32 version and one optimised x64 version with the latter two being chosen so that they are likely to run on most Intel/AMD architectures (for example the p3 and core2 options respectively).

All the build files are generated by running the Python file mpir/msvc/mpir_config.py. This offers a list of target architectures which can be selected after which all the *.vcxproj files are added to the Visual Studio solution file mpir.sln.

Solution files in Visual Studio (.sln files) provide a set of related project files (.vcxproj files) that are loaded into Visual Studio when the solution file is opened.

I hope this explains how it works but please keep asking if anything is unclear!

dimpase commented 6 years ago

I think it is a huge overkill to distribute the output of a python script. Perhaps one can provide an extra file with all these generated outputs (I doubt there is a huge demand for them; I just cannot imagine a (Windows or otherwise) programmer who is not capable of running a Python script.

BrianGladman commented 6 years ago

I am sorry but I have suggested that we can minimise the number of pre-installed build files (a) by removing the Visual Studio 2012 build, and (b) by only providing the generic C/C++ builds for Visual Studio 2015 and 2017. We could still further reduce them by killing off support for Visual Studio 2015.

But removing all pre-installed Visual Studio build files from the distribution is effectively removing the MSVC build capability entirely since, as I have said, many Visual Studio users will expect to be able to build MPIR 'out-of-the-box' - i.e. they will find an mpir.sln file in the distribution that will build MPIR when it is opened in Visual Studio.

Dima, if, as it seems, your aim is to entirely remove the Visual Studio build capability from MPIR distributions, why not just say so!

dimpase commented 6 years ago

Well, the usual Unix build process is two-stage, 1st you run the script called configure, and then you run make. I don't see how telling windows users to run "python foo.py" and then open VS with the resulting .sln file means removing windows support.

Naturally this has to be said in the docs, but it is really no rocket science to run a python script in order to configure things, it is bog-down normal...

On 4 Mar 2018 09:13, "Brian Gladman" notifications@github.com wrote:

I am sorry but I have suggested that we can minimise the number of pre-installed build files (a) by removing the Visual Studio 2012 build, and (b) by only providing the generic C/C++ builds for Visual Studio 2015 and

  1. We could still further reduce them by killing off support for Visual Studio 2015.

But removing all pre-installed Visual Studio build files from the distribution is effectively removing the MSVC build capability entirely since, as I have said, many Visual Studio users will expect to be able to build MPIR 'out-of-the-box' - i.e. they will find an mpir.sln file in the distribution that will build MPIR when it is opened in Visual Studio.

Dima, if, as it seems, your aim is to entirely remove the Visual Studio build capability from MPIR distributions, why not just say so!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/BrianGladman/mpir/pull/7#issuecomment-370190571, or mute the thread https://github.com/notifications/unsubscribe-auth/ABN8HIsd5JjWrymLMsoiZnyVnwa-af5Cks5tazGfgaJpZM4SWpW- .

BrianGladman commented 6 years ago

Users on Windows are even less inclined to read manuals than people who use Linux. In consequence, it is important to make the process of building MPIR as painless as possible in the scenario that the manual won't even be read.

This is not because I have much sympathy for users who are not prepared to do a little reading but rather because I don't want to spend my time answering the questions that will arise if I don't cater for Windows users who know little about development.

They can certainly install Visual Studio. And some will even have discovered that building things with Visual Studio requires double clicking on a *.sln file, waiting for Visual Studio to load, and then clicking on 'Build'. But a significant proportion of them won't know what Python is and certainly won't know how to run Python code.

So I simply want to offer a minimal pre-installed build of the generic C/C++ version of MPIR to allow Windows users to find and use the mpir.sln file to build a working version of the MPIR library without having to install anything else.

Given that you are reluctant to support this need in a common GCC/MSVC distribution, the obvious solution is to offer two distributions for each release, one for GCC and one for MSVC.

This solves your problem of not wanting to support MSVC in a single distribution in the form I want to provide it. And it supports my wish to deliver the MSVC build in a particular way. And we don't need to bother with GIT sub-modules since I simply provide the MSVC release directly from my repository as it is now.

Can we agree on this approach?

dimpase commented 6 years ago

Nobody would expect the user to read the manual, but reading README and INSTALL (I'd suggest them to be converted to Markdown, so that they look better in the browser, and can have proper hyperlinks) would be expected. It is a documentation bug that INSTALL says nothing about installation on Windows.

VS has built-in support for Python. If you selected Python workload while installing VS, you're all set. I'd consider this point moot, it's not 1998, it's 2018 now.

In fact, after the generation is done by mpir_config.py one can also do the build(s) by calling msbuild on the .sln file; this is arguably a less error-prone way. And it is also a way to provide continuous integration tests on Appveyor using VS rather than MSYS.

Providing releases of the same software from different repos is not a good idea. One can have different files linked in https://github.com//mpir/releases for a suitable value of whatever. It's a pity that github user/org name mpir is already taken, and this user has one non-empty repo, tellingly called smiley-poo-emoji-vector :-(

BrianGladman commented 6 years ago

Visual Studio has support for Python only if a user installs it. And it is likely that most users don't.

Yes, msbuild.bat is available if users want to do a build outside Visual Studio.

We can have two separate release distributions - one for GCC and one for MSVC - from a common repository. This has the distinct advantage that they don't have to carry around each other's 'dead weight'.

dimpase commented 6 years ago
wbhart commented 6 years ago

On 4 March 2018 at 23:13, Dima Pasechnik notifications@github.com wrote:

-

about the source repo; regarding dead weight - these vcxproj* files in the source repo are certainly dead weight, and should be removed, as they can be generated using python; when I first raised this point, it has not even crossed my mind that they are generated files! If they were gone I'd be happy with the current structure of the repo.

One does not know what most users have or don't have; anyway, the fact that yasm needs to be installed for building as well easily overweights need for python installation.

Oops, sorry, I overlooked msbuild.bat, as these are in msvc/build/vc1*. Then, why there is no testing on Appveyor done with VS? @wbhart https://github.com/wbhart, can you comment on this?

No idea. I only just found out yesterday that the MSVC tests in Flint were being built on Appveyor, but not run, because they would hang all the time.

I have no objection to it.

-

I am under impression that you don't realise it was perfectly possible. IMHO it's a good idea to have a top-level msbuild.bat in msvc/build too (or instead). It would certainly simplify it for Appveyor tests (the less different scripts to call for different compilers, the smaller the Appveyor config).

The insistence on distributing generated project files is I think in no small part due to absence of detailed source installation docs for mpir on VS (note that currently then manual's Sect. 2.4 does not even have anything about generating solution files---it mentions need for Python (these instructions are outdated, as Python is bundled with VS) and then stops). Once you start writing such a doc you'd see that it is infinitely easier to describe how to install MPIR using batch files than how to install it without them (as the latter involves providing navigation details for the IDE, good luck with this...). And once it is in place, the need to even distribute generated files will be gone.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/BrianGladman/mpir/pull/7#issuecomment-370269160, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOzpreQ1DB4Ri5HqchWWgvXNjH4ueBUks5tbGb7gaJpZM4SWpW- .

BrianGladman commented 6 years ago

I am now confused - can we please distinguish between: (a) what should go in the new MPIR master repository when we set it up, (b) what should go in the GCC distribution of the new MPIR release, and (c) what should go in the MSVC distribution of the new MPIR release. Leaving, for the moment, the issue of whether (b) and (c) are separate or combined into a single distribution.

On (a) I am entirely content that the *.vcxproj files should not be present. They will exist in my repository but not in the MPIR master repository.

On (b) I leave the content entirely in the hands of those who are managing this part of the build system.

On (c) I would expect to see (1) those files within the msvc directory that are necessary to support the build of MPIR using MSVC; (2) the vs12, vs15 and vs17 sub-directories and their content, including at least the *.vcxproj files needed to build MPIR with MSVC 'out-of the box' using the generic C/C++ MPIR source code.

If (b) and (c) can be sensibly combined into a single distribution, great! If not, the GCC and MSVC distributions will need to be delivered as separate distributions in order to preserve the ability to build MPIR 'out of the box' using Visual Studio alone (note here that this does not require YASM or Python).

dimpase commented 6 years ago

It seems that you think that the distribution can only have files committed in the repo, but this is not the case. In fact github allows you to create almost arbitrary binary data to be distributed (only size is the limit, more or less). See https://help.github.com/articles/editing-and-deleting-releases/

There is no need for separate git repos (only if we needed git submodules it would be necessary). In view of the above there is no need to keep *.vcxproj* files in the repo.

dimpase commented 6 years ago

(b) and (c) can be combined in one distribution if you drop distributing *.vcxproj* and require using Python for bootstrap.

By the way, even Microsoft itself stopped distributing project file templates in favour of sln files, so it seems that MPIR is stuck in the past here.

BrianGladman commented 6 years ago

Yes you are right, I don't know what can and cannot be added to a distribution. I was trying to specify what is needed in the MSVC part of the distribution without setting constraints on how this is achieved.

I realise that (b) and (c) can be combined if I drop the 'out-of-the box' build requirement. But, as I hope you might have noticed by now, I am not dropping it!

BrianGladman commented 6 years ago

Incidentally, I am not sure what you mean when you say that Microsoft has stopped distributing project file templates in favour of sln files since sln files are just lists of .proj files (.vcxproj for C/C++, .pyproj for Python, ...).

dimpase commented 6 years ago

Incidentally, I am not sure what you mean when you say that Microsoft has stopped distributing project file templates in favour of sln files

Oops, sorry, never mind, I mistook vdproj (VS installer, now gone) for vcxproj. I looked a bit on how in 2018 one is supposed to package Windows projects, you know...