1Stop fails due to missing .git/config when using download .zip option

[wably]

Using Linux (Debian), a 1Stop build fails because the script cannot find the .git/config file in the hyperion directory tree. This occurs when choosing the download .zip option from the green "clone or download" button on github. The resulting hyperion-master.zip file does not contain a .git directory.

When you clone with a git client, you get the .git directory in the cloned copy and 1Stop proceeds without an issue.

Whatever the reason that 1Stop needs the config file should perhaps be reconsidered. Many users would download via the .zip option and not clone via a git client.

This issue probably occurs on Windows as well given the same circumstances although I have not specifically tried it.

[jphartmann]

A UNIX oldie would say that anyone downloading a zip file needs to be sent somewhere for re-education. However, I am well aware that many Linux users have the Windows mindset.

1Stop tries to install missing prerequisites. To do that, it needs to know whence it was cloned to find the path to the master repository; it also needs to know the method of cloning (https vs. git). The only place to find that information is the git configuration file.

Since you don't give any clue as to what happens other than "it doesn't work", it is difficult for me to offer any specific advice, or indeed make a fix, but here is a list of what you might try:

1. Install and build all prerequisites before building Hyperion. Currently that list is short: SoftFloat3a, but Hercules itself requires several things to be in place; refer to the install instructions for the utility that will alert you to most of them. You likely also need libz-devel, even if the library itself is present, which it is in most distros.
2. Remove the 1Stop shell script from the zip file so that this perceived failure cannot occur. But leave 1Stop.cmd unless Steve says otherwise.
3. Don't use 1Stop; it is only a convenience. (Doctor, doctor it hurts when I ...)

The windows version, 1Stop.cmd, is a Windows batch file, not a shell script. I would not assume the behaviour of one from that of the other.

[wably]

My apologies, I should have included the error message. However, there isn't much to it; it essentially said that it could not find .git/config and the script stopped running at that point. It was fairly early in the process.

Your response goes on to talk about work-arounds and other things that can be done instead of 1Stop. Thanks; but I was able to get past the problem at the time. I had all of the other necessary packages installed, including SoftFloat and everything else.

I think this could be viewed more from an end-user ("customer") point of view: I just want to install Hercules and the doc told me to use this 1Stop thing. It did not tell me not to download the .zip file. It did not tell me to use a git client to clone the files. This may simply be a documentation issue.

But if 1Stop is to be the way forward, it probably needs to provide a better customer experience with regard to clarifying what is required. I have to say that once I resolved the issue, using 1Stop was indeed very convenient.

I hope that my comments will be appreciated as constructive criticism, rather than just criticism. As someone very green in Linux use, my experience with 1Stop and all of the issues encountered with loading the required packages are the very issues that many first time Hyperion installers would experience.

[srorso]

Hi Bill:

Yeah, the message would have been helpful, even if only to help us localize where in the 1Stop/Stop.cmd file things are happening.

But it is also helpful that you bring a new builder's experience to the party. Harold and I have been discussing how the documentation might be made more useful to new installers, and even with my limited experience I have built enough times that I have difficulty returning to the new installer's perspective. I certainly had not tried 1Stop.cmd from a zip download.

Existing instructions (README.BuildUNIX and README.BuildWin) both say to start by cloning the repositories, so running 1Stop.cmd or 1Stop from a .zip might reasonably be expected to fail. A note in the documentation and/or some revision of the message in the script might be sufficient to address any confusion that might result..

The goal for 1Stop is to provide a middle ground between a complete binary distribution, which does not and may never exist for open source platforms, and a developer-skills build of Hercules , which skills seem to be relatively rare in the Windows world. I did not have them 18 months ago.

Both 1Stop.cmd and 1Stop use the git configuration file, included in the clone but not in the zip, to locate any other required components that might be needed, at present 1Stop, with the idea that required components should be downloaded from the same git project. Enhancement to support a zip download would require some decisions regarding action to be taken if building from a .zip:

1. Where to retrieve the required components? The only answer I can think of would be the Hyperion project's known URL: https://github.com/Hercules-390/SoftFloat-3a for example.

2. How to retrieve the required components? git clone would be the best choice because that command works the same on all open source and Windows environments. Downloading a zip means 1Stop.cmd or 1Stop would have to deal with differences in the installed zip utility's command line (Winzip, winrar, 7zip, etc on Windows).

The first question raises, now that I think about it, questions about configuration management. For example, if you are building using 1Stop.cmd from your fork of Hyperion, your project needs to include a SoftFloat 3a For Hercules repository as well or the build will fail. And if we enable 1Stop.cmd from zip, if that's possible, we might be building Hyperion from a local fork and linking it to SoftFloat 3a For Hercules from Hercules-390, with the potential for a capabilities mismatch.

And this in turn raises the possibility that your initial issue building Hercules from your own fork using 1Stop.cmd may have failed because there is no S3FH repo in the project containing your Hyperion fork. Also an opportunity to be addressed with documentation. You can always clone SoftFloat 3a For Hercules from Hercules-390 and use its 1Stop to build it; when you do, Hyperion 1Stop will find it present and not clone it again. By doing the S3FH build yourself, you relieve Hyperion 1Stop of configuration management responsibilities.

Any or all of this can be done...the question is what should be done. And your opinion here is valuable. What do you think?

Thanks for giving this a try....

Best Regards, Steve Orso

[jphartmann]

So you got this message with the variable substituted?

"Hyperion git configuration file '$gitconf' not found."

We should perhaps test for being a clone up front and tell people to do the old-fashioned build even if SoftFloat is installed and built.

Not including the 1Stop files in the zip will just have users complain that it is not there, so documentation is still needed, even if nobody reads it. Then we can at least point to it.

Note that including the git configuration file in the zip will not fly. The information we require is created by the client during git clone processing.

It is regrettable that a user who chooses to build from zip, might well be the one who needs the convenience of 1Stop the most.

Bob, also be warned that there is some controversy in the community about what Steve (and I to a lesser extent) do to the build process. Ivan is very much against it, but anyone who does pay attention to the licensing issue will be forced to conclude that we then have to write everything we import for IEEE float (both binary and decimal) up from scratch if it is being shipped along with stuff that is under the Q licence (clean room implementation as we said in IBM).

On the other hand, the user is free to install and link with a library she has built (at least that is how I interpret Harold's gospel; and anyone can do what they want in the comfort of their bedroom, at least while the blinds are down).

[jphartmann]

Forgot: Not all Linux distros ship with git installed, though this is now the trend; and I think I had to install it on FreeBSD. So that also needs testing and diagnostic; no doubt much more so on Windows.

The requirement for a very uplevel CMAKE is also an issue; I cannot build SoftFloat on the system I'm using at the moment.

[srorso]

It is becoming clear (to me anyway) that bldlvlck needs to be updated to reflect current requirements, that a corresponding Windows bldlvlck.cmd needs to be written, both need to set return codes, and 1Stop should call bldlvlck (resp. 1Stop.cmd and bldlvlck.cmd).

Bob: you have inspired much constructive thought...thanks!

[wably]

With regards to the original error, I have recreated it and provided more information below. The error occurs immediately after entering 1Stop:

  bob@pc1:~/hypx/hyperion$ ./1Stop
  Hyperion git configuration file '.git/config' not found.
  bob@pc1:~/hypx/hyperion$ _

In order to recreate this, these are the simple steps required (this was using Debian 8):

1. Download Hyperion via the download .zip option.
2. Make a test directory to hold this stuff: mkdir hypx
3. cd hypx
4. Unzip the download into this test directory.
5. The unzip action creates a subdirectory called "hyperion-master"; I renamed this to "hyperion".
6. cd hyperion
7. ./1Stop

With regards to all of the commentary above, I am not necessarily pushing for a solution that works with download .zip. I simply did not know I couldn't do that, and so it was the easier option for me at the time since I did not have a git client installed on Linux. But if it is not going to operate from the download zip option then we need to say so in the build documentation.

As for CMAKE, I also had issues with that (this is not a Hyperion problem necessarily). Aptitude on Linux only reported version 3.0.2 of Cmake available. After a bunch of web searches I found that I needed to add another ftp: link to the Aptitude sources.list file in order to get it to find later versions of Cmake. Then it found 3.7 of Cmake and I was past that problem.

From a new user perspective under Linux (and I am not picking on Linux it is just that it was a relatively fresh install of Debian), I had to load lots of other packages as well prior to building Hercules. I needed zlib for example, as well as many of the other packages that bldlvlck indicates. When you do an apt-cache search for zlib you get dozens and dozens of hits. Which one do I need? That was a bit of a challenge to find it, and I never had certainty I had the right one (until Hercules told me later I had zlib support). I'm not picking on zlib either; many other packages were confusing in this regard. Who would have thought I needed zlib1g-dev versus all of the other zlibxx stuff that was presented as a choice?

I will also say that I really didn't want to bother with the hubbub of installing the SoftFloat stuff. So it was very nice that 1Stop did all that for me and without further issues! (This of course was after I resolved the .git/config issue.)

These are the challenges that new users face upon an install. I wish that our doc could name the specific packages that we need to get from aptitude, by their exact name. But I also realize that this probably isn't possible because there are so many platforms and versions and it would be a formidable effort to keep it all straight.

I do agree that bldlvlck should be called by 1Stop. Also, bldlvlck should report on Cmake, which it does not do currently.

As for my opinion on what should be done, I do not have an easy answer. But let's look at it from the point of view of what I wanted to accomplish. I was interested in running Hercules on Linux so I could quickly QA changes in that environment that I had made to ECPS. I didn't want a long ordeal; I just needed 30 minutes or so to run a suite of tests. Instead what I experienced was two somewhat frustrating days of install this package and get that package along with their uncertainties. But eventually I got there. And now that I have all that stuff installed and set up, I'm good to go and I can rebuild Hercules very quickly on Linux now. My point is this: for a typical end user who just found out about Hercules and wants to try out some mainframe software himself on his own PC or whatever, this is an ordeal. I wonder how many give up in frustration? I don't readily have an answer: the packages are required. But getting there is painful.

Regards, Bob

[srorso]

Perhaps we can add/update README.distroXXX files to include the distro software repository-specific names for the required packages...For example: README.DistroDebian.txt.

[jphartmann]

By all means include the ones you know. You have to be specific as to release and package name.

I'd suggest a single tabular file.