Closed TommyMurphyTM1234 closed 6 years ago
the build process on a clean Ubuntu 16.04 64 bit installation. ... the script failed
that's bad. the script has (had?) some tests at the beginning to check the prerequisites, and should not start if there is something missing.
could you list the executables that failed, to extend the initial tests?
installing things on a trial and error basis is probably OK?
in my opinion it is not ok, the script should complain from the beginning.
DEPLOY_FOLDER_NAME
that's bad too, it means something was wrong in the first part of the script, where variables are set.
while the openocd script installs and uses them on the host
good point. I thought most of the interesting actions happen inside the container, where things should already be present.
could you remove those packages from the ubuntu host, retry the script, and take a closer look where things break? if the actions are ok to happen on the host, then the executables need to be tested at the beginning. if the actions are executed on the host by mistake, the script needs to be fixed.
Yes - I will do this later on and post feedback on it.
Apologies for the delay on this but I only got to look at it again now. I double checked with a clean (VM) install of Ubuntu 16.04 64 bit desktop, nothing additional installed other than available updates. I found that the script would not run successfully until the following were installed on the host:
Perhaps it is a mistake that the script looks for these on the host rather than in the docker container(s)?
Also as mentioned above at the end of the build process I got this again:
Changing owner to non-root Linux user...
/Host/Work/openocd/scripts/build-helper.sh: line 743: DEPLOY_FOLDER_NAME: unbound variable
P.S. one other small thing is that the website instructions refer to the working folder as "work" when, in fact, it is "Work" (e.g. ~/Work).
More detailed info if needed:
I started with a clean (VM) install of Ubuntu 16.04 64 bit desktop mode. I didn't add anything other than installing all available updates.
I followed these instructions for installing/configuring docker:
https://gnu-mcu-eclipse.github.io/developer/build-binaries-prerequisites/
The RISC-V tools build instructions link to this page and also to the docker installation instructions but the OpenOCD build instructions only link to the latter - the former are simpler to follow so maybe the OpenOCD page could also link to them?
I then followed the instructions here for building OpenOCD. While the page does mention some required packages it also says that the script should check for and install them if necessary so I didn't manually install any packages at this stage. I simply ran:
bash ~/Downloads/openocd-build.git/scripts/build.sh --win32
This failed because automake was not installed (note that the RISC-V build script did not fail on this):
user@ubuntu-16:~$ bash ~/Downloads/openocd-build.git/scripts/build.sh --win32
Work folder: "/home/user/Work/openocd".
Helper script: "/home/user/Downloads/openocd-build.git/scripts/helper/build-helper.sh".
Script "/home/user/Downloads/openocd-build.git/scripts/build.sh" started at Fri Sep 1 18:15:14 IST 2017.
Running on Ubuntu 64-bits.
Checking host curl...
curl 7.47.0 (x86_64-pc-linux-gnu) libcurl/7.47.0 GnuTLS/3.4.10 zlib/1.2.8 libidn/1.32 librtmp/2.3
Checking host git...
git version 2.7.4
Checking Docker...
Docker version 17.06.1-ce, build 874a737
Checking host automake...
So I installed automake:
sudo apt-get install automake
and ran the script again but this time it failed on makeinfo:
user@ubuntu-16:~$ bash ~/Downloads/openocd-build.git/scripts/build.sh --win32
Work folder: "/home/user/Work/openocd".
Helper script: "/home/user/Downloads/openocd-build.git/scripts/helper/build-helper.sh".
Script "/home/user/Downloads/openocd-build.git/scripts/build.sh" started at Fri Sep 1 18:15:47 IST 2017.
Running on Ubuntu 64-bits.
Checking host curl...
curl 7.47.0 (x86_64-pc-linux-gnu) libcurl/7.47.0 GnuTLS/3.4.10 zlib/1.2.8 libidn/1.32 librtmp/2.3
Checking host git...
git version 2.7.4
Checking Docker...
Docker version 17.06.1-ce, build 874a737
Checking host automake...
automake (GNU automake) 1.15
Checking host patch...
GNU patch 2.7.5
Checking host tar...
tar (GNU tar) 1.28
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by John Gilmore and Jay Fenlason.
Checking host unzip...
UnZip 6.00 of 20 April 2009, by Debian. Original by Info-ZIP.
Checking host makeinfo...
/home/user/Downloads/openocd-build.git/scripts/build.sh: line 444: makeinfo: command not found
So I installed texinfo which provides makeinfo:
sudo apt-get install texinfo
I ran the script again but this time it failed on libtool:
user@ubuntu-16:~$ bash ~/Downloads/openocd-build.git/scripts/build.sh --win32
Work folder: "/home/user/Work/openocd".
Helper script: "/home/user/Downloads/openocd-build.git/scripts/helper/build-helper.sh".
Script "/home/user/Downloads/openocd-build.git/scripts/build.sh" started at Fri Sep 1 18:16:41 IST 2017.
Running on Ubuntu 64-bits.
Checking host curl...
curl 7.47.0 (x86_64-pc-linux-gnu) libcurl/7.47.0 GnuTLS/3.4.10 zlib/1.2.8 libidn/1.32 librtmp/2.3
Checking host git...
git version 2.7.4
Checking Docker...
Docker version 17.06.1-ce, build 874a737
Checking host automake...
automake (GNU automake) 1.15
Checking host patch...
GNU patch 2.7.5
Checking host tar...
tar (GNU tar) 1.28
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by John Gilmore and Jay Fenlason.
Checking host unzip...
UnZip 6.00 of 20 April 2009, by Debian. Original by Info-ZIP.
Checking host makeinfo...
texi2any (GNU texinfo) 6.1
/home/user/Downloads/openocd-build.git/scripts/build.sh: Error: libtool is required
So I installed libtool
sudo apt-get install libtool
and ran the script again - this time it worked.
Hope this helps.
thank you for your detailed message. I'll take a look and try to fix this, but right now I'm very busy with the future project template.
OK - thanks. Just to clarify is it a mistake that the script is trying to find these tools on the host rather than in the docker container? If so and I can spot the difference between the RISC-V GCC build script and the OpenOCD build script I'll see if I can do anything to address this issue (and the unbound variable issue).
Just to clarify is it a mistake that the script is trying to find these tools on the host rather than in the docker container?
I guess the problem is a wrong assumption that the script runs on macOS. in this case the script should check the prerequisites on the host, since there are no macOS Docker containers.
probably a pair of if/fi to test the host will solve the problem.
I guess the problem is a wrong assumption that the script runs on macOS. in this case the script should check the prerequisites on the host, since there are no macOS Docker containers.
probably a pair of if/fi to test the host will solve the problem.
Yes - that seems to be the problem alright - I commented out the calls to/checks for automake, makeinfo and libtool and did a clean build and it worked fine on Ubuntu. So I guess that these calls/checks are macOS specific. I thought that comparing these to the RISC-V GCC tools build script would make it obvious how to wrap them in a host test but that script doesn't have such a test and yet works fine. I'm also not clear on how to fix the unbound variable issue. So I will leave it to you to make the necessary changes whenever you get time. Thanks for the feedback on this.
I updated the build script and the helper, be sure you update the git and the submodule.
I also updated the build page, which refers to the global Build Prerequisites page.
the page defines the packages required on Ubuntu. it would be great to try again with a fresh Ubuntu with those packages.
I also added the missing DEPLOY_FOLDER_NAME
.
please remove the openocd build folder and try again.
one small note: the checkout-stable and checkout-dev are not implemented, you need to manually checkout the desired branch.
is this issue still actual?
Hi Liviu
There still seem to be problems as far as I can see.
I started with a clean Ubuntu 16.04 64 bit VM installation with the latest VirtualBox guest extensions and all available Ubuntu updates installed - nothing else manually installed.
I pulled the latest (I presume?) script as follows:
git clone --recurse-submodules https://github.com/gnu-mcu-eclipse/openocd-build.git ~/Downloads/openocd-build.git
I kicked off a build of the win32 and deb32 targets but only got this far before the script terminated:
user@ubuntu-16:~$ bash ~/Downloads/openocd-build.git/scripts/build.sh --win32 --deb32
Work folder: "/home/user/Work/openocd".
Helper script: "/home/user/Downloads/openocd-build.git/scripts/helper/build-helper.sh".
Script "/home/user/Downloads/openocd-build.git/scripts/build.sh" started at Tue Sep 26 10:21:30 IST 2017.
Running on Ubuntu 64-bits.
Checking host curl...
curl 7.47.0 (x86_64-pc-linux-gnu) libcurl/7.47.0 GnuTLS/3.4.10 zlib/1.2.8 libidn/1.32 librtmp/2.3
Checking host git...
git version 2.7.4
Checking Docker...
Docker version 17.06.2-ce, build cec0b72
Checking host automake...
I installed automake on the Ubuntu host and kicked the build off again but I got this (not sure if this is significant?):
Copying openocd-build.git from Downloads...
fatal: Not a git repository (or any of the parent directories): .git
and then it failed with this:
bootstrap...
./bootstrap: Error: libtool is required
I installed libtool and tried again but again it failed with an error that seems to relate to the docker container (/Host/Work/...) so I'm not really sure how to work around that.
Running OpenOCD configure...
bash: /Host/Work/openocd/openocd.git/configure: No such file or directory
I don't think that I am doing anything wrong but if I am please let me know. Thanks.
Hope this helps Regards Tommy
Looking at the contents of ~/Work/openocd/openocd.git the problem looks to be that there is no configure script because bootstrap has not been run first.
I'll take a look and fix it during the next few days. (I had a similar problem with the gcc script, the changes are small, I need to identify them and apply to this script too).
I might also change the configuration to use git submodules instead of downloding the helper, at least this keeps the scripts in sync.
bootstrap has not been run first
bootstrap was somewhere in the script, probably something went wrong before trying to run it.
bootstrap was somewhere in the script, probably something went wrong before trying to run it.
build.sh does not call bootstrap for openocd. I modified ~/Downloads/openocd.git/scripts/build.sh to add this:
if [ "${target_name}" == "win" ]
then
cd "${build_folder_path}/openocd"
bash "${work_folder_path}/${OPENOCD_FOLDER_NAME}/bootstrap"
but then it fails with this:
Running OpenOCD configure...
+ aclocal
aclocal: error: 'configure.ac' is required
I'm not sure why because ~/Work/openocd/openocd.git/configure.ac does exist.
build.sh does not call bootstrap for openocd
it looks like I messed the script recently, I remember before it was there.
Actually I may be wrong. The script does seem to call openocd's bootstrap but only if build.sh bootstrap is called. Am I supposed to build as follows?
build.sh bootstrap
build.sh --win32
I tried that but still get the error with configure.ac.
I'm a bit busy now, I'll try to fix it tomorrow afternoon.
OK - thanks.
Is the "if false ..." snippet correct here or could this be relevant to the current problems?
# ----- Process "pull|checkout-dev|checkout-stable" actions. -----
do_repo_action() {
# $1 = action (pull, checkout-dev, checkout-stable)
# Update current branch and prepare autotools.
echo
if [ "${ACTION}" == "pull" ]
then
echo "Running git pull..."
elif [ "${ACTION}" == "checkout-dev" ]
then
echo "Running git checkout gnu-mcu-eclipse-dev & pull..."
elif [ "${ACTION}" == "checkout-stable" ]
then
echo "Running git checkout gnu-mcu-eclipse & pull..."
fi
if [ -d "${PROJECT_GIT_FOLDER_PATH}" ]
then
echo
if [ "${USER}" == "ilg" ]
then
echo "If asked, enter ${USER} GitHub password for git pull"
fi
cd "${PROJECT_GIT_FOLDER_PATH}"
if [ "${ACTION}" == "checkout-dev" ]
then
git checkout gnu-mcu-eclipse-dev
elif [ "${ACTION}" == "checkout-stable" ]
then
git checkout gnu-mcu-eclipse
fi
if false
then
git pull --recurse-submodules
git submodule update --init --recursive --remote
git branch
do_host_bootstrap
rm -rf "${BUILD_FOLDER_PATH}/${APP_LC_NAME}"
echo
if [ "${ACTION}" == "pull" ]
then
echo "Pull completed. Proceed with a regular build."
else
echo "Checkout completed. Proceed with a regular build."
fi
else
echo "Not implemented."
exit 1
fi
exit 0
else
echo "No git folder."
exit 1
fi
}
FWIW I tried build.sh checkout-dev and got this:
user@ubuntu-16:~$ bash ~/Downloads/openocd-build.git/scripts/build.sh checkout-dev
Work folder: "/home/user/Work/openocd".
Helper script: "/home/user/Downloads/openocd-build.git/scripts/helper/build-helper.sh".
Script "/home/user/Downloads/openocd-build.git/scripts/build.sh" started at Tue Sep 26 11:31:21 IST 2017.
Running on Ubuntu 64-bits.
Checking host curl...
curl 7.47.0 (x86_64-pc-linux-gnu) libcurl/7.47.0 GnuTLS/3.4.10 zlib/1.2.8 libidn/1.32 librtmp/2.3
Checking host git...
git version 2.7.4
Checking host automake...
automake (GNU automake) 1.15
Checking host patch...
GNU patch 2.7.5
Checking host tar...
tar (GNU tar) 1.28
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by John Gilmore and Jay Fenlason.
Checking host unzip...
UnZip 6.00 of 20 April 2009, by Debian. Original by Info-ZIP.
Copying openocd-build.git from Downloads...
Running git checkout gnu-mcu-eclipse-dev & pull...
fatal: Not a git repository (or any of the parent directories): .git
I don't know if this is a different problem or related to the original build problem(s) above.
fatal: Not a git repository (or any of the parent directories): .git
this is a tricky issue that I'm not very happy with. the problem is that for convenience the initial *-build.git
is cloned in Downloads
, but later on its content is needed in the Work
folder, which might be on a different file system; so I copy it, but doing so is not that easy, since it includes hidden folders. in the gcc build script I fixed this, we'll do the same here.
the 'Not a git repository (or any of the parent directories): .git' problem was solved:
https://github.com/gnu-mcu-eclipse/openocd-build/commit/8b8e0c1085aff1a7df75e8a003e2a60279e47834
bootstrap
the bootstrap script is always executed right after cloning the openocd git, so it should not be required to do it manually. but if bootstrap fails, you need to run it manually before restarting the script. or you can remove everything and start from scratch.
the bootstrap script requires automake, and this is the reason automake is checked to be present on the host. the Prerequisites page also asks for automake.
I suggest you retry with a clean Ubuntu, install the packages listed in the Prerequisites page, and run the script.
if the script still fails, there must be something else missing, and we need to identify the packages, and add them to the Prerequisites page.
I managed to install a new Ubuntu 16 VM and you were right, libtool
was also required; I updated the Prerequisites page.
in my environment the build script was successful for all 4 platforms.
Thanks Liviu. But I started with a clean Ubuntu 16.04 64 bit VM, did all the prerequisite stuff and then ran
bash ~/Downloads/openocd-build.git/scripts/build.sh --win32
but it failed with this:
Checking host unzip...
UnZip 6.00 of 20 April 2009, by Debian. Original by Info-ZIP.
Copying openocd-build.git from Downloads...
Cloning into '/home/user/Work/openocd/openocd-build.git'...
done.
bzip2: (stdin) is not a bzip2 file.
tar: Child returned status 2
tar: Error is not recoverable: exiting now
Just in case I did
rm -rf ~/Work
and tried again but got the same error.
this looks like a damaged file. define DEBUG=-x before starting the script and you should be able to see all script lines.
Yes - the contents of the libusb-1.0.20.tar.bz2 file are HTML including
We're sorry -- the Sourceforge site is currently in Disaster Recovery mode, and currently requires
the use of javascript to function. Please check back later.
so it's a problem with SourceForge. I'll try again later or tomorrow morning.
hmmm... SourceForge is less and less reliable :-(
probably I should re-post those files on GitHub and update the script to get them from there.
SourceForge is less and less reliable
Is this a common occurrence? I can download the zips using a browser (presumably because it supports JavaScript as the message above says is required) but getting them with curl or wget results in a HTML file with that error message.
I can download the zips using a browser (
Oh now I can't and I get this in a browser:
The sourceforge.net website is temporarily in static offline mode. Only a very limited set of project pages are available until the main website returns to service.
I'll leave it until the morning to try again.
Thanks a lot for the work on this - much appreciated.
I already published the binaries on GitHub:
Thanks - I tried to hack the script to take them from there but I can't seem to figure out the magic URL that will download the raw file and I just keep getting HTML e.g.:
Downloading "libusb-1.0.20.tar.bz2"...
+ cd /home/user/Work/openocd/download
+ curl -L https://github.com/gnu-mcu-eclipse/files/blob/master/libs/libusb-1.0.20.tar.bz2 --output libusb-1.0.20.tar.bz2
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 32729 0 32729 0 0 32565 0 --:--:-- 0:00:01 --:--:-- 32598
+ '[' '!' -d /home/user/Work/openocd/libusb-1.0.20 ']'
+ cd /home/user/Work/openocd
+ tar -xjvf /home/user/Work/openocd/download/libusb-1.0.20.tar.bz2
bzip2: (stdin) is not a bzip2 file.
tar: Child returned status 2
tar: Error is not recoverable: exiting now
user@ubuntu-16:~$ head -20 libusb-1.0.20.tar.bz2
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<link rel="dns-prefetch" href="https://assets-cdn.github.com">
<link rel="dns-prefetch" href="https://avatars0.githubusercontent.com">
<link rel="dns-prefetch" href="https://avatars1.githubusercontent.com">
<link rel="dns-prefetch" href="https://avatars2.githubusercontent.com">
<link rel="dns-prefetch" href="https://avatars3.githubusercontent.com">
<link rel="dns-prefetch" href="https://github-cloud.s3.amazonaws.com">
<link rel="dns-prefetch" href="https://user-images.githubusercontent.com/">
I already updated the script to use the 'magic' urls:
https://github.com/gnu-mcu-eclipse/openocd-build/commit/5a4b4e919e35e8cbe06c41fe6abb72cc65e5116f
Thanks a lot. Works great now - win32 build worked fine for me and deb32 churning away now. Thanks! 👍
Edit: deb32 build completed successfully too. Feel free to close the issue if you see fit to do so. Thanks again.
you're welcome!
I'll probably update the other build script to use GitHub archives too.
I ran through the build process on a clean Ubuntu 16.04 64 bit installation. In case it helps to refine the installation instructions at all I found that the script failed a few times until I installed each of the following in turn:
sudo apt install automake sudo apt install texinfo sudo apt install makeinfo
Maybe it's too much trouble to enumerate all required packages for all possible build platforms in which case running the script and installing things on a trial and error basis is probably OK?
Curiously the GCC build script did not complain about these tools being missing and did not install them (otherwise they would still be available now for the openocd script) so I don't know if it uses and installs them in the docker container while the openocd script installs and uses them on the host or something?!
One other thing - at the very end of the build process I got this but I don't know if it's significant?
/Host/Work/openocd/scripts/build-helper.sh: line 743: DEPLOY_FOLDER_NAME: unbound variable