Lastest terminal script in macOS doesn't work

[ghost]

So, the terminal script in macOS VM doesn't work. You know just after pressing enter for the first time in the macOS terminal. The script does nothing and this caracther ">" appear on the next line. This is a least for the lastest commits on your GitHub repo. I didn't try it right now on the lastest release.

[myspaghetti]

Hi, thanks for the bug report. Can you provide a screenshot of the VM when the > character appears? Additionally attaching the output of ./macos-gueest-virtualbox.sh troubleshoot > log.txt could be helpful.

[SamChatfield]

The terminal script is also not working for me in macOS but for a seemingly different reason.

I am having to run the script like so to use homebrew bash rather than my default shell of zsh:

/usr/local/Cellar/bash/5.0.16/bin/bash macos-guest-virtualbox.sh

I then receive the following output:

                  Push-button installer of macOS on VirtualBox
-------------------------------------------------------------------------------

This script installs only open-source software and unmodified Apple binaries.

The script checks for dependencies and will prompt to install them if unmet.

For iCloud and iMessage connectivity, the script needs to be edited with genuine
or genuine-like Apple parameters. macOS will work without these parameters, but
Apple-connected apps will not.

The installation requires about 40GB of available storage, 20GB for
temporary installation files and 20GB for the virtual machine's dynamically
allocated storage disk image.

Documentation about optional configuration, resuming the script by stages, and
other topics can be viewed with the following command:

  macos-guest-virtualbox.sh documentation | less -R

Press enter to review the script configuration.

vm_name="macOS"                  # name of the VirtualBox virtual machine
macOS_release_name="Catalina"    # install "HighSierra" "Mojave" or "Catalina"
storage_size=80000               # VM disk image size in MB. minimum 22000
cpu_count=2                      # VM CPU cores, minimum 2
memory_size=4096                 # VM RAM in MB, minimum 2048
gpu_vram=128                     # VM video RAM in MB, minimum 34, maximum 128
resolution="1280x800"            # VM display resolution

These values may be customized as described in the documentation.

Press enter to continue, CTRL-C to exit.
Please make sure the following packages are installed:
coreutils    gzip    unzip    xxd    wget
Please make sure the coreutils package is the GNU variant.

It seems that this error message is only displayed if the following check is true:

if [[ -z "$(echo "xxd" | xxd -p 2>/dev/null)" || \
      -z "$(gzip --help 2>/dev/null)" || \
      -z "$(unzip -hh 2>/dev/null)" || \
      -z "$(csplit --help 2>/dev/null)" || \
      -z "$(wget --version 2>/dev/null)" ]]; then

It seems as though this is trying to use the wrong command in my case, where it calls csplit it should be calling gcsplit. So I tried to edit the script myself to call gcsplit instead but I still get the same error which is confusing. I have also tried running all of these commands (with csplit replaced by gcsplit) separately and checking the value from echo $? and each returns 0.

I was wondering if you had any thoughts on what might be going wrong here?

[myspaghetti]

Did you install the packages coreutils, gzip, unzip, xxd, and wget as requested by the error message?

[SamChatfield]

Yes I did, that's what's confusing about it. Especially since I tested all of the commands that part of the script uses to verify whether they were installed and I cannot see why that error message is being printed at all.

[myspaghetti]

Please start an interactive bash shell (/usr/local/Cellar/bash/5.0.16/bin/bash) and post the output of the following commands:

echo "xxd" | xxd -p
gzip --help
unzip -hh
gcsplit --help
wget --version

If possible, please install the package which and post the output of the following commands:

which -a xxd
which -a gzip
which -a unzip
which -a gcsplit
which -a wget

[SamChatfield]

1)

bash-5.0$ echo "xxd" | xxd -p
7878640a

bash-5.0$ gzip --help
Apple gzip 287.100.2
usage: gzip [-123456789acdfhklLNnqrtVv] [-S .suffix] [<file> [<file> ...]]
 -1 --fast            fastest (worst) compression
 -2 .. -8             set compression level
 -9 --best            best (slowest) compression
 -c --stdout          write to stdout, keep original files
    --to-stdout
 -d --decompress      uncompress files
    --uncompress
 -f --force           force overwriting & compress links
 -h --help            display this help
 -k --keep            don't delete input files during operation
 -l --list            list compressed file contents
 -N --name            save or restore original file name and time stamp
 -n --no-name         don't save original file name or time stamp
 -q --quiet           output no warnings
 -r --recursive       recursively compress files in directories
 -S .suf              use suffix .suf instead of .gz
    --suffix .suf
 -t --test            test compressed file
 -V --version         display program version
 -v --verbose         print extra statistics

bash-5.0$ unzip -hh

Extended Help for UnZip

See the UnZip Manual for more detailed help

UnZip lists and extracts files in zip archives.  The default action is to
extract zipfile entries to the current directory, creating directories as
needed.  With appropriate options, UnZip lists the contents of archives
instead.

Basic unzip command line:
  unzip [-Z] options archive[.zip] [file ...] [-x xfile ...] [-d exdir]

Some examples:
  unzip -l foo.zip        - list files in short format in archive foo.zip

  unzip -t foo            - test the files in archive foo

  unzip -Z foo            - list files using more detailed zipinfo format

  unzip foo               - unzip the contents of foo in current dir

  unzip -a foo            - unzip foo and convert text files to local OS

If unzip is run in zipinfo mode, a more detailed list of archive contents
is provided.  The -Z option sets zipinfo mode and changes the available
options.

Basic zipinfo command line:
  zipinfo options archive[.zip] [file ...] [-x xfile ...]
  unzip -Z options archive[.zip] [file ...] [-x xfile ...]

Below, Mac OS refers to Mac OS before Mac OS X.  Mac OS X is a Unix based
port and is referred to as Unix Apple.

unzip options:
  -Z   Switch to zipinfo mode.  Must be first option.
  -hh  Display extended help.
  -A   [OS/2, Unix DLL] Print extended help for DLL.
  -c   Extract files to stdout/screen.  As -p but include names.  Also,
         -a allowed and EBCDIC conversions done if needed.
  -f   Freshen by extracting only if older file on disk.
  -l   List files using short form.
  -p   Extract files to pipe (stdout).  Only file data is output and all
         files extracted in binary mode (as stored).
  -t   Test archive files.
  -T   Set timestamp on archive(s) to that of newest file.  Similar to
       zip -o but faster.
  -u   Update existing older files on disk as -f and extract new files.
  -v   Use verbose list format.  If given alone as unzip -v show version
         information.  Also can be added to other list commands for more
         verbose output.
  -z   Display only archive comment.

unzip modifiers:
  -a   Convert text files to local OS format.  Convert line ends, EOF
         marker, and from or to EBCDIC character set as needed.
  -b   Treat all files as binary.  [Tandem] Force filecode 180 ('C').
         [VMS] Autoconvert binary files.  -bb forces convert of all files.
  -B   [UNIXBACKUP compile option enabled] Save a backup copy of each
         overwritten file in foo~ or foo~99999 format.
  -C   Use case-insensitive matching.
  -D   Skip restoration of timestamps for extracted directories.  On VMS this
         is on by default and -D essentially becames -DD.
  -DD  Skip restoration of timestamps for all entries.
  -E   [MacOS (not Unix Apple)]  Display contents of MacOS extra field during
         restore.
  -F   [Acorn] Suppress removal of NFS filetype extension.  [Non-Acorn if
         ACORN_FTYPE_NFS] Translate filetype and append to name.
  -i   [MacOS] Ignore filenames in MacOS extra field.  Instead, use name in
         standard header.
  -j   Junk paths and deposit all files in extraction directory.
  -J   [BeOS] Junk file attributes.  [MacOS] Ignore MacOS specific info.
  -K   [AtheOS, BeOS, Unix] Restore SUID/SGID/Tacky file attributes.
  -L   Convert to lowercase any names from uppercase only file system.
  -LL  Convert all files to lowercase.
  -M   Pipe all output through internal pager similar to Unix more(1).
  -n   Never overwrite existing files.  Skip extracting that file, no prompt.
  -N   [Amiga] Extract file comments as Amiga filenotes.
  -o   Overwrite existing files without prompting.  Useful with -f.  Use with
         care.
  -P p Use password p to decrypt files.  THIS IS INSECURE!  Some OS show
         command line to other users.
  -q   Perform operations quietly.  The more q (as in -qq) the quieter.
  -s   [OS/2, NT, MS-DOS] Convert spaces in filenames to underscores.
  -S   [VMS] Convert text files (-a, -aa) into Stream_LF format.
  -U   [UNICODE enabled] Show non-local characters as #Uxxxx or #Lxxxxxx ASCII
         text escapes where x is hex digit.  [Old] -U used to leave names
         uppercase if created on MS-DOS, VMS, etc.  See -L.
  -UU  [UNICODE enabled] Disable use of stored UTF-8 paths.  Note that UTF-8
         paths stored as native local paths are still processed as Unicode.
  -V   Retain VMS file version numbers.
  -W   [Only if WILD_STOP_AT_DIR] Modify pattern matching so ? and * do not
         match directory separator /, but ** does.  Allows matching at specific
         directory levels.
  -X   [VMS, Unix, OS/2, NT, Tandem] Restore UICs and ACL entries under VMS,
         or UIDs/GIDs under Unix, or ACLs under certain network-enabled
         versions of OS/2, or security ACLs under Windows NT.  Can require
         user privileges.
  -XX  [NT] Extract NT security ACLs after trying to enable additional
         system privileges.
  -Y   [VMS] Treat archived name endings of .nnn as VMS version numbers.
  -$   [MS-DOS, OS/2, NT] Restore volume label if extraction medium is
         removable.  -$$ allows fixed media (hard drives) to be labeled.
  -/ e [Acorn] Use e as extension list.
  -:   [All but Acorn, VM/CMS, MVS, Tandem] Allow extract archive members into
         locations outside of current extraction root folder.  This allows
         paths such as ../foo to be extracted above the current extraction
         directory, which can be a security problem.
  -^   [Unix] Allow control characters in names of extracted entries.  Usually
         this is not a good thing and should be avoided.
  -2   [VMS] Force unconditional conversion of names to ODS-compatible names.
         Default is to exploit destination file system, preserving cases and
         extended name characters on ODS5 and applying ODS2 filtering on ODS2.

Wildcards:
  Internally unzip supports the following wildcards:
    ?       (or %% or #, depending on OS) matches any single character
    *       matches any number of characters, including zero
    [list]  matches char in list (regex), can do range [ac-f], all but [!bf]
  If port supports [], must escape [ as [[]
  For shells that expand wildcards, escape (\* or "*") so unzip can recurse.

Include and Exclude:
  -i pattern pattern ...   include files that match a pattern
  -x pattern pattern ...   exclude files that match a pattern
  Patterns are paths with optional wildcards and match paths as stored in
  archive.  Exclude and include lists end at next option or end of line.
    unzip archive -x pattern pattern ...

Multi-part (split) archives (archives created as a set of split files):
  Currently split archives are not readable by unzip.  A workaround is
  to use zip to convert the split archive to a single-file archive and
  use unzip on that.  See the manual page for Zip 3.0 or later.

Streaming (piping into unzip):
  Currently unzip does not support streaming.  The funzip utility can be
  used to process the first entry in a stream.
    cat archive | funzip

Testing archives:
  -t        test contents of archive
  This can be modified using -q for quieter operation, and -qq for even
  quieter operation.

Unicode:
  If compiled with Unicode support, unzip automatically handles archives
  with Unicode entries.  Currently Unicode on Win32 systems is limited.
  Characters not in the current character set are shown as ASCII escapes
  in the form #Uxxxx where the Unicode character number fits in 16 bits,
  or #Lxxxxxx where it doesn't, where x is the ASCII character for a hex
  digit.

zipinfo options (these are used in zipinfo mode (unzip -Z ...)):
  -1  List names only, one per line.  No headers/trailers.  Good for scripts.
  -2  List names only as -1, but include headers, trailers, and comments.
  -s  List archive entries in short Unix ls -l format.  Default list format.
  -m  List in long Unix ls -l format.  As -s, but includes compression %.
  -l  List in long Unix ls -l format.  As -m, but compression in bytes.
  -v  List zipfile information in verbose, multi-page format.
  -h  List header line.  Includes archive name, actual size, total files.
  -M  Pipe all output through internal pager similar to Unix more(1) command.
  -t  List totals for files listed or for all files.  Includes uncompressed
        and compressed sizes, and compression factors.
  -T  Print file dates and times in a sortable decimal format (yymmdd.hhmmss)
        Default date and time format is a more human-readable version.
  -U  [UNICODE] If entry has a UTF-8 Unicode path, display any characters
        not in current character set as text #Uxxxx and #Lxxxxxx escapes
        representing the Unicode character number of the character in hex.
  -UU [UNICODE]  Disable use of any UTF-8 path information.
  -z  Include archive comment if any in listing.

funzip stream extractor:
  funzip extracts the first member in an archive to stdout.  Typically
  used to unzip the first member of a stream or pipe.  If a file argument
  is given, read from that file instead of stdin.

funzip command line:
  funzip [-password] [input[.zip|.gz]]

unzipsfx self extractor:
  Self-extracting archives made with unzipsfx are no more (or less)
  portable across different operating systems than unzip executables.
  In general, a self-extracting archive made on a particular Unix system,
  for example, will only self-extract under the same flavor of Unix.
  Regular unzip may still be used to extract embedded archive however.

unzipsfx command line:
  <unzipsfx+archive_filename>  [-options] [file(s) ... [-x xfile(s) ...]]

unzipsfx options:
  -c, -p - Output to pipe.  (See above for unzip.)
  -f, -u - Freshen and Update, as for unzip.
  -t     - Test embedded archive.  (Can be used to list contents.)
  -z     - Print archive comment.  (See unzip above.)

unzipsfx modifiers:
  Most unzip modifiers are supported.  These include
  -a     - Convert text files.
  -n     - Never overwrite.
  -o     - Overwrite without prompting.
  -q     - Quiet operation.
  -C     - Match names case-insensitively.
  -j     - Junk paths.
  -V     - Keep version numbers.
  -s     - Convert spaces to underscores.
  -$     - Restore volume label.

If unzipsfx compiled with SFX_EXDIR defined, -d option also available:
  -d exd - Extract to directory exd.
By default, all files extracted to current directory.  This option
forces extraction to specified directory.

See unzipsfx manual page for more information.

bash-5.0$ gcsplit --help
Usage: gcsplit [OPTION]... FILE PATTERN...
Output pieces of FILE separated by PATTERN(s) to files 'xx00', 'xx01', ...,
and output byte counts of each piece to standard output.

Read standard input if FILE is -

Mandatory arguments to long options are mandatory for short options too.
  -b, --suffix-format=FORMAT  use sprintf FORMAT instead of %02d
  -f, --prefix=PREFIX        use PREFIX instead of 'xx'
  -k, --keep-files           do not remove output files on errors
      --suppress-matched     suppress the lines matching PATTERN
  -n, --digits=DIGITS        use specified number of digits instead of 2
  -s, --quiet, --silent      do not print counts of output file sizes
  -z, --elide-empty-files    remove empty output files
      --help     display this help and exit
      --version  output version information and exit

Each PATTERN may be:
  INTEGER            copy up to but not including specified line number
  /REGEXP/[OFFSET]   copy up to but not including a matching line
  %REGEXP%[OFFSET]   skip to, but not including a matching line
  {INTEGER}          repeat the previous pattern specified number of times
  {*}                repeat the previous pattern as many times as possible

A line OFFSET is a required '+' or '-' followed by a positive integer.

GNU coreutils online help: <https://www.gnu.org/software/coreutils/>
Full documentation <https://www.gnu.org/software/coreutils/csplit>
or available locally via: info '(coreutils) csplit invocation'

bash-5.0$ wget --version
GNU Wget 1.20.3 built on darwin19.0.0.

-cares +digest -gpgme +https +ipv6 +iri +large-file -metalink +nls
+ntlm +opie -psl +ssl/openssl

Wgetrc:
    /usr/local/etc/wgetrc (system)
Locale:
    /usr/local/Cellar/wget/1.20.3_2/share/locale
Compile:
    clang -DHAVE_CONFIG_H -DSYSTEM_WGETRC="/usr/local/etc/wgetrc"
    -DLOCALEDIR="/usr/local/Cellar/wget/1.20.3_2/share/locale" -I.
    -I../lib -I../lib -I/usr/local/opt/openssl@1.1/include -DNDEBUG -g
    -O2
Link:
    clang -DNDEBUG -g -O2 -lidn2 -L/usr/local/opt/openssl@1.1/lib -lssl
    -lcrypto -ldl -lz ftp-opie.o openssl.o http-ntlm.o ../lib/libgnu.a
    -liconv -lintl -Wl,-framework -Wl,CoreFoundation -lunistring

Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://www.gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Originally written by Hrvoje Niksic <hniksic@xemacs.org>.
Please send bug reports and questions to <bug-wget@gnu.org>.
bash-5.0$

2)

bash-5.0$ which -a xxd
/usr/bin/xxd

bash-5.0$ which -a gzip
/usr/bin/gzip

bash-5.0$ which -a unzip
/usr/bin/unzip

bash-5.0$ which -a gcsplit
/usr/local/bin/gcsplit

bash-5.0$ which -a wget
/usr/local/bin/wget

[ghost]

The bug in the macOS script is just a ">" after the command is executed. So the command wont do something. It's not in the Linux termnial, it's only in the mac terminal (well I think). https://imgur.com/a/WZJAj3X

[myspaghetti]

That's an Ubuntu WSL terminal, and you're experiencing issues with bash that I won't be able to troubleshoot. It appears there's an issue with delimiters or quoting, as if a quote has been opened before the newline and left unterminated.

As for the macOS host, you are definitely using Apple gzip and unzip, you should be using FSF gzip and Info-ZIP unzip, and in general not using the Apple-supplied binaries, although xxd produces the expected output. gcsplit and wget are the correct versions. Check the homebrew documentation for installing the non-Apple versions of gzip, unzip and xxd.

[ghost]

This is your script in macOS that is not working, not in WSL. I used a WSL bash because I was certainly not to retry to reinstall macOS after a sucessfull install. Just try your lastest .SH by yourself and you'll see that this doesn't work at all.

[myspaghetti]

This is not the script, it's an unrelated bash issue.

The issue on your macOS configuration is that you are using Apple binaries. You need to use the binaries I specified in the post above.

[myspaghetti]

% which -a gzip
/usr/bin/gzip
% gzip --help 2>/dev/null
%

The above means you are using the wrong version of gzip. Install gzip and the rest of the required packages with your package manager and make sure their executables' path is in the bash PATH variable.

[SamChatfield]

Okay thanks, it seems to be working now. I got the impression from the README that any version of gzip, unzip, wget and xxd would be fine, rather than the versions from Homebrew being required specifically.

[ghost]

Ok, thanks. Maybe create a warning in the README about that. Or auto install them when creating dmg files.