Closed cstacy closed 1 year ago
I see this:
../configure: line 3967: syntax error near unexpected token `libusb,'
../configure: line 3967: ` PKG_CHECK_MODULES(libusb, libusb-1.0,'
I believe the fix might be to install the homebrew package pkg-config
.
Oh, this too: ./autogen.sh: line 22: aclocal: command not found
Another package needs installing: automake
Installing those two brews gets a lot farther!
But now it blows up on the shared memory. Attached are the details.
(There is also some minor issue with "touch".)
(THANK YOU!)
The touch thing is strange but harmless. The script applies timestamps to a hundred files or so. Why would it fail only with 198904020211.39?
The shared memory problem is serious. I don't have a Mac to test this. @eswenson1, do you have KLH10 running on a Mac?
I'm able to build KLH10 on my Mac, and I can run it with a previously created DB-ITS disk. I do always run kn10-ks-its as root (with sudo) because otherwise the networking stuff gives me errors such as this:
[dpchaos: Warning - cannot set high priority - Permission denied]
[dpchaos: ifc "" => chaos 5461]
[dpchaos: chaos 5401 => ip 127.0.0.1:42044]
[dpimp: Warning - cannot set high priority - Permission denied]
[dpimp: *** Must usually run as superuser; networking may fail! ***]
[dpimp: Warning - cannot lock memory]
[dpimp: pcap_geterr: (cannot open BPF device) /dev/bpf0: Permission denied - Operation not permitted]
sudo make EMULATOR... gets me the same results as before.
Hey, I thought it didn't use pcap on macOS. When I try to build the "simh" target, it fails because I don't have the pcap libraries. But when I build "pdp10-ja", it does build (and I think everything runs, too, but I didn't test networking). But both of those are simh, right?
I am on macOS Big Sur with the latest XCode. This is old hardware, though, circa 2014.
I ran ITS on a much much older version of OSX, but that was like 2009 and much may have changed.
I meant "pdp10-ka"; although for me it's "pdp10-ja"...
I don't use "sudo" to make KLH10 -- only to run it.
Sorry, there's not much I can do to help here.
Yes, both "simh" and "pdp10-ka" are SIMH, as well as "pdp10-kl". The difference is that the first is Bob Supnik's KS10, and the latter two are Rich Cornwell's KA10 and KL10.
@eswenson1, are you also running Big Sur, or something older? That might explain the difference.
@rmaldersoniii, do you have KLH10 running on macOS? And if so which version of macOS?
I found this: https://github.com/PDP-10/klh10/issues/47
https://forum.xojo.com/t/shared-memory-app-sandbox/46405
thought I’d just post this here; in the hopes that I can save others wasted time.
“shmget”, “shmat”, “shdt” & “shctl” use System V semaphores underneath, and therefore BLOCKED by the App Sandbox
The docs say that shared memory is possible, they provide a warning about “System V semaphores”, but don’t make it clear which functions I should or shouldn’t use for shared memory.
I'm running the latest released macOS -- Big Sur 11.4. And KLH10 builds fine for me, and runs fine as well.
Very odd. Can you send a list of Homebrew installed things and a build transcript?
Maybe I can spot the difference.
Very odd. Can you send a list of Homebrew installed things and a build transcript? Maybe I can spot the difference.
Here is a list of my homebrew installed packages:
➜ ~ brew list
==> Formulae
abcm2ps graphviz libzip pygobject
adns gsettings-desktop-schemas little-cms2 pygobject3
aom gsl llvm pygtk
apr gst-plugins-base lz4 pyqt
apr-util gstreamer lzo pyqt5-webkit
asciidoctor gtk+ m4 pyside
asciinema gtk+3 markdown pyspatialite
atk gts matplotlib python@2
autoconf guile maven python@3.8
autojump harfbuzz minikube python@3.9
automake hdf5 mpdecimal qca
bash hicolor-icon-theme mpfr qgis
bash-completion htop msodbcsql17 qgis-res
bdw-gc icu4c mssql-tools qhull
bison ilmbase mysql qjson
boost imagemagick ncurses qscintilla2
bzip2 iproute2mac netpbm qt
c-ares isl nettle qt5-webkit
cairo jansson nghttp2 qtkeychain
cgal jasper ninja qwt
cmake jemalloc nmap qwtpolar
conan jenv node rabbitmq
coreutils jpeg npth readline
coursier jpeg-turbo nspr rtmpdump
curl jq nss rubberband
cython json-c numpy ruby
desktop-file-utils jsonnet oniguruma sbcl
docbook krb5 openblas sbt
docbook-xsl kubernetes-cli opencore-amr scipy
dos2unix kustomize openexr sdl
dotty lame openjdk sdl2
duf lapack openjdk@11 sdl_image
easy-rsa leptonica openjpeg sfcgal
eigen libagg openssl@1.1 shared-mime-info
emacs-head@28 libass opus sip
enscript libassuan oracle-client-sdk six
erlang libbluray orc snappy
espeak libcroco osgeo-filegdb-api sops
exiv2 libde265 osgeo-gdal spatialindex
expat libepoxy osgeo-gdal-python speedtest-cli
ffmpeg libev osgeo-oracle-client-sdk speex
flac libevent osgeo-postgis sphinx-doc
flex libffi osgeo-pyqt-webkit sqlite
fontconfig libgcrypt osgeo-pyspatialite subversion
freetds libgeotiff osgeo-qgis-res svg2pdf
freetype libgpg-error osgeo-qt-webkit szip
freexl libheif osgeo-qtkeychain task
frei0r libiconv osgeo-six tcl-tk
fribidi libidn2 p11-kit telnet
fzf libksba p7zip terraform
gcc liblqr pandoc tesseract
gd libmpc pango texinfo
gdal2 libogg pcre theora
gdal2-python libomp pcre2 tldr
gdb libpng perl tmux
gdbm libpq pg_top unbound
gdk-pixbuf librsvg pgloader unixodbc
geos libsamplerate pinentry utf8proc
gettext libsndfile pixman watch
ghostscript libsoxr pkg-config webp
giflib libspatialite poppler wget
git libssh portaudio wxmac
git-extras libsvg postgis2 wxpython
git-flow libsvg-cairo postgresql x264
git-gui libtasn1 postgresql@11 x265
glib libtiff proj xmlto
glslang libtool prometheus xvid
gmp libunistring protobuf xz
gnu-getopt libusb pstree yarn
gnu-sed libvorbis pwgen you-get
gnupg libvpx py2cairo z
gnutls libxml2 py3cairo zlib
gobject-introspection libxmlsec1 pyenv
golangci-lint libxslt pyenv-virtualenv
graphite2 libyaml pyenv-virtualenvwrapper
==> Casks
adoptopenjdk8 emacs java11 mactex
➜ ~
I'm redoing a macOS build now because I appear to have overwritten my previous transcript. However, just starting the build showed the same issue with touch
:
build/stamp.sh build/timestamps.txt
touch: invalid date format ‘198904020211.39’
touch: invalid date format ‘198904020211.45’
touch: invalid date format ‘198904020211.46’
Above is a macOS build log -- not of the entire build of DB ITS, but the start, including the build of kn10-ks-its (klh10), and demonstrating the successful boot of ITS from klh10.
I looked in on the build progress from above, and the build (not of KLH10, but of DB) failed. It failed on a timeout running DUMP to load the ../../out/klh10/sources.tape
tape. This has happened to me in the past, and I've just increased the timeout in the TCL scripts. But I wanted to point it out here too. Perhaps we should conditionally increase this timeout for the macOS build -- unless anyone is able to successfully build it with the current settings. I seem to recall (back when I was regularly building on macOS, that I had patched the timeout in my local copy of the sources to get around this issue).
Interestingly, without changing the default timeout, I did manage to get past the load of the sources.tape. However, now I'm timing out here:
EMACS Editor, version 162 - type Help(^_H) for help.
M-X run Library$einit$? Generate
Compressing file DSK: EMACS1; DOC >
Compressing file DSK: EMACS1; USRCOM >
Compressing file DSK: EMACS1; ^RBASE >
Compressing file DSK: EMACS1; WRDLST >
Compressing file DSK: EMACS1; INDENT >
Compressing file DSK: EMACS1; SEARCH >
Compressing file DSK: EMACS1; FILES >
Compressing file DSK: EMACS1; SUPPRT >
Compressing file DSK: EMACS1; ISEARC >
Compressing file DSK: EMACS1; WINDOW >
Compressing file DSK: EMACS1; BUFFER >
Compressing file DSK: EMACS1; CRL >
Compressing file DSK: EMACS1; VARS >
-> DSK: EMACS; [PURE] >
Compressing file DSK: EMACS1; PURIFY >
The last command timed out.
make: *** [out/klh10/rp0.dsk] Error 1
I seem to recall we had this issue before, but it's been so long since I've built on macOS that I don't remember the cause.
I can see that the expect script was waiting for:
expect -timeout 1000 -exact { -> DSK: EMACS; [PRFY]}
expect -timeout 1000 -exact { -> DSK: EMACS; EINIT}
neither of which ever showed up. Any ideas?
Tried it again, and this time got this:
M-X run Library$einit$? Generate
Compressing file DSK: EMACS1; DOC >
Compressing file DSK: EMACS1; USRCOM >
Compressing file DSK: EMACS1; ^RBASE >
Compressing file DSK: EMACS1; WRDLST >
Compressing file DSK: EMACS1; INDENT >
Compressing file DSK: EMACS1; SEARCH >
Compressing file DSK: EMACS1; FILES >
Compressing file DSK: EMACS1; SUPPRT >
Compressing file DSK: EMACS1; ISEARC >
Compressing file DSK: EMACS1; WINDOW >
Compressing file DSK: EMACS1; BUFFER >
Compressing file DSK: EMACS1; CRL >
Compressing file DSK: EMACS1; VARS >
-> DSK: EMACS; [PURE] >
Compressing file DSK: EMACS1; PURIFY >
Compressing file DSK: EMACS1; CCL >
-> DSK: EMACS; [PRFY] >
Compressing file DSK: EMACS1; EINIT >
-> DSK: EMACS; EINIT ^Q:EJ
M-X generate Library$emacs;aux$
The last command timed out.
make: *** [Makefile:139: out/klh10/rp0.dsk] Error 1
eswenson@swenson:~/ITS/ws/its$```
Notice, the first time, it appeared that TECO hung. We never got the output line confirming the compression of EMACS1; CCL >. And thus, never finished PURIFY, and thus never emitted "EMACS; [PRFY] >", which is what expect was looking for. Any ideas?
The previous time we had a problem in the EMACS part was due to file timestamps, so maybe check that. See #640. Given that touch
complains, maybe it doesn't get the timestamps right?
On 6/28/21 3:01 AM, Lars Brinkhoff wrote:
The previous time we had a problem in the EMACS part was due to file timestamps, so maybe check that. See #640 https://github.com/PDP-10/its/pull/640. Given that |touch| complains, maybe it doesn't get the timestamps right?
The touch problem is specific to those files.
Touching other (i.e. non-extant files) with those timestamps works.
Maybe it's something about the files (such as their date when they are created) that is causing the problem.
Last guess: command (transcript) is not what it appears to be.
Well, it appears to be the timestamps. The three errors I get under macOS are:
touch: invalid date format ‘198904020211.39’
touch -h -t "198904020211.45" -- "emacs1/window.77"
touch: invalid date format ‘198904020211.45’
touch -h -t "198904020211.46" -- "emacs1/wordab.624"
touch: invalid date format ‘198904020211.46’
Note that the three errors are with emacs sources, and this is why the compressing of emacs code fails to complete correctly.
These three files end up with current timestamps:
-rwxr-xr-x 1 eswenson CORP\Domain Users 1354 Jun 23 09:38 vt52.4*
-rwxr-xr-x 1 eswenson CORP\Domain Users 8266 Jun 23 09:38 window.77*
-rwxr-xr-x 1 eswenson CORP\Domain Users 71015 Jun 23 09:38 wordab.624*
The weird thing is that there are other files with similar timestamps that work:
-rwxr-xr-x 1 eswenson CORP\Domain Users 4414 Apr 2 1989 trmtyp.39*
-rwxr-xr-x 1 eswenson CORP\Domain Users 40017 Apr 5 1983 usrcom.514*
-rwxr-xr-x 1 eswenson CORP\Domain Users 4073 Apr 2 1989 vars.24*
-rwxr-xr-x 1 eswenson CORP\Domain Users 13741 Apr 2 1989 vt100.49*
The entries in timestamps.txt for those four files are:
emacs1/trmtyp.39 198904020154.03
emacs1/usrcom.514 198304051155.08
emacs1/vars.24 198904020154.20
emacs1/vt100.49 198904020154.21
I'm no longer using macOS touch, I'm using a brew-installed gnu touch:
/usr/local/opt/coreutils/libexec/gnubin/touch
This should preclude the issues being in macOS touch.
Very weird:
➜ its git:(master) ✗ touch -h -t 198904020104.03 src/emacs1/vt52.4
➜ its git:(master) ✗ touch -h -t 198904020204.03 src/emacs1/vt52.4
touch: invalid date format ‘198904020204.03’
➜ its git:(master) ✗
The first worked, the second didn't. The difference is in the value of HH. "01" worked and "02" didn't. Clearly something strange is involved. I'm not sure I have what it takes to debug touch on macOS. If it were linux, I'd have no issues, but I've had problems on macOS with gdb in the past, and I've have to make sure I found the right sources for touch to recompile with symbols. Anyone else have the wherewithal to debug this? If not, I may try to find some time to do it.
I see things differently than Chris. For me, the "bad" timestamp doesn't work on any files, existing or non-existing:
➜ its git:(master) ✗ touch ~/tmp/foo
➜ its git:(master) ✗ touch -h -t 198904020204.03 src/emacs1/vt52.4
touch: invalid date format ‘198904020204.03’
➜ its git:(master) ✗ touch -h -t 198904020204.03 ~/tmp/foo
touch: invalid date format ‘198904020204.03’
➜ its git:(master) ✗ touch -h -t 198904020204.03 ~/tmp/ffjfjf
touch: invalid date format ‘198904020204.03’
➜ its git:(master) ✗
That last file didn't exist.
OH! DST? Looks like April 2, 1989 was "leap forward" day in the US, and 02:04 didn't happen.
I think I blew it on some of my earlier tests.
Turns out, the BSD version of touch works; GNU touch breaks!
========================================= build/stamp.sh build/timestamps.txt touch: invalid date format ‘198904020211.39’
% rm -f vt52.4 % /usr/local/gnubin/touch -t 198904020211.39 vt52.4 touch: invalid date format ‘198904020211.39’
% rm -f vt52.4 % /usr/bin/touch -t 198904020211.39 vt52.4 % ls --full-time vt52.4 -rw-r--r-- 1 cstacy staff 0 1989-04-02 03:11:39.000000000 -0400 vt52.4
========================================= % which touch /usr/local/gnubin/touch
% /usr/local/gnubin/touch --version touch (GNU coreutils) 8.23
/usr/bin/touch is BSD:
@(#)PROGRAM:touch PROJECT:file_cmds-321.100.11 t@$FreeBSD: src/usr.bin/touch/touch.c,v 1.25 2010/03/28 13:16:08 ed Exp $ @(#)touch.c 8.1 (Berkeley) 6/6/93
How can I install BSD touch on my Mac?
BSD adjusted the time! `
1989 04 02 02 11.39 1989-04-02 03:11:39.000000000 -0400
`
macOS is BSD; you should have the default touch in /usr/bin.
I did install XCode, but I doubt that got me a new version of touch. I do see that we are using slightly different toolchains, though.
I installed the GNU touch from Homebrew.
The trick now seems to be how to NOT use GNU here.
Oh, ok. So I should just uninstall brew's touch. I'll try that.
ozymandias 2897 # touch -t 198904020059.59 /tmp/t
ozymandias 2898 # touch -t 198904020100.00 /tmp/t
ozymandias 2899 # touch -t 198904020159.59 /tmp/t
ozymandias 2900 # touch -t 198904020200.00 /tmp/t
touch: formato de fecha inválido «198904020200.00»
ozymandias 2901 # touch -t 198904020259.59 /tmp/t
touch: formato de fecha inválido «198904020259.59»
ozymandias 2902 # touch -t 198904020300.00 /tmp/t
ozymandias 2903 #
Hmm.... using bsd make gives me errors on all the links. Is that expected? I don't remember seeing this under linux:
touch: syseng/chsdef.999999: No such file or directory
touch: syseng/fsdefs.999999: No such file or directory
touch: turnip/view.start: No such file or directory
are some examples (lots more).
Looks like the BSD touch will create a file if it doesn't exist, but not a directory.
The -h flag inhibits file creation (and does change symlink properties).
My touch thoughts today ... 2021-0628-touch.txt
@cstacy Given the actual timestamps involved, I would guess someone set up this build to try to maintain historical timestamps. Whether they actually had any valid historical timestamps is an fair question. :)
After significant digging around in the sources for OS X and gnulib mktime() and friends, my main conclusion is that the OS X code is significantly older. It wouldn't be a surprise if the differences resulted in the former giving back a value that's merely ballpark, where the latter bails because it's "illegal".
touch ends up calling posixtime() on gnulib, or mktime on OS X, and if the date parsing around that operation fails, the message is emitted. The wording isn't entirely accurate, but it's a red herring.
Regardless, the issue arises in some of the cases because the supplied timestamp is invalid: valid times on that date in the USA went from 01:59:59 to 03:00:00. (And the error doesn't arise in other cases apparently because the platform isn't handling the date-time values in quite the same way.) You can do env TZ=UTC touch -t 198904020259.59 /tmp/t
and it works where it would error without the env TZ=UTC
part.
I'd think the errors should be solved one way or another, as otherwise they potentially break some build somewhere. Whether the solution is to quit setting timestamps, or to fix the timestamps, I'll leave to @larsbrinkhoff or @eswenson1 or ???.
The time stamps hack that Lars added to the build is there to solve three problems: 1) we wanted to preserve the original time stamps on files we retrieved from original backup tape images. I believe it was Git that would not preserve the timestamps on checkout, so the touch mechanism puts back the original time stamp. Having an original time stamp is helpful to know the provenance of a file, allowing us to distinguish recent changes versus original files. It also provides a more authentic experience -- if this wee a hardware-based ITS, timestamps would have been maintained accurately.
2) Correct relative timestamps on emacs sources is important because this is one place where files are compressed IFF they need to be (by the TECO macros used to purify the code). If library files from the sources tape have later timestamps than the non-compressed sources that are used to make those library files, then the emacs TECO macros will skip building the library. This is what we're seeing in the MacOS build.
3) A similar thing happens with the LISP runtime -- if the source file date is later than the FASL file date, I think the source file is loaded rather than the FASL file.
These last two issues are exacerbated by the fact that we don't have all sources and sometimes need to use binary files. Or we need binary files for bootstrapping (lisp, emacs, Macsyma) but later rebuild from source.
I'd guess an hour or so of difference in the applied timestamp wouldn't impair the build. That'd be a fairly quick fix. Are there cases where files changed versions multiple times per day in the historical archive? How did ITS sites manage DST?
I agree about the importance of preserving the timestamps generally; glad to hear other people think that too.
As to the accuracy of the "original" timestamps, remember that these have already been through several preservation and conversion/transportation steps by the time the files appear in the project.
Adjustment (e.g. by BSD/touch) would only cause a problem if a dependent (binary) file was later than an (adjusted) source file timestamp.
(The adjustment is one hour, so the files would have to have been modified on the same day, which is quite reasonable. But the source file must have been modified during the one-hour DST change window, and the binary must have been produced in the hour after the window. Otherwise the source would still be newer than the binary.)
02:11 source FOO modified Happened 02:56 i.e. :TECO 03:02 FOO :EJ dumped Nope 2021 touch FOO -> 03:11:39
Late-night hacking notwithstanding, it seems that fate is not quite THAT cruel.
So the BSD touch adjustment is apparently okay. (Unless someday more files are added to the project and they have this same issue.)
Just for the sake of forensic curiosity, I wonder what the binary's timestamp really was. And who ran the job. (...and what machine it was, where was the hacker logged in from, and who else was on at the time. Theoretically we could actuall know this, if certain files were preserved!)
Well, I did say that I appreciated the historical preservation...
I don't remember how time zone changes worked on ITS. I do believe it was an event. Maybe someone even had to run LOCK or something.
Interesting that the timestamp on that file was illegal. Doesn't that mean it was illegal while sitting on at least one Unix medium in the conservation process? I guess system calls to set times on files didn't care. Just the command-line interface.
PS. I now remember running LOCK to fix the time, and I do think it was probably for DST.
I'll say it out loud so it's here, though I'm sure it's obvious: the "invalid" timestamp could have resulted on the original system, or it could be due to an issue in the modern conversion tooling.
While sitting on the unix filesystem, it had a "seconds since epoch" date, not a struct tm
or "human string".
Re: timeouts.
Not happened to me, but can't they just be upped to some insane amount? Why the emulator would seem (to the terminal that expect is looking at) to hang is interesting. Maybe a bug in expect or in the terminal. I bet ITS didn't really hang.
@cstacy Regarding your comments regarding the accuracy of the dates: I think we can trust the dates. The file that we use the timestamps hack on have come from images of backup tapes of the real ITS machines. The original tapes accurately captured the modification dates of the files (although times were all Eastern Time). The tools used to capture the "bits" off the tape and produce tape image files in ToTS also preserved these dates accurately. And the ITSTAR program that reads these tape image files correctly sets the modification dates in the target operating system file system as they were on the tape image.
There are some cases where dates are in the current century -- that is usually because we HAVE modified them recently. And there are probably a few files that may have NOT come directly from a tape image, but from a file system (say on XX or some TOPS-20 machine) tape image, where the dates may have gotten corrupted. But, in general, the timestamps should be accurate -- or at least as accurate as they were on the ITS machine from which they originated.
I may have missed something, but why were the timestamps illegal? As far as I knew they seemed like perfectly valid dates in the Eastern Standard (or Daylight) time zone (2 or 3 o'clock in the morning). Are you saying that because of a time change on those dates, there was no 2:xx AM on those dates, hence illegal?
Exactly.
Maybe my problem is autoconf.
Story and outputs attached. (You might want to read the end of the file first, and if something needs untangling, go back to the beginning of the attachment.)
Executive summary: pdp10-ka works. klh10 blows up building with a syntax error in a configure file.
klh10-fail.wall.txt