Open larsbrinkhoff opened 4 years ago
THE STORY OF SIMTEL20
SIMTEL20 is well known among the Internet community as the world’s largest on-line repository of freely accessible software and documentation on the Internet. What is not well known today, is how it came into being.
In 1979, CP/M (Control Program for Microcomputers) was the primary operating system on smaller (micro) computers. One summer evening back then, Frank Wancho, an employee of WSMR (White Sands Missile Range) in New Mexico, and at the time, the volunteer sponsor of the MIT (Massachusetts Institute of Technology) INFO-CPM mailing list, made a phone call from his home in El Paso, Texas to a newly published phone number for a computer bulletin board system outside Detroit, Michigan. The 2:00 am phone call was answered by the BBS operator, Keith Petersen, rather than by the computer, as expected.
Keith was, at the time, employed by the local Detroit CBS affiliate as a technical engineer, but was also an avid computer enthusiast. As a result of his computing interests, Keith had started collecting CP/M utilities, files, software, etc. from other enthusiasts and keeping them on his system. He was informally encouraging others to send him new software to add to his collection. The unusual part of this activity was that he provided a way for other users to SECURELY do this using the actual remote CP/M commands, by using multiple directories for uploading and downloading.
As a result of that phone call, Keith began to be an active contributor to the MIT INFO-CPM mailing list. This was at a time when the PDP-10 at MIT had no file transfer protocol that was compatible with CP/M. Keith used “blind uploads” via modem to send new public domain CP/M software, patches, and documentation to the MIT machine on a daily basis. The files themselves were sent directly to the MIT mailing list as messages (i.e. each person on the list received a separate copy of the file).
It soon became clear that all this traffic was straining the MIT mail delivery system. MIT agreed to set aside disk space to hold the CP/M collection, and only the announcements of the new software were sent to the mailing list. Several volunteers at MIT, notably Gail Zacharias, soon managed to write mainframe versions of the MODEM2 file transfer protocol and other CP/M utility programs for the MIT machine. These significantly improved the accuracy of the file contents that Keith uploaded.
In 1983, as availability and access to the MIT computer was removed, Frank Wancho (the System Administrator for the WSMR computer), arranged for the contents of the MIT CP/M and the newly formed MS-DOS collections to be moved to a DECSYSTEM 20 that had excess CPU and Disk capacity at WSMR. This computer was also on the ARPANET. Because of the existing network requirements at that time, ARPANET hosts had to have 8 character names. SIMTEL20 was named for the SIMulation and TELeprocessing organization’s DECSYSTEM 20 machine by a vote of the local employees at WSMR.
Previous discussion in #126. The Simtel archive is preserved on the Walnut Creek CP/M CD-ROM, but the structure is very different from what was on MC; it's not clear to me whether the files were moved over as-is or whether some stuff was lost in the transition. So there may be files on the MIT tapes that didn't make it to the preserved archives.
Thanks, I was sure it had been discussed before but didn't find it.
Here's a list of archive files from the CPM directory. There are also lots of other files, but I'm guessing these would be the bulk of the directory.
AR0 DOCFIL
AR1 COMFIL
AR2 HLPFIL
AR3 SQ%USQ
AR4 APPLE
AR5 MODEM
AR6 ASMFIL
AR8 UGLIBS
AR9 FAST2
AR10 GENDOC 7/26/82 04:52:33.5
AR11 GENASM 6/19/82 17:44:38
AR12 GENCOM 7/8/82 03:55:54.5
AR13 HEX 8/1/82 13:28:01.5
AR14 NEWS 3/19/82 23:59:30.5
AR15 CATABS 7/19/81 16:42:10.5
AR16 MSOFT 4/29/82 02:52:23
AR17 FORTH
AR18 RATFOR
AR19 CATLOG
AR20 DSKUTL 4/28/82 04:26:14.5
AR21 TXTUTL 5/31/82 04:59:22
AR22 DIRUTL 8/5/82 04:59:35
AR23 SYSLIB 3/3/82 01:30:12
AR24 FILUTL 8/1/82 13:21:07.5
AR25 DISASM 6/13/82 03:59:11
AR26 SORT 5/31/82 04:52:46
AR27 LIST 6/27/82 05:05:20.5
AR28 CPMARC 7/28/82 14:01:47.5
AR30 BASIC 11/21/81 03:32:41.5
AR31 PASCAL 4/28/82 04:28:17.5
AR32 FORTH 12/20/81 03:10:39
AR33 RATFOR 11/9/81 03:23:33
AR34 BDSC-1 7/19/82 03:15:43.5
AR35 BDSC-2 7/14/82 00:33:16.5
AR36 BDSC-3 7/14/82 00:34:27.5
AR37 BDSC-4
AR40 APPLE 5/20/82 05:14:30
AR41 OSBORN 6/30/82 03:26:45
AR42 TRS-80 3/29/82 02:08:29
AR43 UNIX 7/11/82 20:32:05.5
AR44 HEATH 6/8/82 22:12:50
AR45 ATARI 8/1/82 14:05:40.5
AR46 IBM-PC
AR47 NSTAR
AR50 HELP 3/1/82 03:08:29
AR51 SQ/USQ 4/2/82 12:25:01
AR52 VOICE 2/6/82 15:51:01.5
AR53 CATLOG 1/31/82 11:38:52.5
AR54 FAST2 10/10/81 05:04:35
AR55 CCP 5/29/82 19:51:26.5
AR56 ZCPR 8/1/82 18:34:53
AR57 SUBMIT
AR58 MACLIB 4/2/82 01:23:49.5
AR59 SPELL
AR60 MODEM2 5/15/82 04:10:37.5
AR61 MODEM7 5/29/82 10:12:27
AR62 YAM 2/23/82 22:54:37
AR63 RCPM 6/25/82 11:48:28.5
AR64 TERM 4/24/82 07:10:50.5
AR65 SMODEM 4/8/82 02:43:10
AR66 MICNET 7/6/82 02:10:00.5
AR69 RBBS
I don't see AR67, 68, or 91 mentioned in #126.
Here's a purported MIT-MC archive: http://www.classiccmp.org/cpmarchives/ftp.php?b=cpm/Software/WalnutCD/cpm
It's hard to find anything in common with files from MC, but the CPM; AR45 ATARI file shares some file names with the atari directory on the Walnut Creek CD:
AMODEM 42BAS 1825 1983-04-12 02:17:27(1983-05-22) 36
AMODEM 42DOC 1058 1983-03-13 14:09:55(1983-05-20) 36
AUT850 BAS 126 1982-08-01 14:04:23(1983-05-20) 36
File contents do match.
It seems archive files were rearranged at some points. So we have e.g. both AR4 APPLE and AR40 APPLE, AR19 CATLOG and AR53 CATLOG, etc.
Hello @w8sdz,
I'm researching the ITS operating system, and have come across information about the CP/M collection on MIT-MC. Do you have any recollections about how this collection was moved from MIT-MC to SIMTEL20? For example, were file names changed?
Thank you,
Lars Brinkhoff
This ZIP file has listings from the CPM directory as well as listings of the archive files.
Thanks, @larsbrinkhoff!
Comparing those listings with 00-index.txt from the Walnut Creek CP/M CD, there seem to be a lot of files in \cpm that came from MC, dated September 1985.
For example, here's AR66: MICNET;:
BUFEXC ASM 5578 1982-06-13 14:57
CIS QDOC 486 1982-09-23 21:02
CIS SYSDOC 1299 1982-03-20 03:11
DOW 12COM 225 1982-04-02 01:05
DOW 12HEX 507 1982-04-02 01:27
DOW 12MAC 3031 1982-04-02 00:50
DOWPMM DIF 432 1982-04-23 08:25
MNET PROTO 2720 1982-03-22 06:32
MNET SIGNUP 324 1982-07-06 02:10
MNEXEC DOC 414 1982-03-15 05:15
PROTOC CIS 2944 1983-04-21 01:30
UPLOAD 12COM 385 1982-04-02 01:06
UPLOAD 12HEX 867 1982-04-02 01:28
UPLOAD 12MAC 4715 1982-04-02 00:55
UPPMMI DIF 458 1982-04-23 08:23
And here's \cpm\cis:
CIS and Related Programs
path: \cpm\cis
--------------------------------------
Name Size Date Description
------------ ---- ---- -----
BPROTO.LBR 54272 11-30-86 No description available
BUFEXC.AQM 19456 02-09-85 No description available
BUFEXC.ASM 27904 07-13-94
CIS-SYS.DOC 6528 07-13-94
CIS-SYS.DQC 4224 02-09-85 No description available
CISQUICK.DOC 2432 07-13-94
CISQUICK.DQC 1792 02-09-85 No description available
DOW12.COM 896 02-09-85 No description available
DOW12.MAC 15232 07-13-94
DOW12.MQC 8960 02-09-85 No description available
DOWPMM.DIF 2063 09-09-83 No description available
MNEXEC.DOC 2176 07-13-94
MNEXEC.DQC 1536 02-09-85 No description available
PROTCISA.TXT 14720 07-13-94
PROTCISA.TZT 6784 07-05-87 No description available
PROTOC.CIS 14720 07-13-94
PROTOC.CQS 8448 02-09-85 No description available
UPLOAD12.COM 1536 02-09-85 No description available
UPLOAD12.MAC 23680 07-13-94
UPLOAD12.MQC 13824 02-09-85 No description available
UPPMMI.DIF 2179 09-09-83 No description available
VIDTEX.ARK 23424 03-22-87 No description available
Most of the ITS-sourced files are there, dated 02-09-85, in squeezed form; the unsqueezed versions were added in 1994 (for the CD?). The files have been renamed manually into CP/M 8.3 form throughout the tree, e.g. DOW 12COM becomes DOW12.COM.
But there are also earlier-dated files mixed in there, and the directory names are different. And some other directory names have been taken over directly from ITS (TXTUTL, DSKUTL...) with more files added later. So perhaps the ITS files were renamed and merged into an existing SIMTEL collection in 1985?
Edit: Or later; they don't appear in the "New files on SIMTEL20 between 29-Aug-85 and 26-Sep-85" post on INFO-CPM.
Timestamps on the CPM files I found range from 1980-09-02 (MENU MAC in CPM; AR6 ASMFIL), to 1983-08-03 (CPM; AR55 CCP).
From the TOPS-20 Kermit source:
SET ITS-BINARY (format) ON or OFF
Specify whether ITS-Binary file headers are to be recognized or ignored. ITS binary format is a way (devised at MIT) of storing foreign 8-bit binary data on a 36-bit machine to allow automatic recognition of these files when sending them out again, so that you don't have to depend on the file byte size, or to issue explict SET FILE BYTESIZE commands to KERMIT.
An ITS format binary file contains the sixbit characters "DSK8" left-adjusted in the first 36-bit word. [...]
Several of the CP/M-related ITS tools - CRC, COMIFY, HEXIFY, TYPE8, USQ - have support for this convention, as does SAIL XMODEM.
ToTS tape 7007332, file MMB; MMB MAIL5
Date: Mon, 2 Apr 84 16:49:15 EST
From: Dave Towson (info-cpm) <cpmlist@Amsaa.ARPA>
To: info-cpm@Amsaa.ARPA
Subject: SIMTEL20 ARCHIVES.
Public Domain Software Lovers - At long last, the Great Archive Blurb is
finished! This updated version covers ALL of the archives on SIMTEL20, and
will (I hope) answer most of the questions that have been pouring in since the
announcement of the new PC-BLUE archive.
I especially want to thank Gail Zacharias <GZ@MIT-MC> for a very thorough
job of proof-reading and error reporting. The blurb is considerably improved
as a result of her efforts.
Thanks are also due to Frank Wancho <WANCHO@SIMTEL20> for making the
archives possible, and to Keith Petersen <W8SDZ@SIMTEL20> and Richard Conn
<RCONN@SIMTEL20> for maintaining them. Without the efforts of these people,
I wouldn't have had to write this thing (hmmmmm).
Finally, thanks to all of you who have been patiently waiting for this
information, and wishing I'd hurry up so that you could get at the goodies.
This blurb is about thirteen-thousand characters long. It's a lot to
absorb at one time if you don't already know most of it. My advice is to print
it, and keep it on hand for reference.
Dave Towson
INFO-CPM-REQUEST @ AMSAA
-------------------------------------------------------------------------------
THE SIMTEL20 ARCHIVES
OVERVIEW
There is a collossal amount of free public domain CP/M software in
several archives on SIMTEL20, a PDP-20 running TOPS-20 at White Sands Missile
Range. Files may be obtained using FTP (described in a following paragraph)
with user-name "anonymous". For a password, use your host-name (or any string
of printing characters). Throughout this message, FTP examples are given in a
GENERIC syntax. You will have to consult either a local documentation file or
your friendly system wizard to learn the actual syntax used with your local
mainframe operating system.
[...hundreds of lines...]
ADDITIONS, IMPROVEMENTS AND CORRECTIONS
Suggestions for additions, improvements and corrections to this message
are always welcome. Please send them to INFO-CPM-REQUEST. Additional
information concerning FTP is, however, beyond the scope of this message. If
you need help with FTP, please read the documentation for your local system,
or see your friendly system wizard.
Contributions of public domain software are actively solicited. If you
have something that seems appropriate for inclusion in the <CPM> archive,
please contact Keith Petersen <W8SDZ@SIMTEL20>. Likewise, contributions to the
<UNIX> archive can be addressed to Richard Conn <RCONN@SIMTEL20>.
Happy hacking!
Dave Towson
INFO-CPM-REQUEST @ AMSAA
Also known as the .COM header:
'Way back when SIMTEL20 wasn't holding all that public domain software, MIT had the honor. And The Powers At Mit wanted some way to distinguish between MIT binary and microprocessor binary.
What evolved out of all this was the placement of a four character string at the beginning of each binary file. This string is "DSK8". And it is known as the .COM header. People who download files containing the .COM header must somehow strip off those first four bytes in order to have just the data file.
You probably downloaded the file with the .COM header. USQ, not knowing what what that was, gave you an error message and aborted. The second time you probably told MMODEM not to send the .COM header. And USQ then worked.
A quick grep through ToTS reveals some more DSK8 aware program: LMODEM, MMODEM, DSK8 (Lisp code).
This net.micro.cpm message marks the transfer of the CPM archive from MIT-MC to SIMTEL20.
https://usenetarchives.com/view.php?id=net.micro.cpm&mid=PDQ3NzBAc3JpLWFycGEuVVVDUD4%3D
07/09/1983 02:04:004 UTC
From: Frank J. Wancho
We are in the process of verifying the collection of public domain
files on MC have all been moved to the SIMTEL20. Therefore, we would
like those contributors to the collection to hold off uploading any
new files for a few days until we announce new directions for where to
put the files on the SIMTEL20. Please bear with us.
--Frank
Longer message by Frank J. Wancho about the CP/M archive, posted 4 September 1986 17:02:46.
https://groups.google.com/g/net.micro.cpm/c/rZz-dStwbh0/m/8CrsRwUQgQsJ
This message marks an historic moment, and a bit of background history
which has brought us to this moment is in order. Please bear with me
while you read this lead-in to details you have all been eagerly
awaiting at the end of this message.
Seven years ago this past Friday, August 29, 1979, INFO-CPM was formed
as a spin-off of INFO-MICRO, both homed at the time on MIT-MC. The
following month Keith Petersen's Royal Oak RCP/M system was
discovered, and Keith was invited to directly participate in the
INFO-CPM discussions. Through Keith's direct access, he was able to
upload, crudely at first, selected files for FTP from MIT-MC from his
vast up-to-date RCP/M collection. As new files became available,
Keith made the announcements to the INFO-CPM list. New contributions
and updates to existing files were likewise made available from other
ARPANET users. Thus, the CP/M archives was born, with disk space
courtesy of the MIT-MC management.
This activity caused the development of several mainframe versions of
CP/M utilities, such as the first mainframe implementation of the
Christensen Protocol, in MacLisp, by Ed Barton, called LMODEM. Then
Gail Zacharias developed MMODEM, USQ, HEXFIY, COMIFY, MAKLBR, DE-LBR
and others, some of which were modified for use on TOPS-20 systems.
Bill Westfield wrote the original and invaluable MODEM program for
TOPS-20, which has recently been overhauled into TMODEM.
Disk space was inherently limited on MIT-MC, and when the Macsyma
Consortium was dissolved at the end of September, 1983, SIMTEL20 at
White Sands Missile Range (WSMR) was already online and had disk space
to spare on the required 176MB RP06 boot disk. (SIMTEL20 is a
contraction of the name of the branch which then owned the machine,
SIMulation and TELeprocessing, DECSYSTEM-20.) During that month, the
CP/M collection was moved from MIT-MC to the RP06, designated as the
MICRO: structure, on SIMTEL20. Already purchased for a closely
related project, the then named DARCOM Microcomputer Software Sharing
System (DMSSS), were the entire sets of the CPMUG, SIG/M, and PC/BLUE
distributions, which were uploaded as-is to MICRO:. Since then we
managed to get placed on the tail end of the regional distribution of
both the SIG/M and PC/BLUE update sets which periodically extend both
of those collections.
As this was all going on, it became evident that a collection of
Unix/C versions of the CP/M utilities would be required and that
collection was started. That collection has since come under the
sponsorship of another organization within the Army Materiel Command
(AMC) to which WSMR belongs. That organization, Logistics Systems
Support Activity, LSSA, provided the funds for the 512MB disk drive
known as PD:, to which all the collections residing on the out-grown
MICRO: structure were moved in November, 1985.
In November, 1984, Rick Conn volunteered to start the extremely
popular Ada Software Repository, originally on PS: as there was no
room on MICRO:, and then moved to PD: when that device came online.
As with the CP/M archives in PD:<CPM.*> being considered the best and
most recent, the MSDOS archives are also in the process of being
similarly organized in PD:<MSDOS.*>, with files culled from the latest
releases of the PC/BLUE collection as well as the INFO-IBMPC
collections on ISI-B and Pete Galvin's collection on UTEXAS-20.
When the CP/M collection first started, there was only one directory
with subdirectories, and several not-necessarily CP/M-related
directories lived under that tree. Recently, a new top-level
directory was created on PD:, named PD:<MISC>. Now those generic
subdirectories in PD:<CPM.*>, PD:<UNIX.*>, and PD:<MSDOS.*> are being
moved to that new directory tree.
The original INFO-CPM list was maintained on MIT-MC until 1983 and
then moved to BRL, where it was maintained by Ron Natalie for a short
while. Ron, who has been the maintainer of INFO-MICRO, among other
lists, drafted Dave Towson to maintain the INFO-CPM list on AMSAA.
Dave has been continuously maintaining the list since November 1983,
periodically rewriting and distributing the SIMTEL20 Archive Users
Guide, commonly referred to as the "Archive Blurb" and answering
numerous user requests for information on access to the SIMTEL20
collections.
As the subscription list to INFO-CPM grew, it was gatewayed into the
USENET community, first as fa.info-cpm, and now as a two-way
newsgroup, net.micro.cpm. Later, members from other communities
joined the list from CSNET, BITNET, and others. Meanwhile, new files
and entire collections were added to the SIMTEL20 archives and
announced to the INFO-CPM list. Access to these files was inherently
limited to those with Internet FTP access to SIMTEL20, unless some
kind soul volunteered to mail selected files on request.
Recently, there was a flurry of messages pleading for some form of
automated access to the SIMTEL20 archives via net mail. About two
weeks ago, while reading these pleadings while on travel in the
mountains of Colorado, I said that all the pieces were falling into
place to be put together to provide that service. Maybe it was the
altitude that made me say that, but everything eventually did fall
into place in the two weeks since then.
A Mail File Server was written in C using a beta version of an about
to be released compiler for TOPS-20 systems. If it wasn't for the
quick turnaround on reported bug fixes by the two principal
maintainers of that compiler and run-time library, Ken Harrenstien and
Ian Mackey at SRI-NIC, this program could not have been developed in
such a short time by a person whose only other claim to fame in C
programming is one other C program he co-authored two years ago.
The original concept for this service came from a similar service
developed by Jack Dongarra of the Argonne National Laboratory and Eric
Grosse of AT&T Bell Laboratories called NetLib, which is available
from ANL-MCS for mail access into its collection of well-indexed and
documented high quality public domain mathematical software routines.
Their work was supported, in part, by the National Science Foundation.
From their sources came that part of the package which extracts the
requestor's return address from the request file.
The SIMTEL20 version of this service has been in beta test for almost
two weeks now, with new features added and many bugs fixed since then.
While credits are being handed out, I wish to thank Matt Kimmel for
checking out access from the BITNET side, and Eric Hildum from the
USENET side. And, to Keith Petersen, Dave Towson, Bernie Eiben, Rick
Conn, and Mark Crispin for participating in a lively discussion of the
principles of operation of this service.
Now, before I bombard you with details on how to access the files, let
me caution you that this system is experimental. There is no such
thing as finding the "last" bug in any program. Furthermore, I sit in
awe and fear that even selective and judicious use of this system by
the potential audience this message will eventually reach may overload
this machine or some of the fragile mail links between hosts on the
various networks connecting us all together.
This means that those of you already with FTP access to SIMTEL20 must
not use this service and continue using FTP. There is no blocking
mechanism in place right now, but I will consider taking the time to
install one if you choose to ignore this request. Those of you on
USENET, BITNET, and CSNET hosts should consider appointing one point
of contact through which you should funnel your requests and the
reconstructed files from the replies from this service should be made
available to all your local users. This particularly applies to the
HELP, INFO, and BOOTSTRAP messages and the files they point to.
This message is being sent only to the readers of INFO-CPM so that we
can gauge the impact on the system. Please do not redistribute this
message to any other mailing list or newsgroup. Their time will come.
What follows is the message you get in response to the SEND HELP
command...
Enjoy!
Frank
I have made a suggestion to MIT Libraries to make the ToTS copy of the CPM archive public,
FYI @johnsonjh
MC used to host an archive of CP/M software. It was later moved to the SIMTEL20 twenex.
There was also a mailing list, CPM at MIT-MC.