PDP-10 / its

Incompatible Timesharing System
Other
846 stars 80 forks source link

CP/M archive #1770

Open larsbrinkhoff opened 4 years ago

larsbrinkhoff commented 4 years ago

MC used to host an archive of CP/M software. It was later moved to the SIMTEL20 twenex.

There was also a mailing list, CPM at MIT-MC.

larsbrinkhoff commented 4 years ago

https://www.cni.org/resources/historical-resources/farnet-stories-project/geographical-index/story149-nm/

THE STORY OF SIMTEL20

SIMTEL20 is well known among the Internet community as the world’s largest on-line repository of freely accessible software and documentation on the Internet. What is not well known today, is how it came into being.

In 1979, CP/M (Control Program for Microcomputers) was the primary operating system on smaller (micro) computers. One summer evening back then, Frank Wancho, an employee of WSMR (White Sands Missile Range) in New Mexico, and at the time, the volunteer sponsor of the MIT (Massachusetts Institute of Technology) INFO-CPM mailing list, made a phone call from his home in El Paso, Texas to a newly published phone number for a computer bulletin board system outside Detroit, Michigan. The 2:00 am phone call was answered by the BBS operator, Keith Petersen, rather than by the computer, as expected.

Keith was, at the time, employed by the local Detroit CBS affiliate as a technical engineer, but was also an avid computer enthusiast. As a result of his computing interests, Keith had started collecting CP/M utilities, files, software, etc. from other enthusiasts and keeping them on his system. He was informally encouraging others to send him new software to add to his collection. The unusual part of this activity was that he provided a way for other users to SECURELY do this using the actual remote CP/M commands, by using multiple directories for uploading and downloading.

As a result of that phone call, Keith began to be an active contributor to the MIT INFO-CPM mailing list. This was at a time when the PDP-10 at MIT had no file transfer protocol that was compatible with CP/M. Keith used “blind uploads” via modem to send new public domain CP/M software, patches, and documentation to the MIT machine on a daily basis. The files themselves were sent directly to the MIT mailing list as messages (i.e. each person on the list received a separate copy of the file).

It soon became clear that all this traffic was straining the MIT mail delivery system. MIT agreed to set aside disk space to hold the CP/M collection, and only the announcements of the new software were sent to the mailing list. Several volunteers at MIT, notably Gail Zacharias, soon managed to write mainframe versions of the MODEM2 file transfer protocol and other CP/M utility programs for the MIT machine. These significantly improved the accuracy of the file contents that Keith uploaded.

In 1983, as availability and access to the MIT computer was removed, Frank Wancho (the System Administrator for the WSMR computer), arranged for the contents of the MIT CP/M and the newly formed MS-DOS collections to be moved to a DECSYSTEM 20 that had excess CPU and Disk capacity at WSMR. This computer was also on the ARPANET. Because of the existing network requirements at that time, ARPANET hosts had to have 8 character names. SIMTEL20 was named for the SIMulation and TELeprocessing organization’s DECSYSTEM 20 machine by a vote of the local employees at WSMR.

atsampson commented 4 years ago

Previous discussion in #126. The Simtel archive is preserved on the Walnut Creek CP/M CD-ROM, but the structure is very different from what was on MC; it's not clear to me whether the files were moved over as-is or whether some stuff was lost in the transition. So there may be files on the MIT tapes that didn't make it to the preserved archives.

larsbrinkhoff commented 4 years ago

Thanks, I was sure it had been discussed before but didn't find it.

Here's a list of archive files from the CPM directory. There are also lots of other files, but I'm guessing these would be the bulk of the directory.


AR0    DOCFIL
AR1    COMFIL
AR2    HLPFIL
AR3    SQ%USQ
AR4    APPLE
AR5    MODEM
AR6    ASMFIL
AR8    UGLIBS
AR9    FAST2
AR10   GENDOC    7/26/82 04:52:33.5
AR11   GENASM    6/19/82 17:44:38
AR12   GENCOM    7/8/82 03:55:54.5
AR13   HEX       8/1/82 13:28:01.5
AR14   NEWS      3/19/82 23:59:30.5
AR15   CATABS    7/19/81 16:42:10.5
AR16   MSOFT     4/29/82 02:52:23
AR17   FORTH
AR18   RATFOR
AR19   CATLOG
AR20   DSKUTL    4/28/82 04:26:14.5
AR21   TXTUTL    5/31/82 04:59:22
AR22   DIRUTL    8/5/82 04:59:35
AR23   SYSLIB    3/3/82 01:30:12
AR24   FILUTL    8/1/82 13:21:07.5
AR25   DISASM    6/13/82 03:59:11
AR26   SORT      5/31/82 04:52:46
AR27   LIST      6/27/82 05:05:20.5
AR28   CPMARC    7/28/82 14:01:47.5
AR30   BASIC     11/21/81 03:32:41.5
AR31   PASCAL    4/28/82 04:28:17.5
AR32   FORTH     12/20/81 03:10:39
AR33   RATFOR    11/9/81 03:23:33
AR34   BDSC-1    7/19/82 03:15:43.5
AR35   BDSC-2    7/14/82 00:33:16.5
AR36   BDSC-3    7/14/82 00:34:27.5
AR37   BDSC-4
AR40   APPLE     5/20/82 05:14:30
AR41   OSBORN    6/30/82 03:26:45
AR42   TRS-80    3/29/82 02:08:29
AR43   UNIX      7/11/82 20:32:05.5
AR44   HEATH     6/8/82 22:12:50
AR45   ATARI     8/1/82 14:05:40.5
AR46   IBM-PC
AR47   NSTAR
AR50   HELP      3/1/82 03:08:29
AR51   SQ/USQ    4/2/82 12:25:01
AR52   VOICE     2/6/82 15:51:01.5
AR53   CATLOG    1/31/82 11:38:52.5
AR54   FAST2     10/10/81 05:04:35
AR55   CCP       5/29/82 19:51:26.5
AR56   ZCPR      8/1/82 18:34:53
AR57   SUBMIT
AR58   MACLIB    4/2/82 01:23:49.5
AR59   SPELL
AR60   MODEM2    5/15/82 04:10:37.5
AR61   MODEM7    5/29/82 10:12:27
AR62   YAM       2/23/82 22:54:37
AR63   RCPM      6/25/82 11:48:28.5
AR64   TERM      4/24/82 07:10:50.5
AR65   SMODEM    4/8/82 02:43:10
AR66   MICNET    7/6/82 02:10:00.5
AR69   RBBS
larsbrinkhoff commented 4 years ago

I don't see AR67, 68, or 91 mentioned in #126.

larsbrinkhoff commented 4 years ago

Here's a purported MIT-MC archive: http://www.classiccmp.org/cpmarchives/ftp.php?b=cpm/Software/WalnutCD/cpm

It's hard to find anything in common with files from MC, but the CPM; AR45 ATARI file shares some file names with the atari directory on the Walnut Creek CD:

AMODEM 42BAS     1825  1983-04-12 02:17:27(1983-05-22)  36
AMODEM 42DOC     1058  1983-03-13 14:09:55(1983-05-20)  36
AUT850 BAS        126  1982-08-01 14:04:23(1983-05-20)  36

File contents do match.

larsbrinkhoff commented 4 years ago

It seems archive files were rearranged at some points. So we have e.g. both AR4 APPLE and AR40 APPLE, AR19 CATLOG and AR53 CATLOG, etc.

larsbrinkhoff commented 4 years ago

Hello @w8sdz,

I'm researching the ITS operating system, and have come across information about the CP/M collection on MIT-MC. Do you have any recollections about how this collection was moved from MIT-MC to SIMTEL20? For example, were file names changed?

Thank you,
Lars Brinkhoff

larsbrinkhoff commented 4 years ago

This ZIP file has listings from the CPM directory as well as listings of the archive files.

mit-mc.zip

atsampson commented 4 years ago

Thanks, @larsbrinkhoff!

Comparing those listings with 00-index.txt from the Walnut Creek CP/M CD, there seem to be a lot of files in \cpm that came from MC, dated September 1985.

For example, here's AR66: MICNET;:

BUFEXC ASM       5578  1982-06-13 14:57
CIS    QDOC       486  1982-09-23 21:02
CIS    SYSDOC    1299  1982-03-20 03:11
DOW    12COM      225  1982-04-02 01:05
DOW    12HEX      507  1982-04-02 01:27
DOW    12MAC     3031  1982-04-02 00:50
DOWPMM DIF        432  1982-04-23 08:25
MNET   PROTO     2720  1982-03-22 06:32
MNET   SIGNUP     324  1982-07-06 02:10
MNEXEC DOC        414  1982-03-15 05:15
PROTOC CIS       2944  1983-04-21 01:30
UPLOAD 12COM      385  1982-04-02 01:06
UPLOAD 12HEX      867  1982-04-02 01:28
UPLOAD 12MAC     4715  1982-04-02 00:55
UPPMMI DIF        458  1982-04-23 08:23

And here's \cpm\cis:

CIS and Related Programs
path: \cpm\cis
--------------------------------------
Name             Size    Date    Description
------------     ----    ----    -----
BPROTO.LBR      54272  11-30-86  No description available
BUFEXC.AQM      19456  02-09-85  No description available
BUFEXC.ASM      27904  07-13-94  
CIS-SYS.DOC      6528  07-13-94  
CIS-SYS.DQC      4224  02-09-85  No description available
CISQUICK.DOC     2432  07-13-94  
CISQUICK.DQC     1792  02-09-85  No description available
DOW12.COM         896  02-09-85  No description available
DOW12.MAC       15232  07-13-94  
DOW12.MQC        8960  02-09-85  No description available
DOWPMM.DIF       2063  09-09-83  No description available
MNEXEC.DOC       2176  07-13-94  
MNEXEC.DQC       1536  02-09-85  No description available
PROTCISA.TXT    14720  07-13-94  
PROTCISA.TZT     6784  07-05-87  No description available
PROTOC.CIS      14720  07-13-94  
PROTOC.CQS       8448  02-09-85  No description available
UPLOAD12.COM     1536  02-09-85  No description available
UPLOAD12.MAC    23680  07-13-94  
UPLOAD12.MQC    13824  02-09-85  No description available
UPPMMI.DIF       2179  09-09-83  No description available
VIDTEX.ARK      23424  03-22-87  No description available

Most of the ITS-sourced files are there, dated 02-09-85, in squeezed form; the unsqueezed versions were added in 1994 (for the CD?). The files have been renamed manually into CP/M 8.3 form throughout the tree, e.g. DOW 12COM becomes DOW12.COM.

But there are also earlier-dated files mixed in there, and the directory names are different. And some other directory names have been taken over directly from ITS (TXTUTL, DSKUTL...) with more files added later. So perhaps the ITS files were renamed and merged into an existing SIMTEL collection in 1985?

Edit: Or later; they don't appear in the "New files on SIMTEL20 between 29-Aug-85 and 26-Sep-85" post on INFO-CPM.

larsbrinkhoff commented 4 years ago

Timestamps on the CPM files I found range from 1980-09-02 (MENU MAC in CPM; AR6 ASMFIL), to 1983-08-03 (CPM; AR55 CCP).

atsampson commented 4 years ago

From the TOPS-20 Kermit source:

SET ITS-BINARY (format) ON or OFF

Specify whether ITS-Binary file headers are to be recognized or ignored. ITS binary format is a way (devised at MIT) of storing foreign 8-bit binary data on a 36-bit machine to allow automatic recognition of these files when sending them out again, so that you don't have to depend on the file byte size, or to issue explict SET FILE BYTESIZE commands to KERMIT.

An ITS format binary file contains the sixbit characters "DSK8" left-adjusted in the first 36-bit word. [...]

Several of the CP/M-related ITS tools - CRC, COMIFY, HEXIFY, TYPE8, USQ - have support for this convention, as does SAIL XMODEM.

larsbrinkhoff commented 4 years ago

ToTS tape 7007332, file MMB; MMB MAIL5

Date:     Mon, 2 Apr 84 16:49:15 EST
From:     Dave Towson (info-cpm) <cpmlist@Amsaa.ARPA>
To:       info-cpm@Amsaa.ARPA
Subject:  SIMTEL20 ARCHIVES.

     Public Domain Software Lovers - At long last, the Great Archive Blurb is
finished!  This updated version covers ALL of the archives on SIMTEL20, and
will (I hope) answer most of the questions that have been pouring in since the
announcement of the new PC-BLUE archive.

     I especially want to thank Gail Zacharias <GZ@MIT-MC> for a very thorough
job of proof-reading and error reporting.  The blurb is considerably improved
as a result of her efforts.

     Thanks are also due to Frank Wancho <WANCHO@SIMTEL20> for making the
archives possible, and to Keith Petersen <W8SDZ@SIMTEL20> and Richard Conn
<RCONN@SIMTEL20> for maintaining them.  Without the efforts of these people,
I wouldn't have had to write this thing (hmmmmm).

     Finally, thanks to all of you who have been patiently waiting for this
information, and wishing I'd hurry up so that you could get at the goodies.

     This blurb is about thirteen-thousand characters long.  It's a lot to
absorb at one time if you don't already know most of it.  My advice is to print
it, and keep it on hand for reference.

Dave Towson
INFO-CPM-REQUEST @ AMSAA

-------------------------------------------------------------------------------

                            THE SIMTEL20 ARCHIVES

OVERVIEW

     There is a collossal amount of free public domain CP/M software in
several archives on SIMTEL20, a PDP-20 running TOPS-20 at White Sands Missile
Range.  Files may be obtained using FTP (described in a following paragraph)
with user-name "anonymous".  For a password, use your host-name (or any string
of printing characters).  Throughout this message, FTP examples are given in a
GENERIC syntax.  You will have to consult either a local documentation file or
your friendly system wizard to learn the actual syntax used with your local
mainframe operating system.

[...hundreds of lines...]


ADDITIONS, IMPROVEMENTS AND CORRECTIONS

     Suggestions for additions, improvements and corrections to this message
are always welcome.  Please send them to INFO-CPM-REQUEST.  Additional
information concerning FTP is, however, beyond the scope of this message.  If
you need help with FTP, please read the documentation for your local system,
or see your friendly system wizard.

     Contributions of public domain software are actively solicited.  If you
have something that seems appropriate for inclusion in the <CPM> archive,
please contact Keith Petersen <W8SDZ@SIMTEL20>.  Likewise, contributions to the
<UNIX> archive can be addressed to Richard Conn <RCONN@SIMTEL20>.

     Happy hacking!

                                        Dave Towson
                                        INFO-CPM-REQUEST @ AMSAA
larsbrinkhoff commented 4 years ago

Also known as the .COM header:

'Way back when SIMTEL20 wasn't holding all that public domain software, MIT had the honor. And The Powers At Mit wanted some way to distinguish between MIT binary and microprocessor binary.

What evolved out of all this was the placement of a four character string at the beginning of each binary file. This string is "DSK8". And it is known as the .COM header. People who download files containing the .COM header must somehow strip off those first four bytes in order to have just the data file.

You probably downloaded the file with the .COM header. USQ, not knowing what what that was, gave you an error message and aborted. The second time you probably told MMODEM not to send the .COM header. And USQ then worked.

larsbrinkhoff commented 4 years ago

A quick grep through ToTS reveals some more DSK8 aware program: LMODEM, MMODEM, DSK8 (Lisp code).

larsbrinkhoff commented 3 years ago

This net.micro.cpm message marks the transfer of the CPM archive from MIT-MC to SIMTEL20.

https://usenetarchives.com/view.php?id=net.micro.cpm&mid=PDQ3NzBAc3JpLWFycGEuVVVDUD4%3D


07/09/1983 02:04:004 UTC
From: Frank J. Wancho

We are in the process of verifying the collection of public domain
files on MC have all been moved to the SIMTEL20. Therefore, we would
like those contributors to the collection to hold off uploading any
new files for a few days until we announce new directions for where to
put the files on the SIMTEL20. Please bear with us.
--Frank
larsbrinkhoff commented 3 years ago

Longer message by Frank J. Wancho about the CP/M archive, posted 4 September 1986 17:02:46.

https://groups.google.com/g/net.micro.cpm/c/rZz-dStwbh0/m/8CrsRwUQgQsJ


This message marks an historic moment, and a bit of background history
which has brought us to this moment is in order. Please bear with me
while you read this lead-in to details you have all been eagerly
awaiting at the end of this message.

Seven years ago this past Friday, August 29, 1979, INFO-CPM was formed
as a spin-off of INFO-MICRO, both homed at the time on MIT-MC. The
following month Keith Petersen's Royal Oak RCP/M system was
discovered, and Keith was invited to directly participate in the
INFO-CPM discussions. Through Keith's direct access, he was able to
upload, crudely at first, selected files for FTP from MIT-MC from his
vast up-to-date RCP/M collection. As new files became available,
Keith made the announcements to the INFO-CPM list. New contributions
and updates to existing files were likewise made available from other
ARPANET users. Thus, the CP/M archives was born, with disk space
courtesy of the MIT-MC management.

This activity caused the development of several mainframe versions of
CP/M utilities, such as the first mainframe implementation of the
Christensen Protocol, in MacLisp, by Ed Barton, called LMODEM. Then
Gail Zacharias developed MMODEM, USQ, HEXFIY, COMIFY, MAKLBR, DE-LBR
and others, some of which were modified for use on TOPS-20 systems.
Bill Westfield wrote the original and invaluable MODEM program for
TOPS-20, which has recently been overhauled into TMODEM.

Disk space was inherently limited on MIT-MC, and when the Macsyma
Consortium was dissolved at the end of September, 1983, SIMTEL20 at
White Sands Missile Range (WSMR) was already online and had disk space
to spare on the required 176MB RP06 boot disk. (SIMTEL20 is a
contraction of the name of the branch which then owned the machine,
SIMulation and TELeprocessing, DECSYSTEM-20.) During that month, the
CP/M collection was moved from MIT-MC to the RP06, designated as the
MICRO: structure, on SIMTEL20. Already purchased for a closely
related project, the then named DARCOM Microcomputer Software Sharing
System (DMSSS), were the entire sets of the CPMUG, SIG/M, and PC/BLUE
distributions, which were uploaded as-is to MICRO:. Since then we
managed to get placed on the tail end of the regional distribution of
both the SIG/M and PC/BLUE update sets which periodically extend both
of those collections.

As this was all going on, it became evident that a collection of
Unix/C versions of the CP/M utilities would be required and that
collection was started. That collection has since come under the
sponsorship of another organization within the Army Materiel Command
(AMC) to which WSMR belongs. That organization, Logistics Systems
Support Activity, LSSA, provided the funds for the 512MB disk drive
known as PD:, to which all the collections residing on the out-grown
MICRO: structure were moved in November, 1985.

In November, 1984, Rick Conn volunteered to start the extremely
popular Ada Software Repository, originally on PS: as there was no
room on MICRO:, and then moved to PD: when that device came online.

As with the CP/M archives in PD:<CPM.*> being considered the best and
most recent, the MSDOS archives are also in the process of being
similarly organized in PD:<MSDOS.*>, with files culled from the latest
releases of the PC/BLUE collection as well as the INFO-IBMPC
collections on ISI-B and Pete Galvin's collection on UTEXAS-20.

When the CP/M collection first started, there was only one directory
with subdirectories, and several not-necessarily CP/M-related
directories lived under that tree. Recently, a new top-level
directory was created on PD:, named PD:<MISC>. Now those generic
subdirectories in PD:<CPM.*>, PD:<UNIX.*>, and PD:<MSDOS.*> are being
moved to that new directory tree.

The original INFO-CPM list was maintained on MIT-MC until 1983 and
then moved to BRL, where it was maintained by Ron Natalie for a short
while. Ron, who has been the maintainer of INFO-MICRO, among other
lists, drafted Dave Towson to maintain the INFO-CPM list on AMSAA.
Dave has been continuously maintaining the list since November 1983,
periodically rewriting and distributing the SIMTEL20 Archive Users
Guide, commonly referred to as the "Archive Blurb" and answering
numerous user requests for information on access to the SIMTEL20
collections.

As the subscription list to INFO-CPM grew, it was gatewayed into the
USENET community, first as fa.info-cpm, and now as a two-way
newsgroup, net.micro.cpm. Later, members from other communities
joined the list from CSNET, BITNET, and others. Meanwhile, new files
and entire collections were added to the SIMTEL20 archives and
announced to the INFO-CPM list. Access to these files was inherently
limited to those with Internet FTP access to SIMTEL20, unless some
kind soul volunteered to mail selected files on request.

Recently, there was a flurry of messages pleading for some form of
automated access to the SIMTEL20 archives via net mail. About two
weeks ago, while reading these pleadings while on travel in the
mountains of Colorado, I said that all the pieces were falling into
place to be put together to provide that service. Maybe it was the
altitude that made me say that, but everything eventually did fall
into place in the two weeks since then.

A Mail File Server was written in C using a beta version of an about
to be released compiler for TOPS-20 systems. If it wasn't for the
quick turnaround on reported bug fixes by the two principal
maintainers of that compiler and run-time library, Ken Harrenstien and
Ian Mackey at SRI-NIC, this program could not have been developed in
such a short time by a person whose only other claim to fame in C
programming is one other C program he co-authored two years ago.

The original concept for this service came from a similar service
developed by Jack Dongarra of the Argonne National Laboratory and Eric
Grosse of AT&T Bell Laboratories called NetLib, which is available
from ANL-MCS for mail access into its collection of well-indexed and
documented high quality public domain mathematical software routines.
Their work was supported, in part, by the National Science Foundation.
From their sources came that part of the package which extracts the
requestor's return address from the request file.

The SIMTEL20 version of this service has been in beta test for almost
two weeks now, with new features added and many bugs fixed since then.
While credits are being handed out, I wish to thank Matt Kimmel for
checking out access from the BITNET side, and Eric Hildum from the
USENET side. And, to Keith Petersen, Dave Towson, Bernie Eiben, Rick
Conn, and Mark Crispin for participating in a lively discussion of the
principles of operation of this service.

Now, before I bombard you with details on how to access the files, let
me caution you that this system is experimental. There is no such
thing as finding the "last" bug in any program. Furthermore, I sit in
awe and fear that even selective and judicious use of this system by
the potential audience this message will eventually reach may overload
this machine or some of the fragile mail links between hosts on the
various networks connecting us all together.

This means that those of you already with FTP access to SIMTEL20 must
not use this service and continue using FTP. There is no blocking
mechanism in place right now, but I will consider taking the time to
install one if you choose to ignore this request. Those of you on
USENET, BITNET, and CSNET hosts should consider appointing one point
of contact through which you should funnel your requests and the
reconstructed files from the replies from this service should be made
available to all your local users. This particularly applies to the
HELP, INFO, and BOOTSTRAP messages and the files they point to.

This message is being sent only to the readers of INFO-CPM so that we
can gauge the impact on the system. Please do not redistribute this
message to any other mailing list or newsgroup. Their time will come.

What follows is the message you get in response to the SEND HELP
command...

Enjoy!
Frank
larsbrinkhoff commented 3 years ago

I have made a suggestion to MIT Libraries to make the ToTS copy of the CPM archive public,

FYI @johnsonjh