Open IvanExpert opened 8 years ago
knghtbrd
commented
8 years ago This is on my todo list. As is making it not even require Linux to do it! I want this on my Mac! (I realize I may need to modify some stuff to handle resource forks and ProDOS filetypes in a couple of different ways for different other tools you might use these things with.)
REALLY wish we could get some standardization on that!
IvanExpert
commented
8 years ago Yeah, I didn't really mean Linux, I just meant as a standalone tool as opposed to an integrated part of the A2SERVER installer. I doubt there's anything in there that makes it Linux-specific in a way that wouldn't run on a Mac, except possibly a couple of BSD vs GNUisms in tools like date and touch. Rewriting in Python (as I did for cppo) would be one way of addressing.
knghtbrd
commented
8 years ago Your use of sed -i -e ... was the one thing I noticed that breaks.
Compatible would be sed -i '' ... My Mac's user environment has
been heavily GNUified, but I may test stuff out on Katie's old
MacBook which runs 10.6 and isn't really suited to upgrading. If it
works there, it'll work on a current, up-to-date Mac.
Apparently 10.3 was "the most connectable" OS X release for Macs. I don't know if that still had LocalTalk support if your Mac had the serial ports for it or not. It is perhaps equivalent to A2SERVER's version of netatalk.
No objection to rewriting the installer in Python, though I'm not
100% sure about a few things I'd rather not break regarding reading
installation scripts and the like, and I admittedly don't understand
the "resource fork" used for ProDOS files (other than to say that I
know it's not a resource fork and cannot be treated as one when
interoperating between a real Mac and netatalk. If they could, I'd
be lobbying for netatalk to learn/prefer ._<filename> since Apple
has finally made that a de-facto standard for AppleDouble.
In fact, I note this should already be possible with the adouble option. Since the netatalk documentation page does not indicate what a few options do very well (like prodos for example, which I observe to mostly be about defining which volume to boot off of since I see no other visible changes and you apparently can have but one), there's some guesswork until I have time to go pore over the code a bit.
Speaking of AppleVolumes.default options, an IRC log snippet I don't want to have to go digging for again is:
2015-10-07 01:44:17 Tony /media/sdd1/Fishhead Fishhead options:prodos,limitsize,upriv usedots:invisibledots veto:/_/ perm:0777
2015-10-07 01:44:44 Tony Optimal netatalk AppleVolumes.default entry for a ProDOS share.
2015-10-07 01:45:07 Landon optimal for P8, too?
2015-10-07 01:45:17 Tony yes.I'm not 100% sure I agree with Tony on that based on my reading of the docs, but I can't argue with his experience. I see he uses neither casefold:toupper nor ea:ad. The latter probably gets chosen by default on a non-ro volume. I couldn't guess about casefold. I assume that P8 either can't see mixed case files (haven't tested) or you were concerned about mixed case issues with the installer.
Joseph
On Mon, Oct 26, 2015 at 06:35:13AM -0700, IvanExpert wrote:
Yeah, I didn't really mean Linux, I just meant as a standalone tool as opposed to an integrated part of the A2SERVER installer. I doubt there's anything in there that makes it Linux-specific in a way that wouldn't run on a Mac, except possibly a couple of BSD vs GNUisms in tools like date and touch. Rewriting in Python (as I did for cppo) would be one way of addressing.
Reply to this email directly or view it on GitHub: https://github.com/RasppleII/a2server/issues/31#issuecomment-151134748
IvanExpert
commented
8 years ago I think there's actually more than one use of GNU sed syntax that isn't available in BSD sed.
OS X 10.3 still offered AFP over AppleTalk, just like Netatalk and Classic Mac OS does, on Ethernet and Wi-Fi (but NOT LocalTalk, which is not supported in any version of OS X). 10.4 supported AFP over AppleTalk client connections but would only serve files over TCP; 10.5 retained an AppleTalk stack for printing only; 10.6 excised AppleTalk completely.
ea:ad I'm pretty sure I put in because because several of my tools (e.g. afpsync and cppo -ad) are dependent on having the netatalk AD method of storing files.
The guidance for casefold was from accrued wisdom in comp.sys.apple2. I can't remember what happens if you don't use it, but the issue is that ProDOS doesn't support lowercase at all, and GS/OS is case insensitive. Thus if GS/OS copies something with a lowercase name to Netatalk, and it doesn't get converted to uppercase by casefold, ProDOS won't be able to access it, which means some parts of GS/OS can't either (e.g. START.GS.OS I think). You can probably get away with not using casefold if you're only ever using ProDOS 8, because then you'll never have occasion to copy a lowercase filename onto the volume; or if you're using GS/OS but not network boot, no casefold is probably fine (even desirable).
This is one major behavior break between Netatalk and native AppleShare server; with a native AppleShare server, you get truly case insensitive behavior even on a shared ProDOS volume. GS/OS can do this with Netatalk by using the GSFILES volume, which doesn't have casefold, but I don't think the OS will run without all of its names converted to uppercase.
Anything you want to know about resource fork files I'm happy to explain. As for the resource fork, it's absolutely a resource fork, it's just a GS/OS rather than a Mac resource fork (when not split into component files on a foreign volume). When a ProDOS volume file has a fork, it's called an extended file. Mac OS won't know what to do with it, but GS/OS will. (ProDOS 8 won't even know what it is, however.) Both GS/OS and classic Mac OS with the ProDOS File System or File Exchange 2.x extensions will show an extended file on an ProDOS or AFP volume as a single file, just like Mac OS does. ProDOS 8 will too but will just yield I/O errors if you try to access it.
There are Apple tech docs explaining the installer script format, the ProDOS extended file format, and more related stuff. See: http://www.1000bit.it/support/manuali/apple/technotes/tn.0.html
knghtbrd
commented
8 years ago I think there's actually more than one use of GNU sed syntax that isn't available in BSD sed.
All the more reason to turn it into something that runs more places.
Ideally turning it into something that explains what its doning
better than regexes tend to. I had to go RTFM to figure out what
some of the netboot script's sed commands do, and I'd still almost
prefer to rewrite the functionality than translate those sed commands
to python or something.
OS X 10.3 still offered AFP over AppleTalk, just like Netatalk and Classic Mac OS does, on Ethernet and Wi-Fi (but NOT LocalTalk, which is not supported in any version of OS X). 10.4 supported AFP over AppleTalk client connections but would only serve files over TCP; 10.5 retained an AppleTalk stack for printing only; 10.6 excised AppleTalk completely.
Good to know. This is probably why 10.2 and 10.3 were described as most universal for bridging the generation gap. I assumed serial AppleTalk wouldn't be doable on any OS X, but I wasn't 100% sure if that remained true on hardware with real Zilog serial ports.
It crosses my mind that we could possibly get point to point AFP over a character stream. If so, it'd be possible to send it over a serial port, though at nothing close to the 230.4kbps LocalTalk runs at--not to any Apple // anyway. Part of me still wants to network a //c at some point. :D
ea:ad I'm pretty sure I put in because because several of my tools (e.g. afpsync and cppo -ad) are dependent on having the netatalk AD method of storing files.
I figured as much. And ea:ad is the fallback if your FS doesn't have
something you could stick a resource fork into. However netatalk 2
can store those forks using OS X ._<filename>, which would be my
preference provided that's also correct for a ProDOS filetype. It'd
be a little easier to make nulib2 and friends support that, too.
I'd recommend standardizing on Netatalk 2.2.4 rather than 2.2.5, which was the last 2.x release. 2.2.5 won't even compile out of the box on Debian and was reputed to have other issues. It seemed like sort of a surprise release after they were well into 3.x.
I'm game for that. If we need something from 2.2.5 for security or whatever, we can backport.
The guidance for casefold was from accrued wisdom in comp.sys.apple2. I can't remember what happens if you don't use it, but the issue is that ProDOS doesn't support lowercase at all, and GS/OS is case insensitive. Thus if GS/OS copies something with a lowercase name to Netatalk, and it doesn't get converted to uppercase by casefold, ProDOS won't be able to access it, which means some parts of GS/OS can't either (e.g. START.GS.OS I think). You can probably get away with not using casefold if you're only ever using ProDOS 8, because then you'll never have occasion to copy a lowercase filename onto the volume; or if you're using GS/OS but not network boot, no casefold is probably fine (even desirable).
Is that true for all versions of ProDOS? After all, if you specify lowercase filenames in ProDOS, it converts them to uppercase does it not? Again, if this is something that can be fixed in netatalk, I'd rather fix it there.
Ideally, I'd like to restore the case-retentive features of GS/OS without breaking ProDOS 8. It may not be trivial to do it, though.
This is one major behavior break between Netatalk and native AppleShare server; with a native AppleShare server, you get truly case insensitive behavior even on a shared ProDOS volume. GS/OS can do this with Netatalk by using the GSFILES volume, which doesn't have casefold, but I don't think the OS will run without all of its names converted to uppercase.
Anything you want to know about resource fork files I'm happy to explain. As for the resource fork, it's absolutely a resource fork, it's just a GS/OS rather than a Mac resource fork (when not split into component files on a foreign volume). When a file has a fork, it's called an extended file. ResEdit won't know what to do with it, but GS/OS will. Both GS/OS and classic Mac OS with the ProDOS File System or File Exchange 2.x extensions will show an extended file on an ProDOS or AFP volume as a single file.
My understanding is that ProDOS 8 cannot see any "extended" files
(with resource forks), but afptype explicitly creates a file in
.AppleDouble/ containing something other than a resource fork.
What I don't yet understand is that if this the proper AppleDouble
encoding of a ProDOS file's type info, or if it's a netatalk
extension to do it that way. (If the latter, how does the Mac handle
ProDOS type info?)
I figure it might use the Mac type/creator space and just include a magic number to indicate a ProDOS file instead of a Mac file, but I haven't really dug into the AppleDouble format nor even afptype to really figure out if I'm even in the right ballpark.
Joseph
IvanExpert
commented
8 years ago I had to go RTFM to figure out what some of the netboot script's sed commands do, and I'd still almost prefer to rewrite the functionality than translate those sed commands to python or something.
I can read regexes in my sleep, but if you wanna make it more legible and yield the same result, I have no objection.
It crosses my mind that we could possibly get point to point AFP over a character stream. If so, it'd be possible to send it over a serial port, though at nothing close to the 230.4kbps LocalTalk runs at--not to any Apple // anyway. Part of me still wants to network a //c at some point. :D
I'd like to see that too, but considering that the Workstation Card apparently required a coprocessor, and the IIc lacks decent hardware handshaking (from what I understand), and most of all there currently doesn't exist a ProDOS 8 AppleShare client that could talk to the IIc's serial port rather than the Workstation Card, I'd say you have a pretty heavy lift there.
ea:ad I'm pretty sure I put in because because several of my tools (e.g. afpsync and cppo -ad) are dependent on having the netatalk AD method of storing files.
I figured as much. And ea:ad is the fallback if your FS doesn't have something you could stick a resource fork into. However netatalk 2 can store those forks using OS X
._<filename>, which would be my preference provided that's also correct for a ProDOS filetype. It'd be a little easier to make nulib2 and friends support that, too.
Sure, using the OS X format is well worth investigating, and less clumsy to work with. What I don't know is whether the ._ format supports the same 741 byte header format that ea:ad does; if not, several tools would have to be carefully adapted.
The guidance for casefold was from accrued wisdom in comp.sys.apple2. I can't remember what happens if you don't use it, but the issue is that ProDOS doesn't support lowercase at all, and GS/OS is case insensitive. Thus if GS/OS copies something with a lowercase name to Netatalk, and it doesn't get converted to uppercase by casefold, ProDOS won't be able to access it, which means some parts of GS/OS can't either (e.g. START.GS.OS I think). You can probably get away with not using casefold if you're only ever using ProDOS 8, because then you'll never have occasion to copy a lowercase filename onto the volume; or if you're using GS/OS but not network boot, no casefold is probably fine (even desirable).
Is that true for all versions of ProDOS? After all, if you specify lowercase filenames in ProDOS, it converts them to uppercase does it not? Again, if this is something that can be fixed in netatalk, I'd rather fix it there.
All ProDOS 8, and I think GS System Software before GS/OS 5, behaves the same. It converts them to uppercase before it makes any request for the file on the disk. Netatalk will try to go get whatever it's requesting, so if the file has a lowercase name in Linux, it can't find it.
Ideally, I'd like to restore the case-retentive features of GS/OS without breaking ProDOS 8. It may not be trivial to do it, though.
Sure, I agree. The only way I could think of to do this would be to either to have Netatalk, if it fails to find a file, try it again in all caps before returning a file not found.
My understanding is that ProDOS 8 cannot see any "extended" files (with resource forks), but
afptypeexplicitly creates a file in.AppleDouble/containing something other than a resource fork. What I don't yet understand is that if this the proper AppleDouble encoding of a ProDOS file's type info, or if it's a netatalk extension to do it that way. (If the latter, how does the Mac handle ProDOS type info?)I figure it might use the Mac type/creator space and just include a magic number to indicate a ProDOS file instead of a Mac file, but I haven't really dug into the AppleDouble format nor even afptype to really figure out if I'm even in the right ballpark.
Correct, ProDOS 8 can't. Netatalk writes a 741-byte file header in .AppleDouble, followed by any resource fork data if there is one. The 741-byte file header contains all the file metdata including the ProDOS file type. It is a standard AppleDouble header in which Netatalk also writes some custom data into fields available for that. I found an email I wrote years ago that semi-documents it, and includes references. There's a specific format for indicating a ProDOS file typeand aux type written into the Mac file type field (even though AppleDouble defines fields for ProDOS types, Netatalk doesn't use it).
Here's that email describing the .AppleDouble file:
AppleDouble v2 reference: http://web.archive.org/web/20120316094420/http://users.phg-online.de/tk/netatalk/doc/Apple/v2/AppleSingle_AppleDouble.pdf
Netatalk CNID backend reference: http://netatalk.sourceforge.net/2.1/htmldocs/configuration.html#CNID-backends
It's possible to safely manipulate a file in the .AppleDouble folder of a directory to change the ProDOS Type and AuxType. A2SERVER's afptype tool does this. This makes it possible to introduce Apple II files from scratch on the Linux side, such as the required netboot configuration files.
File metainfo (and resource fork, if any) is stored in the corresponding file in .AppleDouble. I traced through these files in a hex editor against Apple's ADv2 spec and found they conform nicely. In the file, nine specific AFP data fields are defined, and then there are four netatalk fields for the CNID database info (device, inode, db-stamp, and CNID). You can see all this presented neatly by running 'apple_dump' (one of the included netatalk utillities) on a file inside .AppleDouble.
One notable thing in netatalk's AppleDouble implementation is that even though Apple defines a field defined specifically for ProDOS file access, type, and auxtype, it's empty. Instead, Netatalk uses the convention Apple uses in their ProDOS File System and PC Exchange extensions for Mac OS: ProDOS type and auxtype are stored in the Macintosh File Type as 'pXYY' where X is Type and YY is AuxType (big-endian), while creator is always 'pdos'.
In principle, this makes it possible to see the type with 'ad ls -l' (in the files folder, not inside .AppleDouble) though since the Mac file types display as characters, it's not very helpful. It should also be possible to change them using the 'achfile' except that utility actually writes the type and creator in the wrong place! It's apparently out of date; perhaps it is for AppleDouble v1.
The eight bytes for type/creator appear to be pretty consistently at offset $28D in the AppleDouble file, but rather than relying on that, you can actually figure it out by scanning the entry headers at the beginning of the file, looking for the one with type 9 (which corresponds to Finder Info), and then getting the offset from the bytes which immediately follow; this is what afptype does.
As for the database, AFP refers to files by ID, so netatalk keeps the CNID database to map ID's to files. Each entry in the CNID database stores the device and inode of the data fork and the corresponding AppleDouble file, so when AFP asks for file 25 or whatever, the database can say "oh that's this file".
If you bring in a file from elsewhere (that is, in the shell, via SMB, etc), there's not going to be a corresponding .AppleDouble file and the database won't know about it either, so you can run 'sudo dbd -r
The only catch as far as a "bare Debian" install goes is that for dbd to work, there needs to be a
knghtbrd
commented
8 years ago On Wed, Oct 28, 2015 at 05:00:58AM -0700, IvanExpert wrote:
It crosses my mind that we could possibly get point to point AFP over a character stream. If so, it'd be possible to send it over a serial port, though at nothing close to the 230.4kbps LocalTalk runs at--not to any Apple // anyway. Part of me still wants to network a //c at some point. :D
I'd like to see that too, but considering that the Workstation Card apparently required a coprocessor, and the IIc lacks decent hardware handshaking (from what I understand), and most of all there currently doesn't exist a ProDOS 8 AppleShare client that could talk to the IIc's serial port rather than the Workstation Card, I'd say you have a pretty heavy lift there.
My understanding is that the workstation card had to do a fair bit of heavy lifting for timing and buffering reasons more than anything else. LocalTalk is pretty fast by serial link standards, and definitely so by 8 bit CPU standards. The //c's flow control is a little wonky, but it does exist.
Whatever speed VSDRIVE operates at should be possible for the //c to do LocalTalk. Of course I understand that VSDRIVE can bog down a bit, especially if the disk is larger than a given size which is somewhat dependent upon your stable serial link speed.
Option #2 I can see for the //c would involve something like an AVR.
You could prototype this with an Arduino, an ethernet shield, and a
max3232 adapter. You'd have the AVR connect to the netatalk server
and do all of the standard operations there, and then rewrite the bit
of ProDOS needed to make it talk to the serial port instead of to the
Workstation card and make things talk back on the AVR side. Again,
it'd be slower than the //e and IIgs, but it'd be the most modern
AppleTalk implementation on any Apple.
AND, it'd be a solution that could be relatively easily adapted to make a new (ethernet-based) AppleTalk card for the //e and IIgs. If done "right" maybe Glenn could be dragged into this to make it the basis of an Uthernet III that provides both on one card.
Even if the Uthernet functionality couldn't be folded in, I'd be kind of interested in an Ethernet-based AppleTalk solution for the IIgs, especially for the ROM01 where you have to stupidly sacrifice two slots for serial.
Thinking out loud.
I figured as much. And ea:ad is the fallback if your FS doesn't have something you could stick a resource fork into. However netatalk 2 can store those forks using OS X
._<filename>, which would be my preference provided that's also correct for a ProDOS filetype. It'd be a little easier to make nulib2 and friends support that, too.Sure, using the OS X format is well worth investigating, and less clumsy to work with. What I don't know is whether the ._ format supports the same 741 byte header format that ea:ad does; if not, several tools would have to be carefully adapted.
I'm betting that 741 byte header is just AppleDouble metadata.
I find http://forensicswiki.org/wiki/AppleDouble_header_file but the header file needed appears to be a broken link at least at the moment.
AppleDouble v2 reference: http://web.archive.org/web/20120316094420/http://users.phg-online.de/tk/netatalk/doc/Apple/v2/AppleSingle_AppleDouble.pdf
Seems to be the same file as [this one][appledouble_pdf] I found earlier. Page 17 of that file says the magic number for all AppleDouble files is 0x00051607 and earlier it said I should look for a format like so:
| Field | Length |
|---|---|
| Magic number | 4 bytes |
| Version number | 4 bytes |
| Filler | 16 bytes |
| Number of entries | 2 bytes |
Followed by entry headers:
| Field | Length |
|---|---|
| Entry ID | 4 bytes |
| Offset | 4 bytes |
| Length | 4 bytes |
]appledouble_pdf]: http://kaiser-edv.de/documents/AppleSingle_AppleDouble.pdf
Hmm, /media/A2SHARED/A2FILES/SYSTEM/.AppleDouble/P8:
0000000 05 00 07 16 ; AppleDouble magic 0x05000716
0000004 02 00 00 00 ; AppleDouble version SHOULD BE 0x00020000
0000008 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ; filler
0000018 09 ; # entries
; entry 1
000001a 00 00 00 02 ; resource fork
000001d 00 00 00 e5
0000021 02 00 00 00
; entry 2
0000025 00 00 00 03 ; real name
0000029 00 00 00 b6
000002d 00 00 00 00 ; but zero length
; entry 3
0000031 00 00 00 04 ; Macintosh comment
0000035 00 00 00 b5
0000039 01 00 00 00
; entry 4
000003d 00 00 00 08 ; file datestamp info
0000041 00 00 00 7d
0000045 02 00 00 10
; entry 5
0000049 00 00 00 09 ; "Standard finder info"
000004d 00 00 00 8d
0000051 02 00 00 20
; entry 6
0000055 00 00 00 0b ; ProDOS file info
0000059 00 00 00 c1
000005d 02 00 00 08
; entry 7
0000061 00 00 00 0d ; AFP short name
0000065 00 00 00 b5
0000069 02 00 00 00
; entry 8
000006d 00 00 00 0e ; AFP file attributes
0000071 00 00 00 b1
0000075 02 00 00 04
; entry 9
0000079 00 00 00 0f ; AFP directory ID
0000081 00 00 00 ad
0000085 02 00 00 04
;
0000089 00 00 00 00 00 00 00 00 00 00 00
0000090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
0000270 00 00 00 00 00 00 00 00 00 00 00 00 f3 00 65 7c
0000280 f3 58 65 7c 80 58 00 00 80 00 00 00 70 00 00 ff
0000290 70 00 6f 64 00 73 00 00 00 00 00 00 00 00 00 00
00002a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00
00002b0 00 7c 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00002c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
00002e5That looks awfully similar to what's described in the PDF. But not the same as in troubling ways. ProDOS info is clearly not where the PDF says it should be, and just as clearly one of the entries is being overwritten with timestamp data according to the header.
shrug I dunno. That file was written by afptype. I may need to actually try to install GS/OS on to AppleTalk and see what netatalk creates all on its own.
All ProDOS 8, and I think GS System Software before GS/OS 5, behaves the same. It converts them to uppercase before it makes any request for the file on the disk. Netatalk will try to go get whatever it's requesting, so if the file has a lowercase name in Linux, it can't find it.
A flag for case insensitive globbing perhaps? This is somewhat expensive since any time you want to find a file, you have to read the directory structure component-by-component and case-insensitively search for it in that case. It's a little cheaper if we know we can somehow know the difference between clients that need it and those that don't.
Option #2 is for A2SHARED to be on a case-retentive-but-insensitive filesystem. Linux doesn't have many of those and the idea of using FAT32 annoys me for some reason. ;)
One notable thing in netatalk's AppleDouble implementation is that even though Apple defines a field defined specifically for ProDOS file access, type, and auxtype, it's empty. Instead, Netatalk uses the convention Apple uses in their ProDOS File System and PC Exchange extensions for Mac OS: ProDOS type and auxtype are stored in the Macintosh File Type as 'pXYY' where X is Type and YY is AuxType (big-endian), while creator is always 'pdos'.
I thought it might be done that way, but the AppleDouble docs say that's not how its done, and the file written for P8 I dumped above doesn't make any sense at all.
I really wonder if nobody but Apple (or nobody including Apple) is actually following their own spec. Either way, very little of that AppleDouble header should be actually necessary for a ProDOS file.
If you bring in a file from elsewhere (that is, in the shell, via SMB, etc), there's not going to be a corresponding .AppleDouble file and the database won't know about it either, so you can run 'sudo dbd -r
' and it will solve both of those problems by adding an entry and creating the .AppleDouble file as needed. A2SERVER's "afpsync" utility is a front-end to this. So you need to run that first so you can have a .AppleDouble file to manipulate (it will have zeroes where the type/creator field goes initially).
It seemed like you ran afpsync before afptype in your scripts, I'd wondered about that.
It also seemed that the first time I tried to open a .shk file, Finder didn't know how. Then I ran GSHK and it suddenly did after that. Dunno if that's AFP-related at all.
Joseph
knghtbrd
commented
8 years ago @IvanExpert it seems you recently asked me if you'd sent me some info on the installer scripts. You hadn't. Could you please post it here or send it to me and I can figure out the best way to post it is? Thanks.
IvanExpert
commented
8 years ago On Nov 15, 2015, at 8:03 PM, Joseph Carter notifications@github.com wrote:
@IvanExpert https://github.com/IvanExpert it seems you recently asked me if you'd sent me some info on the installer scripts. You hadn't. Could you please post it here or send it to me and I can figure out the best way to post it is? Thanks.
Reply to this email directly or view it on GitHub https://github.com/RasppleII/a2server/issues/31#issuecomment-156883084.
Spec I followed is here:
http://www.1000bit.it/support/manuali/apple/technotes/iigs/tn.iigs.064.htm
knghtbrd
commented
8 years ago Thanks, I'll have a look. Partly unrelated: I've just committed patches to ProDOS 8 (everywhere it lives in the NetBoot files) for the year table. I can't test the //e boot block because I don't have a //e with a workstation card or a //e card and an early Mac LC. Can you verify I basically didn't screw it up? :)
If I didn't, I'll add it to the change log.
IvanExpert
commented
8 years ago Not immediately, but yes. Bear in mind that a IIe Card in an LC actually loads its Boot Blocks from local software on the Mac, so it doesn't precisely simulate the card.
Year table is a good idea, but I'm a little anxious about it, only because then at this point we're no longer providing standard operating system software, which is one of my strong design goals. Is the year table change something widely understood and accepted? Is it documented? I don't even know specifically what it does, or what its consequences are and I've never used it before in any context. My strong preference would be to make this be an opt-in install option, rather than something everyone silently gets, so that those of us who'd rather have vanilla Apple provided OS can get it.
Thinking we need a working document that articulates design goals. Being able to install an Apple-standard OS is one of them for me.
On Nov 16, 2015, at 5:24 AM, Joseph Carter notifications@github.com wrote:
Thanks, I'll have a look. Partly unrelated: I've just committed patches to ProDOS 8 (everywhere it lives in the NetBoot files) for the year table. I can't test the //e boot block because I don't have a //e with a workstation card or a //e card and an early Mac LC. Can you verify I basically didn't screw it up? :)
If I didn't, I'll add it to the change log.
— Reply to this email directly or view it on GitHub https://github.com/RasppleII/a2server/issues/31#issuecomment-156980880.
knghtbrd
commented
8 years ago There's this:
http://www.techtalkz.com/apple/435946-prodos-patch.html
Perhaps not the best documentation I've found, but it did lead me to ProDOS tech note #28: ProDOS dates--2000 and beyond as the last published tech note covering dates. The tech note may not have been written--but Geoff Body points out that CLOCK.PATCH can be found on SystemTools2 in GS/OS 6.0.1, so patching the year table is something Apple expects to be periodically done.
Of course you can't run the program on the netboot files because they aren't mounted, and that BASIC program can't be run on the boot block files because it has no idea what offsets to use.
Patching the splash screen date was not necessary. My thought there is that the fewer functionally identical files with different SHA1 digests out there.
There's this, shared by Eric Rucker in response to me asking for info about the clock patches:
http://mirrors.apple2.org.za/ground.icaen.uiowa.edu/MiscInfo/Prodos/p8date.patch
Of course the table patched in 6.0.3 was done at compile time.
IvanExpert
commented
8 years ago Ok, so I've done a little research on this to figure out what the most appropriate A2SERVER behavior should be, and here's what appears to me to be the case.
The year tables are ONLY relevant to users of a Thunderclock (or compatible) card on ProDOS 8, not GS/OS. That's the only card ProDOS provides a driver for, and it doesn't return year information, only day of week and date, so the lookup table is what ProDOS uses to determine the year based on that combo.
In a leap year, the year/date combo would correspond to a day later in Mar-Dec, so leap years are entered twice, adjacently; this means that only six years may be represented in the table, or five if the range incorporates two leap years. If the range starts in a year before a leap year, the second leap year is broken in Mar-Dec, because there isn't an additional byte to represent it twice. The program CLOCK.PATCH on GS/OS SystemTools2 will calcluate the year table and patch PRODOS.
For A2SERVER's purposes, patching the year table in the (net)boot blocks is only of benefit to real IIe users with both a Workstation Card and a Thunderclock (or compatible) card, and possibly users of the IIe Card for Mac, since that can emulate both. (I'm not aware of any software-only emulator that emulates the Workstation Card.) Other clock cards, including the IIgs clock, provide the year, and patch ProDOS with their own driver, so the year table is irrelevant for those.
So, this is probably not life and death, and there's probably little point to even GS/OS 6.0.2 and 6.0.3 having a patched P8 file. With that said, there's also obviously no reason not perform the patch, so I'm going to revert the behavior to as you had it, Joseph, which is have it happen automatically without prompting. However, I'm going to port the CLOCK.PATCH program logic into the script, so it can calculate based on the current date, especially since the 6.0.2/6.0.3 year table is wrong for 2016 Mar-Dec.
Of greater mystery is the 9-byte range at offset $26 (38) in ProDOS 2.0.3, which has also been changed in GS/OS 6.0.2, and again differently in 6.0.3. This appears to have been done by the authors of those -- it's not documented by Apple as far as I can tell, unlike the year table -- but I don't know what it does.
2.0.3/6.0.1: (1993-1998) 2026: b0 b6 ad cd e1 f9 ad b9 b3 2F76: 60 5F 5E 5D 62 61 60
6.0.2: (2013-2018, with broken leap year (2016 Mar-Dec -> 2011) 2026: b2 b2 ad ca f5 ee ad b1 b5 2F76: 12 11 0b 10 0f 0e 0d
6.0.3: (2013-2018, with broken leap year (2016 Mar-Dec -> 2011) 2026: b0 b2 ad c1 f5 e7 ad b1 b5 2F76: 12 11 0b 10 0f 0e 0d
knghtbrd
commented
8 years ago If memory serves, I looked at this. If memory serves, you'll find that p8 on disk7 is different from that released with 6.0.1/2.0.3 in exactly that spot and nowhere else. If you strip off bit 7, the cause will become apparent.
When ProDOS is built, the current date for the splash is written in that location. The p8 build on disk7 is different than that which went into the 6.0.1 release in terms of the date it was built.
I set it to match 6.0.3 in my build when patching the year tables because the code I wrote in python to patch the files tests md5 hashes of the files before and after and I wanted to reduce the number of hashes my code had to recognize as being valid.
Patching the date on the splash screen affects nothing, it's just written to the screen when p8 starts and farts, but not after.
Joseph
On Fri, Jan 08, 2016 at 05:14:00AM -0800, IvanExpert wrote:
Ok, so I've done a little research on this to figure out what the most appropriate A2SERVER behavior should be, and here's what appears to me to be the case.
The year tables are ONLY relevant to users of a Thunderclock (or compatible) card on ProDOS 8, not GS/OS. That's the only card ProDOS provides a driver for, and it doesn't return year information, only day of week and date, so the lookup table is what ProDOS uses to determine the year based on that combo.
In a leap year, the year/date combo would correspond to a day later in Mar-Dec, so leap years are entered twice, adjacently; this means that only six years may be represented in the table, or five if the range incorporates two leap years. If the range starts in a year before a leap year, the second leap year is broken in Mar-Dec, because there isn't an additional byte to represent it twice. The program CLOCK.PATCH on GS/OS SystemTools2 will calcluate the year table and patch PRODOS.
For A2SERVER's purposes, patching the year table in the (net)boot blocks is only of benefit to real IIe users with both a Workstation Card and a Thunderclock (or compatible) card, and possibly users of the IIe Card for Mac, since that can emulate both. (I'm not aware of any software-only emulator that emulates the Workstation Card.) Other clock cards, including the IIgs clock, provide the year, and patch ProDOS with their own driver, so the year table is irrelevant for those.
So, this is probably not life and death, and there's probably little point to even GS/OS 6.0.2 and 6.0.3 having a patched P8 file. With that said, there's also obviously no reason not perform the patch, so I'm going to revert the behavior to as you had it, Joseph, which is have it happen automatically without prompting. However, I'm going to port the CLOCK.PATCH program logic into the script, so it can calculate based on the current date, especially since the 6.0.2/6.0.3 year table is wrong for 2016 Mar-Dec.
Of greater mystery is the 9-byte range at offset $26 (38) in ProDOS 2.0.3, which has also been changed in GS/OS 6.0.2, and again differently in 6.0.3. This appears to have been done by the authors of those -- it's not documented by Apple as far as I can tell, unlike the year table -- but I don't know what it does.
2.0.3/6.0.1: (1993-1998) 2026: b0 b6 ad cd e1 f9 ad b9 b3 2F76: 60 5F 5E 5D 62 61 60
6.0.2: (2013-2018, with broken leap year (2016 Mar-Dec -> 2011) 2026: b2 b2 ad ca f5 ee ad b1 b5 2F76: 12 11 0b 10 0f 0e 0d
6.0.3: (2013-2018, with broken leap year (2016 Mar-Dec -> 2011) 2026: b0 b2 ad c1 f5 e7 ad b1 b5 2F76: 12 11 0b 10 0f 0e 0d
IvanExpert
commented
8 years ago I just checked it and I see what you mean! Ok, cool. Thanks for explaining.
So I think, in the end, that a2server-setup should silently patch P8 and the boot blocks, as you had it; there's no actual reason to leave it as it ships, and I'm not sure how many people will even see it anyway.
My inclination is to implement the logic of CLOCK.PATCH in the script so that a) it will be accurate even if it's run in 2019, and b) it fixes the broken 2016 leap year. (That kind of bothers me.) Do you agree?
And, if so, is it preferable to you that the script also update the date on the splash screen, or should I leave it as it is (02-Aug-15)?
knghtbrd
commented
8 years ago If 6.0.3 is broken, then yeah, patch it. You can update the year or not. If you're going to patch 6.0.3 as well, there's no need in particular to patch the splash date since I'll already have multiple versions to hash.
Joseph
On Fri, Jan 08, 2016 at 06:10:25AM -0800, IvanExpert wrote:
I just checked it and I see what you mean! Ok, cool. Thanks for explaining.
So I think, in the end, that a2server-setup should silently patch P8 and the boot blocks, as you had it; there's no actual reason to leave it as it ships, and I'm not sure how many people will even see it anyway.
My inclination is to implement the logic of CLOCK.PATCH in the script so that a) it will be accurate even if it's run in 2019, and b) it fixes the broken 2016 leap year? (That kind of bothers me.) Do you agree?
And, if so, is it preferable to you that the script also update the date on the splash screen, or should I leave it as it is (02-Aug-15)?
Reply to this email directly or view it on GitHub: https://github.com/RasppleII/a2server/issues/31#issuecomment-170012953
IvanExpert
commented
8 years ago Interestingly, I just discovered that "p8" on "Disk 7", while it contains ProDOS 2.0.3, has a splash screen stamp of 02-Apr-93, unlike the 06-May-03 that's in GS/OS 6.0.1 and Apple II System 4.0.2.
knghtbrd
commented
8 years ago That's what I was saying. :)
Joseph Sent via mobile
On Jan 10, 2016, at 03:56, IvanExpert notifications@github.com wrote:
Interestingly, I just discovered that "p8" on "Disk 7", while it contains ProDOS 2.0.3, has a splash screen stamp of 02-Apr-93, unlike the 06-May-03 that's in GS/OS 6.0.1 and Apple II System 4.0.2.
— Reply to this email directly or view it on GitHub.
knghtbrd
commented
7 years ago This may not be a neat Python tool by then, but breaking the install script tool out of the netboot script is slated for 2.0.0.
In other words, break out the guts of what's already in a2server-5-netboot for general purposes.