Reintroduce AppleTalk / DDP support

[rdmark]

To investigate the feasibility and break down the requirements for reintroducing the AppleTalk into the main Netatalk development branch.

Background

AppleTalk / DDP and associated daemons and tooling were deprecated with Netatalk 3.0, in order to streamline the codebase and making it more maintainable / extendable. Fast forward a few years, and AFP over TCP has become a legacy protocol with Apple switching to SMB as the file sharing protocol in macOS since Maverick. This changes the equation for Netatalk in two notable ways: 1, no need to continuously iterate on the capabilities as Apple adds new functionality, and 2, the primary value of Netatalk becomes to support vintage Mac networks. In this context, reinstating AppleTalk makes sense.

Solutions

Two approaches can be considered

1. Add atalkd code back to Netatalk 3.1
2. Backport all Netatalk 3.1 improvements to Netatalk 2.2

Pros with option 1

• Retains all improvements and features in 3.1 by default (AFP 3.4, OSX style EA, Spotlight, ini style config files etc.)

Cons with option 1

• Unknown how compatible atalkd 2.2 is with afpd 3.1
• Parts of the codebase changed significantly since 2.2 so not all original deprecation changes can be simply reverted

Pros with option 2

• AFP over TCP and AppleTalk already works in harmony in 2.2

Cons with option 2

• A decade worth of code changes to backport

Requirements

• Restore module code
  • atalkd
  • papd
  • timelord
  • a2boot
  • AT protocols in libatalk
  • AT tools
• Reimplement branches for AT and older AFP/AD versions in afpd and other modules
• Update autotool hooks
• Update Makefiles
• Update init scripts
• Update documentation

[rdmark]

In the case of option 1, I went through the 2.x code tree and picked out modules that I think should be brought over, and a few that I think are safe to discard. Just to get an idea of the scope of effort.

Save

• bin/aecho/** -- AppleTalk ping
• bin/getzones/** -- AppleTalk zone management
• bin/nbp/** -- AppleTalk service management
• bin/pap/** -- AppleTalk print management
• config/atalkd.conf
• config/papd.conf
• contrib/a2boot/** -- Apple II netboot server
• contrib/timelord/** -- AppleTalk time server
• etc/atalkd/**
• etc/papd/**
• include/atalk/** -- AppleTalk related headers
• libatalk/asp/** -- part of the AppleTalk protocol family
• libatalk/atp/** -- part of the AppleTalk protocol family
• libatalk/nbp/** -- part of the AppleTalk protocol family
• libatalk/netddp/** -- part of the AppleTalk protocol family
• sys/netatalk/** -- native AppleTalk headers etc.

Optional

• bin/adv1tov2/** -- AppleDouble v1 is a way way outdated format.
• bin/megatron/** -- Classic Mac OS file format tooling. Out of place for Netatalk?
• bin/psorder/** -- ancient postscript tooling
• contrib/printing/** -- ancient postscript tooling
• contrib/shell_utils/lp2pap.tmpl -- ancient printer tooling
• etc/papd/showppd.c -- Out of place for Netatalk?
• etc/psf/** -- ancient postscript tooling

edit: updated for the state of v2.4

[ghost]

Hi @rdmark, you're doing some great work reviving Netatalk and giving a future for Vintage Mac networks. I have created a fork that allows networking and file sharing between vintage Macs running Mac OS 9 and modern Intel/Apple Silicon Macs that do not support AFP over TCP. Feel free to implement any changes from there that you see fit :) https://github.com/dgsga/netatalk

[rdmark]

@dgsga Well thank YOU for doing the same, really! This is truly a community effort, keeping Netatalk alive. If there is any clear cut improvement that you think would be helpful upstream, don't hesitate to throw a PR this way! You may be the best person to judge whether something is generally useful, or specific to macOS compatibility.

[rdmark]

Glancing through the main branch commit log in 2011 - 2012: two things that may need more plumbing to restore are encodings (conversion from classic Mac code pages), and support for AFP 2.

The deprecation of DDP seems to have started in earnest with https://github.com/Netatalk/netatalk/commit/d5945415631bb4568400fc55e3095259938c979d

This is a good starting point I think. https://github.com/Netatalk/netatalk/commits/main?after=f76299185252a7505e504e102d157ecd69b437da+1434&branch=main&qualified_name=refs%2Fheads%2Fmain

[NJRoadfan]

Option 1 likely makes sense from a code maintainability point of view.

The biggest feature from that I would like to see from Netatalk 3.x is using OSX style EAs. This is key if you intend on sharing the same folders under Samba. Right now using Netatalk 2.x and accessing the same shares via SMB on macOS is going to lead to a ton of headaches and lost resource forks.

Moving to INI style config scripts for all the "legacy" services (mainly papd, and atalkd) would likely be the biggest hurdle. A lot of legacy AFP 2.x protocol support is still present in the 3.x tree.

[rdmark]

Additional requirement:

Forward-porting support for AppleTalk-only AFP protocol versions. There are branches sprinkled across the codebase that checks for AFP version number and returns different data.

	AFP 1.1	AFP 2.0	AFP 2.1
Version String	AFPVersion 1.1	AFPVersion 2.0	AFPVersion 2.1
Client (roughly)	early Mac?	Apple II, System 6.0	System 7.0, 7.1
Server	early personal file sharing?	AppleShare 2.0	AppleShare 3.0

Edit: Updated for accuracy

[NJRoadfan]

Apple II support (namely the ProDOS attribute support) was introduced in AFP 2.0. Both the Apple IIgs and IIe Workstation clients are AFP 2.0. "Inside AppleTalk" goes into the differences between 1.1 and 2.0, which aren't all that different besides the Apple II support.

[rdmark]

First roadblock encountered: This commit https://github.com/Netatalk/netatalk/commit/fb751e80c0bc12c35675fbf1434ceecb4aad47cb removed support for ASP and PAP child process handling, rewriting it as AFP-only. Very mildly used in asp_getsess.c ...

[NJRoadfan]

Obsolete volume options (prodos, size limit, etc.) need to be added back in and reworked for the new INI configuration system. These options were removed in this commit https://github.com/Netatalk/netatalk/commit/863882cb7af7d0bd2651f957b155fd5aa9e2d9d6

Only some of these are actually important. The automatic CR/LF handling can be ignored.

[rdmark]

@NJRoadfan Thanks for the suggestions. We don't need mswindows either I believe. I'm pretty sure it was part of the ancient "SFM" compatibility layer via a modified Samba version lost to time.

[NJRoadfan]

The mswindows options appears to be for clients. DOS and Windows AFP clients can't handle several characters in file names that are legal on Macintosh.

[NJRoadfan]

Additional Note on volumes: Netatalk 3.x no longer creates several "special folders", likely because they are not used by OS X. These are TheVolumeSettingsFolder, TheFindByContentFolder, "Temporary Items", and "Network Trash". None of these seem to work properly on netatalk 2.x anyway due to permissions issues (they are created as owned by root by default).

[rdmark]

All of the relevant code has been committed to main now, with scaffolding for the build system set up. The code does not yet compile, mainly because of the aforementioned child process handling, so AppleTalk is disabled by default in the build system.

Further work has been branched off into new tickets with the appletalk issue label.