Draft Release - ELKS v0.8.0

ghaerr commented 1 day ago

It's been over a year since ELKS v0.7.0 was released on August 3, 2023. Following are draft release notes for v0.8.0, please comment if something or someone has been left out. Thank you to all that have contributed and are using ELKS!

Draft Release - ELKS v0.8.0

ELKS v0.8.0 Downloads

Prebuilt disk images for floppy and hard disks are downloadable below from Github Assets for this release. These images can be used to boot from PC/XT/AT or PC-98 hardware, or emulators, such as QEMU or dosbox-x. The images are built with network support by default, supporting NE2K, WD and 3Com NICs. See the Wiki HowTo for more information.

ELKS v0.8.0 Enhancements

Thank you to all who contributed to ELKS enhancements, bug identification, testing and fixes. The system continues to be very stable, with no known crashes and long uptimes.

Major Enhancements

Major additions include support for:

Application development using the Open Watcom C compiler and OS/2 executable format, allowing for large multisegment programs to run, using all available memory if necessary. Doom was recently ported.
Add NE2K, WD and 3COM networking support to default build.
Compressed executables by default allowing for many more applications per floppy image.
Asynchronous I/O was added to the kernel along with @Mellvik's Direct Floppy disk driver, which allows for other applications to run while waiting for floppy disk I/O.
All kernel resources dynamically adjustable at boot, allowing for the kernel heap, tasks, inode and file allocations to be adjusted as required for min or max memory usage.
Increased symbolic debugging capabilities with enhancements to the symbolic disassembler, system call and function tracing capabilities within running applications.
Rewrite of L1/L2 buffer cache operation allowing for faster operation using fewer buffers.
Precision timing routines and removal of most all long division in the kernel and C library for fastest operation and better elapsed time measurements.

New Commands

[cmds] Port ttyclock to ELKS
[cmds] Port ttypong to ELKS
[cmds] Port ttytetris to ELKS
[cmds] Port md5 from Minix/NetBSD to ELKS as md5sum by @ccoffing
[kernel,cmds] Add sysctl system call and option get/set command

Updated Commands

[cmds] Add "mouse r" for raw mouse output debugging
[cmds] Add -s, -r, and -p options to shutdown, remove reboot, poweroff
[cmds] Add accurate train timing to sl
[cmds] Add decomp - ELKS executable decompressor
[cmds] Add fsck-dos FAT filesystem checker to build
[cmds] Changes to basic.c for OpenWatcom C compilation
[cmds] Cleanup fsck-dos messages and reduce memory requirement
[cmds] Display usage on invalid mount -t parameter
[cmds] Enable more to recognize line wrapping
[cmds] Enhance ps to better recognize task struct size mismatches
[cmds] Finish fsck-dos port to ELKS, add to distribution
[cmds] Fix ELKS_VERSION= on application builds
[cmds] Fix man page reader for filenames > 14 (MINIX max)
[cmds] Fix more to display correct number of lines w/man
[cmds] Fix off-by-one bug in getty @T command by @vkoskiv
[cmds] Fix open file on spawn of programs from /etc/inittab
[cmds] Fix rmdir -p
[cmds] Get ttyclock working with keyboard commands
[cmds] Improve fsck output formatting for -lvv and -lvvv options
[cmds] Meminfo and ps updates, small fixes
[cmds] Reduce size of ps to 6k
[cmds] Reduce size of ramdisk app, fix error messages
[cmds] Remove fsck-dos FAT32 support allowing it to work on floppies
[cmds] Reorganize clock.c source code, add AST hardware RTC support
[cmds] Replace ifdef ia16 with ifdef _M_I86
[cmds] Replace use of non-standard malloc.h header with stdlib.h
[cmds] Rewrite kill to reduce size
[cmds] Show usage on bad options to mount command
[cmds] Speed up 'sl' - let her rip full speed
[cmds] Speed up the sl choo choo train for slow systems
[cmds] Unmount all volumes in shutdown & reboot by @vkoskiv
[cmds] Update fsck to work with max size 64Mb HD, enhance verbose output
[cmds] Update sys command with new device numbers
[cmds] meminfo enhancements and man page
[nx] Adding monochrome CGA nano-X driver. Modifying nxtetris parameter for CGA. by @tyama501
[disasm] Fix sign extension bug when disassembling large RAM segment offsets
screen: modernize prototypes by @ccoffing
lp and misc cleanups by @ccoffing
Printing by @ccoffing

Networking

[net] Fix connection hang on lost SYN packet
[net] Temp fix ftpd startup with QEMU=1 set

Boot and Runtime Configuration

[config] Add CONFIG_ASYNCIO config option, cleanup INIT_REQUEST
[config] Build distribution images using CONFIG_APPS_xxx image size
[config] Do not read MK_ARCH & MK_CPU if already defined by @and3rson
[config] Fix duplicate CONFIG_EXEC_MMODEL=y output
[config] Update config files from recent changes
[config] Update configurations for standard IBM PC build
[boot] Add enhanced debug display to setup.S for early startup debugging
[boot] Add media descriptor 0xfe and boot signature in image FD1232 for PC-98 by @tyama501
[boot] Allow separate EXT/L2 and XMS buffer overrides to compiled configuration
[boot] Fix boot issues when /linux or /bootopts deleted
[boot] Remove /bootopts requirement for booting from direct floppy driver
[boot] Replace panic with printk when bootopts argv/envp overflow
[bootopts] Add heap= option for dynamic sizing of kernel heap

PC-98 Port

[bioshd] 720KB 2DD support for PC-98 2HD/2DD common drive by @tyama501
[menuconfig] Fix PC98-1232k typo that makes error by @tyama501
[pc98_config] Fix typo of capital Y by @tyama501

Kernel Enhancements and Fixes

[kernel,build] Small L1 buffer enhancement, add man4, cleanups
[kernel,cmds] Fix tty SIGINT not handled when using sercat
[kernel,cmds] Small fixes to ps, mkfs and ramdisk driver
[kernel,docs] Small updates, process format doc, check odd address in structs, names
[kernel,libc] Enhance debug output in kernel and printf %p
[kernel,libc] Rename u_ino_t to ino_t
[kernel/libc] Allow for concurrent operation of libc and kernel precision timer
[kernel] Add %P to printk to display process ID
[kernel] Add @Mellvik's direct floppy driver
[kernel] Add DF driver to standard IBMPC build
[kernel] Add capability to load and execute OS/2 binary executables
[kernel] Add char and network driver init routines to INITPROC
[kernel] Add consistency checks to buffer I/O, kernel stacks and scheduler
[kernel] Add kernel and user mode divide by zero handling
[kernel] Add optional display of kernel buffer status
[kernel] Add physaddr 64k I/O overlap checking to BIOS driver
[kernel] Add precision time measurement routines with printk formatting
[kernel] Add working async I/O for SSD driver
[kernel] Automate detection of QEMU for ftp, ftpd and DF driver
[kernel] Calculate then reduce kernel temp startup stack size
[kernel] Change bioshd minor numbering scheme to conserve kernel data size
[kernel] Cleanup /dev/mem source code
[kernel] Cleanup USE_GETBLK source
[kernel] Cleanup and shorten panic messages to save kernel data space
[kernel] Cleanup basic types
[kernel] Cleanup block driver blk_size code
[kernel] Cleanup ifdefs and unused code in direct floppy driver
[kernel] Cleanup inode code and fix inode count on execve
[kernel] Cleanup register decls in console*.c, cleanup init/main.c, strace
[kernel] Cleanup various casts and types in kernel source
[kernel] Cleanup various kernel sources
[kernel] Consolidate reboot, shutdown and poweroff into single program
[kernel] Determine kernel stack usage when allowing async I/O
[kernel] Disable track caching for hard drives
[kernel] Disallow reschedule during kernel startup
[kernel] Display number of tasks, files and inodes at boot
[kernel] Don't allocate IRQ trampolines from heap
[kernel] Don't lock Z-map and I-map buffers on MINIX fs mounts
[kernel] Don't mark inode dirty in fchown unless uid or gid changed, cleanup
[kernel] Don't use map_buffer for memset on new filesystem blocks
[kernel] Dynamically allocate L1 cache, use /bootopts cache= to override
[kernel] Dynamically allocate inode and file table from bootopts
[kernel] Dynamically allocate task array from bootopts
[kernel] Eliminate compiler warnings during kernel compilation
[kernel] Eliminate tabs from some kernel source files
[kernel] Ensure all words aligned on word boundaries
[kernel] Finalize precision timing routines
[kernel] First pass at adding media change support to kernel
[kernel] Fix /bootopts root=df0 failing
[kernel] Fix MINIX fs indirect block allocation
[kernel] Fix crash when accessing 15+ character filename on FAT
[kernel] Fix kernel crash calling get_ide_data
[kernel] Fix kernel data overwrite bug while parsing /bootopts
[kernel] Fix sys_rename inode count bug on FAT filesystems
[kernel] Fix to allow booting 2.88M floppy on dosbox
[kernel] Implement full track caching and fix track retry error
[kernel] Implement setitimer system call
[kernel] Improve ^N system inode status listing
[kernel] Increase parsed ANSI sequence length to 28
[kernel] Invalidate L1 buffers properly, update block drivers to sync
[kernel] Kernel buffers tune and cleanup
[kernel] Minor reformatting for readability
[kernel] Misc cleanups
[kernel] More cleanup of ifdefs
[kernel] More source reformatting for maintainability
[kernel] More source reformatting, slightly refactor kdev_t.h
[kernel] Move bios_disk_park_all to be PFPROC function
[kernel] Move hd/fd bios driver routines into far BFPROC to free up .text space
[kernel] Prohibit reading from /dev/ssd until initialized
[kernel] Rearrange/cleanup types.h and arch/types.h
[kernel] Reduce precision time routines single routine
[kernel] Reduce size of system buffer headers by 4-6 bytes
[kernel] Refactor bioshd driver in preparation for other block devices
[kernel] Refactor debug events, add display inode table with ^N
[kernel] Release .fartext.init code section to free memory after init
[kernel] Release /bootopts parsing and kernel init_task code after init
[kernel] Release L1 buffers in get_free_buf on second pass
[kernel] Remove CONFIG_APM
[kernel] Remove dedicated buffer for MINIX super block
[kernel] Remove include config.h from types.h
[kernel] Replace CURRENT_TIME macro with function call
[kernel] Replace atoi in ANSI console with very fast version
[kernel] Replace kdevname() with printk %D
[kernel] Retab some source files - no functional changes
[kernel] Retab various kernel source files
[kernel] Rewrite FAT filesystem file read/write to not use L1 buffers
[kernel] Rewrite block requests to use start sector and count rather than block number
[kernel] Rewrite printk for speed, add __divmod fast 32/16-bit divide
[kernel] Rewrite wait_on_buffer for async I/O, trace fixes
[kernel] Show open filenames with ^N kernel inode listing
[kernel] Slightly reorganize boot and kernel messages
[kernel] Small cleanups, add disasm to 1440k distribution
[kernel] Small code cleanups
[kernel] Source cleanup replace #if 0 with #if UNUSED
[kernel] Source code readability and optimization improvements
[kernel] Speed up TTY output processing
[kernel] Support Toshiba T1100 BIOS when INT 13 returns 0 floppies
[kernel] Update examples for precision timer
[kernel] Update linker script to place INITPROC sections at highest addresses
[kernel] Updates to console code from #1980
[kernel] Use BIOS Data Area to determine EGA/CGA/MDA for cursor on/off
[kernel] Use MAJOR_NR in blk.h to select block device
[kernel] Use early temp stack to avoid irqit stack overruns on faults during startup
[kernel] Use outb directly for DMA programming in DF driver
[kernel][libc][cmds] Add devname libc function
[direct floppy] Assume 360k floppy if no CMOS drives set
[direct floppy] Fix block driver media size after probe
[direct floppy] Fix problem with QEMU using 360k floppies on 1.2M drives
[direct floppy] Ifdef clear DIR media change code out of normal compilation
[direct floppy] Improve auto probing
[direct floppy] Introduce more than two floppy probes per CMOS type
[direct floppy] Small updates to DF driver
[direct floppy] Validation of floppy drive format tables
[directfd] Add support for 2880k discs, multiple open/close
[directfd] Dynamically allocate floppy bounce buffer
[driver] Minor floppy driver cleanups
[driver] Optionally place direct floppy driver in far text segment
[drivers] Remove unfinished Meta/UDD driver
[bioshd] Implement park for ancient MFM/RLL disks by @vkoskiv
[bioshd] fix fd_types by @tyama501
[cmds,kernel] Add meminfo -m option to display sorted main memory segments
[console] Add dual monitor support to console-direct driver by @vkoskiv
[console] No display only when fg = 0 and bg = 0 for PC-98 by @tyama501
[console] Use bg color and invert for PC-98 when bg is not zero by @tyama501
[basic cga] Add snakcga.bas by @tyama501
[basic cga] CGA Graphic support for IBM-PC ELKS Basic by @tyama501
[beep] Add -f and -l option by @tyama501
[beep] make it round by @floriangit
[df] Add 300k data xfer rate to 360k floppy FDC data
[direct df] Replace QEMU kluge with implied seek option for 360k/1.2M floppies

C Library

[libc] Add fast divmod.asm to Watcom printf
[libc] Add flush destructor function to tiny_printf.c
[libc] Add fminf and fmaxf math functions
[libc] Add noreturn for Watcom C
[libc] Add precision timer routines to C library
[libc] Add precision timing to --ftrace
[libc] Add small ultoa function, cleanup and reduce size of dd
[libc] Add strftime to C library, enhance ttyclock with colors
[libc] Add thousands grouping to printf
[libc] Change default Watcom compilations to large model
[libc] Clean up termcap lib and some header files
[libc] Cleanup C library header files
[libc] Cleanup errno.h and reduce size of etc/perror file
[libc] Cleanup rindex issues
[libc] Cleanup sleep.c source
[libc] Cleanup some stdio source
[libc] Consolidate compiler specific definitions in header files for Watcom C
[libc] Don't export non-standard types from header files
[libc] First pass at compiling most all of C library with OpenWatcom
[libc] Fix fdopen bug introduced in #1923
[libc] Fix glitch when building elks-libc on launchpad.net by @tkchia
[libc] Get --ftrace option working on all commands
[libc] Minimize differences between vfprintf.c and tiny_printf.c
[libc] More cleanup C library header files
[libc] Partially implement sigaction
[libc] Pass 2: Add system calls to OpenWatcom C library for ELKS
[libc] Reduce dependencies on certain standard header files
[libc] Reinstate large model default for OpenWatcom, support all models
[libc] Remove __const from libc routines, cleanup
[libc] Revert large mode change, default Openwatcom build is compact
[libc] Rewrite getpwent/getgrent routines to cache opens
[libc] Rewrite vfprintf.c and tiny_printf.c to use __divmod for speed
[libc] Use __divmod in ltoa, ultoa, ltostr, ultostr and dd
[libc] exit,abort,longjmp are noreturn by @ccoffing
[libc,cmds] Fix isprint and kilo for CP 437 display
[libc,cmds] Fix perror and rm -f, improve rmdir
[libc,cmds] Misc minor fixes to reduce file sizes
[libc,cmds] Rewrite strrchr for speed/size, update rmdir
[libc,cmds] Various small cleanups
[libc,kernel] Rewrite ptostr for speed, add thousands separator to printk
[libc/cmds] Fix time command hour wrapping problem
[tui] Improve error detection on DSR parsing in TUI library
[tui] Optimize curses library for smaller executables
[debug] Add CONFIG_APPS_FTRACE to build all apps with function tracing
[debug] Add nm (symbol table name lister) to distribution
[debug] Enhance stack backtrace and instrumention library
[debug] Move stack backtrace and symbol profiling to C library
[debug] Rewrite stack tracing code, add disasm to distribution
[debug] Use same C file for C library and kernel precision timer routines
[debug] Write profiling output to stderr

Build

[build,df] Various minor fixups
[build] Add :1232k, :1232c, :1200k and :1200c for app selection
[build] Add NE2K, WD and 3COM NICs to default build
[build] Cleanup Makefiles, add RM, remove Make.rules
[build] Create disk images with compressed executables by default
[build] Don't automatically build host programs
[build] Don't build elksemu on macOS
[build] Don't use dummy boot code in kernel disk image
[build] Fix ROM build when using emu86-rom.config
[build] Fix compilation with CONFIG_NANO set
[build] Fix elf2elks libelf build on Apple M1
[build] Fix non-compressed 1440k build
[build] Free up more space on PC-98 1232k compressed image
[build] Minor cleanup, add "make ctags"
[build] Misc fixes
[build] Reformat Applications file
[build] Remove various unused config options
[build] Set PC-98 default images to compressed
[build] Small change to 360k and 1200k app selections
[build] Speed up ELKS build
[build] Temporarily remove compressed executables from CI build
[build] Update ELKS version to 0.8.0-dev
[build] Update build.c kernel builder with better messages
[build] Use CI actions/upload-artifacts@v4
[build][docs] Miscellaneous fixes
[builds] Revise Applications list for PC-98 1232k image
[CI] Add back in "PR" automated workflow
[CI] Disable second PR run of Github actions, run only on push
actions: add OW libc test build on Ubuntu and OSX by @jmalak
[elkscmd,build] Cleanup various unused programs and config options

Toolchain

[tools] Add fixomf utility to tag symbols weak within Watcom OBJ files
[tools] Add omf2elf, omfdump and os2toelks object file utilities
[tools] Add text/data relocation conversion to os2toelks
[tools] Enhance mfs genfs to include directory prefix in destination
[tools] Fix mfs problem with max inodes on MINIX v2 filesystems
[tools] Fix objdump86 compilation warnings
[watcom] Add I8* routines for 64-bit multiply/divide
[watcom] Add constructors/finalizers to Watcom C runtime
[watcom] Add heap and stack size processing to ewlink and os2toelks
[watcom] Add mount, umount, _exit syscalls, add back putenv
[watcom] Add option processing to ewcc and ewlink
[watcom] Add setjmp/longjmp
[watcom] Add setuid, setgid, move bsr.c to C library
[watcom] Add stackavail function to get alloca working
[watcom] Change \e to \033 in TUI for OWC
[watcom] Disable cannot find os2.lib warning using ewlink
[watcom] Fix malloc/realloc/free, add execl, execle, execlp, execlpe
[watcom] Get some TUI library and curses working
[watcom] Offset data and code sections to identify null pointers
[watcom] Reinstate auto CI test build of libc.lib
[watcom] Remove OpenWatcom default symbols and residual code (dup #1944)
[watcom] Rewrite argv/environ array in large model to far pointers
[watcom] Update ewcc/ewlink, add many system calls
[watcom] correct some C compiler, Librarian and Linker options by @jmalak
[libc,watcom] Add weak symbol support for OpenWatcom to C library
[owc] Add fmemalloc syscall by @tyama501
[owc] Add unsigned to __pseg by @tyama501
ow-libc: use more eficient method #pragma aux aborts for functions wh… by @jmalak
add STACK segment for OW libc by @jmalak

Other

Add export TZ to /etc/profile, TOPDIR check to watcom.inc
Add screenshots to main README.md
Better guidance to run slattach.sh as root by @floriangit
Cleanup system image Makefile
ELKS online link by @toncho11
Elksemu syscall fixes by @ccoffing
Fix a variable logic and add a brace pair by @floriangit
Malloc fixes by @ccoffing
PC-98 beep support by @tyama501
Path cleanups by @ccoffing
Portable build fixes by @ccoffing
Remove test code in ssd driver for changing request addresses
Screen portability fixes by @ccoffing
Screenshot ZDS 286 by @toncho11
Size reductions by @ccoffing
Tidy cal by @ccoffing
Tweaks to allow building on other BSDs by @ccoffing
Update libc test to match current perror strings by @ccoffing
Update meta driver file_ops function signatures by @ccoffing
[scripts] Update qemu.sh for all versions of QEMU
[serial] Use Transmitter Hold Register Empty for buggy 8251 UARTs
[minor] Update qemu.sh for macOS x86, reformat config.h
docker: install patch, required for build by @d33tah
inital commit of elkscmd/beep by @floriangit
popen must be matched with pclose, not fclose by @ccoffing
remove nonsense, author used incorrect OpenWatcom configuration by @jmalak
smaller which by @ccoffing
support for MacOs brew qemu installation by @madejusz

Documentation

[docs] Add fmemalloc man pages for section 2 and 3
[docs] Add man page for fsck-dos
[docs] Update documentation on calling conventions, update license texts for utils
[Documentation] Update function calling conventions and prologues

New Contributors

@floriangit made their first contribution
@madejusz made their first contribution
@and3rson made their first contribution
@d33tah made their first contribution
@jmalak made their first contribution
@vkoskiv made their first contribution

Full Changelog: https://github.com/ghaerr/elks/compare/v0.7.0...v0.8.0

Chief Maintainer: @ghaerr

tyama501 commented 1 day ago

Hello @ghaerr Wow, great list. Could you move the following to the PC-98 Port area when the release? I think that is the major update for the PC-98. [bioshd] 720KB 2DD support for PC-98 2HD/2DD common drive by @tyama501

Thank you!

toncho11 commented 17 hours ago

Maybe not worth mentioning, but there are some documentation updates/additions:

We should put a link to ElksDoom in the release notes. We can also add that ElksDoom proves that big memory consuming applications can be ported to ELKS and VGA game programming is doable (doable before, but maybe not at this scale). We could (just an idea) do as well an ELKS image with some of the memory optimizations to boot directly into ElksDoom for this release. We could call it "hd32-minix-doom.img" for example.

Thank you!

rafael2k commented 13 hours ago

Yay! Congrats to all contributors!

ghaerr commented 9 hours ago

@toncho11: I already mentioned Doom in the release notes (too much to read already, right?), but will add a link as well.

The Wiki documentation isn't really release oriented, but agreed its important that everyone can find the information they'd like. I'll add the links you mentioned to the release notes Documentation area. We could add a "What's New" section to the Wiki as well, but this can be done at any time.

Vutshi commented 6 hours ago

Congrats to @ghaerr and the contributors! This seems like the biggest ELKS update since ~0.5 when I started following it.

ghaerr commented 6 hours ago

Version 0.8.0 is released! Prebuilt images are available here: https://github.com/ghaerr/elks/releases/tag/v0.8.0.

Vutshi commented 6 hours ago

sl appears to be the most important ELKS app, with three entries on the list :)

ghaerr commented 6 hours ago

Well @Vutshi you may have missed my comment to you a week ago in all the madness, but yes, a number of enhancements were made to the console and ascii conversion routines so that your favorite program can run a little bit faster! :)

Vutshi commented 5 hours ago

@ghaerr, I'll be celebrating the release with some ceremonial train benchmarking.

Vutshi commented 5 hours ago

ELKS train