search

KermitProject / ckermit

C-Kermit: Portable OPEN SOURCE Scriptable Network and Serial Communication Software for Unix and VMS
Other
18 stars 9 forks source link

Xmodem Transfers fail with xmodem as external protocol. Doubling 0xff in payload #3

Open cbmbas opened 1 year ago

cbmbas commented 1 year ago

Error is reproduceable. Using Ubuntu Linux with current CKERMIT Version, but older Versions until 9.305alpha2 don't work too. Transfer is done within CKermit from Linux PC to a 2nd (8bit) Machine without Kermit and without Kermit Protocol. Transferprotocol on PC: lrzsz Transferprotocol on 2nd Machine: xmodemCRC Testfile is an old C64 Forth Version of Kermit: C644th.prg This File has a 0xFF within the first 128 Bytes. When the 1st Xmodem Transferblock is sent to the 2nd Machine the 0xff gets doubled to two 0xff, which leads to a checksum mismatch, which then leads to a not working Filetransfer. Same with 1K, ymodem. zmodem also didn't work.

Sending the File with the same setup without using Kermit works perfectly. This is how it should be (working transfer) see e1 ff a2 eb 56 at the end of Block#1 image

This is how it is sent through Kermit with Xmodem as external protocol see the e1 ff ff a2 eb 56 at the end of Block#1 image

I am sending File as binary. Would be great if you could fix that. Or if I'm missing something here, which setting is missing ? Already tried set file type, flow-control, set term byte 8 and more...

jaltman commented 1 year ago

@cbmbas the report does not indicate what connection type is being used. However, the External protocols in C-Kermit post might be applicable.

cbmbas commented 1 year ago

@jaltman: Connection type is tcp/ip, no SSL Kerberos etc. just plain telnet (via C-Kermit) to a private BBS System on a Commodore 64, Ultimate64 to be precise. Both Computers are connected with an Ethernet Cable to my Router, so it's not a WIFI Problem. I also tried all 5 parity settings without success. That was mentioned on a Troubleshooting Page in the C-Kermit Manual. Sending the Files the same way without C-Kermit works. I've also set Protocol to xmodem: SET PROTOCOL XMODEM {} {} {lsz --xmodem -b %s} {lsz --xmodem -a %s} {lrz --xmodem -b %s} {lrz --xmodem -a %s} and PREFIXING all as well as SET STREAMING off.

"...it doubles the IAC (0xFF) on Telnet connections..." but Kermit Protocol is not used in my case. By the way, I also tried Kermit Protocol which is inlcuded in the Terminal Program "Novaterm" on C64 with no success. But my goal is to get Filetransfers working with X-Modem, because X-Modem is the only Filetransfer Protocol which is supported by all known C64 / C128 / Amiga BBS Systems.

Currently I have C-Kermit 10.0 beta 04 installed on my Ubuntu Linux. I saw the Bug Report from Ao Huang from 2021, so I updated from excatly that Version, which comes with the Distro, to 10.0 and hoped it would solve the Problem.

jaltman commented 1 year ago

@cbmbas On a Telnet connection, the telnet client is required to quote (aka double) the IAC (0xFF) byte when transmitted because IAC is the beginning of the command sequence for Telnet protocol. The Telnet server should be converting 0xFF 0xFF to 0xFF but if it doesn't then the Xmodem protocol cannot transmit 8-bit data that includes 0xFF.

This is not a bug in C-Kermit.

cbmbas commented 1 year ago

I'm about to give up. I almost did anything to get xmodem Transfers working, but no chance. I also made a base64 Version of that File to remove all 0xFF in Payload. Partial success... now 13 Blocks are transfered, then 1 Bad Block (now 0x00 gets doubled where the complement of xmodem Block# 0X0D is expected -> $0xF2) which repeats bad and stops after 10 fails. I also tried to stop the Kermit Script and continue the Transfer with a Bash Script instead, but what ever I do, the connection closes or transfer fails as long as ckermit is active. I've made a logfile with tcpdump which I could provide to you I you are interessted and the Script with dozends of settings just to get a clear 8bit telnet session somehow. I don't want to supply the log here on github, but would send it via E-Mail. This is a screenshot out of Wireshark where one can see the double 0x00 is the Datablock. image

This 0x00 is not part of the base64 encodeded Filedata.

jaltman commented 3 months ago

The Telnet protocol is not a transparent pipe. Control characters cannot be sent through it. NUL (0) is a control character which has special meaning to the Telnet protocol.
A raw tcp/ip connection is a clean binary stream. A telnet protocol connection is not.