Display update speed on a 3477

[GorrillaRibs]

Hi! Firstly, thanks for the project, I'd never have been able to get the 3477 working at all without it! I got a PCB printed & am running it off of a teensy 4.0. All seems to be working well save for the update speed of the display, which seems much slower than the example video shown (especially noticeable with cmatrix, far more stuttering and seemingly redrawing entire lines. I can provide a video if that helps).

My question is, do you have any advice on how to go about debugging this? Would the teensy's code or the python script be the bottleneck, or is the 3477's emulation just slower in general? (though setting the slower polling rate seemed to just make it worse). I've some experience with writing for a teensy (a 3.1) and python, so I may be able to take a crack at making my 3477 more usable but I'm not sure where to start.

If there's any debug info I can provide that'd be useful, let me know!

[inmbolmie]

Hi GorrillaRibs.

Did you consider the issue described on the "Programming the Teensy" section? https://github.com/inmbolmie/5250_usb_converter#programming-the-teensy

That is, modern versions of the Arduino + Teensyduino environment generate slow code that will result in stuttering. I have included here a binary release compiled from known good versions: https://github.com/inmbolmie/5250_usb_converter/blob/master/5250_interface/5250_interface.ino.TEENSY40.hex

If the red light on the Teensy goes continuously on, this could be the cause.

If you tried this and still have problems, you can generate a debug file using the "-c" option as described here: https://github.com/inmbolmie/5250_usb_converter#debugging and I could take a look at it.

[GorrillaRibs]

I did consider that section, this does occur using the prebuilt binary & one compiled myself under the same teensyduino environment (1.53) - I also tried to calculate the WAIT_CYCLES* values for a 812MHz overclock but couldn't tell an appreciable difference (although I didn't do any specific tests other than scrolling a man page & cmatrix). For posterity the values I used there are:

const int WAIT_CYCLES_RX = 40600;
const int WAIT_CYCLES_RX_PENDING_TX = 6767;
const int WAIT_CYCLES_RX_SAMPLE = 115; 
const int WAIT_CYCLES_TX = 406; 
const int WAIT_CYCLES_TX_DLY = 203; 

For what its worth, I also didn't see the red light on the teensy turn on at all, but perhaps I missed that. I've generated a debug log linked here though (github's gist service wouldn't take it for some reason, so linked as a download from my own server) in case that's any help!.

One potential issue I just realized though: The only version of the line driver (SN75112) I could find was a surface mount package, so soldering it onto the board required a soic-14 to dip-14 adapter, and I its possible I damaged the chip in the process (surface mount soldering isn't my strong suit) - could these symptoms be a result of that? I did buy 3 of the chips so I have spares to test in this case.

In any case, thanks for your help!

log file: http://gorrillaribs.xyz/5250Conv_debug.log (firefox seems to be able to open it in the browser for me)

[inmbolmie]

Hi again.

About the overclocking, thanks, nice to have the reference values . If the red light never goes on there is no need to overclock as the speed on the Teensy has to be "just enough". You will only benefit of more processing speed on the host side, for example something like a Raspberry Pi 1 is too slow, a faster Raspberry or any decent desktop computer will be fine.

What king of host computer are you running the Python script on?

I will take a detailed look to the log but at first sight:

• It looks clean (no major frame loss, no parity errors)
• The terminal seems to take a long time to process the commands, there are much more "busy:1" streaks of responses that I've ever seen on other terminals. That could indicate that the terminal is slow per se or has any kind of internal problem (?) or that we are sending too many commands.

If we are sending too many commands to the terminal you could try to adjust the value of SLOW_POLL_MILLISECONDS to 1 (from the default value of 5) and try with slow polling enabled. Values of less than 1 millisecond will require some code changes though to make the calculations in microseconds.

Anyway it would be nice to watch a video showing the "cmatrix" performance on your terminal to be able to compare it to my 3488. About my videos, note that the original 5251 was insanely fast, even compared to some modern compatible terminals.

About the SN75112, I don't think it is damaged as the log is so clean and there is not any apparent frame losses. The SN75112 health is not that critical, in fact I once made a proof-of-concept replacing the SN75112 by another SN75110 and it works just the same. I think this is because the protocol is designed to operate at very long cable lengths where the signal degradation will be much higher than the level difference between using a SN75112 and a SN75110 and using two SN75110. SN75112 will give you a signal with just the right levels though, compared to the original 5251 signal it is even cleaner. Anyway the only way to be 100% sure would be to take a look to the signals on the wire using an oscilloscope.

Best regards.

[GorrillaRibs]

Hi, thanks for all the info!

I'm running the script on my desktop, which has a fairly recent i5, so I wouldn't think thats the issue. However, setting the SLOW_POLL_MILLISECONDS value to 1 & enabling slow polling did immediately help! I'm not sure its quite as fast as the original (as fast as this terminal can go yet) but cmatrix for example looks far smoother, and scrolling a man page seems faster as well - would you know off the top of your head what I should change to try to work with lower values? So I could see if there's any difference at fractions of a millisecond. As for videos, I've uploaded them to my server again, but I could upload them to youtube or similar if that's easier.

Videos: http://gorrillaribs.xyz/3477_videos/3477_cmatrix_normal_polling.mp4 http://gorrillaribs.xyz/3477_videos/3477_cmatrix_slow_polling.mp4 http://gorrillaribs.xyz/3477_videos/3477_man_scrolling_normal_polling.mp4 http://gorrillaribs.xyz/3477_videos/3477_man_scrolling_slow_polling.mp4

It could be that's the upper limit of the terminal now, but I don't know enough about it to be sure (the best documentation I could find on it so far is a schematic hosted on archive.org, which while very cool, doesn't explain its capabilities very well).

Thanks for all your help in any case!

[inmbolmie]

OK, watching your videos it should be nearly as good as it could be, To set lower polling interval values the code has to be rewritten to specify the polling in microseconds and not in milliseconds. That is the code around line 1295, the variable "actmillis" has to be changed for a "actmicros" or something like that, and the method to load it and constants around it adjusted accordingly.

If you are familiar with Python the changes should be pretty straightforward, or if you want me to look at it I will do it over this weekend and upload an updated version.

Being able to adjust the polling interval under 1 millisecond will give you the opportunity to find the optimal value to maximize throughput over the terminal link.

Best regards.

[inmbolmie]

Of course you can also test values over 1 millisecond, like 2 or 3. The optimal value could be under 1 millisecond or over it. If it works better with 2 than with 1 it makes no sense to try values under 1. But I suspect that the optimal value will be under 1.

Best regards

[GorrillaRibs]

Hi! Thanks for all the info, I've attempted to change the code around actmillis, but I'm not convinced its working properly yet - essentially all I've changed is adding a zero to the number multiplying time.time() for actmillis and lastmillisresponse - would that make any sense? it just seems to be taking the value in regular seconds & multiplying that to get ms, right? If you have a chance to take a look at it the result'd probably be better, but I'll take another crack at it tomorrow and see if I can wrap my head around python again :).

On another note, I have gotten around to adding the layout for my keyboard, a 122 key english/US layout, based on the 122 key DE layout that was there already - would the best way to submit that be a simple patch or a pull request?

[GorrillaRibs]

Looking again, I should probably give this a shot when I'm awake, I might've forgotten how SI units worked. Hopefully I'll have something functional by this evening

[GorrillaRibs]

After some more poking around, I'm tentatively using actmicros = int(round(time.time_ns() / 1000)) # nanoseconds to microseconds and lastmicroresponse[terminal.getStationAddress()] = int(round(time.time_ns() / 1000)) Which seems to make some sense to me to get an accurate integer microseconds (although types in python still confuse me a bit, coming from C) and it seems to be working well. Although thus far my only "test" is "it looks mostly the same with 1 microsconds vs. 1000 microseconds" so it's not exactly a scientific test, and I'd still like to add a configuration toggle/commandline switch to make both modes available in case it helps someone else.

[inmbolmie]

Hi again. I've just pushed a branch feature/pollmilliseconds where the slow poll interval is specified in microseconds (yes, the branch name is awkward, I've just realized ¯_(ツ)/¯ ).

https://github.com/inmbolmie/5250_usb_converter/tree/feature/poll_milliseconds

That just uses time.time_ns() instead of time.time()

On my 3488 the optimal value is around 500 microseconds, but the improvement is only marginal from 1000 microseconds (1 millisecond).

So please tell me how it goes.

About the keyboard map for 122 key english/US, yes, a pull request would be nice for that.

Best regards

[GorrillaRibs]

Awesome, thanks for the update! On the new branch I've settled around 650 microseconds & it seems to work very well (or as well as I've seen it thus far). I've opened a pull request with the 122 Keyboard mapping (cleaned up on the terminal itself!) - let me know if I made any mistakes, github pull requests are still strange to me vs. just 'git format-patch'.

I'm also happy to close this issue, as it seems to be working well now.

Thanks again!

[inmbolmie]

Thanks, merged everything on the master branch