`xxd` is in `vim-common` and some `esptool.py` quirks

[drawkula]

I had to install the package vim-common (on Debian8/AMD64) to satisfy the Makefile's wish to find xxd.

$ apt-file find bin/xxd
vim-common: /usr/bin/xxd
vim-dbg: /usr/lib/debug/usr/bin/xxd
xxdiff: /usr/bin/xxdiff

...there seems not to be a different source for xxd... :-/

All make targets except the ones for uploading:

$ git describe --always 
6a367a5
$ make clean bins disks systemdiskA
[CC] main.c
[CC] machine.c
[CC] uart.c
[CC] conio.c
[CC] flash.c
[CC] monitor.c
[CC] utils.c
[CC] gpio16.c
[CC] z80/z80emu.c
[AS] nosdk/startup.S
[CC] nosdk/nosdk8266.c
[LINK]
   text    data     bss     dec     hex filename
  12784      10   70408   83202   14502 image.elf
[MAKEDISK] DISK_A.DSK
[MAKEDISK] DISK_B.DSK
[MAKEDISK] DISK_C.DSK
[MAKEDISK] DISK_D.DSK
[MAKEDISK] DISK_E.DSK
[MAKEDISK] DISK_F.DSK
[MAKEDISK] DISK_G.DSK
[MAKEDISK] DISK_H.DSK
[MAKEDISK] DISK_I.DSK
[MAKEDISK] DISK_J.DSK
[MAKEDISK] DISK_K.DSK
[MAKEDISK] DISK_L.DSK
[MAKEDISK] DISK_M.DSK
[MAKEDISK] DISK_N.DSK
[MAKEDISK] DISK_O.DSK
[Z80ASM] BOOT.Z80
[Z80ASM] BIOS.Z80
[Z80ASM] BDOS.Z80
[Z80ASM] CCP.Z80

...so with xxd from package vim-common it looks ok now.

I'm still stuck with uploading:

[UPLOAD] disks/DISK_A.DSK to 0x3c0000
[UPLOAD] disks/DISK_B.DSK to 0x381700
Traceback (most recent call last):
  File "/opt/esp-open-sdk/esptool/esptool.py", line 1270, in <module>
    main()
  File "/opt/esp-open-sdk/esptool/esptool.py", line 1181, in main
    operation_func(esp, args)
  File "/opt/esp-open-sdk/esptool/esptool.py", line 871, in write_flash
    flasher.flash_write(address, image, not args.no_progress)
  File "/opt/esp-open-sdk/esptool/esptool.py", line 574, in flash_write
    assert addr % self._esp.ESP_FLASH_SECTOR == 0, 'Address must be sector-aligned'
AssertionError: Address must be sector-aligned
Makefile:224: recipe for target 'writediskB' failed
make: *** [writediskB] Error 1
$ /opt/esp-open-sdk/esptool/esptool.py version
esptool.py v1.2
1.2

Is my esptool.py too old?

The Minion I'm trying to flash is a NodeMCU-Devboard-v0.9 (ESP-12 + CH340) and I don't think he's the culprit.

[mengstr]

Thank you for the report.... I've always thought that xxd was part of some default binary utilities package, but apparently not. When I wrote the installation guide for this I booted up a virgin debian-8.7.1-i386-netinst and installed only the packages required to get all of the cpm8266 pieces in place. (except for "joe" that still after all this years is my non-gui text editor of choice).

It seems that the Deb8 netinst adds vim by default since that's where I got my xxd from as well. I'll have to add that as a prerequisite to the installation guide unless I can find an easy way to get rid of that dependency. I'm only using it for the deskew/deinterleave of the disk images.

When I started this project I also used the version 1.2 of the esptool, and back then I had the disk images put at even 0x40000 intervals in the flash. I later went over to the 2.0 esptool that compresses the data sent to the esp for flashing which gives a considerable reduction of the time it takes to upload & flash the entire chip.

When I started doing the wifi integration I squeezed the disk images together in the flash in order to not waste any space. The disks are 256256 (0x3e900) bytes so they end up at some oddish places in the flash and seems like the 1.2 esptool doesn't like that.

I'll change the code and Makefile to have the disks placed at locations more agreeable to the 1.2.

[drawkula]

vim ... after installing Debian I always exchange vim with the more traditional nvi because I don't want to get used to functions I will not find in "standardvi".

esptool.py ... if version 1.2 still is the default one when building the SDK, staying compatible with it probably is a good idea.

Deskewing ... python is in ESP-OPEN-SDK's prerequisites, so everyone building cpm8266 has it installed. Deskewing with a python script probably even will be easier than in sh. The other idea is: Would defining a diskdef without skew in /etc/cpmtools/diskdefs be a good idea? Changes in that file will need root rights but everyone building cpm8266 did need to install lots of prerequisites with root rights too... so that should not really be a problem.

Thanks for the fixes!

sd$ܟ|�l�<�l�c|���r�#�c��gn�dog���c8�lrd;lx�n��dcg�|l��b��og�l��l`�nd`gs�ۓo$`p�g�sĜ���co�|c��og�l �'l`o;Ǜ�'ld`8�o�;�����b'�|�d�8��og�$`�gd�l`orǛ�'�bd`{��o�c�H�
cpm8266 - Z80 Emulator and CP/M 2.2 system version 0.3

62K CP/M v2.2 [cpm8266 v0.3 - SmallRoomLabs]

a>dir
A: ASM      COM : DDT      COM : DDTZ     COM : DUMP     COM
A: ED       COM : LOAD     COM : PIP      COM : STAT     COM
A: SUBMIT   COM : XSUB     COM
a>█

Yayayayyy... \o/

[mengstr]

I actually have added a no-skew option in my diskdefs, but requiring people to hack into files owned by root didn't seems like a good idea.

I first made a deskewer by using dd to move the file sector-by-sector into a new file, but that was incredibly slow. Converting the file to hex and manipulating it a bash array and then convert it back is like 10x the speed.

The idea of using python is a very good one! - unfortunately I haven't written a single line of python during my 40 years of coding, so it would be major undertaking for me. Can easily do it in golang, node or C though....

If you're a pythonista and could easily whip up a deskewer in it I'd love to add it as a contribution to this project. :-)

[drawkula]

After thinking a while longer...

The deskewing script is hard coded for the current disk format. That would need changes on every change of the diskdef used in cpm8266.

Maybe adding cpmtools to the repo, compiling and using it in place instead of installing it will be more universal? Then the diskdefs file will not be in /etc and this solution will fit all future disk format changes.

[mengstr]

That is a possible solution as well, but it would introduce a dependency on gcc which is doesn't have today. EDIT: Bzzzzt. Wrong! The open-sdk from pfalcon already requires gcc, so we should be good for gcc at least.

[drawkula]

On Debian9, xxd is in an own package named xxd.