search

vipoo / yellow-msx-series-for-rc2014

V9958 Video Board Designed for RC2014
MIT License
36 stars 4 forks source link

Fatal errors during assembly/linking with Docker #8

Open zomgugoff opened 1 month ago

zomgugoff commented 1 month ago

While running the supplied docker command to build roms, the assmebler and linker report fatal errors and undefined globals. Below are 2 examples

Assembling working/france-with-rtc/spcdev.mac

V                     C     IF  RTC EQ  1
V                     C     IF  CNTRY EQ 11
V                     C     IF  CNTRY EQ 0
V                     C     IF  CNTRY EQ 1
V                     C     IF  CNTRY EQ 2
V                     C     IF  CNTRY EQ 3
V                     C     IF  CNTRY EQ 4
V                     C     IF  CNTRY EQ 5
V                     C     IF  CNTRY EQ 6
V                     C     IF  CNTRY EQ 7
V                     C     IF  CNTRY EQ 8
V                     C     IF  CNTRY EQ 9
V                     C     IF  CNTRY EQ 10
V                     C     IF  (BASVER EQ 1) AND (CNTRY NE 3)
V                     C     IF  (BASVER EQ 1) AND (CNTRY EQ 3)
O 0004                C RTC EQU 1CNTRY EQU 4
U                     C     IF  CNTRY EQ 11
U                     C     IF  CNTRY EQ 0
U                     C     IF  CNTRY EQ 1
U                     C     IF  CNTRY EQ 2
U                     C     IF  CNTRY EQ 3
U                     C     IF  CNTRY EQ 4
U                     C     IF  CNTRY EQ 5
U                     C     IF  CNTRY EQ 6
U                     C     IF  CNTRY EQ 7
U                     C     IF  CNTRY EQ 8
U                     C     IF  CNTRY EQ 9
U                     C     IF  CNTRY EQ 10
U                     C     IF  (BASVER EQ 1) AND (CNTRY NE 3)
U                     C     IF  (BASVER EQ 1) AND (CNTRY EQ 3)

30 Fatal error(s)

No Fatal error(s)

A>L80

Link-80  3.44  09-Dec-81  Copyright (c) 1981 Microsoft

*

*BIOHDR/P:0,SLOT,MSXIO,QUEUETL,MSXGRP,CASET,BIO,MSXCHR,MSXINL
Data    0000    2680    < 9856>

-C7D75  1477    -C7FBE  0210    -C7FC4  0242    
-CHRENT 0011    -DIOERR 1BB2    -FILOUT 1B51    
-GETENT 0029    -J7B61  04EE    -RDWEXP 03E3    
-READYR 0D35    -STOP   0547    -SYNENT 0009    
   12 Undefined Global(s)
33737 Bytes Free

*

*BASHDR,F4,BINTRP,MSXEXP,MACLNG,GENGRP,ADVGRP,BIPTRG,BIPRTU,BIMISC,BISTRS,SPCDSK,DSKCOM,SPCDEV,BASBIO,MSXSTS,INIT
Data    0000    8000    <32768>

 7687 Bytes Free

*

*MAINROM/X/Y/E/N

Data    0000    8000    <32768>

 7687 Bytes Free
[0000   8000     127]
Origin below loader memory, move anyway(Y or N)?

hex2bin v2.5.1, Copyright (C) 2017 Jacques Pelletier & contributors

Allocate_Memory_and_Rewind:
Lowest address:   00000000
Highest address:  00007FFF
Starting address: 00000000
Max Length:       32768

Binary file start = 00000000
Records start     = 00000000
Highest address   = 00007FFF
Pad Byte          = FF
unix2dos: converting file msxsyssrc/base300/mainrom.l80 to file working/aust-with-rtc/mainrom.l80 in DOS format...
unix2dos: converting file msxsyssrc/base300/biohdr.mac to file working/aust-with-rtc/biohdr.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/slot.mac to file working/aust-with-rtc/slot.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/msxio.mac to file working/aust-with-rtc/msxio.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/queuetl.mac to file working/aust-with-rtc/queuetl.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/msxgrp.mac to file working/aust-with-rtc/msxgrp.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/caset.mac to file working/aust-with-rtc/caset.mac in DOS format...
unix2dos: converting file msxsyssrc/basekey/keyfr.mac to file working/aust-with-rtc/keyfr.mac in DOS format...
unix2dos: converting file msxsyssrc/basekey/keyuk.mac to file working/aust-with-rtc/keyuk.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/bio.mac to file working/aust-with-rtc/bio.mac in DOS format...
unix2dos: converting file msxsyssrc/basechr/chrrus.mac to file working/aust-with-rtc/chrrus.mac in DOS format...
unix2dos: converting file msxsyssrc/basechr/chrjapv2.mac to file working/aust-with-rtc/chrjapv2.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/msxchr.mac to file working/aust-with-rtc/msxchr.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/bashdr.mac to file working/aust-with-rtc/bashdr.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/f4.mac to file working/aust-with-rtc/f4.mac in DOS format...
unix2dos: converting file msxsyssrc/basekey/keyint.mac to file working/aust-with-rtc/keyint.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/msxexp.mac to file working/aust-with-rtc/msxexp.mac in DOS format...
unix2dos: converting file base300.inc to file working/aust-with-rtc/msx.inc in DOS format...
11
unix2dos: converting file working/aust-with-rtc/msx.inc to file working/aust-with-rtc/msx.inc in DOS format...
unix2dos: converting file msxsyssrc/base300/bintrp.mac to file working/aust-with-rtc/bintrp.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/maclng.mac to file working/aust-with-rtc/maclng.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/gengrp.mac to file working/aust-with-rtc/gengrp.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/advgrp.mac to file working/aust-with-rtc/advgrp.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/biprtu.mac to file working/aust-with-rtc/biprtu.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/bistrs.mac to file working/aust-with-rtc/bistrs.mac in DOS format...
unix2dos: converting file msxsyssrc/basekey/keyrus.mac to file working/aust-with-rtc/keyrus.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/dskcom.mac to file working/aust-with-rtc/dskcom.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/spcdev.mac to file working/aust-with-rtc/spcdev.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/basbio.mac to file working/aust-with-rtc/basbio.mac in DOS format...
unix2dos: converting file msxsyssrc/basekey/keyger.mac to file working/aust-with-rtc/keyger.mac in DOS format...
unix2dos: converting file msxsyssrc/basekey/keyspa.mac to file working/aust-with-rtc/keyspa.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/biptrg.mac to file working/aust-with-rtc/biptrg.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/bimisc.mac to file working/aust-with-rtc/bimisc.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/spcdsk.mac to file working/aust-with-rtc/spcdsk.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/msxsts.mac to file working/aust-with-rtc/msxsts.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/init.mac to file working/aust-with-rtc/init.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/optrom.l80 to file working/aust-with-rtc/optrom.l80 in DOS format...
unix2dos: converting file msxsyssrc/base300/subrom.l80 to file working/aust-with-rtc/subrom.l80 in DOS format...
unix2dos: converting file msxsyssrc/base300/subrom.mac to file working/aust-with-rtc/subrom.mac in DOS format...
unix2dos: converting file msxsyssrc/basechr/chrint.mac to file working/aust-with-rtc/chrint.mac in DOS format...
unix2dos: converting file msxsyssrc/basechr/chrkor.mac to file working/aust-with-rtc/chrkor.mac in DOS format...
unix2dos: converting file msxsyssrc/basechr/chrger.mac to file working/aust-with-rtc/chrger.mac in DOS format...
unix2dos: converting file msxsyssrc/base300/optrom.mac to file working/aust-with-rtc/optrom.mac in DOS format...
vipoo commented 1 month ago

Hmm....

Not sure whats going on here for you....

If you look at the latest Github Actions build --- that build seems ok -- I don't see any errors in the log outputs (https://github.com/vipoo/yellow-msx-series-for-rc2014/actions/runs/10155854765)

Are you using the same docker image version at the github action script? -- have a look at the file (https://github.com/vipoo/yellow-msx-series-for-rc2014/blob/main/.github/workflows/yellow-msx.yml)

That is using docker image versioned: vipoo/yellow-msx-rc2014-tool-chain:0.0.21

Perhaps I have an issue with the latest docker image. (Will investigate)

Can you share what commands you issued to attempt to build?

Dean

vipoo commented 1 month ago

correction 0.0.21 is the latest image, that's been pushed. So that should be working (as it does in github actions)

So I can only suggest you look at the github actions yml config file, to guide on the build script.

Try running the specific docker run .... commands, as detailed in the action yml - see if that helps or highlights the issue

Dean.

zomgugoff commented 1 month ago

I was running the docker command from the msx subdirectory as directed on https://github.com/vipoo/yellow-msx-series-for-rc2014/tree/main/msx.

Image is the current version:

REPOSITORY                           TAG       IMAGE ID       CREATED        SIZE
vipoo/yellow-msx-rc2014-tool-chain   0.0.21    bddb35852aeb   5 months ago   3.95GB
vipoo/yellow-msx-rc2014-tool-chain   latest    bddb35852aeb   5 months ago   3.95GB

Errors are occurring in the build-msx-roms and build-release-candidate parts. Not every rom fails, but there are many that do. Just a guess, but it sort of looks like either the source was updated for a newer version of the compiler, or the compiler updated and the syntax has changed.

vipoo commented 1 month ago

Very strange indeed.

I am unable to reproduce it.

The version of compiler is locked within the docker image. That's the same docker image that the Github-Action uses.

The docker image contains all the build tools,. compilers, assemblers etc. So should be the same for all environments

Your host's current working directory is mapped/mounted as /src within the running docker image - there sometimes can be user/permission mapping problems from host to docker - but that usually results in errors where it can't write output.

Here is the direct docker run commands to run - they also have the -B to force it to rebuild everything. Run this is the root directory of the project.

docker run --rm --privileged=true -v ./:/src/ -t vipoo/yellow-msx-rc2014-tool-chain:0.0.21 make -C msx apps -B
docker run --rm --privileged=true -v ./:/src/ -t vipoo/yellow-msx-rc2014-tool-chain:0.0.21 make -C msx -B

And also, I assume you were able to eventually pull all the submodule ok?

Another thing to try, is to disable the force parallel builds. I don't have that as a make variable. So you will need to manually modify the various makefiles - search for the switch -j and remove it. (I tried running with -j0, but it doesnt seem to pass down to the child-make invocations)

Search for these lines in the Makefile(s) and Makefile-app.mk

 pmsxsysromimages:
-       @$(MAKE) -j -O msxsysromimages --no-print-directory
+       @$(MAKE) -O msxsysromimages --no-print-directory

-       $(MAKE) ./bin/$1.nextor.rom -O -j --no-print-directory
+       $(MAKE) ./bin/$1.nextor.rom -O --no-print-directory

-       BUILD_TYPE=std $(MAKE) ../bin/mfr.nextor-2.1.1.rom -O -j --no-print-directory -f Makefile-rc2014.mk
+       BUILD_TYPE=std $(MAKE) ../bin/mfr.nextor-2.1.1.rom -O --no-print-directory -f Makefile-rc2014.mk

-       RTC=$2 COUNTRY=$1 $(MAKE) --no-print-directory ./working/$1-$2-rtc/main.rom ./working/$1-$2-rtc/optrom.rom ./working/$1-$
2-rtc/subrom.rom -O -j
-       cp -up ./working/$1-$2-rtc/main.rom    ../bin/$1-$2-rtc-main.rom
+       RTC=$2 COUNTRY=$1 $(MAKE) --no-print-directory ./working/$1-$2-rtc/main.rom ./working/$1-$2-rtc/optrom.rom ./working/$1-$
2-rtc/subrom.rom -O     cp -up ./working/$1-$2-rtc/main.rom    ../bin/$1-$2-rtc-main.rom

-       zip -j ../release/msx-rc2014-$${version}.zip ./bin/*.jed
-       zip -j ../release/msx-rc2014-$${version}.zip ./bin/*.doc
-       zip -j ../release/msx-rc2014-$${version}.zip ./bin/*.com
-       zip -j ../release/msx-rc2014-$${version}.zip ./bin/msxsyssrc-*.rom
-       zip -j ../release/msx-rc2014-$${version}.zip ./bin/yellow-*.rom
-       zip -j ../release/msx-rc2014-$${version}.zip ./bin/rcmusic.rom
+       zip ../release/msx-rc2014-$${version}.zip ./bin/*.jed
+       zip ../release/msx-rc2014-$${version}.zip ./bin/*.doc
+       zip ../release/msx-rc2014-$${version}.zip ./bin/*.com
+       zip ../release/msx-rc2014-$${version}.zip ./bin/msxsyssrc-*.rom
+       zip ../release/msx-rc2014-$${version}.zip ./bin/yellow-*.rom
+       zip ../release/msx-rc2014-$${version}.zip ./bin/rcmusic.rom

-       $(MAKE) --no-print-directory -j -C apps
+       $(MAKE) --no-print-directory -C apps

Can you tell me what host OS you have? CPU/Threads? How many threads does your CPU have?

zomgugoff commented 1 month ago

Yes, I was able to install the submodules. Build errors occurred a lot earlier with submodules missing.

I'm running on Arch, 8 cores/16 threads.

Building with the -j options removed, I'm still getting errors. Strangely, re-running make without -B after the failures results in built files. I tested the msxsyssrc-rc2014-usa-with-rtc.rom file and it seems to work fine on hardware. But, I wonder if some functionality is missing.

vipoo commented 1 month ago

Sounds like it got there in the end -- phew... - sorry that it was such a hassle for you.

I suspect if the build completed without error - then you do have a valid rom image.

I can only guess there is some subtle race condition in the Makefile that you are getting. And it takes a couple of tries to complete. I will try and see if I can reproduce --- but unless I can reproduce - not sure how to 'fix' it.

I will look at adding an option to force single thread for the Make script.

As I mention, this project builds on Github Actions. When a job kicks off (on every push), it builds from a complete clean environment - so I find it strange you have found such a build issue. Docker was meant to 'solve' this. Perhaps its the CPU/thread count difference. Who knows!

At least its kind of working for you. I might find some time tomorrow to have a look.

vipoo commented 3 weeks ago

This original issue, may have been due to a missing dependency rule in the make file. Hopefully the commit bcefd16 will fix this.

I also added (f32941f) to allow the use of an optional variable switch to force the build to run single threaded only - it will take a long time to build first time. Just export an environment variable called JOBS and assigned it to "-j1"

eg: export JOBS="-j1"

zomgugoff commented 2 weeks ago

Last build ran without fatal errors. Make still throws an error at the end of the build but I think it's related to the docs.

vipoo commented 2 weeks ago

I believe you are seeing the Latex Warning messages - I would love to be able to supress them. If you re-run the make - you should not see the Latex warnings - as the docs are unchanged and dont need to be re-built.