search

ps2dev / gsKit

gsKit is a C interface to the PS2 Graphics Synthesizer
Other
103 stars 27 forks source link

Attempting to compile the latest version of OPL gets errors. #21

Closed Jay-Jay-OPL closed 5 years ago

Jay-Jay-OPL commented 5 years ago

@rickgaiser and @sp193 -- Just letting you know...

Today, when I updated gsKIT, to the latest commit: https://github.com/ps2dev/gsKit/commit/813f66342597df950f57fc2c109e341abb0c3dbd

When I attempt to compile the latest OPL commit: https://github.com/ifcaro/Open-PS2-Loader/commit/ad182dcf95e3322206f5c2f5ced99666ac1621f8

It will shoot out these errors (Windows Compiling Environment msys):

Building Open PS2 Loader 0.9.3+.1188-Beta-ad182dc...
-Interface
In file included from src/pad.c:7:
include/opl.h:21:19: gsKit.h: No such file or directory
include/opl.h:22:20: dmaKit.h: No such file or directory
include/opl.h:23:23: gsToolkit.h: No such file or directory
make[1]: *** [obj/pad.o] Error 1
make: *** [all] Error 2

Since even this commit also fails to compile and update gsKit: https://github.com/ps2dev/gsKit/commit/81aeb916f930efed108f3da1b183e0aa86886b23

So it seems that @PGAnonamous tried to fix it in commit: https://github.com/ps2dev/gsKit/commit/813f66342597df950f57fc2c109e341abb0c3dbd But its still broken for compiling OPL under a Windows Compiling Environment.

Hopefully you guys can figure this out?

At least if we revert back to this commit of gsKit OPL will compile: https://github.com/ps2dev/gsKit/commit/ed20b404f17a7ee85d144a05754745c570c70ca2

Plus, I can only imagine that @AKuHAK 's OPL auto-bot script may also get affected by this? Since it compiles everything from scratch on a daily basis. So that may be broken also...

ghost commented 5 years ago

@Jay-Jay-OPL if you didn't use setup.sh for gsKit then you forgot to run make install for gsKit.

As a sanity check I repeated making and installing the latest commits and can confirm this only occurs when running make in gsKit and not installing instead of running setup.sh.

I think commit https://github.com/ps2dev/gsKit/commit/813f66342597df950f57fc2c109e341abb0c3dbd was just a fix for the examples because they were not linking to the gsKit libraries.

Jay-Jay-OPL commented 5 years ago

This is the command I run:

cd $HOME/gsKit/ && make clean && git fetch origin && git reset --hard origin/master && make && make install && make clean

That's always been the command that I run that works, so did something recently changed?

TnA-Plastic commented 5 years ago

I recently build the PS2SDK from scratch and the newest builds (atleast from a few days ago) could still be compiled properly... (I used WSL.)

So it is a problem of the very newest merges?

ghost commented 5 years ago

@Jay-Jay-OPL is GSKIT set in your enviroment? If echo $PS2DEV/gsKit and echo $GSKIT don't output the same thing that that may be the problem. If it's not just enter echo >>~/.profile "export GSKIT=\$PS2DEV/gsKit" && . ~/.profile and it should build successfully.

I am also using WSL, but I don't see how using any other environment would cause the build to fail if it is installed and GSKIT is set in your environment.

sp193 commented 5 years ago

You need to have the GSKIT environmental variable set. Some software like OPL depends on it, otherwise it will never find the header files.

The setup.sh file already existed for gsKit since a long time ago. It exists to simplify the installation process. Although you can also set the necessary environmental variables (as setup.sh does), before issuing the make & make install commands.

TnA-Plastic commented 5 years ago

Alright and the setup.sh is called/started, if someone installs it 'afresh'... That explains it.

Edit: So basically, this is an issue due to a user-error! ;)

Jay-Jay-OPL commented 5 years ago

@PGAnonamous , Yes, a while back when I set up my Windows Compiling Environment I did run this:

echo >>~/.profile "export GSKIT=\$PS2DEV/gsKit"

And since I can compile OPL with an older version of gsKit (see the commit I mentioned in the first post above), that means that it was okay until the commit I mention above that it broke that ability.

If you wish to explore the method I installed my Windows Compiling Environment, here is the guide: http://www.ps2-home.com/forum/viewtopic.php?f=50&t=169

So what is the issue? Do I need to add the setup.sh within the command below:

OLD: cd $HOME/gsKit/ && make clean && git fetch origin && git reset --hard origin/master && make && make install && make clean

NEW: cd $HOME/gsKit/ && make clean && git fetch origin && git reset --hard origin/master && setup.sh && make && make install && make clean

Please let me know if the above command looks proper to run??

Or is there another issue here, I wonder?

Because I think setup.sh might just repeat the following setup:

echo >>~/.profile "export GSKIT=\$PS2DEV/gsKit"

Which I did a while back...

sp193 commented 5 years ago

Could you do: echo $GSKIT

If the path to gsKit is not displayed, then you do not have it set.

The setup.sh file will not set that variable. In fact, there was never any standardized instruction for setting up this global variable, although some projects use it.

Jay-Jay-OPL commented 5 years ago

@sp193, here you go, this is what I get when I run that on msys:

$ echo $GSKIT
/usr/lib/ps2dev/gsKit
sp193 commented 5 years ago

Okay, so we have GSKIT pointed to /usr/lib/ps2dev/gsKit. Did it actually compile completely? When you do make, it should complete with no errors.

Jay-Jay-OPL commented 5 years ago

@SP193,

Okay...

When I try to install the latest version of GSKIT: https://github.com/ps2dev/gsKit/commit/813f66342597df950f57fc2c109e341abb0c3dbd , there are no errors when I run the following command cd $HOME/gsKit/ && make clean && git fetch origin && git reset --hard origin/master && make && make install && make clean

But that is when I get the this error when I try to compile OPL:

Building Open PS2 Loader 0.9.3+.1188-Beta-ad182dc...
-Interface
In file included from src/pad.c:7:
include/opl.h:21:19: gsKit.h: No such file or directory
include/opl.h:22:20: dmaKit.h: No such file or directory
include/opl.h:23:23: gsToolkit.h: No such file or directory
make[1]: *** [obj/pad.o] Error 1
make: *** [all] Error 2

But when I then tried the previous GSKIT bundled update: https://github.com/ps2dev/gsKit/commit/81aeb916f930efed108f3da1b183e0aa86886b23 -- this one did give some errors when I tried to install that previous update. This is why I assumed that the changes made from @PGAnonamous was an attempt to fix that -- which it did.

Here is the command that I used to try the this previous version: cd $HOME/gsKit/ && make clean && git checkout 81aeb91 && make && make install && make clean

Let me know if you wish for me to try the above version? If you need to see what the errors say?

UPDATE: here is the last 7 lines when it fails when I attempt to install the above GSKIT commit:

C:/MinGW/msys/1.0/home/JJ/gsKit/lib\libdmakit.a(dmaInit.o)(.text+0x184):dmaIn
it.c: undefined reference to `puts'
make[2]: *** [basic.elf] Error 1
make[2]: Leaving directory `/home/JJ/gsKit/examples/basic'
make[1]: *** [all-basic] Error 2
make[1]: Leaving directory `/home/JJ/gsKit/examples'
make: *** [all-examples] Error 2

So, since the most current version of GSKIT doesn't allow me to compile OPL and the previous version above of GSKIT doesn't finish installing, my only solution was to revert to an older version: https://github.com/ps2dev/gsKit/commit/ed20b404f17a7ee85d144a05754745c570c70ca2

Here is the command that I ran to install that older GSKIT version that works and allows me to compile OPL:
cd $HOME/gsKit/ && make clean && git checkout master && git checkout ed20b40 && make && make install && make clean

And this older version is the one that I've been using in the previous OPL that I compiled and shared at PS2-HOME.

Barely today, when I tried to update to the latest GSKIT version, is when I came across this issue. (Thinking in my head that I should always try to update my PS2SDK environment whenever I see that new updates are available under the PS2DEV repo at GIT in order to offer the next OPL BETA version with all those updates as well.)

FYI: I also updated PS2SDK, PS2ETH, and also PS2SDK-PORTS today.

Let me know if something isn't clear, or if I confused you? Sorry...

ghost commented 5 years ago

@PGAnonamous , Yes, a while back when I set up my Windows Compiling Environment I did run this:

echo >>~/.profile "export GSKIT=\$PS2DEV/gsKit"

And since I can compile OPL with an older version of gsKit (see the commit I mentioned in the first post above), that means that it was okay until the commit I mention above that it broke that ability.

If you wish to explore the method I installed my Windows Compiling Environment, here is the guide: http://www.ps2-home.com/forum/viewtopic.php?f=50&t=169

So what is the issue? Do I need to add the setup.sh within the command below:

NEW: cd $HOME/gsKit/ && make clean && git fetch origin && git reset --hard origin/master && setup.sh && make && make install && make clean

Please let me know if the above command looks proper to run??

This would be under the assumption that you need the git commands for the environment you have (multiple branches of the repo etc) but it should just be cd $HOME/gsKit/ && make clean && git fetch origin && git reset --hard origin/master && ./setup.sh the script will take care of everything else. It checks your environment for the ps2sdk-ports such as libjpeg and adds functions such as gsKit_texture_jpeg if the setup script found it.

I haven't been able to replicate the issue in wsl so I will try setting up in msys and see if I can recreate it there. It will take a while though as msys configures ridiculously slow.

sp193 commented 5 years ago

This is actually 7 commands: cd $HOME/gsKit/ && make clean && git fetch origin && git reset --hard origin/master && make && make install && make clean

Did you actually scroll through the output to see if all commands really succeeded? If so, then you should have also some files in:

For your convenience, you can do these:

ls $PS2DEV/gsKit/include
ls $PS2DEV/gsKit/lib

You should see header and library files.

ghost commented 5 years ago

@Jay-Jay-OPL So I followed the tutorial you linked above and the only thing I had to do differently was change to the correct free-type and and run dos2unix through libpng before it would build. After that OPL built without an issue.

Jay-Jay-OPL commented 5 years ago

@sp193

Did you actually scroll through the output to see if all commands really succeeded?

Since I use msys and I guess it uses the Windows CMD window, after a while, I have a limit how much I can scroll to view the window. It gets cropped.

But yes, I might be able to do what you suggest, run each command to see if I spot if it gets stuck somewhere. Good tip.

Here are the results when I ran those commands you wanted for me to enter:

$ ls $PS2DEV/gsKit/include
dmaArrays.h  dmaKit.h  gsFontM.h  gsInline.h  gsPrimitive.h   gsToolkit.h
dmaCore.h    dmaSpr.h  gsHires.h  gsKit.h     gsTexManager.h  gsVU1.h
dmaInit.h    gsCore.h  gsInit.h   gsMisc.h    gsTexture.h
$ ls $PS2DEV/gsKit/lib
libdmakit.a  libgskit.a  libgskit_toolkit.a

Do let me know if that looks complete to you?

I may also try what @PGAnonamous suggests, I don't think it will hurt anything running this command: cd $HOME/gsKit/ && make clean && git fetch origin && git reset --hard origin/master && setup.sh

@PGAnonamous

the only thing I had to do differently was change to the correct free-type and and run dos2unix through libpng before it would build.

Can you specify what exact steps did you have to do there? Just to make sure I got that covered also...

sp193 commented 5 years ago

So we're back where we started. This makes no sense. Do these show the exact same path?

dos2unix is a tool for converting the line endings from CRLF to LF. @PGAnonamous used it to fix a compilation problem. So you don't need to do it if you have no problems building the libraries. You have a file not found error while compiling a 3rd party piece of software, which is different.

Do you usually delete the contents of $GSKIT before you build and install gsKit? Even if the Makefile has been installing gsKit in the wrong place, I don't understand why your old library files cannot be found either.

ghost commented 5 years ago

This should be a last resort. I don't have any other ideas at the moment though.

@Jay-Jay-OPL By any chance have you recently gone through windows update? My only thought at this point is maybe the latest update did something that is conflicting with the msys environment. I know that everytime I go through windows update it reenables my camera and microphone and stuff so it's just a thought.

Can you try adding both PS2DEV and GSKIT as system environment variables in windows? Add them in Unix path style ( instead of starting with C:/ start it with /c/ ). So your going to add the eviroment variables PS2DEV /c/(path to PS2DEV folder) and GSKIT /c/(path to GSKIT folder) then reboot windows and try to build it again.

If this works then I would recommend either reinstalling mingw and msys to fix any other potential conflicts or setting up wsl instead.

If you need a tutorial for setting up wsl there is a good one here from @TnA-Plastic wrote that I followed. http://psx-scene.com/forums/f19/quick-dirty-tutorial-install-ps2sdk-windows-subsystem-linux-2017-a-157028/

sp193 commented 5 years ago

MSYS is just another program, so I don't think Windows update will matter too much. I'm also using Windows 10 and did build the MSYS environment recently to work on ps2-packer. Although it is a fact that you're using an ancient version of it, and there's nothing much that can be done since the packages that come with any new version will cause GCC to fault.

But I would recommend that we try fixing this first, before going on to WSL. Moving over is another major task, which will take up a lot of time to set everything up again. Configuration issues can happen under any platform, after all. Unless you already had plans to move over, then you should not even bother with fixing this.

Before adjusting the Windows globals (I don't think you should anyway), you should try to check whether $PS2DEV/gsKit and $GSKIT point to the same thing within MSYS. If there is actually a problem there, then you should be able to fix it within MSYS, hence avoiding any need to mess with Windows.

Jay-Jay-OPL commented 5 years ago

UPDATE:

Okay, good news. I was able to compile OPL with the latest changes to GSKIT: https://github.com/ps2dev/gsKit/commit/813f66342597df950f57fc2c109e341abb0c3dbd

Now before I go through what I did to fix this, here are my replies to your questions:

@sp193

Do these show the exact same path?

Yes, here are the results:

$ echo $PS2DEV/gsKit
/usr/lib/ps2dev/gsKit

$ echo $GSKIT
/usr/lib/ps2dev/gsKit

Do you usually delete the contents of $GSKIT before you build and install gsKit?

Yes, that is what I do. I always run this part of the update process for GSKIT:

rm -rf $GSKIT && mkdir $GSKIT
cd $HOME/gsKit/ && make clean && git fetch origin && git reset --hard origin/master && make && make install && make clean

Now to tell you what I ended up doing to fix this.

Well first I ran the following inside the gskit path:

git checkout master

because if you recall, I had to use an older GSKIT version https://github.com/ps2dev/gsKit/commit/ed20b404f17a7ee85d144a05754745c570c70ca2 in order to make OPL compile.

After that I ran the command that @PGAnonamous recommended above: cd $HOME/gsKit/ && make clean && git fetch origin && git reset --hard origin/master && setup.sh

But even though I did spot some errors when it was updating GSKIT and installing, I was able to compile the new changes made into OPL (yesterday):

This is one of the errors I got when updating/installing GSKIT (and I think I spotted it twice):

C:/MinGW/msys/1.0/home/JJ/gsKit/lib\libdmakit.a(dmaInit.o)(.text+0x184):dmaIn
it.c: undefined reference to `puts'
make[2]: *** [basic.elf] Error 1
make[1]: *** [install-basic] Error 2
make: *** [install-examples] Error 2
Performing post-install cleanup.

I also spotted this message, not sure if I need to install LIBTIFF:

This script will attempt to automatically detect the presence of LIBPNG, LIBJPEG
, LIBTIFF and ZLIB, and will set up all required environmental variables.
If your libraries are not stored in /usr/lib/ps2dev/ps2sdk/ports, please specify
 the location of the libraries manually.
Performing pre-install cleanup.
Libraries:
        LIBJPEG detected.
        LIBPNG detected.
        ZLIB detected.
        LIBTIFF not detected.
Building gsKit.

I'm also wondering at this point, that perhaps the new updates to OPL https://github.com/ifcaro/Open-PS2-Loader/commit/70fb5f58db3a9c3ae117db593ed05553e5b1ae64 from @rickgaiser

Because if you recall, I was having problems compiling OPL with the previous commit: https://github.com/ifcaro/Open-PS2-Loader/commit/ad182dcf95e3322206f5c2f5ced99666ac1621f8

Perhaps the new changes in OPL helped to finally compile with the new changes made to GSKIT, since it was he that updated those two areas that perhaps had something in relation that it required OPL to have the new GSKIT changes? I dunno?

But I am now able to compile the current OPL update" https://github.com/ifcaro/Open-PS2-Loader/commit/70fb5f58db3a9c3ae117db593ed05553e5b1ae64 with the latest GSKIT repo changes: https://github.com/ps2dev/gsKit/commit/813f66342597df950f57fc2c109e341abb0c3dbd , so I guess we can mark this issue solved, but still want to know if the above issues reported during the update to GSKIT should be something I should be concerned about? Let me know?

One thing I didn't do is enter each installation process (step by step) to see if I spot more errors with GSKIT, do let me know if you still want me to do that, I got no problem doing this, to see if there is another issue that can be fixed to make it work better for others. Feel free to use use me for your tests. :)

sp193 commented 5 years ago

If you failed to build gsKit, then nothing would be installed. I am not sure how you got those library files listed when they are not supposed to exist and neither could LD find them... but failing to build gsKit should have meant that they were not actually built and copied over.

As for why it seems to be failing, perhaps you are using an old toolchain (before @rickgaiser changed GCC to include our SDK libc by default). For compatibility, we can do a partial rollback of 3efa030 to include libc and libkernel from Makefile.global.

It is not required for anybody to have the libtiff, libpng nor libjpeg, but the script makes it easier to enable gsKit's built-in support for textures in those formats. Previously, you had to do it manually.

rickgaiser commented 5 years ago

Thinking in my head that I should always try to update my PS2SDK environment whenever I see that new updates are available under the PS2DEV repo at GIT in order to offer the next OPL BETA version with all those updates as well.

Since there are no stable versions (or no versions at all), I would advise to always use the latest versions of ALL sources you're using to build your OPL daily build releases.

Attach the git hash of all used software to the release so we can do bug tracking if newer versions have problems (or are you already doing this?).

I'm sorry I cannot comment on the whole compiling on windows thing, I'm always using Linux.

As for why it seems to be failing, perhaps you are using an old toolchain (before @rickgaiser changed GCC to include our SDK libc by default). For compatibility, we can do a partial rollback of 3efa030 to include libc and libkernel from Makefile.global.

I think having an up-to-date toolchain is a better solution. The change was made over 5 months ago.

It is not required for anybody to have the libtiff, libpng nor libjpeg, but the script makes it easier to enable gsKit's built-in support for textures in those formats.

If you're including libpng and/or libjpeg support, then open-ps2-loader will be larger in size, so it's probably better to exclude them when building a release. If you're including libtiff, then open-ps2-loader will not link, becouse open-ps2-loader does not include the libtiff library.

Jay-Jay-OPL commented 5 years ago

@sp193

I am not sure how you got those library files listed when they are not supposed to exist and neither could LD find them...

If you are referring to these libraries:

Libraries:
        LIBJPEG detected.
        LIBPNG detected.
        ZLIB detected.
        LIBTIFF not detected.

Those libraries got listed when I ran setup.sh -- going by what @PGAnonamous recommended above to complete installing GSKIT...

As for why it seems to be failing, perhaps you are using an old toolchain (before @rickgaiser changed GCC to include our SDK libc by default).

Yes, I am using a very old update of ps2toolchain. https://github.com/ps2dev/ps2toolchain/commit/31392d06e2917aa5c4b584992685dbd370214717 (Jan 14, 2017 ) -- I was under the impression that as long as I updated the others (PS2SDK, GSKIT, PS2SDK-PORTS, PS2ETH, and etc.), that I wouldn't have to run the update to ps2toolchain. I thought that was a one-time deal.