Compiling ps2-packer? - Githubissues

Jay-Jay-OPL commented 5 years ago

I was under the impression that all I needed was to make a git pull to this directory:

C:\MinGW\msys\1.0\home\JJ\ps2-packer

And that's all I needed to update ps2-packer.

But after doing a bit of exploring it seems I have to compile it (run a make) -- the issue is that I can't compile it. I get the following error:

$ make
gcc -O3 -Wall -I. -DVERSION=\"0.4.6\" -DPREFIX=\"/usr/lib/ps2dev\" ps2-packer.c
dlopen.c -o ps2-packer -ldl
c:/mingw/bin/../lib/gcc/mingw32/4.6.2/../../../../mingw32/bin/ld.exe: cannot fin
d -ldl
collect2: ld returned 1 exit status
make: *** [ps2-packer] Error 1

So can anyone tell me how to get around this error, or can they zip up the compiled version so I can replace the files found here in my Windows compiling environment:

C:\MinGW\msys\1.0\share\ps2-packer

Since it seems that is where the compiled files need to go for them to work.

So all this time, I was using a very old version that was provided from this link in a guide:

cd $HOME && wget -c http://web.archive.org/web/20090415101458/http://www.nobis-crew.org/ps2-packer/ps2-packer-1.0rc2-win32.zip && unzip -o ps2-packer-1.0rc2-win32.zip -d /share/ps2-packer

Or you guys can upload it to this repo under it's downloads/releases?

sp193 commented 5 years ago

Try building and installing libdl: https://stackoverflow.com/questions/12455160/using-libdl-so-in-mingw

Jay-Jay-OPL commented 5 years ago

Okay, I did that by downloading the repo, ran a make, it did something, placed some new files in my mingw directory, but still not working for me when I attempt to compile ps2-packer.

Maybe I am missing a step??

sp193 commented 5 years ago

What is the new error? I have not worked with MinGW since 2010, due to new versions not being able to correctly build our ancient version of GCC.

Jay-Jay-OPL commented 5 years ago

Okay, to get specific, when I ran the make for the following repo:

C:\MinGW\msys\1.0\home\JJ\dlfcn-win32

This was the output in the msys cmd window:

$ git clone https://github.com/dlfcn-win32/dlfcn-win32.git dlfcn-win32
Cloning into 'dlfcn-win32'...
remote: Counting objects: 341, done.
Receiving objects: 100% (341/341), 89.20 KiB | 3.57 MiB/s, done.
remote: Total 341 (delta 0), reused 0 (delta 0), pack-reused 341
Resolving deltas: 100% (176/176), done.

$ cd dlfcn-win32

$ ./configure --prefix=/ --libdir=/lib --incdir=/include && make && make install
testing compiler: gcc -shared -o tmptest.dll tmptest.c
prefix: /
libdir: /lib
incdir: /include
ar:     ar
cc:     gcc
ranlib: ranlib
strip:  strip
static: yes
shared: no
wine:
gcc -o dlfcn.o -c dlfcn.c -Wall -O3 -fomit-frame-pointer
ar cru libdl.a dlfcn.o
ranlib libdl.a
mkdir -p /include
install -m 644 dlfcn.h "/include"
mkdir -p /lib
cp libdl.a /lib

It placed two files in this directory:

C:\MinGW\msys\1.0\include\dlfcn.h
C:\MinGW\msys\1.0\lib\libdl.a

And the error I get when I try to compile ps2-packer is:

$ make
gcc -O3 -Wall -I. -DVERSION=\"0.4.6\" -DPREFIX=\"/usr/lib/ps2dev\" ps2-packer.c
dlopen.c -o ps2-packer -ldl
c:/mingw/bin/../lib/gcc/mingw32/4.6.2/../../../../mingw32/bin/ld.exe: cannot fin
d -ldl
collect2: ld returned 1 exit status
make: *** [ps2-packer] Error 1

sp193 commented 5 years ago

Try copying libdl.a to c:/MinGW/lib and dlfcn.h to c:/MinGW/include, if those directories exist.

Jay-Jay-OPL commented 5 years ago

Okay, those two directories did exist and I copied those two files to each of those directories.

I then tried again to compile ps2-packer and got this "new" error:

$ make clean && make
gcc -O3 -Wall -I. -DVERSION=\"0.4.6\" -DPREFIX=\"/usr/lib/ps2dev\" ps2-packer.c
dlopen.c -o ps2-packer -ldl
gcc -fPIC -O3 -Wall -I. -DVERSION=\"0.4.6\" -DPREFIX=\"/usr/lib/ps2dev\" zlib-pa
cker.c -shared -o zlib-packer.so -lz
zlib-packer.c:1:0: warning: -fPIC ignored for target (all code is position indep
endent) [enabled by default]
zlib-packer.c:23:18: fatal error: zlib.h: No such file or directory
compilation terminated.
make: *** [zlib-packer.so] Error 1

sp193 commented 5 years ago

You need to have to add the libraries for zlib1g-dev and libucl-dev, for both the MinGW and ps2sdk sides.

For MinGW, I cannot seem to find a pre-compiled version of libucl-dev, so you may have to compile a copy yourself.

As for zlib1g-dev: extract and copy the library and header files to their respective directories (MinGW, not MSYS): zlib1g-dev: https://sourceforge.net/projects/gnuwin32/files/zlib/1.2.3/ If it is not what I think it is, then you have to also compile a version of zlib for yourself.

As for the PS2 side, you need to check out ps2sdk-ports and build zlib and libucl (make, make install).

Jay-Jay-OPL commented 5 years ago

I noticed that the link you gave me above is for zlib v1.2.3 and I found a newer set zlib v1.2.11 at: http://www.zlib.net/ -- Would that be a better set or not?

When you say to extract and copy the files to their respective directories, can you give me a clue what those paths will be?

A question about the ps2sdk-ports. I've often run the following command to update them, can you confirm if they are correct? Perhaps I already have that installed? The only one I don't spot below is to build: libucl

cd $HOME/ps2sdk-ports/ git fetch origin && git reset --hard origin/master
cd $HOME/ps2sdk-ports/zlib && make clean && make install && make clean
cd $HOME/ps2sdk-ports/libjpeg && make clean && make install && make clean
cd $HOME/ps2sdk-ports/libpng && make clean && make install && make clean
cd $HOME/ps2sdk-ports/freetype-2.4.12/ && ./SetupPS2.sh

Please advise?

sp193 commented 5 years ago

If you can get a newer version, that is fine. What you are looking for, are the development libraries (and header files), not the zlib tool itself.

The respective directories are OS-specific. But I guess they are:

C:/MinGW/lib (for libraries)
C:/MinGW/include (for header files)

Yes, you might already have zlib for the PS2 side built and installed.

Jay-Jay-OPL commented 5 years ago

Hmm?? I'm afraid I hit a wall there, it's mostly due to my inexperience.

When you say the following "If it is for what I think it is, then you have to also compile a version of zlib for yourself."

Are you still referring to the zip file that I need to download and extract? And then select to correct OS (I guess the win32 folder) and then compile it? And then after it finishes compiling, I will need to put certain files in those two paths you listed above?

Am I following you correctly?

sp193 commented 5 years ago

Sorry, but I meant to write "if it is not what I think it is". I couldn't check what the archive contained because I was on my phone. I was expecting it to contain the libraries (libz.a) and the header file(s) (zlib.h and friends). But if it does not, then you need to compile and install a copy of zlib yourself, built from the source code. The link that I provided was for the MinGW package of zlib, so the package was meant for use with MinGW. The question is whether it is the right package for this purpose.

Usually, doing something like make install should copy the necessary files into the right place. However, it just did not seem to work for libdl (and I hope that is the only case).

Jay-Jay-OPL commented 5 years ago

Hmm? That's probably where I made my error when installing libdl -- I had run make instead of make install.

Now then, when I open the zip file: zlib-1.2.3-src

There are many folders and subfolders, plus many files. If you can, can you check which one is the root folder where I should run make install from msys?

sp193 commented 5 years ago

For most software: if there is a configure script, then you just need to run it first, before building and installing:

./configure
make
make install

I opened zlib-1.2.3-lib.zip and it contains the library files and headers, so why not use that instead? No need to build a copy yourself. You should be able to just extract the contents of the lib and include directories to their counterparts under C:\MinGW.

But if you want to build a copy yourself (for one reason or another), why not build the latest version from the zlib page?

Jay-Jay-OPL commented 5 years ago

Ah, I see where my fault was. That is quite simple.

You see, I had downloaded the wrong zip file. I had initially downloaded this one: zlib-1.2.3-src.zip -- but now I see that I should have gotten this one instead: zlib-1.2.3-lib.zip

Okay, I copied those folders/files inside the C:/MinGW directory.

Now I get a different error when I try to compile ps2-packer:

$ make
gcc -O3 -Wall -I. -DVERSION=\"0.4.6\" -DPREFIX=\"/usr/lib/ps2dev\" ps2-packer.c
dlopen.c -o ps2-packer -ldl
gcc -fPIC -O3 -Wall -I. -DVERSION=\"0.4.6\" -DPREFIX=\"/usr/lib/ps2dev\" zlib-packer.c -shared -o zlib-packer.so -lz
zlib-packer.c:1:0: warning: -fPIC ignored for target (all code is position independent) [enabled by default]
zlib-packer.c: In function 'pack_section':
zlib-packer.c:49:22: warning: assignment discards 'const' qualifier from pointer target type [enabled by default]
gcc -fPIC -O3 -Wall -I. -DVERSION=\"0.4.6\" -DPREFIX=\"/usr/lib/ps2dev\" lzo-packer.c minilzo.c -shared -o lzo-packer.so
lzo-packer.c:1:0: warning: -fPIC ignored for target (all code is position independent) [enabled by default]
minilzo.c:1:0: warning: -fPIC ignored for target (all code is position independent) [enabled by default]
gcc -fPIC -O3 -Wall -I. -DVERSION=\"0.4.6\" -DPREFIX=\"/usr/lib/ps2dev\" n2b-packer.c -shared 
-o n2b-packer.so -lucln2b-packer.c:1:0: warning: -fPIC ignored for target (all code is position 
independent) [enabled by default]
n2b-packer.c:23:21: fatal error: ucl/ucl.h: No such file or directorycompilation terminated.
make: *** [n2b-packer.so] Error 1

sp193 commented 5 years ago

Now to have zlib1g-dev installed for MinGW, but you still need to have libucl-dev installed. If you cannot find a pre-built library, then you have to build a copy from the source code.

Remember to also build and install zlib and libucl for the ps2sdk.

Jay-Jay-OPL commented 5 years ago

Do you happen to know a reliable source to get that: zlib1g-dev source files -- this way, I don't end up with someone's weird experimentation.

I've already run the ps2sdk-ports install for PS2SDK -- and though zlib was included, I seem to not find libucl, unless you mean ucl, since I do spot a folder there with that name?

Please advise?

sp193 commented 5 years ago

You just installed it. The library is called zlib. What you want are the development (devel) files, not the pre-compiled, shared libraries. In Linux, the package is known as zlib1g-dev. The shared library package is known as zlib1g.

Yes, libucl can be built from the ucl directory. The project is called "ucl".

Jay-Jay-OPL commented 5 years ago

I see, okay let me know if I did the zlib part correctly...

I downloaded this zip: zlib-1.2.3-src.zip

Extracted the zip file. I got a deep folder named: zlib-1.2.3 - and since I think that is the main folder, I renamed it to zlib and moved it to where I have all my repos I get from github (i.e. $HOME).

I then ran the following commands inside that folder:

JJ@PS2HOME ~/ps2-packer
$ cd ~/zlib

JJ@PS2HOME ~/zlib
$ ./configure
Building static library libz.a version 1.2.3 with /mingw/bin/mingw32-gcc.exe.
Checking for unistd.h... Yes.
Checking whether to use vs[n]printf() or s[n]printf()... using vs[n]printf()
Checking for vsnprintf() in stdio.h... Yes.
Checking for return value of vsnprintf()... Yes.
Checking for errno.h... Yes.
Checking for mmap support... No.

JJ@PS2HOME ~/zlib
$ make
/mingw/bin/mingw32-gcc.exe -O3   -c -o example.o example.c
/mingw/bin/mingw32-gcc.exe -O3   -c -o adler32.o adler32.c
/mingw/bin/mingw32-gcc.exe -O3   -c -o compress.o compress.c
/mingw/bin/mingw32-gcc.exe -O3   -c -o crc32.o crc32.c
/mingw/bin/mingw32-gcc.exe -O3   -c -o gzio.o gzio.c
/mingw/bin/mingw32-gcc.exe -O3   -c -o uncompr.o uncompr.c
/mingw/bin/mingw32-gcc.exe -O3   -c -o deflate.o deflate.c
/mingw/bin/mingw32-gcc.exe -O3   -c -o trees.o trees.c
/mingw/bin/mingw32-gcc.exe -O3   -c -o zutil.o zutil.c
/mingw/bin/mingw32-gcc.exe -O3   -c -o inflate.o inflate.c
/mingw/bin/mingw32-gcc.exe -O3   -c -o infback.o infback.c
/mingw/bin/mingw32-gcc.exe -O3   -c -o inftrees.o inftrees.c
/mingw/bin/mingw32-gcc.exe -O3   -c -o inffast.o inffast.c
ar rc libz.a adler32.o compress.o crc32.o gzio.o uncompr.o deflate.o trees.o zut
il.o inflate.o infback.o inftrees.o inffast.o
/mingw/bin/mingw32-gcc.exe -O3 -o example example.o -L. libz.a
/mingw/bin/mingw32-gcc.exe -O3   -c -o minigzip.o minigzip.c
/mingw/bin/mingw32-gcc.exe -O3 -o minigzip minigzip.o -L. libz.a

JJ@PS2HOME ~/zlib
$ make install
cp zlib.h zconf.h /usr/local/include
chmod 644 /usr/local/include/zlib.h /usr/local/include/zconf.h
cp libz.a /usr/local/lib
cd /usr/local/lib; chmod 755 libz.a
cd /usr/local/lib; if test -f libz.so.1.2.3; then \
          rm -f libz.so libz.so.1; \
          ln -s libz.so.1.2.3 libz.so; \
          ln -s libz.so.1.2.3 libz.so.1; \
          (ldconfig || true)  >/dev/null 2>&1; \
        fi
cp zlib.3 /usr/local/share/man/man3
chmod 644 /usr/local/share/man/man3/zlib.3

Let me know if the above looks correct, so I can now run the ps2sdk-ports updates....

sp193 commented 5 years ago

Yes, you have zlib built and installed. But in case you're getting confused, there are 4 parts that need to be built, for 2 different software libraries: zlib and ucl.

PC side (MinGW), used for building the PC-side packer part: 1a) zlib (aka zlib1g-dev) 1b) ucl (aka libucl-dev)
ps2sdk side , used for building the PS2-side unpacking stub: 2a) zlib 2b) ucl

Where there are pre-compiled libraries, you can just copy them to the right places. For MinGW:

For libraries: C:\MinGW\lib
For header files: C:\MinGW\include

For the PS2SDK (usually, we just build them from source. But FYI):

For libraries: $PS2SDK/ports/lib
For header files: $PS2SDK/ports/include

Otherwise, you have to compile the library from source:

./configure (if a configure script exists)
make
make install

The source code can be obtained from the respective project homepages of the projects.

Fortunately, there are pre-built copies around (although they are not the latest versions): ucl: http://gnuwin32.sourceforge.net/packages/ucl.htm zlib: http://gnuwin32.sourceforge.net/packages/zlib.htm

Earlier, we found that there is a pre-compiled copy of zlib, so you did not actually need to build a copy. But either way, you got the zlib library installed so it doesn't really matter. I found a copy of ucl, which you should be able to use.

As for the PS2SDK, we have a collection of ports (ps2sdk-ports). This is because our compilers are old and/or there are architecture-specific customizations that we need. Hence we modify the 3rd party projects and host them in ps2sdk-ports, so that it is easier for others to use.

Jay-Jay-OPL commented 5 years ago

Okay, I think I can can compile now ps2-packer -- at least I don't see anymore errors, and I do see that some files were placed somewhere within the C:\MinGW directory, but I don't see the files that I need. I should get one exe and a few dll files? Right? -- I do get the exe file, but where are those dll files?

What is the command to make compile? Is it just make? Or something else?

sp193 commented 5 years ago

You should have built ps2-packer with the make command, before installing it with make install. ps2-packer is installed to $PS2DEV/bin.

Congratulations on braving the cancer and getting to the holy grail.

Jay-Jay-OPL commented 5 years ago

Ah, okay, yes it seems that I have successfully installed it. Once again thank you for your tremendous help.

I was expecting it to replace the old ps2-packer files -- because that old version had a different location:

old ps2-packer path: C:\MinGW\msys\1.0\share\ps2-packer

but no now I see that it is where you say it should be under $PS2DEV/bin

new ps2-packer path: C:\MinGW\msys\1.0\lib\ps2dev\bin\

-- and only one single exe file is in there: ps2-packer.exe

Can you verify the MD5 checksum for me? 77f3be2acc0b5ed87ab8c951b2b1e429

Plus when I recompiled wLE at the end it said the following line about ps2-packer:

PS2-Packer v0.4.6 (C) 2004-2005 Nicolas "Pixel" Noble
This is free software with ABSOLUTELY NO WARRANTY.

When before it was giving me the older ps2-packer version.

PS2-Packer v1.0rc2 (C) 2004-2005 Nicolas "Pixel" Noble
This is free software with ABSOLUTELY NO WARRANTY.

Yes, this was quite an ordeal to get to this point. I can see now that this isn't something that the final exe can be shared (unless I am wrong?), since it seems it also has to create other files under the following path:

C:\MinGW\msys\1.0\lib\ps2dev\share\ps2-packer

In case you're wondering how I am able to notice what files seem to get added where, is that I made a backup, and then I am able to compare that backup image with the current state of the HDD, to see what changes between the two states. So I can see what folders, files, paths and etc. were created or modified or even removed.

One last thing. Is there an easier way to do all of what I just went through? Is there a way to make it easier for others? I wonder?

Maybe gathering all those files required and then make a script or part of the installation of ps2toolchain (or part of the ps2-packer) in order to get all this up and running?

If you think there is a way, I'd love for you to help me with that, since I want to make it easier for others....

I'll make a backup of this ticket issue, so I have something to look at if I later need to do this again.

Once again, thank you very much for helping me with this, I'm afraid that without your help, I would still be stuck at square one.

Plus I'd also like to share with you that the C:\MinGW directory has two similar paths that both have the folders lib/include and etc. See here:

C:\MinGW\lib
C:\MinGW\include
C:\MinGW\msys\1.0\lib
C:\MinGW\msys\1.0\include

And during this process, I'm not even sure which of the two similar paths all these files should go to and etc.? So it gets confusing, to say the least.

sp193 commented 5 years ago

Rightfully, it should go into $PS2DEV/bin. By the README of ps2toolchain, it should be located as /usr/local/ps2dev, but @doctorxyz (from your tutorial) instructed you to use /usr/lib/ps2dev for $PS2DEV instead. It's not wrong, but things will be different.

Since ps2-packer could not be as easily built on MinGW/Cygwin and it was a recommendation to just download the pre-built copy, but again doctorxyz did something different (files are unpacked to /share/ps2-packer):

unzip -o ps2-packer-1.0rc2-win32.zip -d /share/ps2-packer

As for getting a hash: unfortunately, that will not really be possible. I use Linux for PS2 development, which will surely result in a different hash for the binary. Even if I installed MinGW and tried building ps2-packer, any difference in the compiler version and libraries could result in a different binary being built. This would also result in the hash being different. If it does not work, you can just open another ticket. As long as this isn't a project that is as complex as GCC, I think we can still fix it...

As for compiling software: there's no real shortcut to... short of just providing a pre-built binary. Sorting out the necessary dependencies is something that has to be done for all projects.

Actually, I think you can share the ps2-packer binary. But I would rather upload it as a release here, which you can link to. I think the only parts of the toolchain that really cannot be shared, are binutils and gcc. At least, not the ancient versions we use, which are compiled to fit the target directory.

For the MinGW environment, there are the MinGW and MSYS parts. MSYS is just the shell. Just recently, I installed it back onto my PC because I wanted to try to build a new toolchain on Windows (as a downloadable, pre-built binary), but I had not sorted out the tools (and the required versions). But the MSYS 1.0 installation has no lib and include directories... Perhaps those contain files related to ps2dev, given that the paths used by the doctorxyz tutorial are different?

Jay-Jay-OPL commented 5 years ago

@sp193

Okay, after some rest, I wanted to investigate if the old ps2-packer directory is no longer needed or dependent so I can pack any elf during the compiling of OPL or wLE. -- My goal was to simply remove that old directory.

So what I did is simply rename the old ps2-packer folder to ps2-packer-bak -- just to see if I could still compile wLE without an issue.

Well, it backfired. I got this error at the end:

ps2-packer BOOT.ELF ULE.ELF
PS2-Packer v0.4.6 (C) 2004-2005 Nicolas "Pixel" Noble
This is free software with ABSOLUTELY NO WARRANTY.

Using special ucl-nrv2e asm (one section) stub
Compressing BOOT.ELF...
The specified module could not be found.
make: *** [ULE.ELF] Error 1

So to me, it seems it's still dependent of the old files -- most likely what's inside the stub folder? So I think we are not out of the woods yet with this issue / ticket.

Here is the old path of where ps2-packer sits -- plus here are the files that are within:

OLD PS2-PACKER EXE and FILES

C:\MinGW\msys\1.0\share\ps2-packer\
|   COPYING
|   lzo-packer.dll
|   n2b-packer.dll
|   n2d-packer.dll
|   n2e-packer.dll
|   null-packer.dll
|   ps2-packer.exe
|   README.txt
|   zlib-packer.dll
|   
\---stub
        lzo-0088-stub
        lzo-1d00-stub
        lzo-kmode-0088-stub
        lzo-kmode-1d00-stub
        n2b-0088-stub
        n2b-1d00-stub
        n2b-kmode-0088-stub
        n2b-kmode-1d00-stub
        n2d-0088-stub
        n2d-1d00-stub
        n2d-kmode-0088-stub
        n2d-kmode-1d00-stub
        n2e-0088-stub
        n2e-1d00-stub
        n2e-asm-1d00-stub
        n2e-asm-one-1d00-stub
        n2e-kmode-0088-stub
        n2e-kmode-1d00-stub
        null-0088-stub
        null-1d00-stub
        null-kmode-0088-stub
        null-kmode-1d00-stub
        zlib-0088-stub
        zlib-1d00-stub

Now here are the NEW paths where ps2-packer folder/files were placed:

NEW PS2-PACKER EXE LOCATION

C:\MinGW\msys\1.0\lib\ps2dev\bin\
    ps2-packer.exe

NEW PS2-PACKER FILES

C:\MinGW\msys\1.0\lib\ps2dev\share\ps2-packer\
|   
+---module
|       lzo-packer.so
|       n2b-packer.so
|       n2d-packer.so
|       n2e-packer.so
|       null-packer.so
|       zlib-packer.so
|       
\---stub
        lzo-0088-stub
        lzo-1d00-stub
        lzo-kmode-0088-stub
        lzo-kmode-1d00-stub
        n2b-0088-stub
        n2b-1d00-stub
        n2b-kmode-0088-stub
        n2b-kmode-1d00-stub
        n2d-0088-stub
        n2d-1d00-stub
        n2d-kmode-0088-stub
        n2d-kmode-1d00-stub
        n2e-0088-stub
        n2e-1d00-stub
        n2e-asm-1d00-stub
        n2e-asm-one-1d00-stub
        n2e-kmode-0088-stub
        n2e-kmode-1d00-stub
        null-0088-stub
        null-1d00-stub
        null-kmode-0088-stub
        null-kmode-1d00-stub
        zlib-0088-stub
        zlib-1d00-stub

As you can see above it put the exe by itself in a different path where the new stub folder and etc. was placed.

Plus also note, that the old ps2-packer folder had several other files that were never compiled with the new ps2-packer version (i.e. dll files and etc.)

So maybe I need to move some of those new files to the old location? Do you think that would work, or do you have a suggestion?

What I am trying to avoid is that perhaps the ps2-packer is still using older (obsolete) files to complete the packing process, because it may not find the newer files that it should use instead... Does that make sense?

sp193 commented 5 years ago

Did you move the new files, after renaming the old directory? Now I am quite convinced that this version cannot be moved because the PREFIX is fixed to $PS2DEV at compile-time... The version that I used to move about, was the lite (single stub, built into ps2-packer itself) version.

The current source code for ps2-packer does not have any rules for Windows, hence the stubs have the .so extension instead of .dll. It should be just a cosmetic difference, however.

sp193 commented 5 years ago

If you did not move the files, then you could try renaming all .so files to .dll. It appears that the Makefile has no rules for supporting Windows, but ps2-packer.c might have a rule for Windows, which would cause it to expect .dll extensions for the packer modules.

Jay-Jay-OPL commented 5 years ago

Okay, I haven't tried moving the files yet, so I'm still keeping them where they were placed when I ran the ps2-packer "make install"

I tried to rename the .so extensions to .dll

C:\MinGW\msys\1.0\lib\ps2dev\share\ps2-packer\
|   
+---module
|       lzo-packer.dll
|       n2b-packer.dll
|       n2d-packer.dll
|       n2e-packer.dll
|       null-packer.dll
|       zlib-packer.dll
|       
\---stub
        lzo-0088-stub
        lzo-1d00-stub
        lzo-kmode-0088-stub
        lzo-kmode-1d00-stub
        n2b-0088-stub
        n2b-1d00-stub
        n2b-kmode-0088-stub
        n2b-kmode-1d00-stub
        n2d-0088-stub
        n2d-1d00-stub
        n2d-kmode-0088-stub
        n2d-kmode-1d00-stub
        n2e-0088-stub
        n2e-1d00-stub
        n2e-asm-1d00-stub
        n2e-asm-one-1d00-stub
        n2e-kmode-0088-stub
        n2e-kmode-1d00-stub
        null-0088-stub
        null-1d00-stub
        null-kmode-0088-stub
        null-kmode-1d00-stub
        zlib-0088-stub
        zlib-1d00-stub

I then renamed the old ps2-packer folder to ps2-packer-bak:

C:\MinGW\msys\1.0\share\ps2-packer-bak

Tried to compile wLE and got this error:

ps2-packer BOOT.ELF ULE.ELF
PS2-Packer v0.4.6 (C) 2004-2005 Nicolas "Pixel" Noble
This is free software with ABSOLUTELY NO WARRANTY.

Using special ucl-nrv2e asm (one section) stub
Compressing BOOT.ELF...
The specified module could not be found.
make: *** [ULE.ELF] Error 1

Plus even Windows gave me a pop-up error window that stated the following:

ps2-packer.exe - System Error: The program can't start because ucl1.dll is missing from your computer. Try reinstalling the program to fix this problem.

So, should I move the files from: C:\MinGW\msys\1.0\lib\ps2dev\share\ps2-packer

To the old ps2-packer path? C:\MinGW\msys\1.0\share\ps2-packer

Or place those files next to the new ps2-packer.exe location? C:\MinGW\msys\1.0\lib\ps2dev\bin\

And should I place them exactly how the older folder structure is or not?

Thinking ahead: In case moving the files work? I wonder if this issue is the way ps2-packer got installed? Perhaps something can be fixed on the Makefile? So those files get placed where they should go?

Please advise?

UPDATE:

I restored the name of the old ps2-packer folder and simply tried to compile wLE and guess what? I still got the same error. So I renamed back the .dll files to .so extensions and I no longer get the error, so right now I am confused. I think the error was due to those .so extensions being renamed to .dll -- Hmm?

But that still doesn't fix the issue. ps2-packer still requires some files inside the old ps2-packer folder??

sp193 commented 5 years ago

Even if you do get away with renaming/moving files, there is likely something we are not understanding. I guess the PREFIX definition within ps2-packer.c somehow evaluates to an empty string. That could explain how the share folder (/share) is somehow selected instead of $PS2DEV/share. But because I don't see any obvious reason for this and neither do I have a working toolchain on Windows (particularly under MinGW), I cannot see why it's not working.

I have Cygwin installed, so I scapped together some code from ps2-packer, just to print what PREFIX becomes during compilation. But it seems to print "/usr/local/ps2dev" just fine.

There could be some obscure differences in the functions provided by some tool (could be Make or some shell tool), which is part of your MinGW installation. Given that your MinGW tools are likely very old as well, I don't even know whether this is a bug or a feature.

For now, at least until somebody can explain why this glitch occurs, I would like to put this on hold.

Jay-Jay-OPL commented 5 years ago

Hmm?

Should I revert back my changes and stay with the older ps2-packer or should I continue using the new version like it is now?

The new version seems to work, but not sure what files it may be fetching from the old ps2-packer version to complete the process? And that is what concerns me.

sp193 commented 5 years ago

With the files bundled with MinGW-get-20120426, I could not replicate the problem with the /shared directory being selected instead. So there should be just something different with your toolchain setup.

With commit 235fd72, it should now be possible to compile ps2-packer under Windows (within Cygwin, MinGW etc).

Libdl was merged into glibc, hence that dependency was removed. There was neither a need for ucl nor zlib from ps2sdk-ports, so that just leaves us to:

Check out the ps2-packer code
Ensure that the zlib and ucl development libraries are installed.

The obsolete MinGW Makefile rules were removed. I did not manage to get them to work. A side effect might be that the distribution package will not have MinGW and Linux versions together.

Jay-Jay-OPL commented 5 years ago

Should I revert all my changes and then update PS2-PACKER or should I just do a git pull and then do a make install to see if it works?

sp193 commented 5 years ago

If you want to try compiling it yourself, then you should revert any changes you made to the source code, before updating (git pull).

If you just want a new version, download the binary from the Releases page. The Windows version was always the "lite" (N2E-only) edition.

Jay-Jay-OPL commented 5 years ago

Sorry, I really did not understand your last instructions/comments...

So here is what I did.... (please bear with me....)

First I deleted the old ps2-packer folder (the one that I had for years): C:\MinGW\msys\1.0\share\ps2-packer (see earlier posts to know what's inside this path)

I then removed the following new paths/files that were created for ps2-packer during the above steps we did:

C:\MinGW\msys\1.0\lib\ps2dev\share\ps2-packer
C:\MinGW\msys\1.0\lib\ps2dev\bin\ps2-packer.exe

I then under the ps2-packer local repo I ran make clean && git pull && make install && make clean with msys cmd window.

I noticed that this time it didn't create the files with the .so extension under this directory, but instead created them as .dll extensions: C:\MinGW\msys\1.0\lib\ps2dev\share\ps2-packer\module

And it placed the .stub files inside this directory (like before) C:\MinGW\msys\1.0\lib\ps2dev\share\ps2-packer\stub

It did put the ps2-packer.exe in this directory: C:\MinGW\msys\1.0\lib\ps2dev\bin\ps2-packer.exe

When I attempted to compile wLE I got the same errors as before (except now it's sporting the new build version (v1.10):

ps2-packer BOOT.ELF ULE.ELF
PS2-Packer v1.1.0 (C) 2004-2005 Nicolas "Pixel" Noble
This is free software with ABSOLUTELY NO WARRANTY.

Using special ucl-nrv2e asm (one section) stub
Compressing BOOT.ELF...
The specified module could not be found.
make: *** [ULE.ELF] Error 1

Plus it gave me the windows popup error as before:

ps2-packer.exe - System Error:
The program can't start because ucl1.dll is missing from your computer. Try reinstalling the program to fix this problem.

I recall in order to fix the windows popup is to rename the .dll files to .so files, so I did that: C:\MinGW\msys\1.0\lib\ps2dev\share\ps2-packer\module

That stopped the Windows Popup, but still got the msys error, and in order to fix that one (if you recall), I have to make sure the old ps2-packer needs to be in this path: C:\MinGW\msys\1.0\share\ps2-packer

But instead of putting the old one, I decided to use the release you uploaded to the ps2-packer repo and grabbed this zip file: ps2-packer-1.1.0.zip -- extracted the files and pretty much emulated the old ps2-packer folder's structure (see earlier replies above to see what's inside this directory).

Now when I run to compile wLE, it will compile without any errors. So at the very least I am using the latest files from ps2-packer.

ps2-packer BOOT.ELF ULE.ELF
PS2-Packer v1.1.0 (C) 2004-2005 Nicolas "Pixel" Noble
This is free software with ABSOLUTELY NO WARRANTY.

Using special ucl-nrv2e asm (one section) stub
Compressing BOOT.ELF...
File compressed, from 1063444 to 452377 bytes, ratio = 57.46%

So somehow it's still requires some files from the older ps2-packer path? It would be nice it didn't need that, but at least it works, right?

FYI: the last version of ps2toolchain I installed was: https://github.com/ps2dev/ps2toolchain/commit/31392d06e2917aa5c4b584992685dbd370214717 (Jan 14, 2017)

So my question to you, do you think I would have to reinstall everything fresh again? In case that would help?

In case something above isn't clear, let me know??

sp193 commented 5 years ago

As for the need for the ucl.dll file, perhaps it has got something to do with the ucl library you installed. It might have something to do with the compile-time options that it was built with. I built new copies of the zlib and ucl libraries, based off the latest versions available.

There are 3 ways for ps2-packer to try loading the packer module:

PREFIX/packer
/packer
./packer

Where "packer" is the name of the packer module. A similar method is used to locate the path to the unpacking stubs, for the full ps2-packer version.

The distribution packages make use of the middle path. This would be why unpacking the distribution package to anywhere seems to work and the stubs folder is with the ps2-packer executable.

On the other hand, the top-most path is used if you built and installed ps2-packer on your own.

I doubt the toolchain matters because things have more or less stayed the same since 2004. Much less changes were made in recent years.

I have attempted to change my $PS2DEV path to /usr/lib/ps2dev, but it does not cause ps2-packer to expect files under /usr/shared. I have no idea why or even how you have been able to make it build for one directory but yet expect files in another. The good news is that unless you want to try building your own copy, just use the pre-built version.

I slipped in a few more fixes today.

Jay-Jay-OPL commented 5 years ago

About the ucl library that I installed? Is there a way to revert that install?

If not, then I was thinking of restoring my old back up prior to doing all of the above changes. Then simply download the new ps2-packer-1.1.0.zip file and simply replacing the old one I had in this path: C:\MinGW\msys\1.0\share\ps2-packer

Your advise?

sp193 commented 5 years ago

If you wish to remove any packages, simply delete the files that were copied over. It is not a really simple thing to do, but that is how things are.

Jay-Jay-OPL commented 5 years ago

Okay, I decided to simply roll back my changes from my last backup -- this backup was done before I started to make the changes in this ticket.

I then downloaded the new ps2-packer: ps2-packer-1.1.0.zip

Deleted the old contents in this folder/path: C:\MinGW\msys\1.0\share\ps2-packer

And then simply extracted the new files into that empty folder.

I then ran a git pull for wLE, since my backup still didn't have the new changes ( https://github.com/AKuHAK/uLaunchELF/commit/3576a2badc544910752114374c40342115f6eb40 ) you made to it.

I was able to compile wLE and the new ps2-packer files did it's magic -- as you can see below.

ps2-packer BOOT.ELF ULE.ELF
PS2-Packer v1.1.0 (C) 2004-2005 Nicolas "Pixel" Noble
This is free software with ABSOLUTELY NO WARRANTY.

Using special ucl-nrv2e asm (one section) stub
Compressing BOOT.ELF...
File compressed, from 1063444 to 452313 bytes, ratio = 57.47%

So now all those ps2-packer files only sit here and nowhere else: C:\MinGW\msys\1.0\share\ps2-packer

So that takes care of that, and I'll just remember to not try to compile ps2-packer myself, so I won't run into that snag -- unless you think I should try again to compile ps2-packer now without having to install the above things I went through? Perhaps something I installed throughout this ordeal got me into that situation??

Meanwhile, I plan to update the Windows Compiling Tutorial and simply point to the new zip file so others can now use the new ps2-packer:

cd $HOME && . .profile
cd $HOME && wget -c https://github.com/ps2dev/ps2-packer/releases/download/1.1.0/ps2-packer-1.1.0.zip && unzip -o ps2-packer-1.1.0.zip -d /share/ps2-packer
cd $HOME && . .profile

Does the cmd above look right to you? Do you know if it will simply rewrite the old files? Or do you recommend that we should include the removal of the old folder prior to extracting the new one? If yes, could you help me rewrite it, so it targets the /share/ps2-packer path?

I also plan to run it, once I am done with something else I am currently doing, but just wanted to bring you up to speed to what I've done so far.

So as soon as I can test that, I guess we can finally close this ticket.

sp193 commented 5 years ago

Right. Glad to hear that it is working fine now!

As for the toolchain stuff:

I don't understand why you need to keep changing directories. This part alone should be sufficient:

cd $HOME && wget -c https://github.com/ps2dev/ps2-packer/releases/download/1.1.0/ps2-packer-1.1.0.zip && unzip -o ps2-packer-1.1.0.zip -d /share/ps2-packer

Overwriting the old files should be fine, since it will expect and use the files that it comes with. If you think that users will not need the other packers, then you could choose to install the lite ps2-packer instead.

If you are updating the tutorial:

You can remove the instructions for installing and setting up Mercurial. Unless you think anybody will be working on a repository that uses that.
Git can be installed to its default path (to Program Files) and it should work.
There should be also no need to prevent the computer's display from being switched off. It's only a problem if the computer goes to sleep, so it is appropriate to prevent the computer from hibernating or entering standby.
What's the "Log Off" feature, which should be disabled under step 4? If you're referring to the Windows function that locks the computer after a period of inactivity, it does not log the user out (any software that the user was running, will continue running).
For Windows, Git is added to the Windows PATH environmental variable. So there should be no need to add Git to MinGW/Cygwin path (part C).

I made amendments to the toolchain's script, so that it'll use all cores if your CPU is multi-core. It may still not build as fast as on Linux, but it might be faster than before.

I didn't need to run MSYS with administrator priviledges. I think it's only going to be a problem if MSYS/MinGW was installed to Program Files, which the Windows UAC protects. It's a reason why some software nowadays are installed to other places like C:\ or %APPDATA% by default, to avoid requiring the user to have admin rights each time the software is launched. But if you had an actual need to run the script as an administrator, do keep it.

I didn't need the fix for collect2. I think there was already some fix for this within the toolchain patches. As for the problem with the uintptr_t type definition, I haven't decided how it should be fixed yet. But I think the fix should be made to the configure scripts instead (the tools seem to not be C99-compliant).

If the user's profile name has spaces in it, then the toolchain scripts will likely fail. If the user's name has spaces in it, it is advisable to check out the toolchain and build it elsewhere instead. Since this is not Linux, the user can even build it in /home instead:

cd ..
_Other commands_

Jay-Jay-OPL commented 5 years ago

Thanks for the extra information above, I will update the guide. And yes, I always wondered what was the purpose for Mercurial? Like I said before, this was the guide I got from doc, and I pretty much kept it like that for years.

Well, I stumbled on another snag.

It seems that my version of MinGW has an outdated wget version (GNU Wget 1.12 built on msys.), since I can't download the ps2-packer since it's under a https: URL. -- if you recall the old link to download that old ps2-packer was on a http: URL. Plus I even tested it by putting the zip file to download from ps2-home and that works also for me -- so I have that option to store it on our site.

Here is the error I get when I attempt to run this:

cd $HOME && wget -c https://github.com/ps2dev/ps2-packer/releases/download/1.1.0/ps2-packer-1.1.0.zip && unzip -o ps2-packer-1.1.0.zip -d /share/ps2-packer

OpenSSL: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol
 version
Unable to establish SSL connection.

And I'm not sure how to update wget or that it will be compatible with my version of MinGW? So if you have any suggestions, how to fix that, thanks in advance.

sp193 commented 5 years ago

I've forgotten if we needed to get OpenSSL installed for this. If you had to download and install/extract OpenSSL (for Windows) to get Wget to work, you might just need to update it.

If you cannot find a newer version of Wget that you can use (not even from newer MinGW packages), then I am afraid that you have to compile a new version on your own.

Jay-Jay-OPL commented 5 years ago

I'd like to try anything at this point, since I can backup my current state of how I got MinGW setup. So if you can give me something to follow and try, I'm willing to give it a shot.

I've tried doing a bit of searching in the internet, but I am not sure what to install or test.

ps2dev / ps2-packer

Compiling ps2-packer? #13