Open snake-biscuits opened 1 month ago
The Dreamcast release of Quake III is going to be a pretty essential test case
D:/Emulators/Sega/Dreamcast/quakeiii.cdi
quake.16.iso
QUAKE3/BASEQ3/MAPS/DC_MAP01/BSP.ZIP
AAS.ZIP/botfiles/botfiles.zip/botfiles/game.h
(QuakeC paired with the map?)maps/dc_map01.bsp
We'll also want to roll extensions.archives
into the core bsp_tool
And to read from archives like Vpk
& RPak
we'll need other libraries for decompressing Oodle & LZHAM
This means bsp_tool
will no longer be builtins only
However, this means we can roll lightmaps
into core as well
And we can use Pillow
for MipTextures
(I think, can we do texture palletes?)
archives should move away from the zipfile.ZipFile
__init__
form
.from_stream
etc. would be better
since we will be opening archives inside other archives in memory
source.valve.PakFile
could make a good base for this
.from_stream
etc. would be better
archives.respawn.Vpk
already kind of has a ._from_stream
method, but it's not a @classmethod
a proper @classmethod
would be much nicer
Here's what our deepest load could end up looking like:
from bsp_tool.archives.padus import Cdi
cdi = Cdi.from_file("D:/Emulators/Sega/Dreamcast/quakeiii.cdi")
from bsp_tool.archives.iso import Iso
iso = Iso.from_bytes(cdi.read("1.0.iso"))
from bsp_tool.branches.valve.source import PakFile # or archives.id_software.Pk3
zip_ = PakFile.from_bytes(iso.read("QUAKE3/BASEQ3/MAPS/DC_MAP01/BSP.ZIP"))
from bsp_tool.id_software import IdTechBsp
from bsp_tool.branches.id_software import quake3
bsp = IdTechBsp.from_bytes(
quake3,
"D:/Emulators/Sega/Dreamcast/quakeiii.cdi/1.0.iso/QUAKE3/BASEQ3/MAPS/DC_MAP01/BSP.ZIP/maps/dc_map01.bsp",
zip_.read("maps/dc_map01.bsp"))
As a one-liner:
from bsp_tool.archives.iso import Iso
from bsp_tool.archives.padus import Cdi
from bsp_tool.branches.id_software import quake3
from bsp_tool.branches.valve.source import PakFile # or archives.id_software.Pk3
from bsp_tool.id_software import IdTechBsp
bsp = IdTechBsp.from_bytes(
quake3,
"D:/Emulators/Sega/Dreamcast/quakeiii.cdi/1.0.iso/QUAKE3/BASEQ3/MAPS/DC_MAP01/BSP.ZIP/maps/dc_map01.bsp",
PakFile.from_bytes(
Iso.from_bytes(
Cdi.from_file("D:/Emulators/Sega/Dreamcast/quakeiii.cdi").read(
"1.0.iso")).read(
"QUAKE3/BASEQ3/MAPS/DC_MAP01/BSP.ZIP")).read(
"maps/dc_map01.bsp"))
Yes, a properly indented one-liner ends up being longer
bsp = IdTechBsp.from_bytes(quake3, "D:/Emulators/Sega/Dreamcast/quakeiii.cdi/1.0.iso/QUAKE3/BASEQ3/MAPS/DC_MAP01/BSP.ZIP/maps/dc_map01.bsp", PakFile.from_bytes(Iso.from_bytes(Cdi.from_file("D:/Emulators/Sega/Dreamcast/quakeiii.cdi").read("1.0.iso")).read("QUAKE3/BASEQ3/MAPS/DC_MAP01/BSP.ZIP")).read("maps/dc_map01.bsp"))
bsp_tool.load_bsp("D:/Emulators/Sega/Dreamcast/quakeiii.cdi/1.0.iso/QUAKE3/BASEQ3/MAPS/DC_MAP01/BSP.ZIP/maps/dc_map01.bsp")
I'm currently focused on quakeiii.cdi
Looking though my Dreamcast folder, I found a few other .cdi
:
Half Life (US).cdi
PARANOIADC1.1.CDi
(Half-Life: Paranoia Dreamcast Port)Quake (US).cdi
NOTE: Other Dreamcast games w/
.bsp
s exist (see #117)
All 3 contain .pak
pakfiles
For #21 the proof of concept that showed we could open .bsp
inside archives was Quake's id1/PAK0.PAK
Kinda funny that my other .cdi
all contain .pak
archives.sega.GDRom
maps directly to the GD-ROM Data Area files
This means we can go skip indexing the .iso
inside the .cdi
when we want to load from a full path
Making our "one-liner" much simpler
from bsp_tool.archives import sega
gdrom = sega.GDRom.from_file("D:/Emulators/Sega/Dreamcast/quakeiii.cdi")
from bsp_tool.branches.valve.source import PakFile
zip_ = PakFile.from_bytes(gdrom.read("QUAKE3/BASEQ3/MAPS/DC_MAP01/BSP.ZIP"))
...
Should be nice for other images too:
.cue
+ .bin
& .raw
.gdi
+ .bin
& .raw
IDK if any full .iso
GD-ROM images are out there
You could make one, in theory (maybe with ISOBuster)
All the empty sectors would make it pretty wasteful though
The new GDRom interface is quite nice
@classmethod
initialisers, .tree()
, .listdir()
& other filesystem utilities
It'd be nice to add things like .is_file(path: str) -> bool
to all archives
Working out a new standard for archives will probably be part of #191
Also, valve.source.PakFile
might work better living in archives
Seeing as it already has .from_bytes
, and filesystem utils could be handy additions
Just have to decide on where to keep it
The archives.cdrom
already breaks the naming convention of developer.FileFormat
.zip
format has a developer: PKWARE Inc.^zip
Not exactly common knowledge, but pkware.Zip
fits archives
& is clearly distinct from zipfile.ZipFile
Looking though my Dreamcast folder, I found a few other
.cdi
:
All of the .cdi
files in my collection have 1 Mode2
Track in their 2nd session
In the code I was assuming the GD Area would always be there, worked out nicely
Other .cdi
GD-ROMs might not fit this assumption tho, so it's good to keep note of it being an assumption
Not going to seek out other
.cdi
GD-ROMs, I've got all we need forbsp_tool
rn
WE HAVE IGNITION
>>> from bsp_tool.archives import sega
>>> gdrom = sega.GDRom.from_file("D:/Emulators/Sega/Dreamcast/quakeiii.cdi")
>>> from bsp_tool.branches.valve.source import PakFile
>>> zip_ = PakFile.from_bytes(gdrom.read("QUAKE3/BASEQ3/MAPS/DC_MAP01/BSP.ZIP"))
>>> from bsp_tool.id_software import IdTechBsp
>>> from bsp_tool.branches.id_software import quake3
>>> bsp = IdTechBsp.from_bytes(
... quake3,
... "D:/Emulators/Sega/Dreamcast/quakeiii.cdi/QUAKE3/BASEQ3/MAPS/DC_MAP01/BSP.ZIP/maps/dc_map01.bsp",
... zip_.read("maps/dc_map01.bsp"))
<IdTechBsp 'dc_map01.bsp' id_software.quake3 (IBSP version 46)>
>>> _.VERTICES
<BspLump(19827 Vertex) at 0x0000021809B10750>
Building on #21, I want free up my harddrives by only extracting maps when I load them Rather than add even more complex infrastructure to
tests/megatest.py
, I'd rather just use paths into archivese.g.
D:/SteamLibrary/steamapps/common/Quake/Id1/PAK0.PAK/maps/e1m1.bsp
Challenges
D:/Emulators/Sega/Dreamcast/quakeiii.cdi/quake.16.iso/QUAKE3/BASEQ3/MAPS/DC_MAP01/BSP.ZIP/maps/dc_map01.bsp
.cdi
->.iso
->.zip
->.bsp
os.listdir
will be useless)RespawnBsp
.ent
&.bsp_lump
).rpak
, extra complexity yayRelated
21