[Feature Request] Add custom texture dumping/injection support to Beetle PSX core.

[saftle]

Bounty Requirements:

• Ability to dump textures from VRAM
• Ability to replace textures with full-color, high-resolution images that the PSX cannot support natively
• Should preferably support use of custom textures of any size. Must at least support injection of textures up to 2k resolution .
• Support use of common image filetypes (such as PNG and JPG)
• Support for loading texture packs

https://www.bountysource.com/issues/70820889-feature-request-add-custom-texture-dumping-injection-support-to-beetle-psx-core

[chuchuw]

I just had a vision of medievil with remade textures ❤️️

[charmandercha]

Ohhhhh!

please, i need this!

I will use waifu2x in my all games hahahahah

[chuchuw]

I will use waifu2x in my all games hahahahah

@gamaghostmagus Thank you for showing me this nice piece of software :)

[charmandercha]

I will use waifu2x in my all games hahahahah

@gamaghostmagus Thank you for showing me this nice piece of software :)

And this is not even the better

[chuchuw]

And this is not even the better

Just found these, are you talking about that @gamaghostmagus ?

SFTGAN, DSFTGAN (Deep SFTGAN) Spatial Feature Transform

SRGAN, ESRGAN (Enhanced SRGAN) Super Resolution Generative Adversarial Network

[autun12]

Is any one working on this?

[charmandercha]

And this is not even the better

Just found these, are you talking about that @gamaghostmagus ?

SFTGAN, DSFTGAN (Deep SFTGAN) Spatial Feature Transform

SRGAN, ESRGAN (Enhanced SRGAN) Super Resolution Generative Adversarial Network

yes it's!

i use ESRGAN, and it is... AWESOME

[autun12]

Would anyone mind if I gave this a go?

[hizzlekizzle]

@autun12 If you have some ideas on how it could be done, go for it. No one else is working on it, AFAIK. If you'd like to discuss possible strategies, drop by our discord or IRC channels and we can talk through it.

[coccofresco]

@autun12 do it, and you'll be an hero for people of this lands: we will sing songs to celebrate your genius and bravery... Please, just do it XD

[chuchuw]

[mcdgray129]

Wow this would be great,

[Meetem]

Does anybody working on it? I can have a try.

[chuchuw]

I think you're safe to go :)

[simias]

I just saw this issue, for anybody working on this I made a very quick and dirty texture dumping PoC a few years ago in beetle: https://github.com/simias/beetle-psx-libretro/tree/texture_dump The main difficulty is handling palettes correctly and figuring the best heuristic to decide what to dump.

I implemented 2 solutions (exposed through core options), each having its problems. The most straightforward solution is to dump the textures one polygon at a time. The obvious inconvenient of this solution is that you end up with many small files with tiny texture chunks. For instance, running the dump on Medievil I end up with tiny texture chunks like:

Note that there appears to be a bug in my texture dumping code because I end up with many mangled textures, I must've forgotten to handle a special case (maybe the texture mask bits?).

The other possibility is to dump one texture page (256x256 pixels) at a time. The inconvenient of this solution is that you end up with a lot of extraneous data (and each page is dumped multiple times because of the many palettes), for instance:

I'm not really sure what's the best way to deal with this. I assume that many other consoles have this type of issues though, so maybe it's worth looking into what other emulators that support texture replacement do? Dolphin in particular might be interesting to look at, because AFAIK GameCube games also often use palettes.

So that's just for the dumping part, for the loading it's a whole other can of worms, especially if you want to make it work with the GL/Vulcan code. I've actually started thinking about how to do that for Rustation, so I have a few ideas on how to do that if anybody wants to brainstorm it.

[simias]

Oh and as a side-note, a potential difficulty with paletted texture replacement is that sometimes the same base texture will be used with multiple palettes. For instance in Crash Bandicoot you have red and purple carnivorous plants, they share the exact same texture but use a different palette. Handling these situations when implementing texture replacements might be tricky. Palettes can sometimes be used to simulate shading, using a darker palette when you get in the shadow and a lighter one when illuminated (this is probably less common on the PSX because the hardware has other, more flexible way of doing that sort of shading).

[Meetem]

@simias looks promising, I've never coded for psx, but coded for openGL, so I came with another solution, palette lookup is not easy for me cause I got really bored with this specifications. Thanks for your code, will try to use your palette lookup

Btw I can miss something due to lack of knowledge, but is there a way to dump textures just from TIMs?

[simias]

Actually I amend my previous comment regarding Crash's plant, it's even more devious than that: the texture and palette are purely black and white. Instead they use texture blending in the draw commands. Basically the way it works is that you can tell to render a texture (paletted or not) by blending it with a solid color. So it's an other layer of potential texture shenanigans on top of the texture+palette. Aku-Aku's feather in the same game are the same thing: black and white textures with 4 different colors blended. Here's the relevant code in the GL fragment shader:

https://github.com/libretro/beetle-psx-libretro/blob/18a9de19f3f079408eb8181aa4c5144e9434c5f7/rsx/shaders_gl/command_fragment.glsl.h#L851-L862

That being said these complicated situations might be left for later. Unfortunately I expect that they're fairly common on the PSX...

@Meetem what is a TIM?

[Meetem]

@simias TIMs are Typical Image Format, PSX textures

[simias]

@Meetem I'm not sure I understand what you have in mind exactly. Do you mean the texture format as stored in the CD-ROM instead of dumping it straight from the GPU?

[Meetem]

Yes, I think this would help. Dumping from play is ok, but you cant replace all the textures then

On Fri, Jan 17, 2020, 18:57 Lionel Flandrin notifications@github.com wrote:

@Meetem https://github.com/Meetem I'm not sure I understand what you have in mind exactly. Do you mean the texture format as stored in the CD-ROM instead of dumping it straight from the GPU?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/libretro/beetle-psx-libretro/issues/492?email_source=notifications&email_token=AB2AF4TYGBSRFU3XODANT33Q6HIOJA5CNFSM4G3JOUMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJID2EY#issuecomment-575683859, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB2AF4XDSQT6UT73KFNI37LQ6HIOJANCNFSM4G3JOUMA .

[simias]

Ah that I've never looked into that so I can't really tell. Replacing the textures as stored in game files would obviously be the simplest approach, although then you won't be able to easily overcome palette limitations or change the texture size.

Beyond that I don't know how standard the texture formats are.

[Meetem]

@simias actually I find more convenient way to dump textures, than taking them from polygon draw commands, but CDs TIMs would even better

Actually I want to dump from TIMs but replace textures not in CD, cause psx tims cant be large than 256x256

[simias]

Well you can always compromise and dump PSX texture uploads, but then I'm not sure how you're going to match with palette data (except if you want to just replace the texture and not the palette I suppose).

In mednafen code texture upload starts with this command: https://github.com/libretro/beetle-psx-libretro/blob/18a9de19f3f079408eb8181aa4c5144e9434c5f7/mednafen/psx/gpu.cpp#L252-L275

You can see that you have the X/Y coordinates in VRAM as well as the dimensions in W and H (the VRAM always assumes 16 bits per pixel).

Then the actual data is received here, 32bits at a time: https://github.com/libretro/beetle-psx-libretro/blob/18a9de19f3f079408eb8181aa4c5144e9434c5f7/mednafen/psx/gpu.cpp#L985-L1019

[Meetem]

Thanks, I've figured that out, now I need some time to make naive implementation, also have some ideas to optimize hash calculations

[simias]

If you have questions about how the PSX GPU operates don't hesitate, I have a fairly in-depth knowledge about it. For texture dumping in general I'm sure you could get better inspiration looking at other project's implementation. Does any PSX emulator out there implements texture dumping and replacement? I seem to recall that gpuBladeSoft had something like that, but I don't really know how thorough it is.

[Meetem]

@simias thanks. Given nor poly vram dumping, nor texture page dumping is optimal, I'am trying to figure now how to catch texture upload. There is a way to catch texture part upload during one transaction (or how does it called), when game code calls FB_WRITE. I'am testing on Metal Gear Solid, and Konami logo seems to be splitted across multiple transactions. With that being said, we can go with with this paths: A) Dump TIMs from cd image, and catch up clut in realtime (if lut changed via writing vram) this is relatively easy to make, but hard to implement cd image reading with headers e.t.c B) Write some sophisticated texture upload tracking system which will contains all transactions history for heuristic and not redundant dumping. This is actually hard way (but seems easy), because we need to catch up all texture uploading cases, and also save/load states, also texture is not always uploaded in the same order e.t.c, e.t.c. C) give up on all that stuff and just handle texture uploads AND drawing with further ability to combine multiple textures at load time.

I'am going to stick with way C, cause I don't have enough knowledge to go with A (cause there multiple TIM viewing software), and way B is really PITA as much as I can say for MGS.

Here's my question, does really game can upload software generated clut, or its locked to tims on disk? I think there is a way to change cluts in vram as you want, creating cool effects, don't know if any games using it. And one more: can I draw polys with any clut I want, e.g can I set clut position while drawing poly, without restriction by texture clut position in TIMs?

UPD: I found some opensource tim viewer, but in C#, it's not hard to me to rewrite it in C. So I might go for path A. Questions still applicable, answers needed to choose which path I'll finally go

[simias]

Well I guess the golden rule of emulation is "if it's possible, somebody must be doing it". There's nothing special about palettes on the PlayStation, they're effectively just linear 16x1 or 256x1 textures somewhere in VRAM (actually "somewhere" has to be in specific portions of the VRAM due to the way they're addressed, but it's still fairly flexible).

As I mentioned previously the PSX supports "texture blending" with a solid color which can be used for certain effects where other consoles would mess with the palette instead (like the carnivorous plant in Crash Bandicoot as mentioned previously). But I'm sure some games must be using palette swapping for some effects.

Unfortunately I think the best way to decide which way to go would be to look at a sample of games and see how they handle textures. Maybe you'll be able to find a solution that works for most of them.

[inactive123]

Very cool to finally see some interest in this. I've seen some of these Dolphin repacks where RE2/RE3-specific Dolphin builds are circulated and textures are replaced (RE Seamless Project), and while they look the part, there still seems to be noticeable audio stutter during scene transitions in those Dolphin builds. That and mods like FF7 Remako project, while they might successfully replace the background textures, they're still based on the old DOS Eidos version, with all the numerous defects of that version. So in the back of my mind I was always thinking it would be much more preferable if we could do this kind of texture replacement with a PSX emulator that already has features like PGXP to combat texture warping and polygon jitter, so I couldn't be more excited about people finally exploring solutions to this.

Decided to drop $100 into this bounty just as a show of appreciation - this isn't really about the money, just as a show of appreciation for the development going into this, so hopefully this won't be taken the wrong way.

[simias]

The FF backgrounds would be a great target for replacement I think but unfortunately it's also not the simplest target. The problem is that, IIRC, the background texture is actually made up of many small square chunks that aren't laid out contiguously in VRAM. You can always dump all these chunks just fine, but you end up with a puzzle. I'm not sure why they do it that way but I think it's because they regroup every background "tile" by palette.

That being said I wouldn't be surprised if the PC port used a similar format, after all it seems like a very straightforward port so maybe they reused most of the assets as-is? If that's the case it would be interesting to understand how they handled it when they modded it.

[Meetem]

So, what I've archieved so far. I'am able to dump textures uploaded and drawn, there are a lot of small textures btw. That's a bit better than simias's solution, cause can dump textures as-is they were uploaded (if the big atlas was uploaded, it will dumped then). But as I said before, I'am testing Metal Gear Solid, and it's HUGE in a way of pushing console's limits. I mean, yeah, we have textures, but there are a freaking lot of palettes this game using (for all kind of effects, mostly lighting ones).

May be replacing all the textures with truecolor images will work just fine with some simple games, but won't work with MGS which is my personal target.

So, I need some help to make a decision, should we: A) replace texture with truecolor instance regardless to the palette B) replace textures with truecolor variants one for palette C) make some tricky format for "faketruecolor". Ex: we can make texture format which will be almost truecolor, but also will respect texture's current rendering palette. May be with HSV offsets or something like that. (Real true color parts of texture can be marked with part of alpha channel for example, left-side for 8bit alpha, right side for palette masking)

I see the ideal variant is to have all of this three, but can't know are there a lot of games which using palette tricks.

[simias]

May be replacing all the textures with truecolor images will work just fine with some simple games, but won't work with MGS which is my personal target.

In my experience very few games use truecolor textures on the PSX because there's not enough VRAM to really make it worthwhile. Truecolor is mainly only used is special cases such as videos decoded with the MDEC or textures that are generated procedurally. In either case you probably don't want to replace these textures anyway.

Note that MGS is a bit special in the way it deals with textures which might actually make it a bit easier to handle than some other games because the palettes are almost all using a linear gradient. For instance if I dump MGS's raw VRAM using monochrome 8bits per pixels I get:

You can see that all the textures are easily identifiable because the palettes are ordered from darker to lighter colors:

That would make the textures easier to edit and replace without touching the palettes: just edit it as a black-and-white texture and it should mostly just work as you reimport it.

Many other games have much more anarchic palettes which would make this approach impossible.

So, I need some help to make a decision, should we: A) replace texture with truecolor instance regardless to the palette

That's probably the easiest implementation but I'm sure it's going to create problems. Might be worth starting with that though, I suspect that it shouldn't be too hard to change the texture selection algorithm later for something more advanced. Also by then you'll have a much clearer idea of what the tricky corner cases are (I suspect that color blending is going to give you trouble for some games, although maybe not MGS).

B) replace textures with truecolor variants one for palette

I guess you could have an hybrid between A and B, where by default the replacement is based only on the texture (which would be fine enough the vast majority of the time) but you allow to sometimes specify multiple replacements based on the palette for tricky corner cases. But again, to begin with I'd stick with A.

C) make some tricky format for "faketruecolor". Ex: we can make texture format which will be almost truecolor, but also will respect texture's current rendering palette. May be with HSV offsets or something like that. (Real true color parts of texture can be marked with part of alpha channel for example, left-side for 8bit alpha, right side for palette masking)

That's actually a very clever solution, I had never thought of that. You reuse the palette but you allow offset per-pixel. I really think that you should keep that for later because it sounds more complicated than the other solutions but it might be worth looking into at some point.

[Meetem]

@simias thanks again. I've thought about anarchic palettes in other games, there are in mgs too, so it's actually a deal breaker for implementaion.

Do you mind answer some questions in discord?

[Meetem]

Omg, I was thinking there are a lot of overlapping texture rewriting and palette swaps in MGS, but no, there was some trick.

When PSX sending poly to draw it uses UV coords of course, but bit shift must be applied to U coordinates (min_u/max_u >>= depth_shift), and after then current page_x added. Hope nobody will waste time on this after me

[simias]

@simias thanks again. I've thought about anarchic palettes in other games, there are in mgs too, so it's actually a deal breaker for implementaion.

I can't think of a solution to make dumping work that doesn't involve hooking yourself into the GPU draw command instead of just dumping the texture upload. You simply won't be able to match the palette otherwise.

Do you mind answer some questions in discord?

Discord makes me sad, but I'm on freenode's IRC (#retroarch channel for instance). You can also shoot me an email if you prefer not to spam this issue (you'll find it in my commits).

When PSX sending poly to draw it uses UV coords of course, but bit shift must be applied to U coordinates (min_u/max_u >>= depth_shift), and after then current page_x added. Hope nobody will waste time on this after me

There's an other texture trick that's used in a few games, it's texture coordinates masking. Basically it lets you do U/V &= some_mask and it can be used with masks like 0xf to repeat the texture every 16 pixels for instance. This is useful when a game wants to tile a small texture over a large polygon.

That being said I don't think it should matter much for texture replacement (it's more of a problem for texture filtering).

[Meetem]

@simias this is strange. Why would they use uv masking if there is a texture window?

Am I wrong with depth_shift? Ut doesnt look like masking (tested on Konami logo which occupaies 2 pages, and drawn with 2 quads)

If so, how do I "unmask" coordinates, I've never seen masking flag for it. (I'am talking about UVs in gpu_polygon.cpp, hooking rsx_intf_push_triangle/quad)

Now the question is: how do mask bit works exactly when data uploading to VRAM? I can't figure that out.

And one more: does PS gpu sample last pixels in uv, is max uv coordinates are inclusive or exclusive?

[Meetem]

Unpack your packages! Proof of concept is ready

[simias]

@simias this is strange. Why would they use uv masking if there is a texture window?

Because the texture window is always 256x256, sometimes they want a smaller window so they use masking.

Am I wrong with depth_shift? Ut doesnt look like masking (tested on Konami logo which occupaies 2 pages, and drawn with 2 quads)

Depth shift is way I awkwardly named the texture depth:

https://github.com/libretro/beetle-psx-libretro/blob/5d4ecd68be89aa8bb7317a97d890e9bd173c91c9/rsx/shaders_gl/command_fragment.glsl.h#L40-L41

I named it that way because the actual texture "depth" is equal to 16 >> depth_shift.

So no it doesn't have anything to with masking.

If so, how do I "unmask" coordinates, I've never seen masking flag for it. (I'am talking about UVs in gpu_polygon.cpp, hooking rsx_intf_push_triangle/quad)

Oh, so after digging into the code I think I now understand why you're probably confused:

https://github.com/libretro/beetle-psx-libretro/blob/5d4ecd68be89aa8bb7317a97d890e9bd173c91c9/rsx/rsx_lib_gl.cpp#L1923-L1926

In the GL shader the masks are written to an attribute called texture_window. I don't remember if it's something I wrote but if it is I apologize, it's a very bad name because the texture window in the context of the PSX graphics is not that at all. And this mask is not necessarily a window either, it's an arbitrary bit mask.

I wouldn't worry too much about it at any rate, it's not used super commonly as far as I know (and even when it is it shouldn't be a problem for replacement as long as the new texture "lines up" correctly with the old one).

Now the question is: how do mask bit works exactly when data uploading to VRAM? I can't figure that out.

Ah yeah, there's that too, but that's a different mask. I really need to make a drawing or something. I'll try thinking of something and I'll get back to you.

And one more: does PS gpu sample last pixels in uv, is max uv coordinates are inclusive or exclusive?

I want to say inclusive but I'm not certain anymore, I need to double check.

[inactive123]

@Meetem Hey that's great to see. This proof of concept right now is specific to one game ?

[Meetem]

@twinaphex I've done texture replacement for MGS, and turns out this game have very good texture work and clean uploadings, so for it texture replacement is easy. But other games I've checked has indistinct texture uploading thus making hard to do texture-to-texture mapping. I've already thought up a concept how to deal with it for almost all games I think. But it will require to write external tool. And texture-packs creators (but not users) will need to use it. Now I'am not sure if anybody need this and will use :( may be I need some confidence if this way is comfortable for anyone.

Tool is not hard to write btw, I will do that if see some interest

[inactive123]

We will do everything we can to build up support and to get the information out to users on whatever external tools you write that would aid people in creating texture packs like this. We are all for this idea.

So consider our interest in this being at an all-time high, and you will have no issues with getting the word out, we will take care of that and there are already people doing N64 texture packs that would no doubt be VERY interested in this.

[Meetem]

@twinaphex Oh, thank you then, this would help, cause I'am not so much about describing things and putting them simple. I'am interested too cause there is texture pack for MGS: Twin Snakes on GameCube (Dolphin), but original MGS is much closer to me in some sense.

[inactive123]

Yeah Twin Snakes is really a different game from MGS1, many of the changes were not for the better IMHO.

[coccofresco]

I don't see any trouble using a software to manage making texture pack. Infact, in n64 world, some packs was never totaly finished because the lack of a software like this (great exemple: Ocarina of Time prerendered backgrounds)

[lyonhrt]

what would the tools be doing? @Meetem even if you can get the basic tools together, i'm sure the community is behind you, and would most likely improve/tweak as time goes by. i know palettes and how the textures is always the problem here, though not attempted anything for a long time.

[Meetem]

@lyonhrt I'am working on the tool which can hook VRAM writes and replace regions with truecolor or pseudopaletted (mentioned above) texture. This tool will allow see and bind VRAM write events to new texture and also view textures added in pack e.t.c. So it's not assembly of basic tools, this is the new one.

This is the skeleton I've done few moments ago

[lyonhrt]

sounds very similar to what i had hacked together years ago using third party tools, the problem i had back then was injecting it back plus performance issues. Though this tool seems very interesting, if its side loaded, and in theory could be put to use on multiple emulators ps2 perhaps (another system i had got texture modding working in a way using the same method as ps1, but bit more easier with the palettes, apart from being bgr format)

[Meetem]

@lyonhrt I think aside from any platform texture replacement is simmilar process, but PS1 require some tricky things because of internal rendering technology. Newer platforms should be more like PC (OpenGL/DirectX) so much easier to patch.

I'am glad somebody interested in what I'am doing. Will keep you up

[InfiniteT8]

A similar tool that may prove to be a good reference is uMod texmod. It worked off directx9 games. For psx emulators like epsxe and pcsxr its possible to use the tool with the next3d 1.4 and 1.5 plugins. The only issue that I came across was random crashes and some upload limitations. (some retextured items wouldn't show up if you have 20 or more). But perhaps this could be incorporated? The sources and tool itself is available here: https://code.google.com/archive/p/texmod/downloads

[Meetem]

@InfiniteT8 this is not the same. This tool hooks uploads as just data and you can modify contents, so you can use only bpp given by game, no truecolor. And cant resize. PSX doesn't have "textures" in common meaning like pc games do, so there is nothing to replace with that way.