Possible solution for texture wildcards

[CadetSparklez]

I started working on a texture pack a while back and found out about the wildcard area in texture names. Having to delete all of them I came across bulk rename utility which used regular expressions to select which part of the file name to change.

Don't know if the system for regular expressions is proprietary to bulk rename utility, but here is what works to remove to wildcard from the name, incase it would be possible to implement into GLide itself on loading textures:

(.+)#(?!\d)[^#]+?(.*) \1\2

[GhostlyDark]

Wildcard support is already implemented (post 4.0, use WIP builds) and works just like in latest Dolphin dev builds:

Perfect Dark#chk1#format#size#chk2_ciByRGBA
option 1: Perfect Dark#$#format#size#chk2_ciByRGBA
option 2: Perfect Dark#chk1#format#size#$_ciByRGBA

Be aware that if you update, some of your textures will not load as they have a wrong format and size information (ciByRGBA is likely #2#0 or #2#1).

[GhostlyDark]

If for whatever reason you requested to have GLideN64 auto rename all files to be wildcarded, then I don't think that's a good idea. A simple renaming script will do. Besides, not all textures should be wildcarded, only the ones that dump infinitely and can be replaced without causing issues.

[gonetz]

@GhostlyDark Am I got you right that no additional wildcards support is actually necessary and this request can be closed?

[GhostlyDark]

@GhostlyDark Am I got you right that no additional wildcards support is actually necessary and this request can be closed?

I don't fully understand what has been requested here, but I am pretty sure there is nothing that needs to be changed about GLideN64. Feel free to close it.

[CadetSparklez]

Hmm I'm using a post 4.0 wip build (rev.35554dab). Are there emulator specific limitations similar to the greater than 1024MiB texture cache limit that only exists for pj64 I believe? (taking an educated guess here since the huge majora's mask retexture project uses that and that only). I'm using a fork of the ancient 1964 0.8.5 so I often run into this kind of stuff.

Is there a better place to ask about things that I'm not sure are actual issues or not than the issues page?

If for whatever reason you requested to have GLideN64 auto rename all files to be wildcarded, then I don't think that's a good idea. A simple renaming script will do. Besides, not all textures should be wildcarded, only the ones that dump infinitely and can be replaced without causing issues.

From my experience this will only occur is you mess with the #x#y section or using the all suffix in the wrong spot. Also rarely there will be an #x#y#z and the z addresses which of a tile to use, I've only run into this one time however with the concrete floor/ceiling texture on the top and bottom floors as well as the roof on datadyne central in perfect dark. This might be game dependant though ¯_(ツ)/¯

[GhostlyDark]

Hmm I'm using a post 4.0 wip build (rev.35554dab).

Build should have wildcard support.

Are there emulator specific limitations similar to the greater than 1024MiB texture cache limit that only exists for pj64 I believe?

HD texture cache is unlimited, but 32-bit GLideN64 will be bound to a max of 1 GB or 3 GB with LAA worth of HD textures in RAM. Not to be confused with texture enhancement cache, which has nothing to do with HD textures.

I'm using a fork of the ancient 1964 0.8.5 so I often run into this kind of stuff.

Which is 32-bit only (1964 GEPD?). You can try to do a 4 GB patch (aka make it LAA) on the 1964.exe to increase the RAM limit.

From my experience this will only occur is you mess with the #x#y section or using the _all suffix in the wrong spot. Also rarely there will be an #x#y#z and the z addresses which of a tile to use, I've only run into this one time however with the concrete floor/ceiling texture on the top and bottom floors as well as the roof on datadyne central in perfect dark.

That's where I'm totally lost. What are you trying to accomplish? What is your issue exactly? Judging from the screenshot of your initial post, the renamed files are completely wrong. The second hash isn't wildcarded with a $ like it should be, it's omitted instead. Textures that can be wildcarded are always _ciByRGBA and never _all.

[gonetz]

Are there emulator specific limitations similar to the greater than 1024MiB texture cache limit that only exists for pj64 I believe? (taking an educated guess here since the huge majora's mask retexture project uses that and that only).

As GhostlyDark explained, "32-bit GLideN64 will be bound to a max of 1 GB or 3 GB with LAA worth of HD textures in RAM." That is, memory cache can't be large 1 GB or 3 GB. If you have larger texture pack, you may use file-based texture cache. The aforementioned huge majora's mask retexture project uses file based cache.

[Xii-Nyth]

That's where I'm totally lost. What are you trying to accomplish? What is your issue exactly? Judging from the screenshot of your initial post, the renamed files are completely wrong. The second hash isn't wildcarded with a $ like it should be, it's omitted instead. Textures that can be wildcarded are always _ciByRGBA and never _all.

Oh yes there are no _all files with the second #12345678, the left is input and right output. Would of actually been alot easier to come up with the regex if that was the case since I wouldn't had to avoid deleting the "#2" (would only be (.+)#, \1 or (.*)#, \1). However I've never seen any textures with a '$' rather than a '#'. Unless what you meant is that rather than deleting it I could instead change the # to a $ and then glide would properly avoid that and load the texture in. I couldn't find a guide on the process and just came up with my own solution of deleting it.

As for what I meant by "this will only occur if.." was the issues you said can happen if deleting it like I am doing, in my case this was getting a black screen on the output. And to clarify, it occurs if I manually added in _all on textures which I deleted the #12345678 from, not on textures which originally had it. (Like I mentioned before I didn't have a guide so I just discovered this by trial and error, since the only textures which were working for me were the ones that ended in _all, so my first idea was to make every file end with that suffix.

[Xii-Nyth]

Are there emulator specific limitations similar to the greater than 1024MiB texture cache limit that only exists for pj64 I believe? (taking an educated guess here since the huge majora's mask retexture project uses that and that only).

As GhostlyDark explained, "32-bit GLideN64 will be bound to a max of 1 GB or 3 GB with LAA worth of HD textures in RAM." That is, memory cache can't be large 1 GB or 3 GB. If you have larger texture pack, you may use file-based texture cache. The aforementioned huge majora's mask retexture project uses file based cache.

This refers to a .hts which is accessed on the drive while .htc is in the ram?

Is it generally faster to have a compressed htc in memory (which lowers the size down to 250 MB, which should be only ~500 when the pack is complete), or an uncompressed hts, assuming the drive is a fast ssd.

Also, I'm assuming the textures are decompressed on the cpu when accessed, if that is the case, is the load independent of the emulated cpu? (is it done on my cpu without slowing down emulation, or does it take up some of the emulated cpu's headroom?)

[GhostlyDark]

Oh yes there are no _all files with the second #12345678, the left is input and right output. Would of actually been alot easier to come up with the regex if that was the case since I wouldn't had to avoid deleting the "#2" (would only be (.+)#, \1 or (.*)#, \1). However I've never seen any textures with a '$' rather than a '#'. Unless what you meant is that rather than deleting it I could instead change the # to a $ and then glide would properly avoid that and load the texture in. I couldn't find a guide on the process and just came up with my own solution of deleting it.

This is not how wildcarded textures work. I explained how the naming works in my initial post. Check out Mario Kart 64 HD, wildcard is used on the first part of the hash: https://github.com/AndratVA/Mario-Kart-64-HD/tree/MARIO-KART-64/Portraits%26Items/shells

This refers to a .hts which is accessed on the drive while .htc is in the ram?

Is it generally faster to have a compressed htc in memory (which lowers the size down to 250 MB, which should be only ~500 when the pack is complete), or an uncompressed hts, assuming the drive is a fast ssd.

Also, I'm assuming the textures are decompressed on the cpu when accessed, if that is the case, is the load independent of the emulated cpu? (is it done on my cpu without slowing down emulation, or does it take up some of the emulated cpu's headroom?)

HTC prefetches all textures in RAM, HTS streams from disk with textures that have been loaded (on demand) stored in VRAM. Decompressing a compressed cache file uses CPU cycles, which can cause stuttering. Reading from a slow storage device may cause stuttering as well. The emulated CPU does not slow down when using HD textures.

[Xii-Nyth]

This is not how wild carded textures work. I explained how the naming works in my initial post. Oh my bad, I thought you were showing some code and those were variables or something. I guess what I've been doing is option 2 but with #$ missing (which seemingly works fine?).

Well you guys can close this issue since that answers all my questions. Ill make sure to add in that #$ just so nothing breaks that I don't notice. And ill use your mario kart pack as reference to figure stuff out.

Also in case you are wondering about this:

Also rarely there will be an #x#y#z and the z addresses which of a tile to use Here is the one place I've seen this done (I didn't make this part of the pack)

It applied to a column of tiles that met a different texture (overriding the texture that applied to all columns). I might experiment with this to create some variation along all the repeating textures in the carrington institute. Although I'm guessing this was unintended and it was just a coincidence that it worked out.

Oh also I don't mean to say #x#y#z as axis, just meant as in a b c or 1 2 3, obviously the #x#y is the #format and #size you mentioned.

Edit: I just noticed all the duplicate textures I had (hands) use the same #chk2, Ill go and use option 1 for these and save 100 MB. Thanks!!

[gonetz]

there is nothing that needs to be changed about GLideN64. Feel free to close it.

Well you guys can close this issue since that answers all my questions.

[Xii-Nyth]

Huh received an email for this a month late lol.

Anyways I've gotten to the point in the pack where I found out multiplayer files are random on both chk1 and 2 so guess I'll have to wait for a pc port.

Will close all other niche pd related issues since there's just no point to do them anymore now that the game is a month away from being decompiled :)