Problems importing large RVZs

[ColetteDiskette]

I'm not sure the exact threshold, but at some point the RVZ seems too large to extract and calculate a CRC (on my machine, at least.) I've noticed my RAM usage was going up as it was extracting, so I figured it was trying to use my RAM. I closed all my browser instances (since that was eating about 20-25% of my RAM) and tried again. It would error in the same place regardless.

The partition has ~150 GiB free (using btrfs) so it's doubtful that's the issue. This is a sample output I'm getting:

Processing "recomp22-Hip Hop Dance Experience, The (USA) (En,Fr,Es).rvz"
Extracting "recomp22-Hip Hop Dance Experience, The (USA) (En,Fr,Es).rvz"
Error: SimpleError { err: "Failed to write the output file \"/tmp/.tmpoxAQL6/recomp22-Hip Hop Dance Experience, The (USA) (En,Fr,Es).iso\".\nCheck that you have enough space available on the target drive.\nError: Conversion failed\n" }

This is using the recently compiled oxyromon.

Version: 0.20.0-beta

Dependencies:
  7-zip: found (17.05)
  bchunk: found (1.2.2)
  chdman: found (0.270)
  ctrtool: not found
  dolphin-tool: found (unknown)
  flips: not found
  maxcso: found (1.13.0)
  nsz: not found
  wit: not found
  xdelta3: not found

Systems: 245
Games: 258566
Roms: 1291399

I know it's possible this is entirely on dolphin-tool and an upstream issue, but I figured I'd check here first in case. (If I use Dolphin itself to verify, it verifies fine.)

[alucryd]

I'll need to do some tests but RAM usage is probably from dolphin itself. I try to use buffered read and writes everywhere.

In this case it looks like it is your /tmp partition that is too small, I assume the 150Gb free space is your rom directory.

If you can't increase the size of your /tmp you can change the TMP_DIRECTORY setting in oxymoron to point to a directory on a larger partition. It can be anything, oxymoron only writes ephemeral hidden directories in it and won't touch any of its contents.

Edit: Actually RAM usage can also be /tmp itself if it is tmpfs on your system.

[ColetteDiskette]

Wow. I've had this Arch install going for months and I entirely forgot that I had set it up this way, but a quick mount | grep /tmp shows that indeed /tmp is mounted as type tmpfs.

I only have 16GB RAM, so that probably wasn't the smartest move on my part. I'm just PEBKAC all around. 😅

That said, even when I have the space free there (which in my case is in RAM) it's not working on a file if it's already failed on it before. I don't know if there's some kind of cache affecting things or if I'm just misunderstanding things.

EDIT: I've masked tmp.mount and rebooted and I am now using a normal /tmp folder on my root partition (which is what I thought I was doing this whole time lol.) When I have the time, I'll mess around with some files, but I get the feeling I won't run into anything else.

I wonder what other misconfigurations I'll discover down the road just cleaning up my library and posting here. 😛 (Thanks for putting up with me.)

[ColetteDiskette]

Yeah, this solved it for me entirely. I appreciate everything!

This tool is great and I keep opening issues thinking I'm gonna point out things to make it better, but it's often something silly I've done instead. 😅

Before discovering this, I was using ROMVault in Wine because nothing Linux native was good enough for me. This surpassed what I was looking for so I just wanted to take the opportunity to put my appreciation out there to everyone contributing!

[alucryd]

Thanks for the kind words! It's not silly, tmpfs is actually the default on arch and probably all of its derivatives. Please do not hesitate to reach out if you run into any issue, github discussions are a great place if you're not sure whether it's a bug or not.