indy_4610 display issue (and crashes)

[darkstar]

Running the indy_4610 driver with the IRIX 6.5 softlist HDD shows a shifted display and ultimately crashes MAME 0.250:

Steps to reproduce:

• remove old NVRAM files
• launch for example mame indy_4610 irix_6_5_hdd (also happens with at least irix_6_5_22_hdd and irix_5_3_hdd softlist entries and indy_4613 and indy_5015 drivers)
• enter PROM monitor and set MAC address (setenv -f eaddr 08:00:69:12:34:56 or similar)
• quit MAME and relaunch it (resetting with F3 also makes it crash, see below)
• wait

MAME crashes as soon as it tries to display the login screen, with the following exception in the RB2 device:

-----------------------------------------------------
Exception at EIP=00007ff6623e47fe (rb2_device::read_pixel()+0x000e): ACCESS VIOLATION
While attempting to read memory at 000001dd02d72000
-----------------------------------------------------
RAX=000001dd02d72000 RBX=000001dd73c6fa10 RCX=0000000000000000 RDX=000000878a4f938c
RSI=000001dd686749f0 RDI=00000000000000c6 RBP=0000000000000329 RSP=000000878a4f9328
 R8=000000878a4f9388  R9=000000878a4f9374 R10=0000000000001e00 R11=000001dd02d1bea0
R12=000001dd68673680 R13=000001dd68674970 R14=000001dd686749f0 R15=0000000000000000
-----------------------------------------------------
Stack crawl:
  000000878a4f9320: 00007ff6623e47fe (rb2_device::read_pixel()+0x000e)
  000000878a4f9350: 00007ff65c7cd17e (std::_Function_handler<unsigned int (unsigned int, unsigned int), devcb_read<unsigned int, 4294967295u>::creator_impl<devcb_read<unsigned int, 4294967295u>::delegate_builder<emu::device_delegate<unsigned int ()> > >::create()::{lambda(auto:1&&)#1}::operator()<devcb_read<unsigned int, 4294967295u>::delegate_builder<emu::device_delegate<unsigned int ()> >::build<{lambda(auto:1&&)#1}>({lambda(auto:1&&)#1}&&)::{lambda(unsigned int, unsigned int)#1}>({lambda(auto:1&&)#1}&&) const::{lambda(unsigned int, unsigned int)#1}>::_M_invoke(std::_Any_data const&, unsigned int&&, unsigned int&&)+0x000e)
  000000878a4f93b0: 00007ff65c7d7578 (devcb_read<unsigned int, 4294967295u>::operator()()+0x0038)
  000000878a4f9440: 00007ff6623ebe76 (newport_base_device::do_rex3_command()+0x0296)
  000000878a4f94b0: 00007ff6623ecf10 (newport_base_device::rex3_w(unsigned int, unsigned long long, unsigned long long)+0x00b0)
  000000878a4f94e0: 00007ff660f75817 (handler_entry_write_delegate<3, 0, emu::device_delegate<void (unsigned int, unsigned long long, unsigned long long)> >::write(unsigned int, unsigned long long, unsigned long long) const+0x0017)
  000000878a4f9510: 00007ff66054998c (handler_entry_write_dispatch<14, 3, 0>::write(unsigned int, unsigned long long, unsigned long long) const+0x001c)
  000000878a4f9540: 00007ff65daa94b5 (address_space_specific<1, 3, 0, (util::endianness)1>::write_qword(unsigned int, unsigned long long, unsigned long long)+0x0025)
  000000878a4f9570: 00007ff65f8448ca (gio64_device::write(unsigned int, unsigned long long, unsigned long long)+0x001a)
  000000878a4f95a0: 00007ff660f75817 (handler_entry_write_delegate<3, 0, emu::device_delegate<void (unsigned int, unsigned long long, unsigned long long)> >::write(unsigned int, unsigned long long, unsigned long long) const+0x0017)
  000000878a4f95d0: 00007ff65dacb39d (address_space_specific<1, 3, 0, (util::endianness)1>::write_dword(unsigned int, unsigned int)+0x004d)
  000000878a4f95d8: 000001dd68873024 (+0x68873024)
  000000878a4f95e0: 000000000000000a (+0x000a)
  000000878a4f9620: 00007ff65c797eda (mcs51_cpu_device::sfr_read(unsigned long long)+0x005a)
  000000878a4f9650: 00007ff65c79b879 (mcs51_cpu_device::iram_read(unsigned long long)+0x0039)

Things to note:

• sometimes, during boot, the messages "First RE may not be null" might be displayed multiple times (I've seen it 0, 2 or 4 times during my tests):
• when resetting MAME with F3, it crashes during boot with an NSCSI error. This is not related to this issue, and I have not investigated it further, but the exception is this:

  -----------------------------------------------------
  Exception at EIP=00007ff65d95c58e (nscsi_cdrom_device::set_block_size(unsigned int)+0x002e): INTEGER DIVIDE BY ZERO
  -----------------------------------------------------
  RAX=0000000000000800 RBX=0000026304abdcc0 RCX=0000000000000000 RDX=0000000000000000
  RSI=0000000000000200 RDI=0000000000000040 RBP=0000000000000000 RSP=000000c82bdb89c0
  R8=000000000000000b  R9=0000000000000000 R10=00007ff65d95c7f0 R11=0000000000000000
  R12=0000026304abdcc0 R13=0000000000000000 R14=00007ff667b92e60 R15=000000c82bdb9ea0
  -----------------------------------------------------
  Stack crawl:
  000000c82bdb8ad0: 00007ff65d95c58e (nscsi_cdrom_device::set_block_size(unsigned int)+0x002e)
  000000c82bdb8b00: 00007ff65d95c85d (nscsi_cdrom_device::scsi_put_data(int, int, unsigned char)+0x006d)
  000000c82bdb8bc0: 00007ff65c95476a (nscsi_full_device::step(bool)+0x195a)
  000000c82bdb8bf0: 00007ff65c954bab (nscsi_full_device::scsi_ctrl_changed()+0x000b)
  000000c82bdb8c40: 00007ff65c951fc8 (nscsi_bus_device::ctrl_w(int, unsigned int, unsigned int)+0x0148)
  000000c82bdb8f50: 00007ff65c97a45e (wd33c9x_base_device::step(bool)+0x0bce)
  000000c82bdb9260: 00007ff65c97a087 (wd33c9x_base_device::step(bool)+0x07f7)
  000000c82bdb9290: 00007ff65c97bb34 (wd33c9x_base_device::scsi_ctrl_changed()+0x0024)
  000000c82bdb92f0: 00007ff65c954d28 (nscsi_full_device::target_recv_byte()+0x0168)
  000000c82bdb93b0: 00007ff65c9536f1 (nscsi_full_device::step(bool)+0x08e1)
  000000c82bdb9470: 00007ff65c953b99 (nscsi_full_device::step(bool)+0x0d89)
  000000c82bdb94a0: 00007ff65c954bab (nscsi_full_device::scsi_ctrl_changed()+0x000b)
  000000c82bdb94f0: 00007ff65c951fc8 (nscsi_bus_device::ctrl_w(int, unsigned int, unsigned int)+0x0148)
  000000c82bdb9800: 00007ff65c979a67 (wd33c9x_base_device::step(bool)+0x01d7)
  000000c82bdb9b10: 00007ff65c97a762 (wd33c9x_base_device::step(bool)+0x0ed2)
  000000c82bdb9b40: 00007ff65c97bafe (wd33c9x_base_device::update_step(int)+0x000e)
  000000c82bdb9bb0: 00007ff65c943b7b (device_scheduler::timeslice()+0x014b)
  000000c82bdb9d10: 00007ff65c940a98 (running_machine::run(bool)+0x0198)
  000000c82bdbf190: 00007ff65ff468cb (mame_machine_manager::execute()+0x020b)
  000000c82bdbf570: 00007ff66293ad2f (cli_frontend::start_execution(mame_machine_manager*, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&)+0x03cf)
  000000c82bdbf830: 00007ff66293b33d (cli_frontend::execute(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&)+0x004d)
  000000c82bdbf890: 00007ff65ff40f79 (emulator_info::start_frontend(emu_options&, osd_interface&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&)+0x0029)
  000000c82bdbfc60: 00007ff663b17b77 (main+0x0197)
  000000c82bdbfd30: 00007ff657ef13b1 (__tmainCRTStartup+0x0231)
  000000c82bdbfd60: 00007ff657ef14e6 (mainCRTStartup+0x0016)
  000000c82bdbfd90: 00007ff8f3b03d2d (BaseThreadInitThunk+0x001d)
  000000c82bdbfe10: 00007ff8f4dfd838 (RtlUserThreadStart+0x0028)

• Running IRIX 6.2 (irix_6_2_hdd) gives a different crash in the recompiler, but that might be due to the fact that this version of IRIX was not supposed to run on Indy's anyway:

  -----------------------------------------------------
  Exception at EIP=00007ff65e6a42df (mips3_device::code_compile_block(unsigned char, unsigned int)+0x061f): ACCESS VIOLATION
  While attempting to read memory at 0000000000000000
  -----------------------------------------------------
  RAX=000000000000000b RBX=000001f554a2c130 RCX=000001f554b313d0 RDX=00000000900cbfe4
  RSI=000001f53b125fe0 RDI=0000000000000000 RBP=0000000f81d69514 RSP=0000000f81d694a0
  R8=0000000000000004  R9=0000000f81d69570 R10=000001f5388c00b8 R11=0000000000000006
  R12=000001f554a4fa10 R13=00000000100cbfe4 R14=00000000100cbfe4 R15=0000000f81d69c60
  -----------------------------------------------------
  Stack crawl:
  0000000f81d69840: 00007ff65e6a42df (mips3_device::code_compile_block(unsigned char, unsigned int)+0x061f)
  0000000f81d69900: 00007ff65e6b0c69 (mips3_device::execute_run()+0x0089)
  0000000f81d69970: 00007ff65c943d18 (device_scheduler::timeslice()+0x02e8)
  0000000f81d69ad0: 00007ff65c940a98 (running_machine::run(bool)+0x0198)
  0000000f81d6ef50: 00007ff65ff468cb (mame_machine_manager::execute()+0x020b)
  0000000f81d6f330: 00007ff66293ad2f (cli_frontend::start_execution(mame_machine_manager*, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&)+0x03cf)
  0000000f81d6f5f0: 00007ff66293b33d (cli_frontend::execute(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&)+0x004d)
  0000000f81d6f650: 00007ff65ff40f79 (emulator_info::start_frontend(emu_options&, osd_interface&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&)+0x0029)
  0000000f81d6fa20: 00007ff663b17b77 (main+0x0197)
  0000000f81d6faf0: 00007ff657ef13b1 (__tmainCRTStartup+0x0231)
  0000000f81d6fb20: 00007ff657ef14e6 (mainCRTStartup+0x0016)
  0000000f81d6fb50: 00007ff8f3b03d2d (BaseThreadInitThunk+0x001d)
  0000000f81d6fbd0: 00007ff8f4dfd838 (RtlUserThreadStart+0x0028)

I will build a debug build from git and try and reproduce it there, but maybe someone else with a debug build handy can already confirm it

Pinging @FlyGoat and @MooglyGuy (this might be related to the recent REX3 changes)

[FlyGoat]

Does revert REX3 change commit help?

[darkstar]

The screen shift can be resolved by setenv monitor h in the PROM. My bad, I remember doing it in the past to fix the issue.

I'm currently bisecting the crash during the login screen

[darkstar]

This is the commit causing the crashes on the login screen

0b513144cbe45330659dae245acc09b2ba7dd368 is the first bad commit
commit 0b513144cbe45330659dae245acc09b2ba7dd368
Author: Jiaxun Yang
Date:   Mon Nov 14 16:19:27 2022 +0000

    newport: Split out iterator setup stage (#10546)

    In REX3, iterator setup should be a dedicated operation that
    is only execuated if
    a). A Draw command is execuated with DoSetup at DRAWMODE0 set
    or
    b). A host write is issued to SETUP register

    At setup stage REX3 will calcuate quadrant for a block or span
    drawcall or octant together with some Bresenham parameters for
    a line drawcall.

    Linux newport_con driver is rely on this behavior to use quadrant
    calculated by previous draw call to render characters.

    Fixes: #9667

 src/devices/bus/gio64/newport.cpp | 70 ++++++++++++++++++++++++++++++++++++---
 src/devices/bus/gio64/newport.h   |  1 +
 2 files changed, 67 insertions(+), 4 deletions(-)

[FlyGoat]

Will try to debug.