Dealing with a mixed width ISA

[snim2]

I'm writing a simulator in Pydgin for an ISA which mixes 16-bit and 32-bit instructions. I can happily update my PC correctly and mix different width instructions in the same program, but only if line 149 in pydgin.storage is commented out, and my instructions are padded to 32bits with xs. This is the LOC in pydgin.storage.iread() that has to be removed to make this work:

assert start_addr & 0b11 == 0  # only aligned accesses allowed

At the moment I'm testing this code with py.test, so I haven't built a simulator with a JIT, so I don't think my code has exercised read() yet, and I'm not sure what would happen if it did.

So, how safe is it to delete this assertion? Do I need to do anything to read()? Is anything else liable to break?

/cc @cfbolz

[dmlockhart]

@snim2 It looks like you got mixed width instructions working with Pydgin. That's great! It isn't something we've tried yet, so we're definitely excited you were able to get this to work . Poking through your implementation now, it looks really cool!

@berkinilbeyi and I were impressed with your handling of 16-bit vs 32-bit instructions by bumping the PC by 2 or 4. Very clean!

[snim2]

Many thanks!

[dmlockhart]

@snim2 My initial plan was to have a create_cisc_decoder in addition to create_risc_decoder since I figured this would be necessary to handle different instruction widths. Considering this isn't needed to support CISC instruction sets since you can just pad the upper bits with x, do you think this would still be a useful feature to have?

[snim2]

Good question, the simulator I am writing is for a RISC architecture, so it's hard to day. Perhaps an easier approach would be to include some pydgin functions to do the 16 / 32/ 64 conversions. We didn't get very far through the ISA, but I will probably want to abstract some of the common operations.

[snim2]

Having gone a little further down this route I'm not sure that our approach quite works. I have a nice test harness that allows me to write a program as a list of Python integers and load that straight into a memory object for the simulator. Up until now the test harness treated each instruction as if it were 32 bits long, padding with zeroes whenever it wrote out a 16 bit instruction. Obviously, that isn't the case in a real ELF file, where the mixed width instructions are just laid out next to one another, and as you would expect my test results from ELF files differed from the results of similar tests using the harness.

Having fixed up the test harness so that it writes the instructions out in memory in a more realistic way, I find that my functional tests now fail. Below is a full, but minimal, simulator which exhibits the problem. This simulator has two 16 bit instructions (nop16 and halt16) and prints out the contents of its memory before it starts running.

The results I get are as follows:

$ PYTHONPATH=. python mycode/example_mixed_width.py 
You must supply a filename
NOTE: Using sparse storage
sparse memory size 400 addr mask 3ff block mask fffffc00
Program in memory:
0: 0000000110100010 (418)  # width=16bits
1: 0000000111000010 (450)  # width=16bits
Exception in execution (pc: 0x00000000), aborting!
Exception message: Invalid instruction 0x1a20000!
DONE! Status = 0
Instructions Executed = 0
Traceback (most recent call last):
  File "mycode/example_mixed_width.py", line 104, in <module>
    assert machine.state.pc == 4, "Expected pc = 4, got pc = {0}".format(machine.state.pc)
AssertionError: Expected pc = 4, got pc = 0

...and the simulator is below. Looking through this it struck me that the issue might be with endianness (the real machine I'm simulating is little-endian), but even if that is the case I'm sure how best to go about fixing this problem, so any advice you might have would be much appreciated.

from pydgin.debug import Debug
from pydgin.misc import create_risc_decoder
from pydgin.sim import Sim, init_sim
from pydgin.storage import Memory, RegisterFile

reg_map = { 'pc' : 0, 'STATUS' : 1 }

encodings = [
    ['nop16',       'xxxxxxxxxxxxxxxxxxxxxx0110100010'],
    ['halt16',      'xxxxxxxxxxxxxxxxxxxxxx0111000010'],
]

def execute_nop16(s, inst):
    """Do nothing but increment the PC.
    16 bit instruction.
    """
    s.pc += 2

def execute_halt16(s, inst):
    """Set a flag in the STATUS register and HALT the machine.
    16 bit instruction.
    """
    s.rf[reg_map['STATUS']] |= 1
    s.pc += 2
    s.running = False

decode = create_risc_decoder(encodings, globals(), debug=True)

class ExampleState(object):

    def __init__(self, memory, debug):
        self.pc        = 0
        self.mem       = memory
        self.debug     = debug
        self.rf        = RegisterFile(constant_zero=False, num_regs=2)
        self.running   = True   # Set False by halt16 instruction.
        self.debug     = debug
        self.rf.debug  = debug
        self.mem.debug = debug
        self.status   = 0
        self.ncycles  = 0
        self.stats_en = True

    def fetch_pc(self):
        return self.pc

class ExampleInstruction(object):

    def __init__(self, bits, str):
        self.bits = bits
        self.str  = str

class ExampleMachine(Sim):

    def __init__(self):
        Sim.__init__(self, "Example", jit_enabled=True) # Breaks if False.

    def decode(self, bits):
        inst_str, exec_fun = decode(bits)
        return ExampleInstruction(bits, inst_str), exec_fun

    def load_program(self, instructions, **args):
        """Load the program into a memory object.
        Instructions should take the form of a list of tuples containing an
        instruction and its width. e.g.:
            [ (0b0, 32) ]
        """
        mem = Memory(size=2**18, byte_storage=False)
        written_so_far = 0
        for data, width in instructions:
            num_bytes = width / 8
            written_so_far += num_bytes
            mem.write(written_so_far, num_bytes, data)
        print 'Program in memory:'
        # We are expecting to see a nop16 and a halt16, i.e.:
        #    1: 418    # 16bits
        #    2: 450    # 16bits
        read_so_far = 0
        for i, (_, width) in enumerate(instructions):
            num_bytes = width / 8
            read_so_far += num_bytes
            inst = mem.read(read_so_far, num_bytes)
            print "{0}: {1:016b} ({2})  # width={3}bits".format(i, inst, inst, width)
        self.state = ExampleState(mem, Debug())

init_sim(ExampleMachine())

if __name__ == '__main__':
    # This test case has been transcribed directly from an ELF file.
    instructions = [(0b0000000110100010, 16),  # nop16
                    (0b0000000111000010, 16),  # halt16
                    ]
    machine = ExampleMachine()
    machine.load_program(instructions)
    machine.run()
    # nop16 increments by 2, halt16 increments by 2.
    assert machine.state.pc == 4, "Expected pc = 4, got pc = {0}".format(machine.state.pc)
    assert not machine.state.running, "Machine not HALTed."

[berkinilbeyi]

Hi Sarah,

This also seems to have the same bug as the one I emailed you. If you put written_so_far += nbytes and read_so_far += nbytes after the write and read respectively, it seems to work.

Berkin

[snim2]

Argh! Even my unit tests need unit tests.

Many thanks for this. I have issued a PR, but I haven't looked closely at optimising this approach.

[berkinilbeyi]

I think this bug is resolved, so closing it.