go-restruct / restruct

Rich binary (de)serialization library for Golang
https://restruct.io/
ISC License
352 stars 17 forks source link

How do I use restruct to deserialize these structs? #5

Open kellabyte opened 7 years ago

kellabyte commented 7 years ago

I have these structs from C# that I want to port to Go. How would I handle these custom annotations with restruct?

[StructLayout (LayoutKind.Sequential, Pack = 1)]
        struct ATOM_ROM_HEADER
        {
            public ATOM_COMMON_TABLE_HEADER sHeader;
            public UInt32 uaFirmWareSignature;
            public UInt16 usBiosRuntimeSegmentAddress;
            public UInt16 usProtectedModeInfoOffset;
            public UInt16 usConfigFilenameOffset;
            public UInt16 usCRC_BlockOffset;
            public UInt16 usBIOS_BootupMessageOffset;
            public UInt16 usInt10Offset;
            public UInt16 usPciBusDevInitCode;
            public UInt16 usIoBaseAddress;
            public UInt16 usSubsystemVendorID;
            public UInt16 usSubsystemID;
            public UInt16 usPCI_InfoOffset;
            public UInt16 usMasterCommandTableOffset;
            public UInt16 usMasterDataTableOffset;
            public Byte ucExtendedFunctionCode;
            public Byte ucReserved;
            public UInt32 ulPSPDirTableOffset;
            public UInt16 usVendorID;
            public UInt16 usDeviceID;
        }

        [StructLayout (LayoutKind.Sequential, Pack = 1)]
        struct ATOM_DATA_TABLES
        {
            public ATOM_COMMON_TABLE_HEADER sHeader;
            public UInt16 UtilityPipeLine;
            public UInt16 MultimediaCapabilityInfo;
            public UInt16 MultimediaConfigInfo;
            public UInt16 StandardVESA_Timing;
            public UInt16 FirmwareInfo;
            public UInt16 PaletteData;
            public UInt16 LCD_Info;
            public UInt16 DIGTransmitterInfo;
            public UInt16 SMU_Info;
            public UInt16 SupportedDevicesInfo;
            public UInt16 GPIO_I2C_Info;
            public UInt16 VRAM_UsageByFirmware;
            public UInt16 GPIO_Pin_LUT;
            public UInt16 VESA_ToInternalModeLUT;
            public UInt16 GFX_Info;
            public UInt16 PowerPlayInfo;
            public UInt16 GPUVirtualizationInfo;
            public UInt16 SaveRestoreInfo;
            public UInt16 PPLL_SS_Info;
            public UInt16 OemInfo;
            public UInt16 XTMDS_Info;
            public UInt16 MclkSS_Info;
            public UInt16 Object_Header;
            public UInt16 IndirectIOAccess;
            public UInt16 MC_InitParameter;
            public UInt16 ASIC_VDDC_Info;
            public UInt16 ASIC_InternalSS_Info;
            public UInt16 TV_VideoMode;
            public UInt16 VRAM_Info;
            public UInt16 MemoryTrainingInfo;
            public UInt16 IntegratedSystemInfo;
            public UInt16 ASIC_ProfilingInfo;
            public UInt16 VoltageObjectInfo;
            public UInt16 PowerSourceInfo;
            public UInt16 ServiceInfo;
        };

        [StructLayout (LayoutKind.Sequential, Pack = 1)]
        unsafe struct ATOM_POWERPLAY_TABLE
        {
            public ATOM_COMMON_TABLE_HEADER sHeader;
            public Byte ucTableRevision;
            public UInt16 usTableSize;
            public UInt32 ulGoldenPPID;
            public UInt32 ulGoldenRevision;
            public UInt16 usFormatID;
            public UInt16 usVoltageTime;
            public UInt32 ulPlatformCaps;
            public UInt32 ulMaxODEngineClock;
            public UInt32 ulMaxODMemoryClock;
            public UInt16 usPowerControlLimit;
            public UInt16 usUlvVoltageOffset;
            public UInt16 usStateArrayOffset;
            public UInt16 usFanTableOffset;
            public UInt16 usThermalControllerOffset;
            public UInt16 usReserv;
            public UInt16 usMclkDependencyTableOffset;
            public UInt16 usSclkDependencyTableOffset;
            public UInt16 usVddcLookupTableOffset;
            public UInt16 usVddgfxLookupTableOffset;
            public UInt16 usMMDependencyTableOffset;
            public UInt16 usVCEStateTableOffset;
            public UInt16 usPPMTableOffset;
            public UInt16 usPowerTuneTableOffset;
            public UInt16 usHardLimitTableOffset;
            public UInt16 usPCIETableOffset;
            public UInt16 usGPIOTableOffset;
            public fixed UInt16 usReserved[6];
        };

        [StructLayout (LayoutKind.Sequential, Pack = 1)]
        struct ATOM_MCLK_ENTRY
        {
            public Byte ucVddcInd;
            public UInt16 usVddci;
            public UInt16 usVddgfxOffset;
            public UInt16 usMvdd;
            public UInt32 ulMclk;
            public UInt16 usReserved;
        };

        [StructLayout (LayoutKind.Sequential, Pack = 1)]
        struct ATOM_MCLK_TABLE
        {
            public Byte ucRevId;
            public Byte ucNumEntries;
            // public ATOM_MCLK_ENTRY entries[ucNumEntries];
        };

        [StructLayout (LayoutKind.Sequential, Pack = 1)]
        struct ATOM_SCLK_ENTRY
        {
            public Byte ucVddInd;
            public UInt16 usVddcOffset;
            public UInt32 ulSclk;
            public UInt16 usEdcCurrent;
            public Byte ucReliabilityTemperature;
            public Byte ucCKSVOffsetandDisable;
            public UInt32 ulSclkOffset;
            // Polaris Only, remove for compatibility with Fiji
        };

        [StructLayout (LayoutKind.Sequential, Pack = 1)]
        struct ATOM_SCLK_TABLE
        {
            public Byte ucRevId;
            public Byte ucNumEntries;
            // public ATOM_SCLK_ENTRY entries[ucNumEntries];
        };

        [StructLayout (LayoutKind.Sequential, Pack = 1)]
        struct ATOM_VOLTAGE_ENTRY
        {
            public UInt16 usVdd;
            public UInt16 usCACLow;
            public UInt16 usCACMid;
            public UInt16 usCACHigh;
        };

        [StructLayout (LayoutKind.Sequential, Pack = 1)]
        struct ATOM_VOLTAGE_TABLE
        {
            public Byte ucRevId;
            public Byte ucNumEntries;
            // public ATOM_VOLTAGE_ENTRY entries[ucNumEntries];
        };

        [StructLayout (LayoutKind.Sequential, Pack = 1)]
        struct ATOM_FAN_TABLE
        {
            public Byte ucRevId;
            public Byte ucTHyst;
            public UInt16 usTMin;
            public UInt16 usTMed;
            public UInt16 usTHigh;
            public UInt16 usPWMMin;
            public UInt16 usPWMMed;
            public UInt16 usPWMHigh;
            public UInt16 usTMax;
            public Byte ucFanControlMode;
            public UInt16 usFanPWMMax;
            public UInt16 usFanOutputSensitivity;
            public UInt16 usFanRPMMax;
            public UInt32 ulMinFanSCLKAcousticLimit;
            public Byte ucTargetTemperature;
            public Byte ucMinimumPWMLimit;
            public UInt16 usFanGainEdge;
            public UInt16 usFanGainHotspot;
            public UInt16 usFanGainLiquid;
            public UInt16 usFanGainVrVddc;
            public UInt16 usFanGainVrMvdd;
            public UInt16 usFanGainPlx;
            public UInt16 usFanGainHbm;
            public UInt16 usReserved;
        };

        [StructLayout (LayoutKind.Sequential, Pack = 1)]
        struct ATOM_POWERTUNE_TABLE
        {
            public Byte ucRevId;
            public UInt16 usTDP;
            public UInt16 usConfigurableTDP;
            public UInt16 usTDC;
            public UInt16 usBatteryPowerLimit;
            public UInt16 usSmallPowerLimit;
            public UInt16 usLowCACLeakage;
            public UInt16 usHighCACLeakage;
            public UInt16 usMaximumPowerDeliveryLimit;
            public UInt16 usTjMax;
            public UInt16 usPowerTuneDataSetID;
            public UInt16 usEDCLimit;
            public UInt16 usSoftwareShutdownTemp;
            public UInt16 usClockStretchAmount;
            public UInt16 usTemperatureLimitHotspot;
            public UInt16 usTemperatureLimitLiquid1;
            public UInt16 usTemperatureLimitLiquid2;
            public UInt16 usTemperatureLimitVrVddc;
            public UInt16 usTemperatureLimitVrMvdd;
            public UInt16 usTemperatureLimitPlx;
            public Byte ucLiquid1_I2C_address;
            public Byte ucLiquid2_I2C_address;
            public Byte ucLiquid_I2C_Line;
            public Byte ucVr_I2C_address;
            public Byte ucVr_I2C_Line;
            public Byte ucPlx_I2C_address;
            public Byte ucPlx_I2C_Line;
            public UInt16 usReserved;
        };

        [StructLayout (LayoutKind.Sequential, Pack = 1)]
        struct ATOM_VRAM_TIMING_ENTRY
        {
            public UInt32 ulClkRange;
            [MarshalAs (UnmanagedType.ByValArray, SizeConst = 0x30)]
            public Byte[] ucLatency;
        };

        [StructLayout (LayoutKind.Sequential, Pack = 1)]
        struct ATOM_VRAM_ENTRY
        {
            public UInt32 ulChannelMapCfg;
            public UInt16 usModuleSize;
            public UInt16 usMcRamCfg;
            public UInt16 usEnableChannels;
            public Byte ucExtMemoryID;
            public Byte ucMemoryType;
            public Byte ucChannelNum;
            public Byte ucChannelWidth;
            public Byte ucDensity;
            public Byte ucBankCol;
            public Byte ucMisc;
            public Byte ucVREFI;
            public UInt16 usReserved;
            public UInt16 usMemorySize;
            public Byte ucMcTunningSetId;
            public Byte ucRowNum;
            public UInt16 usEMRS2Value;
            public UInt16 usEMRS3Value;
            public Byte ucMemoryVenderID;
            public Byte ucRefreshRateFactor;
            public Byte ucFIFODepth;
            public Byte ucCDR_Bandwidth;
            public UInt32 ulChannelMapCfg1;
            public UInt32 ulBankMapCfg;
            public UInt32 ulReserved;
            [MarshalAs (UnmanagedType.ByValArray, SizeConst = 20)]
            public Byte[] strMemPNString;
        };

        [StructLayout (LayoutKind.Sequential, Pack = 1)]
        struct ATOM_VRAM_INFO
        {
            public ATOM_COMMON_TABLE_HEADER sHeader;
            public UInt16 usMemAdjustTblOffset;
            public UInt16 usMemClkPatchTblOffset;
            public UInt16 usMcAdjustPerTileTblOffset;
            public UInt16 usMcPhyInitTableOffset;
            public UInt16 usDramDataRemapTblOffset;
            public UInt16 usReserved1;
            public Byte ucNumOfVRAMModule;
            public Byte ucMemoryClkPatchTblVer;
            public Byte ucVramModuleVer;
            public Byte ucMcPhyTileNum;
            // public ATOM_VRAM_ENTRY aVramInfo[ucNumOfVRAMModule];
        }
jchv commented 7 years ago

I've made a quick demonstration. It may not be 100% accurate, sorry if I messed anything up, I've only verified that it reads the header correctly.

package main

import (
    "encoding/binary"
    "flag"
    "fmt"
    "io/ioutil"
    "log"

    restruct "gopkg.in/restruct.v1"
)

const (
    AtomROMChecksumOffset = 0x21
    AtomROMHeaderPtr      = 0x48
)

type AtomCommonTableHeader struct {
    StructureSize        int16
    TableFormatRevision  byte
    TableContentRevision byte
}

type AtomRomHeader struct {
    Header                    AtomCommonTableHeader
    FirmWareSignature         uint32
    BiosRuntimeSegmentAddress uint16
    ProtectedModeInfoOffset   uint16
    ConfigFilenameOffset      uint16
    CRCBlockOffset            uint16
    BIOSBootupMessageOffset   uint16
    Int10Offset               uint16
    PciBusDevInitCode         uint16
    IoBaseAddress             uint16
    SubsystemVendorID         uint16
    SubsystemID               uint16
    PCIInfoOffset             uint16
    MasterCommandTableOffset  uint16
    MasterDataTableOffset     uint16
    ExtendedFunctionCode      byte
    _                         byte
    PSPDirTableOffset         uint32
    VendorID                  uint16
    DeviceID                  uint16
}

type AtomDataTables struct {
    Header                   AtomCommonTableHeader
    UtilityPipeLine          uint16
    MultimediaCapabilityInfo uint16
    MultimediaConfigInfo     uint16
    StandardVESATiming       uint16
    FirmwareInfo             uint16
    PaletteData              uint16
    LCDInfo                  uint16
    DIGTransmitterInfo       uint16
    SMUInfo                  uint16
    SupportedDevicesInfo     uint16
    GPIOI2CInfo              uint16
    VRAMUsageByFirmware      uint16
    GPIOPinLUT               uint16
    VESAToInternalModeLUT    uint16
    GFXInfo                  uint16
    PowerPlayInfo            uint16
    GPUVirtualizationInfo    uint16
    SaveRestoreInfo          uint16
    PPLLSSInfo               uint16
    OemInfo                  uint16
    XTMDSInfo                uint16
    MclkSSInfo               uint16
    ObjectHeader             uint16
    IndirectIOAccess         uint16
    MCInitParameter          uint16
    ASICVDDCInfo             uint16
    ASICInternalSSInfo       uint16
    TVVideoMode              uint16
    VRAMInfo                 uint16
    MemoryTrainingInfo       uint16
    IntegratedSystemInfo     uint16
    ASICProfilingInfo        uint16
    VoltageObjectInfo        uint16
    PowerSourceInfo          uint16
    ServiceInfo              uint16
}

type AtomPowerplayTable struct {
    Header                    AtomCommonTableHeader
    TableRevision             byte
    TableSize                 uint16
    GoldenPPID                uint32
    GoldenRevision            uint32
    FormatID                  uint16
    VoltageTime               uint16
    PlatformCaps              uint32
    MaxODEngineClock          uint32
    MaxODMemoryClock          uint32
    PowerControlLimit         uint16
    UlvVoltageOffset          uint16
    StateArrayOffset          uint16
    FanTableOffset            uint16
    ThermalControllerOffset   uint16
    _                         uint16
    MclkDependencyTableOffset uint16
    SclkDependencyTableOffset uint16
    VddcLookupTableOffset     uint16
    VddgfxLookupTableOffset   uint16
    MMDependencyTableOffset   uint16
    VCEStateTableOffset       uint16
    PPMTableOffset            uint16
    PowerTuneTableOffset      uint16
    HardLimitTableOffset      uint16
    PCIETableOffset           uint16
    GPIOTableOffset           uint16
    _                         [6]uint16
}

type AtomMClkEntry struct {
    VddcInd      byte
    Vddci        uint16
    VddgfxOffset uint16
    Mvdd         uint16
    Mclk         uint32
    _            uint16
}

type AtomMClkTable struct {
    RevID      byte
    NumEntries byte
    Entries    []AtomMClkEntry ``
}

type AtomSClkEntry struct {
    VddInd                 byte
    VddcOffset             uint16
    Sclk                   uint32
    EdcCurrent             uint16
    ReliabilityTemperature byte
    CKSVOffsetandDisable   byte
    SclkOffset             uint32
    // Polaris Only, remove for compatibility with Fiji
}

type AtomSClkTable struct {
    RevID      byte
    NumEntries byte `struct:"sizeof=Entries"`
    Entries    []AtomSClkEntry
}

type AtomVoltageEntry struct {
    Vdd     uint16
    CACLow  uint16
    CACMid  uint16
    CACHigh uint16
}

type AtomVoltageTable struct {
    RevID      byte
    NumEntries byte `struct:"sizeof=Entries"`
    Entries    []AtomVoltageEntry
}

type AtomFanTable struct {
    RevID                   byte
    THyst                   byte
    TMin                    uint16
    TMed                    uint16
    THigh                   uint16
    PWMMin                  uint16
    PWMMed                  uint16
    PWMHigh                 uint16
    TMax                    uint16
    FanControlMode          byte
    FanPWMMax               uint16
    FanOutputSensitivity    uint16
    FanRPMMax               uint16
    MinFanSCLKAcousticLimit uint32
    TargetTemperature       byte
    MinimumPWMLimit         byte
    FanGainEdge             uint16
    FanGainHotspot          uint16
    FanGainLiquid           uint16
    FanGainVrVddc           uint16
    FanGainVrMvdd           uint16
    FanGainPlx              uint16
    FanGainHbm              uint16
    _                       uint16
}

type AtomPowertuneTable struct {
    RevID                     byte
    TDP                       uint16
    ConfigurableTDP           uint16
    TDC                       uint16
    BatteryPowerLimit         uint16
    SmallPowerLimit           uint16
    LowCACLeakage             uint16
    HighCACLeakage            uint16
    MaximumPowerDeliveryLimit uint16
    TjMax                     uint16
    PowerTuneDataSetID        uint16
    EDCLimit                  uint16
    SoftwareShutdownTemp      uint16
    ClockStretchAmount        uint16
    TemperatureLimitHotspot   uint16
    TemperatureLimitLiquid1   uint16
    TemperatureLimitLiquid2   uint16
    TemperatureLimitVrVddc    uint16
    TemperatureLimitVrMvdd    uint16
    TemperatureLimitPlx       uint16
    Liquid1I2CAddress         byte
    Liquid2I2CAddress         byte
    LiquidI2CLine             byte
    VrI2CAddress              byte
    VrI2CLine                 byte
    PlxI2CAddress             byte
    PlxI2CLine                byte
    _                         uint16
}

type AtomVRAMTimingEntry struct {
    ClkRange uint32
    Latency  [0x30]byte
}

type AtomVRAMEntry struct {
    ChannelMapCfg     uint32
    ModuleSize        uint16
    McRAMCfg          uint16
    EnableChannels    uint16
    ExtMemoryID       byte
    MemoryType        byte
    ChannelNum        byte
    ChannelWidth      byte
    Density           byte
    BankCol           byte
    Misc              byte
    VREFI             byte
    _                 uint16
    MemorySize        uint16
    McTunningSetID    byte
    RowNum            byte
    EMRS2Value        uint16
    EMRS3Value        uint16
    MemoryVenderID    byte
    RefreshRateFactor byte
    FIFODepth         byte
    CDRBandwidth      byte
    ChannelMapCfg1    uint32
    BankMapCfg        uint32
    _                 uint32
    MemPNString       [20]byte
}

type AtomVRAMInfo struct {
    Header                   AtomCommonTableHeader
    MemAdjustTblOffset       uint16
    MemClkPatchTblOffset     uint16
    McAdjustPerTileTblOffset uint16
    McPhyInitTableOffset     uint16
    DramDataRemapTblOffset   uint16
    _                        uint16
    NumOfVRAMModule          byte `struct:"sizeof=VramInfo"`
    MemoryClkPatchTblVer     byte
    VramModuleVer            byte
    McPhyTileNum             byte
    VramInfo                 []AtomVRAMEntry
}

func main() {
    flag.Parse()
    if flag.NArg() != 1 {
        log.Println("Usage: <ROM file>")
        return
    }

    fn := flag.Arg(0)

    data, err := ioutil.ReadFile(fn)
    if err != nil {
        log.Fatalln("Error reading ROM file:", err)
    }

    // Get offset to header
    headerOffset := binary.LittleEndian.Uint16(data[AtomROMHeaderPtr : AtomROMHeaderPtr+2])

    // Unpack header
    header := AtomRomHeader{}
    err = restruct.Unpack(data[headerOffset:], binary.LittleEndian, &header)
    if err != nil {
        log.Fatalln("Error unpacking ROM header:", err)
    }

    fmt.Printf("HEADER: %#v\n", header)
}

Some useful notes:

I hope this will help you get started. I tested it with the RX480 BIOS and was able to get what looked like the header structure. Try it using the demo code above (throw the example into a file in an empty folder, go build, and pass it the filename via command line.)

Also, if you are doing so, please be careful when modding your BIOS :) Obviously, I can't provide any warranty as to the correct operation of Restruct, but I wish you the best of luck in your endeavors.

kellabyte commented 7 years ago

Holy crap. This was like one of the best answers I've ever gotten on GitHub. You actually googled what I was trying to do and read in a BIOS haha :) You described everything as well! Let me try this out! Thanks so much!

kellabyte commented 7 years ago

Why did you have to rename public Byte ucReserved; to _? Just curious why you were forced to do that?

jchv commented 7 years ago

No problem. Of course if you find any bugs or other issues feel free to open those separately of this ticket.

jchv commented 7 years ago

Oh sorry. No I was not forced, I just tried to make it as idiomatic as possible. You can restore those to Reserved if you want to be able to access them, the _ keyword in go is just convenient for when you don't care.

kellabyte commented 7 years ago

Okay great, thanks so much again! This seems to be working :) This library really made it so much simpler.

kellabyte commented 7 years ago

How would I handle deserializing this part?

atom_vram_timing_offset = atom_vram_info_offset + atom_vram_info.usMemClkPatchTblOffset + 0x2E;
atom_vram_timing_entries = new ATOM_VRAM_TIMING_ENTRY[MAX_VRAM_ENTRIES];
for (var i = 0; i < MAX_VRAM_ENTRIES; i++) {
    atom_vram_timing_entries [i] = fromBytes<ATOM_VRAM_TIMING_ENTRY> (buffer.Skip (atom_vram_timing_offset + Marshal.SizeOf (typeof(ATOM_VRAM_TIMING_ENTRY)) * i).ToArray ());

    // atom_vram_timing_entries have an undetermined length
    // attempt to determine the last entry in the array
    if (atom_vram_timing_entries [i].ulClkRange == 0) {
        Array.Resize (ref atom_vram_timing_entries, i);
        break;
    }
}
jchv commented 7 years ago

Hmm... Mostly the same, but it is a little more challenging.

We have to go through the chain of structs to get there. I decided to make another demo because it is a little harder than I expected.

The worst part is determining the size of the VRAM info structure. I created a new ticket to cover making it simpler to get the binary size of a structure.

FWIW, I can't verify that this is actually accurate, so you may want to try to compare this to what Polaris shows for the same ROM.

package main

import (
    "encoding/binary"
    "flag"
    "fmt"
    "io/ioutil"
    "log"

    restruct "gopkg.in/restruct.v1"
)

const (
    AtomROMChecksumOffset = 0x21
    AtomROMHeaderPtr      = 0x48
)

type AtomCommonTableHeader struct {
    StructureSize        int16
    TableFormatRevision  byte
    TableContentRevision byte
}

type AtomRomHeader struct {
    Header                    AtomCommonTableHeader
    FirmWareSignature         uint32
    BiosRuntimeSegmentAddress uint16
    ProtectedModeInfoOffset   uint16
    ConfigFilenameOffset      uint16
    CRCBlockOffset            uint16
    BIOSBootupMessageOffset   uint16
    Int10Offset               uint16
    PciBusDevInitCode         uint16
    IoBaseAddress             uint16
    SubsystemVendorID         uint16
    SubsystemID               uint16
    PCIInfoOffset             uint16
    MasterCommandTableOffset  uint16
    MasterDataTableOffset     uint16
    ExtendedFunctionCode      byte
    _                         byte
    PSPDirTableOffset         uint32
    VendorID                  uint16
    DeviceID                  uint16
}

type AtomDataTables struct {
    Header                   AtomCommonTableHeader
    UtilityPipeLine          uint16
    MultimediaCapabilityInfo uint16
    MultimediaConfigInfo     uint16
    StandardVESATiming       uint16
    FirmwareInfo             uint16
    PaletteData              uint16
    LCDInfo                  uint16
    DIGTransmitterInfo       uint16
    SMUInfo                  uint16
    SupportedDevicesInfo     uint16
    GPIOI2CInfo              uint16
    VRAMUsageByFirmware      uint16
    GPIOPinLUT               uint16
    VESAToInternalModeLUT    uint16
    GFXInfo                  uint16
    PowerPlayInfo            uint16
    GPUVirtualizationInfo    uint16
    SaveRestoreInfo          uint16
    PPLLSSInfo               uint16
    OemInfo                  uint16
    XTMDSInfo                uint16
    MclkSSInfo               uint16
    ObjectHeader             uint16
    IndirectIOAccess         uint16
    MCInitParameter          uint16
    ASICVDDCInfo             uint16
    ASICInternalSSInfo       uint16
    TVVideoMode              uint16
    VRAMInfo                 uint16
    MemoryTrainingInfo       uint16
    IntegratedSystemInfo     uint16
    ASICProfilingInfo        uint16
    VoltageObjectInfo        uint16
    PowerSourceInfo          uint16
    ServiceInfo              uint16
}

type AtomPowerplayTable struct {
    Header                    AtomCommonTableHeader
    TableRevision             byte
    TableSize                 uint16
    GoldenPPID                uint32
    GoldenRevision            uint32
    FormatID                  uint16
    VoltageTime               uint16
    PlatformCaps              uint32
    MaxODEngineClock          uint32
    MaxODMemoryClock          uint32
    PowerControlLimit         uint16
    UlvVoltageOffset          uint16
    StateArrayOffset          uint16
    FanTableOffset            uint16
    ThermalControllerOffset   uint16
    _                         uint16
    MclkDependencyTableOffset uint16
    SclkDependencyTableOffset uint16
    VddcLookupTableOffset     uint16
    VddgfxLookupTableOffset   uint16
    MMDependencyTableOffset   uint16
    VCEStateTableOffset       uint16
    PPMTableOffset            uint16
    PowerTuneTableOffset      uint16
    HardLimitTableOffset      uint16
    PCIETableOffset           uint16
    GPIOTableOffset           uint16
    _                         [6]uint16
}

type AtomMClkEntry struct {
    VddcInd      byte
    Vddci        uint16
    VddgfxOffset uint16
    Mvdd         uint16
    Mclk         uint32
    _            uint16
}

type AtomMClkTable struct {
    RevID      byte
    NumEntries byte
    Entries    []AtomMClkEntry
}

type AtomSClkEntry struct {
    VddInd                 byte
    VddcOffset             uint16
    Sclk                   uint32
    EdcCurrent             uint16
    ReliabilityTemperature byte
    CKSVOffsetandDisable   byte
    SclkOffset             uint32
    // Polaris Only, remove for compatibility with Fiji
}

type AtomSClkTable struct {
    RevID      byte
    NumEntries byte `struct:"sizeof=Entries"`
    Entries    []AtomSClkEntry
}

type AtomVoltageEntry struct {
    Vdd     uint16
    CACLow  uint16
    CACMid  uint16
    CACHigh uint16
}

type AtomVoltageTable struct {
    RevID      byte
    NumEntries byte `struct:"sizeof=Entries"`
    Entries    []AtomVoltageEntry
}

type AtomFanTable struct {
    RevID                   byte
    THyst                   byte
    TMin                    uint16
    TMed                    uint16
    THigh                   uint16
    PWMMin                  uint16
    PWMMed                  uint16
    PWMHigh                 uint16
    TMax                    uint16
    FanControlMode          byte
    FanPWMMax               uint16
    FanOutputSensitivity    uint16
    FanRPMMax               uint16
    MinFanSCLKAcousticLimit uint32
    TargetTemperature       byte
    MinimumPWMLimit         byte
    FanGainEdge             uint16
    FanGainHotspot          uint16
    FanGainLiquid           uint16
    FanGainVrVddc           uint16
    FanGainVrMvdd           uint16
    FanGainPlx              uint16
    FanGainHbm              uint16
    _                       uint16
}

type AtomPowertuneTable struct {
    RevID                     byte
    TDP                       uint16
    ConfigurableTDP           uint16
    TDC                       uint16
    BatteryPowerLimit         uint16
    SmallPowerLimit           uint16
    LowCACLeakage             uint16
    HighCACLeakage            uint16
    MaximumPowerDeliveryLimit uint16
    TjMax                     uint16
    PowerTuneDataSetID        uint16
    EDCLimit                  uint16
    SoftwareShutdownTemp      uint16
    ClockStretchAmount        uint16
    TemperatureLimitHotspot   uint16
    TemperatureLimitLiquid1   uint16
    TemperatureLimitLiquid2   uint16
    TemperatureLimitVrVddc    uint16
    TemperatureLimitVrMvdd    uint16
    TemperatureLimitPlx       uint16
    Liquid1I2CAddress         byte
    Liquid2I2CAddress         byte
    LiquidI2CLine             byte
    VrI2CAddress              byte
    VrI2CLine                 byte
    PlxI2CAddress             byte
    PlxI2CLine                byte
    _                         uint16
}

type AtomVRAMTimingEntry struct {
    ClkRange uint32
    Latency  [0x30]byte
}

type AtomVRAMEntry struct {
    ChannelMapCfg     uint32
    ModuleSize        uint16
    McRAMCfg          uint16
    EnableChannels    uint16
    ExtMemoryID       byte
    MemoryType        byte
    ChannelNum        byte
    ChannelWidth      byte
    Density           byte
    BankCol           byte
    Misc              byte
    VREFI             byte
    _                 uint16
    MemorySize        uint16
    McTunningSetID    byte
    RowNum            byte
    EMRS2Value        uint16
    EMRS3Value        uint16
    MemoryVenderID    byte
    RefreshRateFactor byte
    FIFODepth         byte
    CDRBandwidth      byte
    ChannelMapCfg1    uint32
    BankMapCfg        uint32
    _                 uint32
    MemPNString       string `struct:"[20]byte"`
}

type AtomVRAMInfo struct {
    Header                   AtomCommonTableHeader
    MemAdjustTblOffset       uint16
    MemClkPatchTblOffset     uint16
    McAdjustPerTileTblOffset uint16
    McPhyInitTableOffset     uint16
    DramDataRemapTblOffset   uint16
    _                        uint16
    NumOfVRAMModule          byte `struct:"sizeof=VramInfo"`
    MemoryClkPatchTblVer     byte
    VramModuleVer            byte
    McPhyTileNum             byte
    VramInfo                 []AtomVRAMEntry
}

func main() {
    flag.Parse()
    if flag.NArg() != 1 {
        log.Println("Usage: <ROM file>")
        return
    }

    fn := flag.Arg(0)

    data, err := ioutil.ReadFile(fn)
    if err != nil {
        log.Fatalln("Error reading ROM file:", err)
    }

    // Get offset to header
    headerOffset := binary.LittleEndian.Uint16(data[AtomROMHeaderPtr : AtomROMHeaderPtr+2])

    // Unpack header
    header := AtomRomHeader{}
    err = restruct.Unpack(data[headerOffset:], binary.LittleEndian, &header)
    if err != nil {
        log.Fatalln("Error unpacking ROM header:", err)
    }

    // Unpack data tables.
    dataTables := AtomDataTables{}
    err = restruct.Unpack(data[header.MasterDataTableOffset:], binary.LittleEndian, &dataTables)
    if err != nil {
        log.Fatalln("Error unpacking data tables:", err)
    }
    fmt.Printf("Data tables: %#v\n\n", dataTables)

    // Unpack VRAM info.
    vramInfo := AtomVRAMInfo{}
    err = restruct.Unpack(data[dataTables.VRAMInfo:], binary.LittleEndian, &vramInfo)
    if err != nil {
        log.Fatalln("Error unpacking VRAM info:", err)
    }
    fmt.Printf("VRAM Info: %#v\n\n", vramInfo)

    // HACK: determine sizeof VRAM info.
    // See restruct issue #5.
    vramInfoData, err := restruct.Pack(binary.LittleEndian, vramInfo)
    if err != nil {
        log.Fatalln("Error sizing VRAM info:", err)
    }

    // Loop over VRAM timing entries.
    vramTimingPtr := int(dataTables.VRAMInfo) + len(vramInfoData)
    vramTimingEntries := [16]AtomVRAMTimingEntry{}
    for i := range vramTimingEntries {
        err := restruct.Unpack(data[vramTimingPtr:], binary.LittleEndian, &vramTimingEntries[i])
        if err != nil {
            log.Fatalln("Error unpacking timing entry:", err)
        }
        if vramTimingEntries[i].ClkRange == 0 {
            break
        }
        vramTimingPtr += 0x34
    }

    fmt.Printf("VRAM Info Length: %d\n", len(vramInfoData))
    fmt.Printf("VRAM Timing Entries: %#v\n\n", vramTimingEntries)
}
kellabyte commented 7 years ago

Thanks so much for the help again! I think this is working however I've been trying to figure out how to decode some of the fields and I'm a bit stuck at this spot.

istVRAM.Items.Clear ();
for (var i = 0; i < atom_vram_info.ucNumOfVRAMModule; i++) {
    if (atom_vram_entries [i].strMemPNString [0] != 0)
        listVRAM.Items.Add (Encoding.UTF8.GetString (atom_vram_entries [i].strMemPNString).Substring (0, 10));
}
listVRAM.SelectedIndex = 0;
atom_vram_index = listVRAM.SelectedIndex;

tableVRAM_TIMING.Items.Clear ();
for (var i = 0; i < atom_vram_timing_entries.Length; i++) {
    uint tbl = atom_vram_timing_entries [i].ulClkRange >> 24;
    tableVRAM_TIMING.Items.Add (new ListViewItem (new string[] {
        tbl.ToString () + ":" + (atom_vram_timing_entries [i].ulClkRange & 0x00FFFFFF) / 100,
        ByteArrayToString (atom_vram_timing_entries [i].ucLatency)
    }
    ));
}

This is tripping me up:

Encoding.UTF8.GetString (atom_vram_entries [i].strMemPNString).Substring (0, 10)
jchv commented 7 years ago

That part (the Encoding.UTF8.GetString line) isn't strictly necessary, or at least not to the same degree. In my updated example, I switched MemPNString to be a string type with the Restruct tag [20]byte, which will make it behave like it does in the C# code but allow you to treat it as a UTF-8 string directly.

Specifically, check this bit in AtomVRAMEntry:

MemPNString       string `struct:"[20]byte"`

To get just the first 10 characters, you'd simply do vramEntry.MemPNString[:10].

To do it without setting the type of the field to string, you'd just have to do a little more work. Assuming the field looks like this:

MemPNString       [20]byte

...you should just be able to do this: string(vramEntry.MemPNString[:10]) to get the UTF-8 string of the first 10 bytes.

kellabyte commented 7 years ago

Oh! That looks simple but I'm getting weird output.

`�PS
DD�
�
count := int(vramInfo.NumOfVRAMModule)
for i := 0; i < count; i++ {
    if vramInfo.VramInfo[i].MemPNString[0] != 0 {
        fmt.Printf("%s\n", string(vramInfo.VramInfo[i].MemPNString[:10]))
    }
}
jchv commented 7 years ago

Intersting. Looking at the RX480 BIOS with the previously posted example, i'm getting "K4G80325FB" which looks right to me. Could you possibly try my example with your ROM to make sure that the ROM layout isn't differing slightly somewhere?

kellabyte commented 7 years ago

Yup! Just did, I pasted your example into my app and ran it instead of my code and it shows the same output for the for loop I pasted above.

This ROM is for a RX580 BIOS but Polaris works on it and supports both so this should be working unless there's some offset I'm missing that Polaris is doing.

jchv commented 7 years ago

Hmmm... this is interesting then. I'll get on my Windows computer later and see what Polaris is doing differently. I've probably made a mistake somewhere.

kellabyte commented 7 years ago

I've pushed my more polished implementation of this up to GitHub :) FYI I'm running Polaris via Mono on a Mac, so Windows isn't required. It takes a loooong time to start the first time though.

https://github.com/kellabyte/atitool

jchv commented 7 years ago

OK, thanks, that's actually quite helpful. I'll try to remember to take a look at this tomorrow, I didn't end up getting a look at this tonight.

kellabyte commented 7 years ago

No problem! Thank you so very much for all the help :)

kellabyte commented 7 years ago

Interesting! So I have 2 RX580 GPU kinds and there's 2 filesizes of ROM's for 580's. One is causing me this error one is not.

The VBIOS with 262144 bytes works but the VBIOS with 524288 bytes doesn't work. In PolarisBiosEditor there's a check for these file sizes.

https://github.com/jaschaknack/PolarisBiosEditor/blob/master/PolarisBiosEditor.cs#L513

However I'm not sure how this alters reading data from the file. I must be missing an offset somewhere or something?

jchv commented 7 years ago

I have a feeling the answer lies somewhere else. The way I would go about figuring this out is by instrumenting the Go and C# code with some printf statements that traces out the offsets as determined by each program, all the way up to the top. That way, you would be able to see where it is going awry.

That being said, I've been too busy to actually do that.

kellabyte commented 7 years ago

No problem! I'm outputting some offsets and there seems to be something off here. In PBE:

var atom_vram_entry_offset = atom_vram_info_offset + Marshal.SizeOf (typeof(ATOM_VRAM_INFO));
Console.WriteLine(atom_vram_entry_offset);

41906

In Go code:

// HACK: determine sizeof VRAM info.
// See restruct issue #5.
vramInfoData, err := restruct.Pack(binary.LittleEndian, vramInfo)
if err != nil {
    fmt.Println(chalk.Red, "Error sizing VRAM info: ", err, chalk.Reset)
    os.Exit(1)
}
vramEntryOffset := int(vramInfoOffset) + len(vramInfoData)
fmt.Println(vramEntryOffset)

42098

Perhaps that len trick isn't working.

C#

Console.WriteLine(Marshal.SizeOf (typeof(ATOM_VRAM_INFO)));

20

Go

fmt.Println(len(vramInfoData))

212

Aha! It's reporting the wrong size because you added this field to AtomVRAMInfo.

VramInfo                 []AtomVRAMEntry
jchv commented 7 years ago

Was that the problem?

The C# code was manually decoding VramInfo, I used restruct's sizeof annotation to decode it instead. It's definitely possible that I got the implementation wrong, sizeof doesn't match the behavior, or there's a bug with sizeof, however it would be confusing to me if simply removing VramInfo solved the problem.

However, if it did, feel free to close the issue.

I've been working on the deficiencies in the library that we saw here, especially the lack of a SizeOf function (Issue #5). It unmasked several limitations in the API that I have now created issues for, and if you use the github.com/go-restruct/restruct package instead of the gopkg.in/restruct package, you'll get the ability to use SizeOf instead of Packing and checking len.