solemnwarning / rehex

Reverse Engineers' Hex Editor
https://rehex.solemnwarning.net/
GNU General Public License v2.0
2.3k stars 113 forks source link

Very interesting app but how would I support the MIDI.bt file? #220

Closed perivar closed 7 months ago

perivar commented 9 months ago

I tried to convert the Midi.bt file to this format, but failed. The midi template contains so manye things that I was unable to get working. Like break, sizeof and bitfields. https://www.sweetscape.com/010editor/manual/Bitfields.htm

I would really appriciate if you showed how this could be done: https://www.sweetscape.com/010editor/repository/templates/file_info.php?file=MIDI.bt

Thanks,

solemnwarning commented 9 months ago

There's no support for bit-level addressing in rehex yet, so no bitfields in the templates, but it looks like that MIDI.bt template is only using them for a 24-bit integer, so you could use unsigned char m_usecPerQuarterNote[3]; for the time being, since it doesn't seem to actually depend on its value.

Where the sizeof operator is used, you could instead do something like this:

local unsigned int messages_end = FTell() + m_seclen;
MidiMessage messages[0];

while(FTell() < messages_end)
{
    ArrayExtend(messages);
}
perivar commented 9 months ago

Excellent, thanks!

The last code I am unable to convert is the section using break; How would you conditionally read another variable when break cannot be used? I.e. like the DeltaTime struct: struct DeltaTime { local uint total = 0; char t0; total += t0 & 0x7f; if (!(t0 & 0x80)) break;

total <<= 7;
char t1;
total += t1 & 0x7f;
if (!(t1 & 0x80))
    break;

total <<= 7;
char t2;
total += t2 & 0x7f;
if (!(t2 & 0x80))
    break;

total <<= 7;
char t3;
total += t3 & 0x7f;
if (!(t3 & 0x80))
    break;

};

On Sun, Dec 3, 2023 at 9:27 PM Daniel Collins @.***> wrote:

There's no support for bit-level addressing in rehex yet, so no bitfields in the templates, but it looks like that MIDI.bt template is only using them for a 24-bit integer, so you could use unsigned char m_usecPerQuarterNote[3]; for the time being, since it doesn't seem to actually depend on its value.

Where the sizeof operator is used, you could instead do something like this:

local unsigned messages_end = FTell() + m_seclen; MidiMessage messages[0];

while(FTell() < messages_end) { ArrayExtend(messages); }

— Reply to this email directly, view it on GitHub https://github.com/solemnwarning/rehex/issues/220#issuecomment-1837590581, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHGCFHOQN2OSBNISD6PXYDYHTOC5AVCNFSM6AAAAABAE3RMOSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZXGU4TANJYGE . You are receiving this because you authored the thread.Message ID: @.***>

solemnwarning commented 9 months ago

You can selectively put variable definitions inside 'if' statements, so the lack of a break keyword in struct definitions itself isn't a problem, but 'local' variables declared in structs won't be exposed as members, so where the template does m_length.total wouldn't work.

In rehex templates, I'd suggest doing something like the following at the moment:

unsigned int read_DeltaTime()
{
    local unsigned char byte = ReadU8();
    FSkip(1);

    local unsigned int value = byte & 0x7F;

    for(local int i = 0; i < 3; ++i)
    {
        if((byte & 0x80) != 0)
        {
            byte = ReadU8();
            FSkip(1);

            value <<= 7;
            value |= (byte & 0x7F);
        }
    }

    return value;
}

And then in place of DeltaTime m_length;, you can read/skip the size with local unsigned int length = read_DeltaTime();

perivar commented 9 months ago

I struggled a bit to make it work, and to annotate properly the fields that are missing. Here is a working conversion of the MIDI.bt template. Maybe you could have a repository for working templates? Thanks for your help.

//------------------------------------------------
//--- 010 Editor Binary Template
//
//      File: MIDI.bt
//   Authors: Jack Andersen
//   Version: 1.2
//   Purpose: General MIDI sound file template. Complete with 
//            delta-time and BPM evaluation into local variables. 
//  Category: Audio
// File Mask: *.mid
//  ID Bytes: 4D 54 68 64 //MThd
//   History: 
//   1.2r  2023-12-05 P Nerseth: Converted to REHEX format
//   1.2   2016-05-17 J Andersen: Correctly interpret unset status-bit as repeat message.
//   1.1   2016-02-12 SweetScape Software: Updated header for repository submission.
//   1.0   J Andersen: Initial release.
//------------------------------------------------

BigEndian();

struct MidiHeader
{
    char m_magic[4];
    uint32_t m_seclen;

    enum <int16_t> 
    {
        MIDI_SINGLE = 0,
        MIDI_MULTIPLE = 1,
        MIDI_PATTERN = 2
    } m_format;

    int16_t m_ntracks;
    int16_t m_tickdiv;
};

uint32_t read_DeltaTime()
{
    local uint32_t total = 0;
    local char byte = 0;
    local uint32_t start_pos = FTell();
    local uint32_t end_pos = FTell();

    byte = ReadU8();
    FSkip(1);

    total |= byte & 0x7f;

    for(local int i = 0; i < 3; i++)
    {
        if((byte & 0x80) != 0)
        {
            byte = ReadU8();
            FSkip(1);
            end_pos = FTell();

            total <<= 7;
            total |= byte & 0x7f;
        }
    }

    SetComment(end_pos, end_pos-start_pos, SPrintf("Deltatime: %d", total));
    return total;
}

local char lastStatus = 0;

struct MidiMessage
{
    local uint32_t m_dtime = read_DeltaTime();

    char m_status;
    if (m_status & 0x80)
        lastStatus = m_status;
    else
        FSeek(FTell()-1);

    local char m_channel = lastStatus & 0xf;
    if ((lastStatus & 0xf0) == 0x80)
    {
        struct 
        {
            char m_note;
            char m_velocity;
        } note_off_event;
    }
    else if ((lastStatus & 0xf0) == 0x90) 
    {
        struct 
        {
            char m_note;
            char m_velocity;
        } note_on_event;
    }
    else if ((lastStatus & 0xf0) == 0xA0) 
    {
        struct 
        {
            char m_note;
            char m_pressure;
        } note_pressure_event;
    }
    else if ((lastStatus & 0xf0) == 0xB0) 
    {
        struct 
        {
            char m_controller;
            char m_value;
        } controller_event;
    }
    else if ((lastStatus & 0xf0) == 0xC0) 
    {
        struct 
        {
            char m_program;
        } program_event;
    }
    else if ((lastStatus & 0xf0) == 0xD0) 
    {
        struct 
        {
            char m_pressure;
        } channel_pressure_event;
    }
    else if ((lastStatus & 0xf0) == 0xE0) 
    {
        struct 
        {
            char m_lsb;
            char m_msb;
        } pitch_bend_event;
    }
    else if (lastStatus == -1) 
    {
        struct 
        {
            enum <char> 
            {
                META_SEQUENCE_NUM = 0,
                META_TEXT = 1,
                META_COPYRIGHT = 2,
                META_SEQUENCE_NAME = 3,
                META_INSTRUMENT_NAME = 4,
                META_LYRIC = 5,
                META_MARKER = 6,
                META_CUE_POINT = 7,
                META_PROGRAM_NAME = 8,
                META_DEVICE_NAME = 9,
                META_MIDI_CHANNEL_PREFIX = 0x20,
                META_MIDI_PORT = 0x21,
                META_END_OF_TRACK = 0x2f,
                META_TEMPO = 0x51,
                META_SMPTE_OFFSET = 0x54,
                META_TIME_SIGNATURE = 0x58,
                META_KEY_SIGNATURE = 0x59,
                META_SEQUENCER_EVENT = 0x7f
            } m_type;

            local uint32_t m_length = read_DeltaTime();

            if (m_type == META_SEQUENCE_NUM)
            {
                int16_t m_seqNum;
            }
            else if (m_type == META_TEXT)
            {
                char m_text[m_length];
            }
            else if (m_type == META_COPYRIGHT)
            {
                char m_copyright[m_length];
            }
            else if (m_type == META_SEQUENCE_NAME)
            {
                char m_name[m_length];
            }
            else if (m_type == META_INSTRUMENT_NAME)
            {
                char m_name[m_length];
            }
            else if (m_type == META_LYRIC)
            {
                char m_lyric[m_length];
            }
            else if (m_type == META_MARKER)
            {
                char m_marker[m_length];
            }
            else if (m_type == META_CUE_POINT)
            {
                char m_cuePoint[m_length];
            }
            else if (m_type == META_PROGRAM_NAME)
            {
                char m_programName[m_length];
            }
            else if (m_type == META_DEVICE_NAME)
            {
                char m_deviceName[m_length];
            }
            else if (m_type == META_MIDI_CHANNEL_PREFIX)
            {
                char m_channelPrefix;
            }
            else if (m_type == META_MIDI_PORT)
            {
                char m_port;
            }
            else if (m_type == META_END_OF_TRACK)
            {
            }
            else if (m_type == META_TEMPO)
            {
                // Combine three bytes into a 24-bit unsigned integer
                uint8_t bytes[3];
                local uint32_t m_usecPerQuarterNote = (bytes[0] << 16) | (bytes[1] << 8) | bytes[2];
                local uint32_t m_bpm = 60000000 / m_usecPerQuarterNote;
            }
            else if (m_type == META_SMPTE_OFFSET)
            {
                char m_hours;
                char m_mins;
                char m_secs;
                char m_fps;
                char m_fracFrames;
            }
            else if (m_type == META_TIME_SIGNATURE)
            {
                char m_numerator;
                char m_denominator;
                char m_clocksPerClick;
                char m_32ndPer4th;
            }
            else if (m_type == META_KEY_SIGNATURE)
            {
                char m_flatsSharps;
                char m_majorMinor;
            }
            else
            {
                char m_data[m_length];
            }
        } meta_event;
    }
    else if ((lastStatus & 0xf0) == 0xF0) 
    {
        struct 
        {
            local uint32_t m_length = read_DeltaTime();
            char m_message[m_length];
        } sysex_event;
    }
};

struct MidiTrack
{
    char m_magic[4];
    uint32_t m_seclen;

    local uint32_t messages_end = FTell() + m_seclen;

    struct MidiMessage messages[0];

    // while(!FEof())
    while(FTell() < messages_end)
    {
        ArrayExtend(messages);
    }
};

struct 
{
    struct MidiHeader header;
    struct MidiTrack tracks[header.m_ntracks];
} file;
solemnwarning commented 7 months ago

Maybe you could have a repository for working templates?

I've thought about it a bit, but I don't think there's a demand for it right now. Would you be happy for me to include the above MIDI.bt script if/when it happens?

perivar commented 7 months ago

Yes thats fine

/Per Ivar

On Fri, Feb 9, 2024 at 5:13 PM Daniel Collins @.***> wrote:

Maybe you could have a repository for working templates?

I've thought about it a bit, but I don't think there's a demand for it right now. Would you be happy for me to include the above MIDI.bt script if/when it happens?

— Reply to this email directly, view it on GitHub https://github.com/solemnwarning/rehex/issues/220#issuecomment-1936203777, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHGCFH44WA5DIJ3TNUAJUTYSZDJDAVCNFSM6AAAAABAE3RMOSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZWGIYDGNZXG4 . You are receiving this because you authored the thread.Message ID: @.***>