Add support for MIDI SDS and TM-1

dagargo commented 2 years ago

The goal is to transmit samples to Machinedrum via TM-1.

TM-1 is a standard USB to MIDI (DIN5) interface that can also speed up MIDI transmission if the Elektron devices connected to them allow that.

As it seems that the underlying protocol is standard MIDI SDS, perhaps we could extend this functionality to any other device that supports it too.

Here we have some useful links taken from here.

Format: http://elektron-users.com/index.php?option=com_openwiki&Itemid=43&id=waveforms_samples 94

Batch conversion: http://www.elektronauts.com/t/best-software-for-batch-sample-rate-conversion/510

Discussion: http://www.elektron-users.com/index.php?option=com_fireboard&Itemid=28&func=view&id=30234&catid=9&limit=10&limitstart=0 27

dagargo commented 2 years ago

More official information at Elektronauts but not related to samples.

dagargo commented 2 years ago

Working MIDI SDS pushed to branch sds.

As I don't have a TM-1, I suggest you to work on the connector_handshake_midi_sds function in here.

https://github.com/dagargo/elektroid/blob/bdddae89b1671df2b49c1778ae33590ed9051468/src/connector.c#L2506

dagargo commented 2 years ago

SDS support completed.

dagargo commented 2 years ago

sds branch merged into master and deleted.

dagargo commented 2 years ago

A lot of refactoring has been going on lately so the instructions above are obsolete.

In order to send a custom SysEx message to the TM-1, the code should be added at the end of this function. Other devices will ignore it so it will be safe.

https://github.com/dagargo/elektroid/blob/aff3aab084b89a5dc9492560b7e4d5aee4f8ce45/src/connectors/sds.c#L763-L785

Notice that we are assuming that TM-1 has only one MIDI interface and it's reading and bypassing the messages.

The code should look like this.

static const guint8 SDS_TM1_TURBO_ON[] = { 0xf0,..., 0xf7 };
[...]
backend_tx_raw (backend, SDS_TM1_TURBO_ON, sizeof(SDS_TM1_TURBO_ON));

erroreyes commented 2 years ago

I am going to try to test this out tomorrow (PST). I emphasize "try" because my TM-1 is really unstable in my Linux/Lenovo laptop because I have to do this plug/unplug/plug-while-holding-the-turbo-button dance to get it to stay recognized and connected by the machine/OS, and even then, I haven't been able to get a simple command like amidi -p hw:1,0 -s some_file.syx -i 20 to work with it, while it works fine with an equally as old Native Instruments Audio Kontrol 1 audio/MIDI interface.

PS, this is in response to your recent post on Elektronauts. I didn't want to reply there as I think a technical conversation may be more relevant here.

dagargo commented 2 years ago

PS, this is in response to your recent post on Elektronauts. I didn't want to reply there as I think a technical conversation may be more relevant here.

Abosulutely! Fully agree.

I forgot to mention that, for now, you should change the following line.

#define SDS_BITS 16

I think it'a a 12 there in your case. We should set a variable depending on the MIDI identity reply but we'll do it later.

dagargo commented 2 years ago

Any news on this?

I've just pushed a few commits that should improve the SDS functionality. It's been tested and works well with my Digitakt (only upload)) and my E-Mu ESI-2000 (except when the device doesn't send the last byte).

Now, there are 4 filesystems; for 8, 12, 14 and 16 bits transmissions.

I'm still trying to figure out a couple of issues but hopefully this won't affect you if the Machinedrum MIDI SDS code is the same it's in the Digitakt.

One of the issues is the Receive SysEx option in the menu, it's not working ATM.

erroreyes commented 2 years ago

Hey, sorry about the delay due to unforeseen personal things.

I meant to test this exclusively with my TM-1, but with my current Linux setup, the TM-1 just doesn't want to connect permanently. I will keep trying since I have in fact been able to connect it to the laptop well by doing a bunch of brute force connecting and disconnecting maneuvers. That said, in the times that I have managed to keep it connected, Turbo mode hasn't worked.

With that in mind, do you need me, or is it possible and beneficial, to test the new changes to Elektriod on the Machinedrum without having to use the TM-1, i.e., can I use it with, say, my Native Instruments Komplete Audio 6 MKII?

dagargo commented 2 years ago

Hey, sorry about the delay due to unforeseen personal things.

No worries. First things first.

With that in mind, do you need me, or is it possible and beneficial, to test the new changes to Elektriod on the Machinedrum without having to use the TM-1, i.e., can I use it with, say, my Native Instruments Komplete Audio 6 MKII?

Testing the TM-1 should be the last thing to do. Right now, I need you because I'd like you to test the SDS against the Machinedrum and the integration in Elektroid and using Komplete Audio 6 MKII is perfectly fine. Later, we'll see how to integrate the TM-1 and its turbo mode.

As a side note, Elektroid offers direct sample copying with the GUI (and CLI) but also SysEx sending via the app menu (soon via the CLI too). The latter can be used to send the SysEx files found in these sample packs. BTW, SysEx sending only allows to send a single file but I'm working on sending multiple files at once.

Regarding a better integration with the Machinedrum, could you post here the output of elektroid -vv when selecting it in the device combo? I'd like to add a few tweaks if possible.

TIA.

dagargo commented 2 years ago

FYI, the bugs I mentioned a few days ago are fixed. Also, multiple SysEx uploading is now possible as well as using the CLI to send and receive SysEx.

dagargo commented 2 years ago

A couple of bugs have been fixed.

Also, an occasional error with an E-Mu ESI-2000 when downloading has been addressed with a little hack. Apparently, the sampler was not transmitting the last packet if it had only a single sample which cancelled the whole download. Now, Elektroid uploads an additional 0 sample if this is the case and then downloading just works. This affects all MIDI SDS devices but a single sample in this scenario is not much.

Edit: The additional sample was removed in b5a320f7f50c343831e78735adbc45d75737255d as another workaround was found.

dagargo commented 2 years ago

Any update on this?

dagargo commented 1 year ago

Version 2. 2 is almost ready. I'm planning to release it during the following couple of weeks.

For anyone reading this, have you been able to use the SDS connector in Elektroid with the Machinedrum or other SDS supporting sampler?

dagargo commented 1 year ago

Has anyone been able to test SDS or the TM-1 with Elektroid?

dagargo commented 1 year ago

In case anyone is interested, a few improvements have been added to the SDS code.

WAIT packages are sent now to better control the device and avoid the 20 ms timeout specified in the protocol that caused devices to continue without further handshakings.

Besides, a couple of retry loops have been added to better recover in case of failure.

erroreyes commented 8 months ago

I finally got a stable Linux + TM-1 setup. Previously, I couldn't get the TM-1 to stay connect for longer than 30 seconds no matter what I would do. I can properly look into this now that everything seems almost fine. Anyway, to start, a question:

I don't seem to see any support for SDS transfers using the UI version. Does Elektroid only support SDS transfers via cli?

dagargo commented 8 months ago

Does Elektroid only support SDS transfers via cli?

No, it does support SDS transfers from the GUI too.

Maybe, your problem is in the detection of the SDS device. Elektroid (GUI) tries its best to detect the appropriate device so the available operations match its capabilities. As the CLI forces a filesystem, the automatic detection is never fully triggered. Probably the elektron connector, which is run before the sds connector, is detecting the device erroneously.

In order to know what's actually going on, please, disconnect all your MIDI devices, run elektroid -vvv and post here the output so we can take a proper look.

This is how the handshake works.

In the file connector.c there is a list of the available connectors together with a regular expression and a function to initialize the filesystems and the operations.

https://github.com/dagargo/elektroid/blob/9f46c3b42ea9dcee18c478e83e4f38525eca530e/src/connector.c#L59-L63

All the available connectors are tested for a handshake if the MIDI device name matches the regex. As soon as a handshake is successful, the process finishes. Note that the SDS connector do not have a regex and it's placed at the next-to-last place in the list. The last place is used for a generic MIDI device that only supports SysEx and preset listing.

The goal of the handshake function detect the device and initialize all the needed structures in the backend struct. For the SDS devices, here is the function, but you could take a look at other handshake functions for an easier example.

https://github.com/dagargo/elektroid/blob/9f46c3b42ea9dcee18c478e83e4f38525eca530e/src/connectors/sds.c#L1209

What device are you trying to connect to? A Machinedrum?

It's also possible to create a new connector, create a new handshake function, register that on the connector.c file and redeclare this structure to have a more precise implementation. For instance, if the Machinedrum only accepts 12 bits 44.1 kHz mono samples, then one could specify exactly the operations they want. Or, perhaps, we could modify the elektron connector and its handshake function.

erroreyes commented 8 months ago

What device are you trying to connect to? A Machinedrum?

Yes, I am doing the test with a Machinedrum and a TM-1. I have a Monomachine as well, but I'll stick to the MD to make testing simple for now.

Assuming that I should disconnect all MIDI instruments but leave the TM-1 connected via USB, this is what elektroid -vvv displayed when I selected the TM-1 from the dropdown menu:

DEBUG:preferences.c:132:(preferences_load): Loading preferences from '/home/roby/.config/elektroid/preferences.json'...
DEBUG:audio.c:231:(audio_init): Initializing audio (PulseAudio 16.1.0)...
DEBUG:editor.c:645:(editor_stop_load_thread): Stopping load thread...
DEBUG:audio_pa.c:325:(audio_server_info_callback): Using 48000 Hz sample rate...
DEBUG:audio.c:274:(audio_reset_sample): Resetting sample...
DEBUG:backend_alsa.c:363:(backend_get_system_subdevices): Adding hw:1 (name 'Elektron TM-1', subname 'Elektron TM-1 MIDI 1')...
DEBUG:elektroid.c:247:(elektroid_load_devices): Loading devices...
DEBUG:elektroid.c:267:(elektroid_load_devices): Selecting device -1...
DEBUG:audio_pa.c:290:(audio_connect_playback_stream_callback): Sink index: 1623
DEBUG:audio.c:61:(audio_write_to_output): Writing 256 frames...
DEBUG:audio_pa.c:305:(audio_connect_record_stream_callback): Sink index: 1624
DEBUG:audio_pa.c:252:(audio_set_sink_volume): Setting volume to 1.000000...
DEBUG:audio_pa.c:252:(audio_set_sink_volume): Setting volume to 1.000000...
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 3044, loop end at 3044
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 6074, loop end at 6074
DEBUG:backend_alsa.c:363:(backend_get_system_subdevices): Adding hw:1 (name 'Elektron TM-1', subname 'Elektron TM-1 MIDI 1')...
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 14361, loop end at 14361
DEBUG:elektroid.c:247:(elektroid_load_devices): Loading devices...
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 28722, loop end at 28722
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 54524, loop end at 54524
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 6280, loop end at 6280
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 6847, loop end at 6847
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 28596, loop end at 28596
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 28631, loop end at 28631
DEBUG:elektroid.c:267:(elektroid_load_devices): Selecting device -1...
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 4424, loop end at 4424
DEBUG:editor.c:680:(editor_set_volume_callback_bg): Setting volume to 1.000000...
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 5675, loop end at 5675
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 6331, loop end at 6331
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 3828, loop end at 3828
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 28869, loop end at 28869
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 5251, loop end at 5251
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 28762, loop end at 28762
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 6591, loop end at 6591
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 6271, loop end at 6271
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 7164, loop end at 7164
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 28707, loop end at 28707
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 54858, loop end at 54858
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 27777, loop end at 27777
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 4341, loop end at 4341
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 3674, loop end at 3674
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 48067, loop end at 48067
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 2950, loop end at 2950
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 7059, loop end at 7059
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 14450, loop end at 14450
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 6858, loop end at 6858
DEBUG:editor.c:680:(editor_set_volume_callback_bg): Setting volume to 1.000000...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:notifier.c:32:(notifier_set_dir): Changing local browser path to '/home/roby/Music/Samples/drums/Roland Tr-707'...
DEBUG:notifier.c:140:(notifier_set_active): Starting local notifier...
DEBUG:notifier.c:66:(notifier_run): local notifier running...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:elektroid.c:2352:(elektroid_fill_fs_combo_bg): Selecting first filesystem...
DEBUG:browser.c:620:(browser_load_dir): Browser already loading. Skipping load...
DEBUG:notifier.c:32:(notifier_set_dir): Changing remote browser path to '/'...
DEBUG:notifier.c:140:(notifier_set_active): Starting remote notifier...
DEBUG:notifier.c:66:(notifier_run): remote notifier running...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:backend.c:389:(backend_destroy): Destroying backend...
DEBUG:progress.c:140:(elektroid_new_progress_thread_gsourcefunc): Creating SysEx thread...
DEBUG:backend.c:367:(backend_init): Initializing backend (ALSA) to 'hw:1,0,0'...
DEBUG:backend_alsa.c:85:(backend_init_int): Setting blocking mode...
DEBUG:backend.c:685:(backend_rx_drain): Draining buffers...
DEBUG:editor.c:329:(editor_draw_waveform): Drawing waveform from 0 with 0x zoom...
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:backend.c:378:(backend_init): Stopping device...
DEBUG:connector.c:132:(connector_init_backend): Connector elektron matches the device
DEBUG:connector.c:183:(connector_init_backend): Testing elektron connector...
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (13): f0 00 20 3c 10 00 00 00 00 00 00 01 f7
DEBUG:connectors/elektron.c:751:(elektron_tx): Message sent (5): 00 00 00 00 01
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:connector.c:183:(connector_init_backend): Testing microbrute connector...
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (6): f0 7e 7f 06 01 f7
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:backend.c:116:(backend_midi_handshake): No MIDI identity reply
DEBUG:connector.c:183:(connector_init_backend): Testing phatty connector...
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (6): f0 7e 7f 06 01 f7
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:backend.c:116:(backend_midi_handshake): No MIDI identity reply
DEBUG:connector.c:183:(connector_init_backend): Testing summit connector...
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (6): f0 7e 7f 06 01 f7
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:backend.c:116:(backend_midi_handshake): No MIDI identity reply
DEBUG:connector.c:183:(connector_init_backend): Testing cz connector...
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (10): f0 44 00 00 70 10 60 70 31 f7
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:connector.c:183:(connector_init_backend): Testing sds connector...
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (6): f0 7e 00 7d 00 f7
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (13): f0 00 20 3c 10 00 00 00 00 00 00 01 f7
DEBUG:connectors/elektron.c:751:(elektron_tx): Message sent (5): 00 00 00 00 01
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:backend.c:685:(backend_rx_drain): Draining buffers...
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (21): f0 7e 00 01 68 07 00 00 00 00 00 00 00 00 00 00 00 00 00 00 f7
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (2000)
DEBUG:connector.c:183:(connector_init_backend): Testing efactor connector...
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (6): f0 7e 7f 06 01 f7
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (5000)
DEBUG:connector.c:183:(connector_init_backend): Testing default connector...
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (6): f0 7e 7f 06 01 f7
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:backend.c:116:(backend_midi_handshake): No MIDI identity reply
DEBUG:connector.c:192:(connector_init_backend): Using default connector...
DEBUG:progress.c:60:(progress_stop_running_sysex): Stopping SysEx transfer...
DEBUG:progress.c:41:(progress_join_thread): Stopping SysEx thread...
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 3044, loop end at 3044
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 6074, loop end at 6074
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 14361, loop end at 14361
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 28722, loop end at 28722
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 54524, loop end at 54524
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 6280, loop end at 6280
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 6847, loop end at 6847
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 28596, loop end at 28596
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 28631, loop end at 28631
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 4424, loop end at 4424
DEBUG:browser.c:620:(browser_load_dir): Browser already loading. Skipping load...
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 5675, loop end at 5675
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 6331, loop end at 6331
DEBUG:browser.c:620:(browser_load_dir): Browser already loading. Skipping load...
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 3828, loop end at 3828
DEBUG:browser.c:620:(browser_load_dir): Browser already loading. Skipping load...
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 28869, loop end at 28869
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 5251, loop end at 5251
DEBUG:browser.c:620:(browser_load_dir): Browser already loading. Skipping load...
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 28762, loop end at 28762
DEBUG:browser.c:620:(browser_load_dir): Browser already loading. Skipping load...
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 6591, loop end at 6591
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 6271, loop end at 6271
DEBUG:browser.c:620:(browser_load_dir): Browser already loading. Skipping load...
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 7164, loop end at 7164
DEBUG:browser.c:620:(browser_load_dir): Browser already loading. Skipping load...
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 28707, loop end at 28707
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 54858, loop end at 54858
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 27777, loop end at 27777
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 4341, loop end at 4341
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 3674, loop end at 3674
DEBUG:elektroid.c:2352:(elektroid_fill_fs_combo_bg): Selecting first filesystem...
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 48067, loop end at 48067
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 2950, loop end at 2950
DEBUG:editor.c:645:(editor_stop_load_thread): Stopping load thread...
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 7059, loop end at 7059
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 14450, loop end at 14450
DEBUG:sample.c:492:(sample_set_sample_info): Loop start at 6858, loop end at 6858
DEBUG:notifier.c:32:(notifier_set_dir): Changing local browser path to '/home/roby/Music/Samples/drums/Roland Tr-707'...
DEBUG:audio.c:274:(audio_reset_sample): Resetting sample...
DEBUG:browser.c:620:(browser_load_dir): Browser already loading. Skipping load...
DEBUG:notifier.c:150:(notifier_set_active): Stopping remote notifier...
DEBUG:notifier.c:103:(notifier_run): Finishing notifier...
DEBUG:notifier.c:32:(notifier_set_dir): Changing local browser path to '/home/roby/Music/Samples/drums/Roland Tr-707'...

dagargo commented 8 months ago

This section is the interesting part. Let's break it in pieces.

Here, the elektron connector regex matches the system device name but the modern handshake doesn't work as it's quite an old machine.

DEBUG:connector.c:132:(connector_init_backend): Connector elektron matches the device
DEBUG:connector.c:183:(connector_init_backend): Testing elektron connector...
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (13): f0 00 20 3c 10 00 00 00 00 00 00 01 f7
DEBUG:connectors/elektron.c:751:(elektron_tx): Message sent (5): 00 00 00 00 01
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)

As this fails, other connectors are tested. But fail too.

DEBUG:connector.c:183:(connector_init_backend): Testing microbrute connector...
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (6): f0 7e 7f 06 01 f7
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:backend.c:116:(backend_midi_handshake): No MIDI identity reply
DEBUG:connector.c:183:(connector_init_backend): Testing phatty connector...
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (6): f0 7e 7f 06 01 f7
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:backend.c:116:(backend_midi_handshake): No MIDI identity reply
DEBUG:connector.c:183:(connector_init_backend): Testing summit connector...
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (6): f0 7e 7f 06 01 f7
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:backend.c:116:(backend_midi_handshake): No MIDI identity reply
DEBUG:connector.c:183:(connector_init_backend): Testing cz connector...
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (10): f0 44 00 00 70 10 60 70 31 f7
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)

Here, during the sds connector handshake, everything fails, including an additional check for Elektron devices supporting SDS.

DEBUG:connector.c:183:(connector_init_backend): Testing sds connector...
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (6): f0 7e 00 7d 00 f7
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (13): f0 00 20 3c 10 00 00 00 00 00 00 01 f7
DEBUG:connectors/elektron.c:751:(elektron_tx): Message sent (5): 00 00 00 00 01
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:backend.c:685:(backend_rx_drain): Draining buffers...
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (21): f0 7e 00 01 68 07 00 00 00 00 00 00 00 00 00 00 00 00 00 00 f7
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (2000)

Another connector here...

DEBUG:connector.c:183:(connector_init_backend): Testing efactor connector...
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (6): f0 7e 7f 06 01 f7
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (5000)

After all of these, the default connector fails to determine the device too but it's the one being selected.

DEBUG:connector.c:183:(connector_init_backend): Testing default connector...
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (6): f0 7e 7f 06 01 f7
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:backend.c:116:(backend_midi_handshake): No MIDI identity reply
DEBUG:connector.c:192:(connector_init_backend): Using default connector...

But there is something really weird here. There is no response for the Identity Request Universal SysEx message (f0 7e 7f 06 01 f7). It's all timeouts.

Can you use the sds connector from the CLI? Or have you just used the send SysEx functionality? I thought the detection of the Machinedrum was working at some point. Can you by any chance try to connect the Machinedrum without the TM-1?

By the way, in the right lower corner you can find the detected device. It's just the default.

erroreyes commented 8 months ago

Can you use the sds connector from the CLI?

I admit that I am confused about the command line implementation and I am not really sure what commands and options to issue in the terminal. If you don't mind, could you give me a few sample command lines to try out?

Or have you just used the send SysEx functionality?

I just sent a Sysex firmware file through the TM-1 and the Machinedrum is receiving it.

Can you by any chance try to connect the Machinedrum without the TM-1?

Unfortunately I cannot at the moment since the TM-1 is the only MIDI interface I have on hand at the moment.

I am curious though, what is the UI expected to look like for a TM-1 + Machinedrum setup, or for a Machinedrum connected to any other MIDI interface? What functionality should be made available in the UI once a connection is made? Is the expectation to send a Sysex dump of samples or SDS files? Can individual SDS files be transmitted?

dagargo commented 8 months ago

I just sent a Sysex firmware file through the TM-1 and the Machinedrum is receiving it.

Nice!

I admit that I am confused about the command line implementation and I am not really sure what commands and options to issue in the terminal. If you don't mind, could you give me a few sample command lines to try out?

Then let's try to send a sample from the CLI.

First, list the devices. From version 3.0, the device 0 is always the computer. In my case, device 1 is a MIDI to USB interface hooked up to my E-mu ESI-2000.

$ elektroid-cli ld
0: id: SYSTEM_ID; name: asuka
1: id: hw:1,0,0; name: hw:1,0,0: M-Audio MIDISPORT Uno, M-Audio MIDISPORT Uno MIDI 1

Let's try to guess what's in there by running the command info. It'll take a while as it will check all the connectors. In my case, the sds connector handshake succeeds and returns the initialized values. This won't work in your case as we already know something happens with the handshake, or will it?

$ elektroid-cli info 1
Type: MIDI
Device name: SDS sampler
Device version: 
Device description: 
Connector name: sds
Filesystems: program,16bits1c,14bits1c,12bits1c,8bits1c,44.1k16b1c,32k16b1c,16k16b1c,8k16b1c

Anyway, let's try to send a sample. All the commands have the same form: connector-fs-operation. In this case, we are using the sds connector, the 16 bits and sample rate as stored in the file, and the upload operation.

$ elektroid-cli -vv sds-16bits1c-ul sample.wav 1:/1:name

Does that work? Could you post the output of the last command here? Note that you might need to use other filesystem. Isn't the Machinedrum 12 bits? In fact, perhaps this is the issue.

I decided to implement the handshake for SDS devices by sending a dump header and listening for a reply. This header contains the bit depth of the sample to be sent.

This is the line.

https://github.com/dagargo/elektroid/blob/9f46c3b42ea9dcee18c478e83e4f38525eca530e/src/connectors/sds.c#L1235

Does it make any difference if you change that 16 with a 12 and recompile? Would the info command work then?

dagargo commented 8 months ago

I am curious though, what is the UI expected to look like for a TM-1 + Machinedrum setup, or for a Machinedrum connected to any other MIDI interface?

The GUI should look the same. Depending on the filesystem selected on the device side, on the computer side are shown only the appropriate files. In the case of SDS devices, you should see samples there. And in this case, the sampler editor will be shown too.

What functionality should be made available in the UI once a connection is made? Is the expectation to send a Sysex dump of samples or SDS files?

You'll just be able to drag and drop samples from left to right to certain slot. All the needed conversions will apply.

Can individual SDS files be transmitted?

You wouldn't work with SDS files but with actual samples.

If you wanted to send SDS files, which are just a bunch of SysEx messages, you'd need to use the menu. You could send several SysEx files at once. Just be reminded that you'd have no control over the slot they'd have been sent to. How a device will handle this depends on the particular case.

erroreyes commented 8 months ago

❯ elektroid-cli ld
0: id: SYSTEM_ID; name: rb-linux
1: id: hw:1,0,0; name: hw:1,0,0: Elektron TM-1, Elektron TM-1 MIDI 1

Calling info in the TM-1 returns a connector name of default, most likely after all the timeouts, and only displays program as file system, which explains why the UI only shows the "Program" option in the drop down menu.

❯ elektroid-cli info 1
Type: MIDI
Device name: MIDI device
Device version: 
Device description: 
Connector name: default
Filesystems: program

I am assuming that because the only file system available is program, sending a file fails; the sds-16bits1c-ul is not available:

❯ elektroid-cli -vv sds-16bits1c-ul ~/Music/Samples/sqr_wv.wav 1:/1:default
DEBUG:elektroid-cli.c:830:(main): Connector: "sds"; filesystem: "16bits1c"; operation: "ul"
DEBUG:backend_alsa.c:363:(backend_get_system_subdevices): Adding hw:1 (name 'Elektron TM-1', subname 'Elektron TM-1 MIDI 1')...
DEBUG:backend.c:367:(backend_init): Initializing backend (ALSA) to 'hw:1,0,0'...
DEBUG:backend_alsa.c:85:(backend_init_int): Setting blocking mode...
DEBUG:backend.c:685:(backend_rx_drain): Draining buffers...
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:backend.c:378:(backend_init): Stopping device...
DEBUG:connector.c:132:(connector_init_backend): Connector elektron matches the device
DEBUG:connector.c:171:(connector_init_backend): Testing sds connector...
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (6): f0 7e 00 7d 00 f7
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (13): f0 00 20 3c 10 00 00 00 00 00 00 01 f7
DEBUG:connectors/elektron.c:751:(elektron_tx): Message sent (5): 00 00 00 00 01
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:backend.c:685:(backend_rx_drain): Draining buffers...
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (21): f0 7e 00 01 68 07 00 00 00 00 00 00 00 00 00 00 00 00 00 00 f7
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (2000)
DEBUG:backend.c:389:(backend_destroy): Destroying backend...
ERROR:elektroid-cli.c:894:(main): Error: No such device

In the File Sample Manager, the Machinedrum waits for a sample, I send the commend from the terminal, the MD waits a bit, and then "cancels" the transmission (displays "CANCELLED" on the screen).

Isn't the Machinedrum 12 bits? In fact, perhaps this is the issue.

16 bit samples can be transferred to the Machinedrum and they are converted internally to 12-bit. I can send 16-bit WAV and SDS files with amidi and they get converted.

dagargo commented 8 months ago

Calling info in the TM-1 returns a connector name of default, most likely after all the timeouts, and only displays program as file system, which explains why the UI only shows the "Program" option in the drop down menu.

Exactly. That is by design. The GUI won't be able to do anything special but will allow selecting presets and sending and receiving SysEx.

I am assuming that because the only file system available is program, sending a file fails; the sds-16bits1c-ul is not available:

You're right.

However, notice the CLI bypasses the connector detection stage and uses the connector indicated by the user. What happens after is that the chosen connector runs the handshake anyway and it fails because no messages are received.

16 bit samples can be transferred to the Machinedrum and they are converted internally to 12-bit. I can send 16-bit WAV and SDS files with amidi and they get converted.

So 16 bits is OK.

In the File Sample Manager, the Machinedrum waits for a sample, I send the commend from the terminal, the MD waits a bit, and then "cancels" the transmission (displays "CANCELLED" on the screen).

Is the Machinedrum showing something on the screen during the connector detection or handshake?

I suspect the issue is just that no packages are received. Is the computer receiving MIDI data from the Machinedrum? You said that sending SysEx works, can you use amidi to check if it's receiving anything?

erroreyes commented 8 months ago

Is the Machinedrum showing something on the screen during the connector detection or handshake?

When it is in the file manager and I select the sample slot to receive a sample to, the only thing that shows is CANCELLED after the connector routine fails. When the Machinedrum is not in the file manager and its normal view, it displays the message "ERROR INVALID POSITION". That made me think that maybe the syntax for the path was wrong so I tried:

elektroid-cli -vv sds-16bits1c-ul ~/Music/Samples/sqr_wv.wav 1:/44:sqr

assuming that 1 is the device, 44 is the sample slot position in the Machinedrum where I want a new sample to be stored, and sqr is the name of I want to give to the sample, but that didn't work either.

can you use amidi to check if it's receiving anything?

I can receive data from the Machinedrum to the laptop with amidi.

dagargo commented 8 months ago

When it is in the file manager and I select the sample slot to receive a sample to, the only thing that shows is CANCELLED after the connector routine fails.

You don't need to setup anything on the SDS devices. The whole process is controlled from the computer.

I can receive data from the Machinedrum to the laptop with amidi.

Great.

When the Machinedrum is not in the file manager and its normal view, it displays the message "ERROR INVALID POSITION". That made me think that maybe the syntax for the path was wrong so I tried:

Then, the issue is in the handshake message and no matter what parameters you pass the handshake will be the same.

But you guessed out all the parameters correctly.

The issue is very likely in the line 1235. At least I added a meaningful comment.

Change the 1000 value for 44, which is the slot you were using but any other would do. Recompile and test the CLI again. Does this make any difference?

https://github.com/dagargo/elektroid/blob/9f46c3b42ea9dcee18c478e83e4f38525eca530e/src/connectors/sds.c#L1233-L1248

erroreyes commented 8 months ago

I started a fork.

After I changed the value to 44 and issued

elektroid-cli -vvv sds-16bits1c-ul audio.wav 1:/44:sqr

I now get an "ERROR INVALID SMPL RATE" error on the Machinedrum. The audio.wav file is in 16-bit, 44.1kHz. For comparison, I can convert that to SDS, transfer it via amidi, and it loads in the MD without issues.

dagargo commented 8 months ago

I now get an "ERROR INVALID SMPL RATE" error on the Machinedrum. The audio.wav file is in 16-bit, 44.1kHz. For comparison, I can convert that to SDS, transfer it via amidi, and it loads in the MD without issues.

We're getting closer.

Let's keep in mind that the SDS connector has only been tested on my ESI-2000 and probably there will be bugs. Also, this was my attempt at detecting the SDS feature and might not work with other devices and might need to be redesigned.

The issue is now on the sample rate, which is not initialized. Same line as before.

https://github.com/dagargo/elektroid/blob/9f46c3b42ea9dcee18c478e83e4f38525eca530e/src/connectors/sds.c#L1235

This creates a mock message to test if the device responds as an SDS sampler would.

The 3rd parameter is NULL, so it doesn't initialize properly the message, including the sample rate.

https://github.com/dagargo/elektroid/blob/9f46c3b42ea9dcee18c478e83e4f38525eca530e/src/connectors/sds.c#L258-L285

Try passing an initialized struct sample_info * value.

erroreyes commented 8 months ago

I think I created the struct correctly (I'm not a C/C++ person):

struct sample_info *si;
si->rate = 44100;
si->channels = 1;
tx_msg = sds_get_dump_msg (44, 0, si, 16);

When I send the command line, I don't get the error message. I think we are on to something like you said

❯ elektroid-cli -vvv sds-16bits1c-ul audio.wav 1:/44:sqr
DEBUG:elektroid-cli.c:830:(main): Connector: "sds"; filesystem: "16bits1c"; operation: "ul"
DEBUG:backend_alsa.c:363:(backend_get_system_subdevices): Adding hw:1 (name 'Elektron TM-1', subname 'Elektron TM-1 MIDI 1')...
DEBUG:backend.c:367:(backend_init): Initializing backend (ALSA) to 'hw:1,0,0'...
DEBUG:backend_alsa.c:85:(backend_init_int): Setting blocking mode...
DEBUG:backend.c:685:(backend_rx_drain): Draining buffers...
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:backend.c:378:(backend_init): Stopping device...
DEBUG:connector.c:132:(connector_init_backend): Connector elektron matches the device
DEBUG:connector.c:171:(connector_init_backend): Testing sds connector...
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (6): f0 7e 00 7d 00 f7
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (13): f0 00 20 3c 10 00 00 00 00 00 00 01 f7
DEBUG:connectors/elektron.c:751:(elektron_tx): Message sent (5): 00 00 00 00 01
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:backend.c:685:(backend_rx_drain): Draining buffers...
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
fish: Job 1, 'elektroid-cli -vvv sds-16bits1c…' terminated by signal SIGSEGV (Address boundary error)

Is Raw message sent (6): f0 7e 00 7d 00 f7 the Sysex init message for the ESI?

dagargo commented 8 months ago

You're using a pointer that has not been initialized, hence the SIGSEGV.

struct sample_info *si;
si->rate = 44100;
si->channels = 1;
tx_msg = sds_get_dump_msg (44, 0, si, 16);

You should have done it this way. (The memory is allocated in the stack and you can pass a pointer to it to other functions.)

struct sample_info si;
si.rate = 44100;
si.channels = 1;
tx_msg = sds_get_dump_msg (44, 0, &si, 16);

Anyway, I have pushed a commit to try to improve all this. Now, 3 different handshake methods are used and the original one is now the fallback, as it's very tricky. I can not tell if this will work on the Machinedrum.

In case it wouldn't, we could try to look for a SysEx request that the Machinedrum implements and use the reply to detect this. The Kit Sysex Dump commented here looks promising for this purpose. The https://github.com/jmamma/MCL repo might have something interesting too.

So, is the last commit fixing the Machinedrum detection?

dagargo commented 8 months ago

Oops... I forgot to answer this.

Is Raw message sent (6): f0 7e 00 7d 00 f7 the Sysex init message for the ESI?

That's the MIDI inquiry message. It''s the official way to ask what device is there. But not all devices implement this.

Elektroid tries this first because an official response would be the best way to ensure what's the actual device but even modern machines do no reply to this message. For instance, all the modern Elektron machines. And, as seen in the log, the TM-1/Machinedrum does not do this either. If the MIDI inquiry request fails, then the different connectors could try different approaches to identify the devices these work with.

This is the code of a simple handshake function that relies purely on the result of the MIDI inquiry request. (Internally, some fields are set so only some checks need to be done.)

https://github.com/dagargo/elektroid/blob/35ca06be3b04fe9b9ee5288866d546b79d1675d7/src/connectors/microbrute.c#L505-L527

The handshake functions in the sds connector is really tricky because it's quite difficult to guess if the device implements the SDS standard. That's what I tried to improve in the last commit when adding other checks in it.

erroreyes commented 7 months ago

Sorry for the gap in response. I need to make that fix to my branch/fork, but for now, I just git pull your latest changes, built on them, and ran elektroid-cli -vv sds-16bits1c-ul ~/Music/Samples/sqr_wv.wav 1:/1:default and this is the result I got:

DEBUG:elektroid-cli.c:829:(main): Connector: "sds"; filesystem: "16bits1c"; operation: "ul"
DEBUG:backend_alsa.c:363:(backend_get_system_subdevices): Adding hw:1 (name 'Elektron TM-1', subname 'Elektron TM-1 MIDI 1')...
DEBUG:backend.c:367:(backend_init): Initializing backend (ALSA) to 'hw:1,0,0'...
DEBUG:backend_alsa.c:85:(backend_init_int): Setting blocking mode...
DEBUG:backend.c:685:(backend_rx_drain): Draining buffers...
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:backend.c:378:(backend_init): Stopping device...
DEBUG:connector.c:132:(connector_init_backend): Connector elektron matches the device
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (6): f0 7e 7f 06 01 f7
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:backend.c:116:(backend_midi_handshake): No MIDI identity reply
DEBUG:connector.c:173:(connector_init_backend): Testing sds connector...
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (6): f0 7e 00 7d 00 f7
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (13): f0 00 20 3c 10 00 00 00 00 00 00 01 f7
DEBUG:connectors/elektron.c:751:(elektron_tx): Message sent (5): 00 00 00 00 01
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:backend.c:685:(backend_rx_drain): Draining buffers...
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1000)
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (8): f0 7e 00 05 04 01 00 f7
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1500)
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (10): f0 7e 00 05 02 01 00 01 00 f7
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (1500)
DEBUG:backend_alsa.c:205:(backend_tx_sysex_internal): Raw message sent (21): f0 7e 00 01 68 07 00 00 00 00 00 00 00 00 00 00 00 00 00 00 f7
DEBUG:backend.c:454:(backend_rx_raw_loop): Timeout (2000)
DEBUG:backend.c:389:(backend_destroy): Destroying backend...
ERROR:elektroid-cli.c:893:(main): Error: No such device

where ERROR:elektroid-cli.c:893:(main): Error: No such device is a new error message.

I still get the "CANCELLED" message when I put the MD to receive a sample and issue that elektroid-cli command send a sample.

dagargo commented 7 months ago

I still get the "CANCELLED" message when I put the MD to receive a sample and issue that elektroid-cli command send a sample.

We don't need to do that. Sample transmission is always started by Elektroid. We had to check if the sent messages were breaking the communication and so it was but it is a requirement that the receiving sample state is not activated during Elektroid handshake process.

So... We haven't fixed anything. Let's try something different.

As the problem is detecting the MD, we just need a message that identifies this particular machine and the best way to test different messages is with the amidi command.

This is an example of a preset download with an Arturia MicroFreak.

$ amidi -l
Dir Device    Name
IO  hw:0,0,0  Arturia MicroFreak Arturia Micr

$ amidi -p hw:0,0,0 -S "f0 00 20 6b 07 01 00 03 19 00 00 00 f7" -d -t 3
F0 00 20 6B 07 01 00 23 52 00 00 00 00 00 00 00 00 00 00 02 10 4E 65 72 76 6F 75 73 4B 65 79 73 00 00 00 00 00 00 00 00 00 00 00 00 F7
45 bytes read

If there is a response and it matches the expected data, then there is a MicroFreak. What we need to do is finding a message that fulfills these requirements.

From the official MD manual from Polynominal, appendix C.

Conventions:
Data printed with a $ sign is written in hexadecimal format.
Data printed with a % sign is written in a binary bitfield
format.
--------------------------
Machinedrum SYSEX messages
--------------------------
All SYSEX messages start with this sequence.
MIDI Byte | Purpose
------------+----------------------------
$f0 | SYSEX Start
$00 | Europe/USA ID
$20 | Europe ID
$3c | Elektron ESI ID
$02 | Machinedrum SPS-1 ID
$00 | Base channel (Padding)
A complete SYSEX message looks like this:
$f0,$00,$20,$3c,$02,$00,command,...,$f7

SYSEX global setting dump request:
MIDI Byte | Purpose
------------+----------------------------
(SYSEX init)|
$51 | Global setting dump request ID
%0000aaaa | Send global setting %aaaa (0 to 7) by sysex
$f7 | SYSEX end

Does the last SysEx return something? You should test this with a command like this.

$ amidi -p hw:0,0,0 -S "f0 00 20 3c 02 00 51 00 f7" -d -t 3

erroreyes commented 7 months ago

Here you go:

❯ amidi -p hw:1,0,0 -S "f0 00 20 3c 02 00 51 00 f7" -d -t 3

F0 00 20 3C 04 00 02 01 F7
F0 00 20 3C 02 00 50 06 01 00 06 06 06 06 06 06 06 06 06 06 06 06 06 06 06 06 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 54 7F 00 7F 01 7F 02 03 54 7F 04 7F 05 7F 06 07 52 7F 08 7F 09 0A 7F 0B 52 7F 0C 7F 0D 0E 7F 0F 4A 7F 10 11 7F 12 7F 13 4A 7F 14 15 7F 16 7F 17 2A 18 7F 19 7F 1A 7F 1B 29 1C 7F 1D 7F 1E 1F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 60 7F 7F 7F 00 11 13 01 61 01 10 10 01 01 00 00 00 00 7F 7F 00 01 77 20 01 3B F7
206 bytes read

dagargo commented 7 months ago

The first message is interesting. Looks like it is the TM-1 responding based on the information from this Elektronauts post. By the way, there are lots of information about speed negotiation there.

Considering that we have the SysEx messages for globals, kits and other types of data, I'm pondering creating a new connector as it would be easier to add new functionalities there.

dagargo commented 7 months ago

A new branch called machinedrum includes the code to work with the MD samples. Both GUI and CLI should work but the connector has not been tested.

Anyway, this is a starting point.

A few notes:

The connector is called machinedrum.
The filesystem for samples is called sample.
All the internal functionality has been tested but the renaming operation.
Adding a filesystem for the global settings looks really easy. We could try to implement this if you want.
The sample amount limit and the sample name length can be tweaked from these constants.

https://github.com/dagargo/elektroid/blob/f323db2d18f82dff03e777eba94775a435477424/src/connectors/machinedrum.c#L25-L26

Could you post here the output of these commands?

$ elektroid-cli -vv info 1

$ elektroid-cli -vv machinedrum-sample-ls | head

So, is this working for you?

dagargo commented 7 months ago

I forgot to mention that getting the sample names could be possible.

As with the renaming functionality, this code is already implemented but has never been tested.

If you're interested check if this works.

$ amidi -p hw:1,0,0 -S "f0 7e 00 05 04 00 00 f7" -d -t 3

dagargo commented 6 months ago

@erroreyes, did you manage to try the machinedrum branch?

We are not in a hurry, it's just that wanted to track the status of this issue. Don't mean to put pressure on you.

If, for whatever reason, you can't work on this we might ask in Elektronauts if anyone else is interested.

johnnywagner commented 5 months ago

I've got 2 working TM-1's, a Machinedrum, Monomachine, Windows PC & Macbook and would be happy to help test this as C6 is getting harder for everyone to use. Especially for those on MacOS, there are no options for getting TurboMIDI speeds from the TM-1. I'm a bit of a novice programmer though & not familiar with autotools, is it possible to compile for Windows and Mac? Or would I need to set up a VM to run this?

dagargo commented 5 months ago

Thanks for wanting to try this.

First, a bit of a recap.

Basic sample transmission should already work with the MachineDrum. The code resides in its own branch.
Additional things like renaming are still in the works.
Turbo MIDI, from the application side, seems to work but it's still unknown if something else needs to be done to work in the turbo mode. There are some notes about all these in the previous comments.

Regarding the compilation, there are many options.

Last Tuesday I checked the Windows compilation under MSYS2 UCRT64 and added a few commits to address some minor issues. Now, seems to work nicely but the MSYS2 layer makes it very slow and compiling and running it's not really user friendly.
Elektroid should compile and work under OSX and autotools.
The development environment and where Elektroid has been fully tested is GNU/Linux. I could help you easily if we choice this path.

Perhaps, the best choice now is going for a Linux VM and, once it's done, then I could help you building this on OSX.

Is this good for you?

If it is, let's follow this.

Install a Debian or Ubuntu VM. (I've tried WSL on Windows and works quite well too. This is another option.)
Clone the repo and change to the machinedrum branch.
Install the dependencies indicated from the instructions.
Try running the compilation instructions.
Run Elektroid (you'll need to map the USB devices to the guest OS in your VM manager).

Let me know at which point you are so I can help you with that.

dagargo commented 5 months ago

There is the issue #108 related to building on OSX.

johnnywagner commented 5 months ago

Sounds great, thanks for all of the info. It may be a few days before you hear back from me as I have never set up a Linux VM or build environment before but am eager to figure it out. Is this the comment thread an appropriate place to reach out for troubleshooting/help or should we move to some kind of direct messaging (if needed)?

dagargo commented 5 months ago

Take the time you need. This issue was created almost 2 years ago so we're definitely not in a hurry.

As a team, we've done as much as we could but we haven't accomplished much yet. Still, I think we are really close.

Is this the comment thread an appropriate place to reach out for troubleshooting/help or should we move to some kind of direct messaging (if needed)?

I'd say so. If we end up cluttering this issue so much, we'll find another way.

dagargo commented 5 months ago

@johnnywagner, in case it makes things easier for you, support for OSX with Homebrew has been added.

johnnywagner commented 5 months ago

Great, sorry I haven't had much time to try and set this up. Thanks for the update, I'll post here soon.

dagargo commented 3 months ago

For anyone reading this, has there been any advance in this issue?

BTW, branch machinedrum is way behind master so I should probably reabase it.

johnnywagner commented 3 months ago

I haven't had a chance to test it yet but I haven't forgotten!

adsr commented 2 months ago

Hi, I'm a Linux user with a Machinedrum and can help test / debug. I misplaced my TM-1 a while ago but I will look for that.

I figured I'd try a non-turbo transfer on the machinedrum branch and that worked:

$ ./elektroid-cli machinedrum-sample-ul ~/downloads/untitled.wav 1:/34:34

This uploaded untitled.wav to slot 35 (34 0-indexed). The name appears as "....". machinedrum-sample-ls also worked, showing 0 thru 127.

This helps me as Elektron C6 crashes under wine. Thanks!

Btw I needed the following to compile the machinedrum branch on my setup.

diff --git a/src/connectors/system.c b/src/connectors/system.c
index d2e5c83..098ad13 100644
--- a/src/connectors/system.c
+++ b/src/connectors/system.c
@@ -19,6 +19,7 @@
  */

 #include <errno.h>
+#include <glib/gstdio.h>
 #include <glib/gi18n.h>
 #if defined(__linux__)
 #include <sys/statvfs.h>
diff --git a/src/editor.c b/src/editor.c
index 4e8c017..2811ac0 100644
--- a/src/editor.c
+++ b/src/editor.c
@@ -19,6 +19,7 @@
  */

 #include <gtk/gtk.h>
+#include <glib/gstdio.h>
 #include <glib/gi18n.h>
 #include "editor.h"
 #include "sample.h"

You mentioned you need to rebase so maybe that's already fixed.

If I find my TM-1 I will test with that and update.

dagargo commented 2 months ago

This is great news!

TM-1 is something we can deal with later as this will only speed up the transfers.

Regarding sample names, we need to know how the machine names the samples. My guess is that they're using an extension of MIDI SDS for this so 've enabled it in the code. Are the names there now? You should see the names in the GUI too. Consider this naming feature has never been tested so there might be errors. The specifications are here.

As you're saying the samples are 0-indexed, I've tried to fix it in the last commit.

Branch machinedrum has been rebased too. Those compilation errors were already fixed in master, just as you said.

Please, check everything again.

dagargo / elektroid

Add support for MIDI SDS and TM-1 #64