RPI?

[marek-memsql]

Is there any way how to run this in a headless mode so it could be preconfigured and started on a device like a Raspberry Pi? - the RPI would act like a bridge between the pioneer link and the ableton link so no computer is needed.....

thx!

[brunchboy]

That is an interesting idea! Just to confirm the details: You are looking for only a very specific subset of the features offered by beat-link-trigger, to have it set the tempo of an Ableton Link session based on what it is seeing on a Pioneer Pro DJ Link network.

There are really two parts to this question, and if you are willing to investigate the first half, then I can help with the second. That bridging is done by a combination of Carabiner (to talk to Link), and beat-link (to talk to the Pioneer gear). Carabiner is already a headless daemon, so if you can get it up and running on your Raspberry Pi, we are halfway there.

Could you try that? I don’t have the hardware to experiment on, so I can’t. (Can you even build and run C++ code on a Pi?)

Once that is working then I could help you write a similar daemon that does the Pioneer half. We would not use beat-link-trigger at all (since it is just an example of how to build a user interface to do cool stuff with the beat-link library); we would use beat-link directly and re-implement beat-link-trigger’s carabiner.clj as a headless daemon.

[marek-memsql]

Exactly! What I am thinking about is having a feature limited version of the beat-link-trigger + carabinier running on my rpi, same like it does on my mac. The idea is having the RPI connected via lan to the same network as my CDJ's. There would be the simplified version of the beat-link-trigger running, only getting the beat data from the linked players and providing the information to the carabinier running there as well, which would be taking care of the ableton link stuff. [Pro DJ Link -> RPI running {simplified beat-link-trigger -> Carabinier} <- any ableton link friendly app] On the RPI 3 or Zero W, I could also run the hostapd so the RPI would act as a wireless AP so virtually any device (like an iPad running a link enabled app) could connect to it and get the Ableton link data without needing any computer on the network..
Haven't tried anything yet but when thinking about it, this should be doable - sure we can compile the c++ as we have a regular gcc. I will try to compile && run the carabinier etc. in few days on and let you know...

[brunchboy]

That sounds great, I look forward to hearing what you discover. 👍

[brunchboy]

The information linked from this issue on the Ableton Link project might also be helpful, because part of compiling Carabiner is compiling Ableton’s open source Link implementation: https://github.com/Ableton/link/issues/7

[marek-memsql]

So I have compiled the gcc and cmake from source, but failing on the last step of the carabinier compilation... Maybe I'd need to change some flags or whatever... EDIT: Found that I'd need to add "-lpthread" as I am building on linux while this is not needed on osx - where is the right place to add this flag? cmake.txt

[brunchboy]

Unfortunately, I have not done any significant work in the C/C++ world for far too many years to be able to offer any help in figuring out things like this. The first time I even heard of cmake was when I tried writing Carabiner. Your best bet might be to try to comment on the Link building issue I linked to above, since the person who offered that solution may have some good ideas.

[marek-memsql]

@brunchboy

Got it... I had to add set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -pthread") to carabiner/CMakeLists.txt file. Now I am able to build && run the Carabinier :-)

pi@raspberrypi:~/carabiner/build $ cmake --build .
Scanning dependencies of target gflags_nothreads_static
[  9%] Building CXX object gflags/CMakeFiles/gflags_nothreads_static.dir/src/gflags.cc.o
[ 18%] Building CXX object gflags/CMakeFiles/gflags_nothreads_static.dir/src/gflags_reporting.cc.o
[ 27%] Building CXX object gflags/CMakeFiles/gflags_nothreads_static.dir/src/gflags_completions.cc.o
[ 36%] Linking CXX static library libgflags_nothreads.a
[ 36%] Built target gflags_nothreads_static
Scanning dependencies of target Carabiner
[ 45%] Building C object CMakeFiles/Carabiner.dir/mongoose.c.o
[ 54%] Building CXX object CMakeFiles/Carabiner.dir/carabiner.cpp.o
[ 63%] Linking CXX executable bin/Carabiner
[ 63%] Built target Carabiner
Scanning dependencies of target gflags_static
[ 72%] Building CXX object gflags/CMakeFiles/gflags_static.dir/src/gflags.cc.o
[ 81%] Building CXX object gflags/CMakeFiles/gflags_static.dir/src/gflags_reporting.cc.o
[ 90%] Building CXX object gflags/CMakeFiles/gflags_static.dir/src/gflags_completions.cc.o
[100%] Linking CXX static library libgflags.a
[100%] Built target gflags_static
pi@raspberrypi:~/carabiner/build $ ll
total 52
drwxr-xr-x 5 pi pi  4096 Mar 19 18:48 .
drwxr-xr-x 8 pi pi  4096 Mar 19 18:48 ..
-rw-r--r-- 1 pi pi 12880 Mar 19 18:48 CMakeCache.txt
drwxr-xr-x 6 pi pi  4096 Mar 19 18:51 CMakeFiles
-rw-r--r-- 1 pi pi  8632 Mar 19 18:48 Makefile
drwxr-xr-x 2 pi pi  4096 Mar 19 18:51 bin
-rw-r--r-- 1 pi pi  1515 Mar 19 18:48 cmake_install.cmake
drwxr-xr-x 4 pi pi  4096 Mar 19 18:51 gflags
pi@raspberrypi:~/carabiner/build $ cd bin
pi@raspberrypi:~/carabiner/build/bin $ ll
total 4620
drwxr-xr-x 2 pi pi    4096 Mar 19 18:51 .
drwxr-xr-x 5 pi pi    4096 Mar 19 18:48 ..
-rwxr-xr-x 1 pi pi 4721288 Mar 19 18:51 Carabiner
pi@raspberrypi:~/carabiner $ uname -a
Linux raspberrypi 4.4.38-v7+ #938 SMP Thu Dec 15 15:22:21 GMT 2016 armv7l GNU/Linux
pi@raspberrypi:~/carabiner/build/bin $ ./Carabiner
Starting Carabiner on port tcp://127.0.0.1:17000
Link bpm: 120 Peers: 0 Connections: 0

Ableton started on my laptop:

pi@raspberrypi:~/carabiner/build/bin $ ./Carabiner
Starting Carabiner on port tcp://127.0.0.1:17000
Link bpm: 130 Peers: 1 Connections: 0

I am also able to hook any Link enabled app on my iPad and it connects to the Carabinier!

pi@raspberrypi:~/carabiner/build/bin $ ./Carabiner
Starting Carabiner on port tcp://127.0.0.1:17000
Link bpm: 137 Peers: 1 Connections: 0

So all we need now is the simplified beat link trigger which would connect to the pioneer link, and will be sending the beat data from the master player to the Carabinier. It would me nice to have be able to bind the beat-link-trigger and the Carabinier to a specific network interface. (what is far behind my knowledge).

[brunchboy]

That’s great! I wonder if there is a way to have it automatically detect that so that a single source tree can build properly in both environments?

[brunchboy]

As far as the last paragraph, you don’t want to bind beat-link to a particular interface, it is part of its job to figure out what interface contains the Pioneer network, and communicate properly with it on that interface. And that daemon and Carabiner will want to communicate with each other on the loopback interface (127.0.0.1) since that is most efficient. I should be able to work up an example of how to build such a simple beat-link daemon for you shortly, perhaps even today.

[marek-memsql]

No idea.... This was my very first contact with the CMake :( Maybe gflags/CMakeLists.txt line 616 and further - there is some if/then based on the architecture so maybe there the decision could be made but really not sure...

Thanks for helping with the beat-link part - no hurry - I am leaving tomorrow for 2 weeks so probably won't be able to test it until I am back. I can take my rpi but won't have any pro link there to test... So in case there are multiple interfaces (like when the mentioned wlan acts as the access point), there is no need to care about the listening interface at all for both of the apps? If so, then it will be a really simple task... If this works in its simplest form and we have the proof of concept, I have few ideas what else could be done there.

[brunchboy]

All right, I can wait for you to return… what is your time zone, though? Is there any hope of getting it tested today if I can do it quickly? 😉

[brunchboy]

And yes, I believe you should not need to worry about the listening interface, because beat-link figures it out for the Pioneer network, and I am pretty sure Ableton Link works on all interfaces as well.

[marek-memsql]

I am CET - so I have got about 2 hours before going to the bed :)

[brunchboy]

That’s what I feared. Ok, I will work on it more sensibly, and you can give it a try with your actual players when you return.

[brunchboy]

I’m glad I decided to wait and think a bit before diving in to write this. I have started now, but I am working in a direction that will be more useful for headless operation than I had originally planned.

It is going to sit and wait for Pioneer gear to appear on any interface, and start working with that, until it disappears, and then go back to watching all interfaces again. Similarly, it will watch for Carabiner to be running on the port you configure. When both are present, it will synchronize the tempo and beats (and, unless you specify otherwise, the bars as well).

I have set up a skeleton project, beat-carabiner, and implemented the command-line argument parsing, as well as pulling in the necessary libraries. It will probably only be another couple of evenings work to get it basically operational for an initial release.

[brunchboy]

All right, I have a release of beat-carabiner working! Once you get home and have a chance to try it with your Pioneer gear, if it is working for you we can close this issue. In the mean time, if you did bring your Pi, you could try running it and see if it seems to be connecting properly to Carabiner and producing reasonable log output anyway.

[PouleR]

I like this idea! Maybe it's a good idea to make beat-link-trigger configurable via the CLI or a configuration file and don't hard link it to Carabiner?

For e.g. java -jar beat-link-trigger.jar --config=<config_file>

Where a configuration file contains all triggers with it's Expressions? Especially a RPI would be great since it's so small

[brunchboy]

That is a much larger project which I am unlikely to undertake, because Beat Link Trigger is written with its user interface bindings spread throughout the code. I misunderstood your use case, I thought you just wanted a way to bridge between Pioneer Pro DJ Link and Ableton Link without a user interface.

[brunchboy]

I think for such applications you are better off simply writing your own programs that interact with Beat Link, MIDI, OSC, and/or Carabiner directly, and implement your “triggers” in your own code.

[marek-memsql]

@brunchboy Cool! I will try if my vpn works so I could ssh to my rpi and try running it. How does the latency works? - can this be changed during the runtime or the process needs to be restarted? If so, is it a good idea to search for a player on the network while starting up the script, ping it, get the avg. response ms and then use the value as the latency value?

@PouleR What I am thinking about for the future is running some ngix (or maybe a simple python server) which would handle the configuration using some REST or whatever.. - you know this doesn't expect you need to do any changes and in fact the only parameter you need to mess with is the latency.. If both of the apps run ok on the rpi, I will write a services for both of them and demonize them while placing the config file in /boot so you can do changes simply by inserting the sdcard to your computer.

[marek-memsql]

@brunchboy Looks more than good to me.... :-D Sadly I can't test with my players atm...

pi@raspberrypi:~ $ java -jar beat-carabiner.jar
^Z
[1]+  Stopped                 java -jar beat-carabiner.jar
pi@raspberrypi:~ $ bg
[1]+ java -jar beat-carabiner.jar &

pi@raspberrypi:~/carabiner/build/bin $ ./Carabiner
Starting Carabiner on port tcp://127.0.0.1:17000
2017-Mar-21 20:36:28 INFO [beat-carabiner.core:187] - Will align Ableton Link session to bars and beats.
2017-Mar-21 20:36:31 INFO [beat-carabiner.core:207] - Waiting for Pro DJ Link devices...
2017-Mar-21 20:36:31 INFO [beat-carabiner.core:229] - Trying to connect to Carabiner daemon on port 17000 with latency 20
2017-Mar-21 20:36:31 INFO [beat-carabiner.carabiner:81] - Connected to Carabiner.
Link bpm: 120 Peers: 0 Connections: 1

[brunchboy]

@marek-memsql: Whoops, that will teach me to respond in a hurry in the morning before I really have time to pay attention! I thought @PouleR’s question came from you! In any case, that output is definitely looking promising as far as things can go without your players, so I look forward to what you find when you get home.

The latency cannot currently be reconfigured without killing and restarting the daemon, but that should be a fairly quick and harmless thing to do. Honestly I would be surprised if you need to adjust it at all, but I added that “knob” because Ableton is so careful about accounting for latency in their own protocol. The basic idea is that the CDJ reports a beat happening through a UDP packet. It takes some time for it to write that UDP packet, for it to be sent over the network, delivered to us, and for us to recognize it. I don’t think there is any way you can measure that latency using mechanical tools; a ping would measure round-trip time so perhaps you could divide that in half, but the result might still need some adjustment to account for processing time. But exactness doesn’t really matter; the point is to let you adjust it if it sounds like beats in the Link world are not happening at the same time as you hear beats coming out of your Pioneer players. I have never had to adjust it so far, but I wanted people to have the option if their network was very different than mine.

As for the bigger question from @PouleR, the fundamental problem is that beat-link-trigger as it is written relies on the user interface objects everywhere. When a packet is received from a player, the Triggers window is walked, looking at each row to find the triggers. Each trigger configuration is examined by looking at the actual menu, checkbox, and field values to determine if the packet will affect it, and so on. There would be hundreds of places in the code that would need to be changed if it was going to work in a headless mode where none of those user interface objects could exist. A significant rewrite that would take weeks or more.

That said, I have already been unable to resist thinking about how I would go about doing it, creating a more abstract structure to use instead of the UI objects (which could be based on the structure that gets saved to the preferences), updating that whenever the UI is changed, and only trying to provide graphical status information when there is a UI present. I can see ways to do it, but cannot predict if or when I would find the time to undertake such a major rewrite.

But if I did, once it was successful, it would also be possible to think about offering an embedded web interface as an option when running headless, so you could make configuration changes or check on the status of your triggers using your smart phone… See? I can’t help thinking about it! 😉

[marek-memsql]

@brunchboy AD the latency - I asked the question because I really need to do the latency adjustments when running the standard beat-link-trigger and Carabiner on my mac - the default value is too high that I am heavily off beat unless I lower the latency to 2-5ms. Anyway - kill and restart sounds fair. I will give it a try and see how that plays with the measured roundtrip and if its good to have it there or not.

Btw I like the idea of "make configuration changes or check on the status of your triggers using your smart phone" - thats exactly what I was thinking about when I mentioned the rest api & simple set/get.

[Kristey]

Hello guys.

Nice "add-on"for beat link trigger, but I have a question. This just sync Bar_cdj with Bar_ableton, right? (works fine in my setup congrats ) but... How hard make a trigger to Start Play (on ableton) and a perfect match between 1st cdj bar and ableton 1st bar, 2st cdj bar and ableton 2st bar, (...) 25st cdj bar and ableton 25st bar,.... etc ? The Beat link trigger knows exactly which bar where the music is playing. If I made a jump(hot cue) or a loop it was perfect that the ableton accompanies the CDJ. Is it possible?

[marek-memsql]

@Kristey The ableton link doesn's align to the absolute position but only keeps the audio sources in sync.... If you use the hotcue on the cdj, the ableton link will sync the ableton so the beats are in sync - it won't jump to the 25st bar of the clip playing from the ableton to sync the position.

[brunchboy]

Thanks, @Kristey ! It took a huge amount of research (with contributions from several people) and a great amount of new code in Beat Link to get to this point, so I am glad to hear people are liking it. @marek-memsql is exactly right, the Ableton Link protocol is not designed to support that level of tight binding of musical timelines. But now that Beat Link Trigger can track CDJs at that level (thanks to the discovery of how to download the beat grids in Beat Link), I am planning to add support for synchronization via MIDI Time Code in the near future. You can use that to bind tracks together (video, audio, whatever). I think Ableton Live supports MTC? You’ll need to check the manual to be certain. But you may be disappointed with the results of trying to track loops, because even the Pioneer protocol is not designed to support that.

[Kristey]

@brunchboy I really love your work and research .THANK YOU it's little. About MTC: https://help.ableton.com/hc/en-us/articles/209071149-Synchronizing-Live-via-MIDI "If the master sequencer can generate MIDI Time Code (MTC), this would be the recommended option since MTC has a better resolution. However, as MIDI Time Code does not transmit any tempo information, the BPM will have to be set manually in Live to match the BPM of the master sequencer."

In this time I can't try if works, but in next week I will try this.

About LOOP: See this: https://youtu.be/wMQw_zZVKas I take a loop of 4 beat, and the BLT send de time correct. The timescode is the SIGNAL of time HH:MM:SS:FF, that way the timecode signal is generate with this information HH, MM, SS, e Frames. If you send 00:01:12:00 (for exemple), the signal of timecode send precisely that information. If num JUMP in music, the time information make a timecode with the new time.

Sorry my English is a little poor, you understand my point?

[brunchboy]

Hi, thanks, I think I understand your points. You won’t be able to try anything with MTC until a future release when I write new code in Beat Link Trigger to send it, but I hope there will be a way you can set both the BPM and track position in Live, perhaps using both Link and MTC.

The problem with loops is that during a loop, it is sometimes impossible to know the correct track time. In your example, where you were looping four beats, things work well. But the only time we know precisely where in the track a player has reached is immediately after the player sends us a beat packet, because we can then look that up in the beat grid table, and translate it to a precise track position in milliseconds. In between beats, we can only interpolate the time, which is a fancy mathematical way of saying “guess.” 😄 We look at how much time has passed since the last beat packet, and we look at the player’s reported pitch (speed) in its status packets, and calculate what track time it has reached. But if it loops in the middle of a beat, or the DJ does a needle jump to the middle of a beat, our calculation of the track time will be incorrect until the player plays the next beat and sends us the corresponding beat packet. If it is playing in reverse, it sends no beat packets at all, so our idea of the time is even worse.

There is nothing we can do about that, because the DJ Link protocol was designed to allow beat matching during normal forward playback, not the kind of precise sub-beat synchronization we might try to do.

[Kristey]

Ok I understand your point about looping. But this is for especial use, the artist previous know this. I talk about loop for talk with audience or something like this.

When I said about try if MTC works in ableton I was thinking in try with a program outside ableton with MTC generator. After this I told you if works. Thank you for your help

[brunchboy]

Very good, I am glad that loop tracking seems like it might work for your special purposes, and I look forward to hearing what you discover with MTC. As an experiment, I tried forcing Ableton to jump back four beats using Link by forcing the Carabiner timeline back four beats, and as Marek and I expected, that had no effect. So MTC will be required, and some other mechanism for keeping the tempo synced. Could we move this discussion to the gitter chat, though, since it doesn’t have much to do with the Issue Marek opened?

[frankbash]

Hey guys is there a ready build "carabiner" for raspberry pi ???

[brunchboy]

Not that I am aware of. If you would like to build and host one, I would be happy to link to it.

[frankbash]

Ah i am sorry my skill in programming are extraordinary low :(

[brunchboy]

No worries, perhaps someone else who has access to a Raspberry Pi can see this and help. I don’t think it will require any programming, simply a knowledge of the build and packaging tools used on that platform, so more of a system-administrator focus.

Cheers!

[brunchboy]

I think I remember someone talking about this on the Gitter chat so I’ve asked there if anyone has a build they can share.

[marek-memsql]

I think I might have it somewhere - will try to find the sd card... The problem is that it simply doesn't work well - it keeps the bpm in sync nicely, yet I always struggled to get the cdj's and ableton in sync as they were always off-beat. I didn't make any measurements etc. but I guess that the java packages simply do not work well on rpi, the timing is simply off and it wasn't useable. We might get better results with some non-java version of beat-carabiner which would be compiled for arm and there will be no latency caused by the jvm..

[marek-memsql]

....also found my 2 years old notes on how to compile on rpi... not sure if these steps are still valid but someone might decipher and find them somehow helpful...

`GCC:

https://bitbucket.org/sol_prog/raspberry-pi-gcc-6.1.0-binary/overview

CMAKE https://osdevlab.blogspot.cz/2015/12/how-to-install-latest-cmake-for.html

sudo apt-get install p7zip-full git git clone https://bitbucket.org/sol_prog/raspberry-pi-gcc-6.1.0-binary.git 7z x gcc-6.1.0.tar.7z tar xvf gcc-6.1.0.tar gcc-6.1.0/ sudo cp -r gcc-6.1.0 /usr/local/ wget https://cmake.org/files/v3.8/cmake-3.8.0-rc2.tar.gz tar -xvzf cmake-3.8.0-rc2.tar.gz sudo ./bootstrap

git clone https://github.com/brunchboy/carabiner.git cd carabinier

git submodule update --init --recursive

EDIT: carabiner/CMakeLists.txt ADD: set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -pthread”)

openjdk-8-jre-headless`

[brunchboy]

Interesting, thanks so much! I hope someone can investigate this further. As long as the JDK and Carabiner are using the same system monotonic clock, latencies in Java should not matter much in terms of converging on a good sync. OpenJDK 8 is ancient now (and not supported by Beat Link Trigger any longer), so it might be worth trying with OpenJDK 12. Also the new Raspberry Pi 4 has more resources and may be able to run a full Java/Clojure environment well. I may have to pick one up to experiment with myself and see if I can’t offer official packages someday.

[brunchboy]

There have been a lot of fixes in Beat Link and Beat Link Trigger since then too. And a lot more capabilities, like letting Ableton Link be tempo master for the CDJs. That would require updating beat-carabiner, and maybe using some of the GPIO pins on the Pi to give you buttons and LEDs as a UI. Although rather than updating beat-carabiner, I would get more long-term benefit from working on my project of separating the UI from the engine in Beat Link Trigger itself, to make it easier to use all of its capabilities in a headless environment. So many potential projects, so little time!

If you do want to explore non-Java implementations of beat-carabiner, the two starting points I am aware of at the moment are prolink-go and Python ProDJ Link, in Go and Python respectively. Go has somewhat more of a chance of reducing latency, although these days once you get past startup and warm-up/JIT time, it is really hard for any language to beat Java.

[marek-memsql]

The prolink-go might be the way to go atm- seems lightweight enough to run nicely on a rpi and should be simple enough to tweak so it sends events to carabiner. I wasn't aware something like this exists :)

[brunchboy]

Cool, if you experiment with it and have any success, let me know, and I’ll link people to the results!