search

supercollider / supercollider

An audio server, programming language, and IDE for sound synthesis and algorithmic composition.
http://supercollider.github.io
GNU General Public License v3.0
5.43k stars 746 forks source link

cmake needs to install HUT directory on OSX #2116

Closed sensestage closed 7 years ago

sensestage commented 8 years ago

From @adcxyz :

On linux, the cmake install script copies a folder “hut” from external lib hidapi) into SCClassLibrary, which is looked up as

HIDUsage.hutDirectory;

/SuperCollider.app/Contents/Resources/SCClassLibrary/Common/Control/HID_support/hut/ On OSX this folder is not copied, and thus hutDirectory is missing, so you cannot ask HIDs for semantic element types by name. IMO, it would be cleaner to put that folder into String.scDir, have HIDUsage.hutDirectory point to it with: /SuperCollider.app/Contents/Resources/HID_support/hut/ I don’t know how to safely fix that in the cmake script, but I am sure it is simple for someone who does :-) BTW, for people who want to add the missing folder to an existing install of SC, see here: https://github.com/ModalityTeam/Modality-toolkit/blob/master/Modality/HelpSource/Tutorials/Fix_Missing_hutDirectory_osx.schelp
sensestage commented 8 years ago

It is currently set in the class as:

hutDirectory = PathName( HID.class.filenameSymbol.asString ).pathOnly +/+ "HID_support/hut/";

It could be made dependent on 

thisProcess.platform.systemAppSupportDir

if that would solve it for the suggestion that @adcxyz is mentioning for OSX.

Important is that it needs to be installed system-wide and found in a platform-transparent way.

adcxyz commented 8 years ago

For finding the best option - Where are these folders on linux?

String.scDir 
Platform.systemAppSupportDir
Main.filenameSymbol.asString.dirname.dirname

On OSX, Platform.systemAppSupportDir is hardly ever used. All static files/resources the app needs are in String.scDir within the app folder, that is why I would suggest putting the HID_Support next to SCClassLibrary in any case.

sensestage commented 8 years ago 
*scDir {
        ^Platform.resourceDir
    }

Platform.systemAppsupportDir and String.scDir are the same on Linux. We should also check for Windows!

Main.filenameSymbol.asString.dirname.dirname
--> /usr/local/share/SuperCollider/SCClassLibrary
adcxyz commented 8 years ago

OK, I have access to a windows 7 machine and can test later today.

In our April 1 discussion, we had some of it figured out, and rereading it, I guess what needs to be done in

external_libraries/hidapi/CMakeLists.txt: — ... // turn installing hut dir ON - was OFF option(HID_INSTALL_HUT "install hid usage tables" ON)

...

// and at the bottom: // was: if ( HID_DEBUG_PARSER )

if( HID_INSTALL_HUT ) set(HID_HUT_PATH ${CMAKE_INSTALL_PREFIX}/share/hidapi CACHE STRING "Installation path for the HID usage tables")

install(DIRECTORY hut DESTINATION ${HID_HUT_PATH} FILES_MATCHING PATTERN "*.yaml" ) endif()

—

best a

Begin forwarded message:

From: marije marije@nescivi.nl Subject: Re: [Modality] hutDirectory - was generic or parent descriptions? Date: 01/Apr /2016 at 10:27:24 GMT+2 To: "alberto.decampo" alberto.decampo@gmail.com

Hi,

I think there was an issue somewhere how to get it to a place sc finds it and not a systemwide dependent place.

HID_DEBUG_PARSER is a test programming for HID_API library, you don't normally want it.

So it seems installing this directory in the sc build process got lost somewhere or doesn't work on all platforms...

OK... I'll have a look...

Sincerely, Marije

"alberto.decampo" alberto.decampo@gmail.com wrote:

ok, copied it to its place, and now the sematic HID stuff is working … amazing that nobody found that for how long now?

It seems the relevant lines are in CMakeLists.txt :

// there is a line which seems to turn the HID_INSTALL_HUT flag off,
// but that flag is never used: option(HID_INSTALL_HUT "install hid usage tables" OFF)

// and then there is this, which only installs the hut dir when HID_DEBUG_PARSER is on (why?): if( HID_DEBUG_PARSER ) set(HID_HUT_PATH ${CMAKE_INSTALL_PREFIX}/share/hidapi CACHE STRING "Installation path for the HID usage tables")

install(DIRECTORY hut DESTINATION ${HID_HUT_PATH} FILES_MATCHING PATTERN "*.yaml" ) endif()

should it not just install in any case? or when is there a reason not to install it - embedded systems?

best a

On 22/05/2016, at 12:13 , sensestage notifications@github.com wrote:

*scDir { ^Platform.resourceDir }

Platform.systemAppsupportDir and String.scDir are the same on Linux. We should also check for Window!

Main.filenameSymbol.asString.dirname.dirname --> /usr/local/share/SuperCollider/SCClassLibrary

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub

bagong commented 8 years ago

@adcxyz , do you want this for 3.7.2, or is 3.8 fine? We (well mainly @llloret) are currently getting stuff ready for HID on Windows. It will also include Marije's latest commits for Linux which haven't sofar made it into SC... Would you like to keep your OSX build system stuff separate (and targeted at 3.7.2) or could we create an "integrated" PR that includes all of this, get's tested by all, and is targeted at 3.8?

That you have a Windows machine sounds exciting :D You might want to try it mit Luis' brand new Windows code. It's actually accessible as a pre-pre-version in a Windows branch in the new hidapi repo in SC org.

adcxyz commented 8 years ago

@bagong, it seems very unlikely to introduce any instabilities (a folder of .yaml files which get read on demand), so I would prefer to get it done for 3.7.2. OTOH, if the windows HID batch is for 3.8, maybe it is better there - hm.

I will get very little time on that windows machine, and installation things will be frowned upon, so not that exciting ... will try to improve that though.

bagong commented 8 years ago

Ah, setting up a build environment for Windows is likely to take a couple of hours the first time, and you'd have to build from scratch to test anything concerning HID on Windows. So if your time slice on the Windows server is below a couple of hours it is likely not realistic to test anything on Windows (the very first time)... ;) And it wouldn't make much sense either, as HID doesn't work on SCWin 3.7 and could only be added for 3.7.3 if really wanted on 3.7. It does feel that comprehensive HID support sounds like something nice to put on the 3.8 poster?

bagong commented 8 years ago

I've had a look at this now, also because I'm hoping 3.7.2 can come out soon. The problem with bringing this into 3.7.2 is not so much that it is difficult or risky per se, but that putting something into this submodule without merging other commits as well is pretty tricky. There are 4 commits in master that haven't been added to SC yet and that also need testing. So may I propose to keep this for 3.8. For self builders, btw I think there is a nice and useful way to test the result, even when using 3.7 (the submodule can locally be updated independently of the SC branch).

As to this issue: I think the consensus is the install folder should be Platform.resourceDir, right? May I propose to call it hidapi? "In cmake" that could be said as:

"${CMAKE_INSTALL_PREFIX}/${auxresourcesdir}/hidapi"

Which I think is cross-platform. install(Directory hut Destination ${HID_HUT_PATH} will create a subfolder in hidapi.

So I'll send a PR to the hidapi submodule that installs the folder as suggested, and another one to SC, that set's

hutDirectory = Platform.resourceDir +/+ "hidapi/hut/";

Does that look right? (alongside that I'll add hid to the sclang target_link_libraries for Windows)

I would like to do the following: push the submodule additions at the end of a larger existing pull-request that allows HID on Windows to work. The PR is targeted at 3.8, and next to Windows support, de facto also implies a few older, mainly linux related commits, that haven't been added to SC yet, and haven't had much exposure. Ideally we would accumulate the remaining stuff needed for even better HID support on a single branch. That will make testing a PR in a submodule significantly easier, and could make testing across OS frontiers fun. ;) I think everybody here has the rights to write directly to the branch and thereby add to the PR. The branch is called hid3.8 and is in this repo:

https://github.com/supercollider/hidapi.git

If we agree on this, I'll put it in action (it's almost ready, so any minute ;) ), and add an explanation how to activate the branch in the submodule...

And: once we have all of that in the submodule, I think it could also be used in 3.7 if so wished. It would then just be a matter of setting the module to the right commit there, and backporting the tiny additions to sc itself.

bagong commented 8 years ago

Hehe, sorry, I was not patient enough. So these are the two PRs in question:

SC: https://github.com/supercollider/supercollider/pull/2123 hidapi submodule: https://github.com/supercollider/hidapi/pull/1

adcxyz commented 8 years ago

Dear Rainer,

thanks so much for taking care of this!

On 25/05/2016, at 20:40 , Rainer Schütz notifications@github.com wrote:

I've had a look at this now, also because I'm hoping 3.7.2 can come out soon. The problem with bringing this into 3.7.2 is not so much that it is difficult or risky per se, but that putting something into this submodule without merging other commits as well is pretty tricky. There are 4 commits in master that haven't been added to SC yet and that also need testing. So may I propose to keep this for 3.8. For self builders, btw I think there is a nice and useful way to test the result, even when using 3.7 (the submodule can locally be updated independently of the SC branch).

As to this issue: I think the consensus is the install folder should be Platform.resourceDir, right? May I propose to call it hidapi? "In cmake" that could be said as:

for OSX, yes.

"${CMAKE_INSTALL_PREFIX}/${auxresourcesdir}/hidapi"

Which I think is cross-platform. install(Directory hut Destination ${HID_HUT_PATH} will create a subfolder in hidapi.

So I'll send a PR to the hidapi submodule that installs the folder as suggested, and another one to SC, that set's

hutDirectory = Platform.resourceDir +/+ "hidapi/hut/";

Does that look right? (alongside that I'll add hid to the sclang target_link_libraries for Windows)

The dir name was “HID_Support/hut/“, which we could keep (it says clearly what it is); otherwise, yes, looks right. Should be tested that the build script puts it in the equivalent places on all 3 platforms.

// The quick tests would be: HIDUsage.hutDirectory.pathMatch.postcs.notEmpty;
HIDUsage.getUsageDescription( 1, 5 ).postcs == [ 'GenericDesktop', 'GamePad' ];

I would like to do the following: push the submodule additions at the end of a larger existing pull-request that allows HID on Windows to work. The PR is targeted at 3.8, and next to Windows support, de facto also implies a few older, mainly linux related commits, that haven't been added to SC yet, and haven't had much exposure. Ideally we would accumulate the remaining stuff needed for even better HID support on a single branch. That will make testing a PR in a submodule significantly easier, and could make testing across OS frontiers fun. ;)

// Quick tests for hutDirectory: HIDUsage.hutDirectory.pathMatch.postcs.notEmpty;
HIDUsage.getUsageDescription( 1, 5 ).postcs == [ 'GenericDesktop', 'GamePad' ];

I think everybody here has the rights to write directly to the branch and thereby add to the PR. The branch is called hid3.8 and is in this repo:

https://github.com/supercollider/hidapi.git

If we agree on this, I'll put it in action (it's almost ready, so any minute ;) ), and add an explanation how to activate the branch in the submodule…

+1

And: once we have all of that in the submodule, I think it could also be used in 3.7 if so wished. It would then just be a matter of setting the module to the right commit there, and backporting the tiny additions to sc itself.

thanks, best adc

bagong commented 8 years ago

The dir name was “HID_Support/hut/“, which we could keep (it says clearly what it is);

Added.

On Windows, MinGW build:

// Quick tests for hutDirectory:
HIDUsage.hutDirectory.pathMatch.postcs.notEmpty;
-> true
HIDUsage.getUsageDescription( 1, 5 ).postcs == [ 'GenericDesktop', 'GamePad' ];
-> true

Just to show off (with Luis' work) ;)

HID.postAvailable
HID: found 3 devices
-> IdentityDictionary[ (1 -> a HIDInfo(, , IDs:1054, 4099, \\?\hid#vid_041e&pid_1003#7&24e0b3a3&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}, , 272, -1)), (2 -> a HIDInfo(, Controller (XBOX 360 For Windows), IDs:1118, 654, \\?\hid#vid_045e&pid_028e&ig_01#8&2cc7b128&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}, , 0, -1)), (0 -> a HIDInfo(ACRUX, SL-6632 Dark Tornado Joystick, IDs:6708, 2053, \\?\hid#vid_1a34&pid_0805#7&321877c&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}, , 304, -1)) ]
0:  Usage name and page:    Joystick,   GenericDesktop
    Vendor name:    ACRUX
    Product name:   SL-6632 Dark Tornado Joystick
    Vendor and product ID:  6708, 2053
    Path:   \\?\hid#vid_1a34&pid_0805#7&321877c&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}
    Serial Number:  
    Releasenumber and interfaceNumber:  304, -1
1:  Usage name and page:    GamePad,    GenericDesktop
    Vendor name:    
    Product name:   
    Vendor and product ID:  1054, 4099
    Path:   \\?\hid#vid_041e&pid_1003#7&24e0b3a3&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}
    Serial Number:  
    Releasenumber and interfaceNumber:  272, -1
2:  Usage name and page:    GamePad,    GenericDesktop
    Vendor name:    
    Product name:   Controller (XBOX 360 For Windows)
    Vendor and product ID:  1118, 654
    Path:   \\?\hid#vid_045e&pid_028e&ig_01#8&2cc7b128&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}
    Serial Number:  
    Releasenumber and interfaceNumber:  0, -1
-> HID

Please teeeeeest!

bagong commented 8 years ago

On OSX the hut folder doesn't install yet, and on Linux it's installed to /usr/local//HID... instead of share. I'll try to fix that.

bagong commented 8 years ago

Wrong, on Linux it does install properly. I had just forgotten to use my PR ;)

Ubuntu Studio 16.04:

HIDUsage.hutDirectory
-> /usr/local/share/SuperCollider/HID_Support/hut/

HIDUsage.hutDirectory.pathMatch.postcs.notEmpty
[ "/usr/local/share/SuperCollider/HID_Support/hut/" ]
-> true

HIDUsage.getUsageDescription(1, 5).postcs == [ 'GenericDesktop', 'GamePad' ]
[ 'GenericDesktop', 'GamePad' ]
-> true

This is from HID-branch latest including Marije's yet unused commits:

HID: found 2 devices
-> IdentityDictionary[ (1 -> a HIDInfo(, , IDs:1054, 4099, /dev/hidraw1, , 272, 0)), (0 -> a HIDInfo(ACRUX, SL-6632 Dark Tornado Joystick, IDs:6708, 2053, /dev/hidraw0, , 304, 0)) ]
0:  Usage name and page:    undefined,  undefined
    Vendor name:    ACRUX
    Product name:   SL-6632 Dark Tornado Joystick
    Vendor and product ID:  6708, 2053
    Path:   /dev/hidraw0
    Serial Number:  
    Releasenumber and interfaceNumber:  304, 0
1:  Usage name and page:    undefined,  undefined
    Vendor name:    
    Product name:   
    Vendor and product ID:  1054, 4099
    Path:   /dev/hidraw1
    Serial Number:  
    Releasenumber and interfaceNumber:  272, 0
-> HID
bagong commented 8 years ago

I hope that Mac works too, now:

HIDUsage.hutDirectory
-> /Users/rainer/Projects/supercollider/buildmaster/Install/SuperCollider/SuperCollider.app/Contents/Resources/HID_Support/hut/

// Quick tests for hutDirectory:
HIDUsage.hutDirectory.pathMatch.postcs.notEmpty;
-> true
HIDUsage.getUsageDescription( 1, 5 ).postcs == [ 'GenericDesktop', 'GamePad' ];
-> true
bagong commented 8 years ago

The difficulty in testing this is that the both PRs must be used in conjunction to work.

getting the one to SC is simple (assuming the sc repo is origin)

git fetch origin pull/2123/head:hidapiTest
git checkout hidapiTest

Getting the branch in the submodule is a bit trickier. You need to be sure first, that the hidapi submodule is already initialized to pull from the hidapi repo in the supercollider org (rather than still from Marijes):

[submodule "external_libraries/hidapi"]
    path = external_libraries/hidapi
    url = git://github.com/supercollider/hidapi.git

You might additionally need to do these steps to really switch:

`git submodule deinit external_libraries/hidapi`
then delete the folder .git/modules/external_libraries/hidapi (it will be recreated in the next step with fresh data)
`git submodule init`
`git submodule update`

After this you can checkout the hid3.8 branch in the submodule:

cd external_libraries/hidapi
git checkout -b hid3.8 origin/hid3.8

If you want to return to the old state, just do:

git submodule update

Happy testing!

adcxyz commented 8 years ago

hi @bagong,

fails here in line:

git checkout hid3.8 origin/hid3.8 with: error: pathspec 'hid3.8' did not match any file(s) known to git. error: pathspec 'origin/hid3.8' did not match any file(s) known to git.

?

On 25/05/2016, at 23:58 , Rainer Schütz notifications@github.com wrote:

The difficulty in testing this is that the both PRs must be used in conjunction to work.

getting the one to SC is simple (assuming the sc repo is origin)

git fetch origin pull/2123/head:hidapiTest git checkout hidapiTest

Getting the branch in the submodule is a bit trickier. You need to be sure first, that the hidapi submodule is already initialized to pull from the hidapi repo in the supercollider org (rather than still from Marijes):

• the file scsource/.gitmodules must contain "supercollider" instead of "sensestage" in the address (this is the case on latest master): [submodule "external_libraries/hidapi"] path = external_libraries/hidapi url = git://github.com/supercollider/hidapi.git

You might additionally need to do these steps to really switch:

git submodule deinit external_libraries/hidapi then delete the folder .git/modules/external_libraries/hidapi (it will be recreated in the next step with fresh data) git submodule init git submodule update

After this you can checkout the hid3.8 branch in the submodule:

cd external_libraries/hidapi git checkout hid3.8 origin/hid3.8

If you want to return to the old state, just do:

git submodule update

Happy testing!

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub

llloret commented 8 years ago

Hi, Alberto,

you may need to do a "git fetch --all", and then "git checkout -t origin/hid3.8".

adcxyz commented 8 years ago

ok, thanks, tried that: "hut" directory is not copied anywhere. semi-related: the app did not open, because OSX 10.11.5 seems to require qt5.6, so I will try that next.

bagong commented 8 years ago

@adcxyz , if the hut directory is not copied anywhere, you probably haven't got the right branch.

bagong commented 8 years ago

Eeeeeh, sorry A. there is a mistake in that line you quoted:

git checkout hid3.8 origin/hid3.8

It has to be:

git checkout -b hid3.8 origin/hid3.7

Soooory... (I just changed it above)

bagong commented 8 years ago

@adcxyz did you get there? Sorry for that mishap earlier. At this point positive feedback would be as helpful as problem-reports.

Also the SC part of the complementary commits is merged to master now, so getting it going is a bit simpler. I am tempted to clean up this thread to avoid scaring off people. Once you have that gotcha with submodules, it's actually pretty simple...

Currently the following would be the quickest/easiest way to get the full set of HID innovations:

git clone --recursive https://github.com/supercollider/supercollider.git
cd supercollider/external_libraries/hidapi
git fetch origin hid3.8:testhid3.8
git checkout testhid3.8

Ready to build!

adcxyz commented 8 years ago

just tested fresh build with your instructions now, git branch seems right:

adc:hidapi adc$ git branch
  master
* testhid3.8
adc:hidapi adc$

build seems ok, but Library does not compile (likely unrelated) with:

inumClassDeps 0   gNumClasses 82
Library has not been compiled successfully.
Library has not been compiled successfully.

no HID_Support/hut in Contents/Resources/, script install it to /usr/local//HID_Support/hut :

-- Installing: /Users/adc/scgit/testonly/supercollider/build/Install/SuperCollider/CHANGELOG.md
-- Installing: /Users/adc/scgit/testonly/supercollider/build/Install/SuperCollider/COPYING
-- Installing: /Users/adc/scgit/testonly/supercollider/build/Install/SuperCollider/AUTHORS
-- Up-to-date: /usr/local//HID_Support/hut
-- Installing: /usr/local//HID_Support/hut/hut_11_telephony.yaml
-- Installing: /usr/local//HID_Support/hut/hut_12_consumer.yaml
-- Installing: /usr/local//HID_Support/hut/hut_13_digitizers.yaml
-- Installing: /usr/local//HID_Support/hut/hut_1_generic_desktop.yaml
-- Installing: /usr/local//HID_Support/hut/hut_20_alphanumeric_display.yaml
-- Installing: /usr/local//HID_Support/hut/hut_2_simulation_controls.yaml
-- Installing: /usr/local//HID_Support/hut/hut_3_vr_control.yaml
-- Installing: /usr/local//HID_Support/hut/hut_4_sport_controls.yaml
-- Installing: /usr/local//HID_Support/hut/hut_5_game_controls.yaml
-- Installing: /usr/local//HID_Support/hut/hut_64_medical_instrument.yaml
-- Installing: /usr/local//HID_Support/hut/hut_6_generic_device.yaml
-- Installing: /usr/local//HID_Support/hut/hut_7_keyboard_keypad.yaml
-- Installing: /usr/local//HID_Support/hut/hut_8_ledpage.yaml

so no luck yet on OSX :-(

bagong commented 8 years ago

@adcxyz , sorry, yes, I can reproduce that. Something went wrong in my tests. Will try to fix tonight...

bagong commented 8 years ago

Should be fixed now. It's kind of a provisional fix, as the install should better be defined in sc-land, where the fundamental variables for install are visible. So I had to reassign them in the submodule.

I made clean builds on Windows, OSX and Linux, and it all looks fine now.

Happy testing and sorry for the annoying detours I caused ;)

adcxyz commented 8 years ago

looks like it is installing the HID_Support/hut in Resources/ . Yay! @bagong, did the classlib compile correctly on your OSX build? here, it still does not. building with: 

cmake -DCMAKE_PREFIX_PATH=`brew --prefix qt5`  ..
make -j 8
make install
bagong commented 8 years ago

Thanks for being patient! Sorry, I forgot to mention: I don't get any error of the sort you observe... As to building, my commands were:

cmake -GXcode -DCMAKE_PREFIX_PATH=/usr/local/opt/qt5 .. cmake --build . --target install --config Release

I am on 10.11.5 and Xcode 7.3... I hope that's not an issue here...

bagong commented 8 years ago

I've posted testbuilds using that hid3.8 branch within 3.7.2. As Windows is still very unstable on master it doesn't make sense to post a build of the master branch. I added an OSX build as well (should work down to OSX 10.7):

https://github.com/bagong/supercollider/releases/tag/Version-3.7.2-win-HID

I think @llloret is very eager for input (any platform), an opportunity we should not let pass by ;). There is some additional stuff in that branch relevant for the other platforms, especially @sensestage 's Linux related commits before Luis' work. Unfortunately I don't know how to create a debian package, so for Linux it's still biy ;)

adcxyz commented 7 years ago

just tested building current master and hutDirectory is where it should be - yay! thanks @llloret @bagong @sensestage !