Cater for moved .sc files on update

[antonhornquist]

It seems like moved sc files (such as me moving CroneGenEngine.sc from the sc engine folder to ”~/dust/lib/sc/abstractions”) might mess up/cause sclang discrepancies in the update process. This might be due to users git cloning https://github.com/monome/dust in between updates, I dunno. Anyhow, is there any way to fix this during update? (update is a process I’m not familiar with)

[catfact]

adding/removing/moving .sc files requires running the norns/sc/install.sh script (currently). this is actually not really necessary given that i've finally realized that you can add arbitrary search paths for SC class files (see Quark.sc for example)

i also have been wondering about the update process, since some upcoming changes require more than just copying files (e.g. internet connection)

@tehn is there a reason not to have update.sh in git?

i see update.lua -

[tehn]

update.sh changes per update... though much is the same.

i'll add it

[tehn]

can you clarify what you mean by adding arbitrary search paths?

ref: https://github.com/supercollider/supercollider/blob/develop/SCClassLibrary/Common/Quarks/Quark.sc

trying to comprehend how we can have .sc files scattered around dust, ie, encapsulated within a "project" re #446

would it be possible to just have the search path include a whole subtree ie ~/dust/scripts/?

[catfact]

would it be possible to just have the search path include a whole subtree ie ~/dust/scripts/?

Yes I think so

[antonhornquist]

include a whole subtree ie ~/dust/scripts/ ?

yeah that's what i'm thinking

[tehn]

ok, so my experiments using EXPORT to set paths failed. what's the magic word?

but i'm also confused about what the actual problem is. i had success changing install.sh to link /dust instead of /dust/lib/sc and i tried moving .sc files around the dust repo, without re-executing install.sh and all seems fine. was the earlier problem @antonhornquist reported about duplicate files?

[catfact]

sorry i linked the wrong file https://github.com/supercollider/supercollider/blob/develop/SCClassLibrary/Common/Quarks/Quarks.sc#L219

*link { |path|
        path = path.withoutTrailingSlash;
        if(LanguageConfig.includePaths.includesEqual(path).not, {
            path.debug("Adding path");
            LanguageConfig.addIncludePath(path);
            LanguageConfig.store(LanguageConfig.currentPath);
            installedPaths = installedPaths.add(path.withoutTrailingSlash);
            ^true
        });
        ^false
}

i made install.sh link individual files in the first place because of various frustrations. SC will attempt to link HTML (as help files) and temporary editor files (#Foo.sc#) and other stuff. there seems to be some caching behavior or something that makes things sometimes not work when you have a whole directory symlinked and you add/remove things. it was hair-pulling. but give it a try

[antonhornquist]

@tehn

was the earlier problem @antonhornquist reported about duplicate files?

See https://llllllll.co/t/norns-updates/14179/57. I guess somehow the update script together with cloning / hacking dust locally caused two CroneGenEngine.sc files to exist in dust. (I moved the file during development to ~/dust/lib/sc/abstractions/).

Not sure if the source of the error repoted on lines is ./install.sh but the ./install.sh step seems unnecessary to me - I'd recommend somehow adding the norns sc root ~/dust/lib/sc to the search path in some other way. Either using LanguageConfig.addIncludePath() (as suggested by @catfact) or by setting the Platform.userExtensionDir to ~/dust/lib/sc on sc startup.

[catfact]

setting the Platform.userExtensionDir

don't think you can do this, it's baked into the language (_Plaform_userExtensionDir is a primitive?)

i'll try adding all the necessary include paths directly before crone class init. it's a little more involved than implied so far b/c there are also .so's from waf. these need to be seen by the server. what we've discussed so far is fine for language components but scsynth and supernova have different rules for loading plugins

[tehn]

relevant to https://github.com/monome/norns/pull/664

[antonhornquist]

setting the Platform.userExtensionDir

don't think you can do this, it's baked into the language (_Plaform_userExtensionDir is a primitive?)

sorry, you're right, i thought this was possible to override. anyhow, on my laptop (Windows) I make a symlink from the userExtensionDir to the dust/lib/sc directory and it works well.

[catfact]

there is a mechanism for adding include paths. found this by looking in Quarks.sc. this works for me (on mac, and lacking a path to the built .so/.scx files for ugens)

~addIncludePath = { |path|
    path = path.withoutTrailingSlash;
    if(LanguageConfig.includePaths.includesEqual(path).not, {
        path.debug("Adding path");
        LanguageConfig.addIncludePath(path);
        LanguageConfig.store(LanguageConfig.currentPath);
        ^true
    });
    ^false
};

~addIncludePath.value("/Users/emb/norns/sc/core");
~addIncludePath.value("/Users/emb/norns/sc/ugens");
~addIncludePath.value("/Users/emb/dust/lib/sc");

but, of course theres a bit of a chicken-and-egg problem, we have to at least install one .sc file to set this up. (or use Quarks somehow.)

[tehn]

would it make sense to just have a single file norns-config.sc (or whatever) with the path setting above, for which install.sh becomes solely responsible for?

[tehn]

this is just me clomping around, but almost works:

norns/sc/norns-config.sc:

Norns {
  *addIncludePath { arg path;
    if(LanguageConfig.includePaths.includesEqual(path).not, {
        path.debug("Adding path");
        LanguageConfig.addIncludePath(path);
        LanguageConfig.store(LanguageConfig.currentPath);
        ^true
        });
    ^false
  }

  *initClass {
    Norns.addIncludePath("/home/we/norns/sc/core");
    Norns.addIncludePath("/home/we/norns/sc/engines");
    Norns.addIncludePath("/home/we/norns/sc/ugens");
    Norns.addIncludePath("/home/we/dust");
  }
}

install.sh

#!/bin/bash

sc_ext_dir=~/.local/share/SuperCollider/Extensions
cp norns-config.sc $sc_ext_dir/

unfortunately the ugens don't get found?

Exception when reading synthdef: Cannot load synth faust_comp: Unit generator FaustCompressor not installed
Exception when reading synthdef: Cannot load synth faust_verb: Unit generator FaustZitaRev not installed

[tehn]

oh, but maybe this doesn't matter entirely, since new-crone includes all of this functionality and we can remove this all?

[catfact]

yeah the ugens wont be needed. but if you want to link them you need to point at wherever the .so files are, which is norns/build/[something]

[tehn]

fixed by #664

https://github.com/monome/norns/pull/664/commits/9cb309d9518f7cef016e507203581ddd6390400b