Custom block libraries, 2.0

[towerofnix]

See also #52 for the original and main ideas, and #55 for the original PR plus some Real(tm)-user feedback.

Input is wanted! (ping @heyitsmeuralex, @bobbybee, @everyone)

There's probably been some good discussion by Jens and friends on the Snap! repository.

---

• Give custom block libraries their own colors (user-specifiable).
• Separate custom block libraries into different "subcategories", like extensions.
• Make library block creation simpler.
  • An "add to library" button in the custom block dialog?
  • Could be a dropdown containing a list of library names, and a "create new library..." option.
• Let two sprites use a given library, and synchronize libraries between the sprites that are using them.
  • Need to consider "live editing" library custom blocks; if two sprites use a library block, then the code for that block should update in both sprites, as soon as the user modifies (e.g. drops a block) the actual custom block in either of the sprites using the custom block.
• What about libraries that depend on other libraries?
  • E.g. a "add item to lists [str]" library, which uses a "string utilities" library's "split string" block to split "[str]" by commas
• Metadata can be stored in Scratch-based project comments (the "add comment" button), if necessary/convenient.

[bates64]

I like the idea of general-purpose libraries having somewhere like npm to be stored so you can just add coolstrings to your project and they would always be there. This should also solve dependencies. In this case libraries would be per-project rather than per-sprite, but they could always be per-sprite and distributed as .sprite2 files...

We could even just utilise Scratch itself as a place to store scrap libs - the scrap-library tag could denote a Scrap library and they'd be referenced by project ID by dependents.

[towerofnix]

That's an interesting idea!

In this case libraries would be per-project rather than per-sprite, but they could always be per-sprite and distributed as .sprite2 files...

I've thought of storing them as separate "library" sprites. (See below section.)

We could even just utilise Scratch itself as a place to store scrap libs - the scrap-library tag could denote a Scrap library and they'd be referenced by project ID by dependents.

I'm curious - why should Scrap-library projects have a tag? Just to make it more convenient, e.g., by the Explore page? (..Which doesn't even consider tags anymore, I believe..)

---

I'm thinking - you add a dependency by clicking a button in the UI. This requests a project ID (or project URL, for convenience) and the name of the wanted library's dedicated sprite in that project. Then Scrap downloads the project.json for the given ID, and copies the given sprite into your project.

We also need to make it clear to Scrap whether a library-sprite is a "local" library (i.e. one you created yourself) or an "imported" library (i.e. one you added from an online project). I think the easiest way to do that would be to have Scrap automatically add a metadata-comment to imported library sprites, giving them a remote URL and sprite ID (i.e., what you filled into the comment).

(I thought of making those metadata-comments just be in the online project.sb2, but we can't really do that, because Scrap doesn't know the URL of a project before it's uploaded.)

In NPM, you can publish new versions of a library. Comparatively, in Scratch, you can update a shared project. So, should Scrap library dependencies (imported libraries) automatically be updated? I'm thinking no; instead some user interaction should be given, e.g. an "update library" button.

In NPM, different versions of a library are actually different versions, and all published versions are stored online. But in Scratch, only one version of a project is stored (or at least, only one version is visible to users): the version with the most recent changes. Because of that, you can't really swap between different versions of a library, and an "update library" button might actually break your program. I think it'd be best to at least warn the user before pressing update library, and maybe to even make a hidden backup sprite in the project for the version before the update library button was pressed.

(Another crazy idea would be to store various versions of a library in your project, but I'm wondering if maybe that's overkill, and not realistically useful? After all, you can just have sprite2 exports of the libraries, or have different versions be published in different online Scratch projects.)

[bates64]

I'm curious - why should Scrap-library projects have a tag? Just to make it more convenient, e.g., by the Explore page? (..Which doesn't even consider tags anymore, I believe..)

So we can have a little search engine for Scrap libraries (built right into the editor?)! :shipit:

I think that we shouldn't bother with versioning other than "update libs might break your code!" as people will likely make a separate project for each new version of their library. All libraries should be essentially statically linked into your project - i.e. all libraries live as hidden sprites in your project - and should you wish to update you can do so. They shouldn't be loaded on-demand, obviously.

[towerofnix]

So we can have a little search engine for Scrap libraries (built right into the editor?)! :shipit:

Agreed! But I don't think there's a Scratch API endpoint for searching for projects with a given tag.

[bates64]

We could use Google site searches?

https://www.google.com/search?q=site%3Ahttps%3A%2F%2Fscratch.mit.edu+"[scrap]"+Better+Strings

idk. That's probably a bad idea.

[towerofnix]

It'd be both unreliable (pages would need to be indexed by Google before they show up) and evil (relying on external sites/services is not good!). :rainbow:

[alyssarosenzweig]

Few thoughts:

• Statically linking libraries is necessary for backwards compatibility.
• While the project is still living within Scrap, it is not; we can version as much as want.
• You can afford to keep all that extra metadata and old versions lying around in the scrap file. Just because it's backwards compatible doesn't mean we can't do this kind of thing -- just remember to provide a cleanup button once in a while.
• Special tags in Scratch would be ideal. Barring that, we can host metadata on our own site and just use Scratch for the actual JSON. Alternatively, host everything on our own site (including versioned copies!) and just syndicate to Scratch.
• Some library updates break code. Others don't. Read up on semantic versioning.
• Dependency handling is a must, but luckily it's not too difficult if you don't do anything crazy.
• If you want to have library sync across sprites without an internal scrap format, metadata in the comments and some crazy UI hacks is the way to go, I guess..

[towerofnix]

You can afford to keep all that extra metadata and old versions lying around in the scrap file. Just because it's backwards compatible doesn't mean we can't do this kind of thing -- just remember to provide a cleanup button once in a while.

I'm thinking that backup library versions could be displayed in the same place as normal/active libraries, just greyed out a little to show that they aren't actually being used. They'd otherwise behave just like normal library sprites (e.g. "export library to sprite2" and "remove library [backup]" would be options).

Special tags in Scratch would be ideal. Barring that, we can host metadata on our own site and just use Scratch for the actual JSON. Alternatively, host everything on our own site (including versioned copies!) and just syndicate to Scratch.

That would be fun, but it would mean me maintaining a server, and forcing/requesting users to rely on that server, which I don't think is good. Plus it sort of takes away from Scrap just being a standalone Scratch mod.

[alyssarosenzweig]

That would be fun, but it would mean me maintaining a server, and forcing/requesting users to rely on that server, which I don't think is good. Plus it sort of takes away from Scrap just being a standalone Scratch mod.

Oh, uh... store the database as comments on a Scratch project? :P

[towerofnix]

Oh, uh... store the database as comments on a Scratch project? :P

Then I'd have to do crazy hacks to deal with the 500 character limit.. and I'd have to re-implement it when the commenting system's API changes (which isn't going to be that long from now) :P

[alyssarosenzweig]

Then I'd have to do crazy hacks to deal with the 500 character limit

Since when are project IDs 500 characters? ;)