Should we rename trio.hazmat?

[njsmith]

On the one hand, the name hazmat is definitely effective at getting people to pay attention and be careful. OTOH, I'm not sure it's quite communicating what we want: people seem scared of it, and uncertain whether they can actually use it. A bit of that is fine, but... I'm not sure it's communicating what we want. The block of text at the beginning helps of course, but you can't really fix up a confusing name with good docs.

Maybe trio.lowlevel? Though trio.socket is kind of low-level too. Does lowlevel have the kind of "if you see this during a code review, ask for a comment justifying its usage" implication that we're going for?

trio.foundation? trio.core?

[lisael]

Hi, and thanks for the great work with trio.

While reading the docs, I had the feeling that trio API is sometimes not non-english-native-speaker friendly. I read a lot of technical documentation and I usually barely struggle to understand technical English but to be honest, I have exactly no idea what hazmat means. Not scary at all for more than the 2/3 of potential users. I've also noticed cushion and tiebreaker, that don't sound very technical, even though I can infer the meaning of those, from my French background or by word composition.

OTOH, the naming in the API is somehow unusual, refreshing, and sometimes very explicit (do_stuff_before_foo()), but the refreshing part shouldn't hinder the readability. Don't make me wrong, I do think that the documentation is incredibly extensive, readable and sometimes amusing for such a young project.

May I suggest trio.internals ? core and foundation read good too. Maybe internals reads too much like don't you dare using this, you mortal

[njsmith]

@lisael

I read a lot of technical documentation and I usually barely struggle to understand technical English but to be honest, I have exactly no idea what hazmat means. Not scary at all for more than the 2/3 of potential users.

That's a great point – thank you! In colloquial English, hazmat is "hazardous materials"; it suggests that when using this module you might want to wear a suit like this. I definitely want things to be accessible, but as a native English speaker I'm at a disadvantage: it's hard for me to guess which things are common knowledge and which things are weird idioms that most people don't know :-). So, I really appreciate this kind of feedback! If you notice any other places where trio's naming is confusing, please speak up; it's a really valuable contribution. (And just to confirm: you're saying that cushion and tiebreaker surprised you, but were understandable, right?)

I am kinda liking trio.foundation... it's long, which nudges you not to use it, and it has that hint of "you don't want to use this directly, you want to use the things that are built on top of it" without being outright scary.

[njsmith]

(Possibly interesting trivia: I stole the hazmat name from the cryptography library.)

[smurfix]

or we could take a hint from git and use plumbing, though foundation make sense too.

[sorcio]

I will just contribute some personal thoughts just because I found myself thinking of this naming more than a couple times. I'm going to look very opinionated but I just have some time to write about something that makes me think.

1) I met cryptography before Trio so the intentions in the naming were easy to translate, and yet required some translation. Because in cryptography the name is meant to actively discourage people who are not experts to implement their own cryptographic solutions, but the hazmat part of the library is mostly extension. It makes the library more extensive, it exposes more ("a larger number of") useful algorithms and protocol implementations. In Trio it feels more like foundation, because it lies on an explicitly different level of abstraction and you could potentially build different abstractions. It's only hazardous because it's lower-level.

1) In both libraries, I feel that telling the user what is dangerous carries the risk of transmitting a patronizing message. The name choice itself tones this down a bit by resulting "ironically patronizing", although 1) this might even be more triggering for some folks and 2) the humor could easily be lost in translation anyway. There is a difference between documenting the non-obvious hazards of something and labelling something in a way that might imply "grownups only, you must be this tall for this ride". Taking a shortcut to remind of dangers has value, to be sure, but it also has implications.

1) Reading Trio code for high level components like streams and sync feels like there is some cheating going. You see that what would be hazmat to commoners (and is documented as such) is actually _core to the library author. What is it, then? Is this actually the core of Trio, or something which is better avoided? Is this just a way the maintainers of the library are telling us they don't support users of this layer? And what about the idea of the foundational layer, if the library itself seems to be playing on a different ground? (It's not, it's pretty much just calling hazmat a different name)

I like names that emphasize "core", "kernel", "low-level" because it's a widely known model to programmers. But: it's common to the point of overuse. It'd be easy to confuse "Trio Core API" or "trio.core API" (the API currently known as trio.hazmat) with "Trio's core API" (some API which is considered to be fundamental to most Trio users, like nurseries). "Low-level" is already taken. "Kernel" is meaningful but way too loaded (is it about interacting with the OS kernel? Is it because it includes a scheduler?). "Foundation" is a good idea because it suggests building upon a layer; it also suggests solidity, which might or might not be the right message. "Internals" is not exactly right (why is it public then? Internal to what? Is this subject to change without warning? Is this less clean?).

One question would be: what is Trio? Is it the whole thing? Is it the API? Is the split we are talking about between two API sets for the same underlying beast, or is it more like between two libraries working one on top of the other? It's actually a mix of both, but is that an implementation detail? I like the split because it says something like "as long as you only use stuff from the main API, some properties of your program are guaranteed". It's great that this is clearly documented in the name itself. Maybe the two parts should be exposed with different names, one as trio and the other as something else?

[Fuyukai]

I personally don't see the point of even having trio.hazmat. Why not just let people access the core through a normal package, instead of various startup redirects? It feels overly protectionist.

[njsmith]

@Fuyukai can you clarify what you're thinking of when you say we could not have hazmat at all? I assume you don't mean: delete that module and don't let people use those functions at all :-). Do you mean, expose trio._core as trio.core or something like that?

[Fuyukai]

The second one; maybe with a big warning saying "Anything in trio.core can change at any point without complaint".

[njsmith]

So, everything in trio._core is exposed as public API -- we actually have a test checking this. But it gets split up between trio, trio.testing, and trio.hazmat. And the things in trio.hazmat aren't supposed to be any less stable than the rest of trio, they're just lower-level: intended for implementing nice tools rather than being nice tools.

I want to be able to say things like: "in trio, await always means a checkpoint" (but in trio.hazmat, that's not true: checkpoint_if_cancelled and cancel_shielded_checkpoint aren't checkpoints, but rather tools for building checkpoints). And "in trio, we don't use callbacks" (but in trio.hazmat, that's not true: wait_task_rescheduled does). And "in trio, concurrency always uses a nursery object" (but in trio.hazmat, that's not true: spawn_system_task and TrioToken.run_sync_soon don't). And "in trio, our APIs try hard to be as friendly and easy to use as possible" (but that's not exactly true for trio.hazmat in the same way it is for the rest of the library). Also, trio.hazmat is where introspection and debugging stuff lives, things you can use to pry open the normal abstractions and look inside.

The rough analogy here is that trio.hazmat is something like the os module: normally in Python if you want to open a file you use __builtins__.open, which gives you a sensible high-level interface. But if you really need to access the raw APIs your operating system exposes, you can use os.open. Along with other wacky stuff like os.setsid, os.posix_fadvise, os.sched_rr_get_interval.

So... hmm. Some things it would be good to communicate to users about this submodule (whatever we call it):

• When you're first learning trio, you can ignore it (and lots of people may never need to learn it at all)

• If you're trying to extend trio or implement your own "primitives", you should look at it

• It's a real public API, in the sense that it's documented / supported / stable (or at least, no more unstable than the rest of the library)

• But you may need some expertise and care to use it correctly.

[shoyer]

I like trio.lowlevel.

[cjerdonek]

I am kinda liking trio.foundation... it's long, which nudges you not to use it, and it has that hint of "you don't want to use this directly,

Oddly, trio.foundation to me communicates the opposite feeling of being very safe and stable to use, like the tried-and-true, can't-go-wrong core part of an API. After reading the last comments, it sounds like trio.expert or trio.advanced might be better. trio.plumbing was in fact the first thing that popped into my mind (as @smurfix also suggested).

By the way, I'm an English speaker and was also confused by what hazmat was going for. I interpreted it to mean something like "use at your own risk because these functions might break at any time." (But that was without reading the docs.)

[sorcio]

Foundation is a good metaphor but maybe the word itself is not the best option. Following synonyms chains and random ideas... trio.groundwork, which as a non-native is not entirely clear but suggestive trio.base, well, feels like a good lead to me, but maybe too neutral trio.ground, ough, don't know trio.dirt, sounds... interesting but maybe... avoidable?

"Plumbing", in Git lingo, is contrasted to "porcelain". Again a good metaphor. In Git, though, "porcelain" comes with less stability guarantees, because it's meant to be used by humans, while "plumbing" is sort of an API (or scripting interface) and tries to be more stable. Also, porcelain is implemented only in terms of plumbing and some glue. This contrasts a bit with the split in Trio's API. Anyway I wouldn't mind adopting trio.plumbing and I don't think it would be too misleading. My only worry is the idea of pipes implied by "plumbing", which is kind of delicate in an I/O library.

The split in Git also reminded me that it's not only about bases and building upon them, but also about fronts and backs. trio.backend might be another good lead trio.backstage trio.backoffice (ugh) trio.behind (?)

Also... trio.extend trio.toolkit trio.unsafe trio.system trio.unprotected trio.panelboard

I'm bad at self-explanatory names.

[Fuyukai]

trio.scary perhaps.

[kyrias]

trio.plumbing is probably the name I like the most, personally.

[ncoghlan]

Given the descriptions above, trio.plumbing would make sense to me as someone relatively unfamiliar with the details of the Trio API. Even without the git precedent, the underlying analogy of "If you're just using the bathroom, you don't need to care about this layer, but if you're building a bathroom, or water is unexpectedly pouring over the bathroom floor, this layer is important" is a good one.

[touilleMan]

From my point of view there is no need to rename the hazmat module.

As @njsmith said, the point is to describe a module that's part of the public API but should be used with caution. For this task I think hazmat is a pretty good name:

• it scares people which is good because you want them to avoid using it...
• ...until they need to do something they cannot do otherwise. At this point your user should be reading the documentation, that's the time they encounter the block of text describing what hazmat is.

The fact this name is already used in a really big and popular project for similar need also help enforcing this. As a matter of fact I find this name so good I've got a hazmat module in a project of mine (Python binding on the Godot game engine), however I don't recall if I named it after or before discovering the trio.hazmat module.

On the other hand things like trio.core or trio.kernel are far too generic, beside plenty of libraries are not as well organized as trio and require multiple imports to work, so doing from trio.core import whatever can feel totally legit to a newcomer.

trio.fundation is a bit better (because the user is less likely to make quick assumption like it would for generic names as core), but still focus more on the fact it is a building block than it is a dangerous thing.

Another important thing that come to my mind is the TrioInternalError exception that tells you to submit a bug to the trio project. Given this exception can be triggered when using hazmat stuff, the name comes handy "You play with hazmat, you get a violent crash. What did you expect ?".

[njsmith]

What do the non-native English speakers here think about "plumbing"? (Or "foundation" or "low-level", for that matter.)

On Mar 20, 2018 15:20, "Emmanuel Leblond" notifications@github.com wrote:

From my point of view there is no need to rename the hazmat module.

As @njsmith https://github.com/njsmith said, the point is to describe a module that's part of the public API but should be used with caution. For this task I think hazmat is a pretty good name:

• it scares people which is good because you want them to avoid using it...
• ...until they need to do something they cannot do otherwise. At this point your user should be reading the documentation, that's the time they encounter the block of text describing what hazmat is.

The fact this name is already used in a really big and popular project for similar need also help enforcing this. As a matter of fact I find this name so good I've got a hazmat module https://github.com/touilleMan/godot-python/tree/master/pythonscript/embedded/godot/hazmat in a project of mine (Python binding on the Godot game engine), however I don't recall if I named it after or before discovering the trio.hazmat module.

On the other hand things like trio.core or trio.kernel are far too generic, beside plenty of libraries are not as well organized as trio and require multiple imports to work, so doing from trio.core import whatever can feel totally legit to a newcomer.

trio.fundation is a bit better (because the user is less likely to make quick assumption like it would for generic names as core), but still focus more on the fact it is a building block than it is a dangerous thing.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/python-trio/trio/issues/476#issuecomment-374776848, or mute the thread https://github.com/notifications/unsubscribe-auth/AAlOaHppb_PsVIUZlvAe9ysQiHrD-1hnks5tgYCjgaJpZM4Svcpu .

[nicoddemus]

What do the non-native English speakers here think about "plumbing"?

Since it was explicitly asked of non-native English speakers, I like "lowlevel" and "plumbing", in that order.

[smurfix]

The problem I have with "hazmat" is that it connotes "dangerous unless you take special precautions", not "dangerous unless you know what you're doing". There's nothing dangerous about most of the stuff in trio.hazmat, you simply need to use them correctly. By contrast, something that requires me to don a hazmat suit simply cannot be handled safely without one.

NB: I do have an atrocious German accent. ;-)

[Shivelight]

I'm not native English speaker, and I think "plumbing" is a good name. Also, the word is quite popular.

I just discovered trio woohoo!

[nwp90]

Given your comment beginning "I want to be able to say things like:...", it seems like what you're saying is that these functions should not be regarded as part of the trio API, but rather as internals that you are free to use if you wish. So trio.internals?

[njsmith]

Looking at this again... definitely of the suggestions so far, trio.plumbing is the one that speaks to me the most. If we didn't also have APIs for working with pipes, then I'd pick it right now :-). I think @ncoghlan's message here is the best so far at capturing what we're going for...

Like, most APIs in trio come to you as pre-assembled, consumer-friendly things with electrical safety marks and a return policy and all that stuff (metaphorically speaking). But sometimes that's not what you want. Sometimes you want to build something new from raw materials. trio.<whatever we call it> is the place where we keep our lathes and soldering irons and table saws and a big pile of oddly-shaped scraps of sheet metal, and you can do all kinds of cool stuff but you'd better take off your jewelry and wear eye protection.

trio.lumberyard? trio.machineshop, trio.makerspace, trio.scrapyard, trio.fab ... trio.mill? trio.some_assembly_required... trio.carpentry, trio.machinery?

Or pushing harder on the construction/plumbing metaphor... trio.wiring? trio.jobsite, trio.hardhats_required_beyond_this_point. (Actually, using one of those images to illustrate the appropriate manual chapter might be good :-))

trio.powertools? trio.workshop?

[dhirschfeld]

Ohhh, a naming bikeshed!

workbench, bare_metal, foundations

...but I'm happy enough with hazmat and if not that then I think the rather pedestrian core or lowlevel are also fine.

[njsmith]

trio.bikeshed

(okay not really)

[smurfix]

lowlevel. Yeah, the machinery thing is cute, but frankly I'd have a somewhat-hard[-hat] time remembering which.

[njsmith]

My problem with lowlevel is that we have a whole section of the manual already talking about the "high level" interface (Stream etc.) versus the "low level" interface (trio.socket)...

[pquentin]

I'm not sure two new cute words (nursery + something else) are within our budget. :)

+1 for plumbing. I don't think any I/O library uses plumbing for something I/O related? (And when I think about pipes, I mostly think about os.pipe().)

Fun fact: the French term for plumbing (plomberie) is used in software development to mean glue: "that's not difficult, you just need to glue those components together" would translate to "rien de compliqué, ce n'est que de la plomberie entre ces composants". But we have many other false friends anyway, so I don't think this should be held against plumbing!

[pquentin]

In the meantime, I think the warning on top of trio.hazmat should explain why we probably don't want to use this module.

[njsmith]

I'm not sure two new cute words (nursery + something else) are within our budget. :)

The problem is that hazmat is already so cute that 70% of people misunderstand it, and the other 30% don't understand it at all :-).

Fun fact: the French term for plumbing (plomberie) is used in software development to mean glue: "that's not difficult, you just need to glue those components together" would translate to "rien de compliqué, ce n'est que de la plomberie entre ces composants".

That's actually fine in English too: "you just need to plumb those components together"...

---

I kind of like trio.powertools – that at least means something to non-native speakers (I hope?), and has the dual connotation of "these are powerful" and "these are maybe a bit scary". Especially if we add some evocative picture to the manual, like this one. Doesn't that capture the feel of using wait_task_rescheduled or run_sync_soon?

[pquentin]

(I was not counting hazmat because it's already used elsewhere.)

+1 for trio.powertools + an evocative picture

[oremanj]

I came to this issue from #832, so it's possible my comments are too late to be helpful, but I'll leave them anyway just in case.

I pretty strongly prefer plumbing to powertools:

• it's a shorter name (I don't mind going longer than the six characters of hazmat, but all the way to the ten of powertools feels like a lot)
• more fundamentally, I don't feel like powertools is an accurate description of all the things in this module? wait_task_rescheduled, permanently_detach_coroutine_object, TrioToken: sure, I totally get it for these. But Task? current_statistics? current_root_task? Current hazmat is both "tools for making new primitives" and "tools for introspection and debugging", and I don't feel like powertools implies inclusion of the latter set, compared to plumbing which I feel can cover both.

[njsmith]

@oremanj I'm not 100% attached to powertools, but I'm pretty reluctant to have "plumbing" and "pipes" right next to each other with completely different meanings.

Is "implementing my own debugger" really not something that you associate with "power tools"?

[sorcio]

Maybe "wiring" as a metaphor works similarly to "plumbing". Or there could be other trades to borrow from.

[njsmith]

trio.drywall trio.grout

[oremanj]

Writing your own debugger is definitely power tools. :-) I don't feel like statistics or instruments or checkpoint_if_cancelled are power tools as much, and Task can be used for fairly simple things like "save the name of the task that's using this resource so you get better error messages on a conflict". It's possible I have a skewed idea of what's in scope for "normal" library users. It's also true that the powertools name sits better with me now than it did when I first saw it, and I'm not going to be notably sad if it gets adopted; it's just not my first choice.

I'm not very worried about the plumbing/pipes conflict, because:

• I agree with @pquentin that I don't think I've ever seen "plumbing" used to refer to IPC pipes
• we don't literally have a trio.pipes submodule
• we don't have public pipe APIs at all right now
• if and when we do have public pipe APIs, they'll probably be limited in scope, maybe even just make_pipe() returning a stream pair and serve_named_pipe / open_stream_to_named_pipe for Windows

But I do understand that it's a possible point of confusion.

wiring seems good, though if I saw it without explanation I wouldn't know what it was. More brainstorm: backplane, switchboard, reactor, guts, rivets, engine, gears, ...

I actually think kernel works pretty well for Curio as a term for this sort of thing, and I don't predict much confusion if we were to borrow it, but I recognize that might be a minority opinion.

[oremanj]

Another idea: trio.loop? (loop like "run loop" or "event loop")

It's short and I don't see anything in current-hazmat that's not reasonably interpretable as "tell the run loop to do something at a low level". The connotations of "normal users don't need this" are less present, and there's the danger that people coming from asyncio with so many normal operations performed via loop.foo() will look in trio.loop for those normal operations. But I think that might be addressable with documentation. "Low-level control of the run loop: trio.loop" or something.

[oremanj]

A friend suggests (and I like) trio.control: you don't always want this level of control, but when you do...

[ncoghlan]

Nudging folks in the direction of talking about async kernels rather than async event loops would actually make a lot of sense to me, as does the idea of having separate low level kernel APIs (akin to making your own syscalls directly against an OS kernel rather than using libc convenience wrappers).

[energizah]

From the above, the criteria describing this submodule seem to be:

So... hmm. Some things it would be good to communicate to users about this submodule (whatever we call it):

• When you're first learning trio, you can ignore it (and lots of people may never need to learn it at all)
• If you're trying to extend trio or implement your own "primitives", you should look at it
• It's a real public API, in the sense that it's documented / supported / stable (or at least, no more unstable than the rest of the library)
• But you may need some expertise and care to use it correctly.

To my ear, this suggests trio.primitives. It tells me everything I need to know.

powertools, by contrast, is confusing -- I don't know if it's more or less high level; if power tools are automated machines that take care of the manual work, that's sort of the opposite of what this module is.

[njsmith]

I just read over this again, and now I'm leaning towards lowlevel. It seems broadly unobjectionable, and the only concrete argument against it was the one I raised about having a "high level" and "low level" I/O layer in the docs... but now I think we need to reorganize that part of the docs anyway, so it's not a big deal :-). And it's the same length as plumbing, shorter than powertools or primitives.

Maybe we'll still put the power tools picture in the docs though :-)

[ncoghlan]

@njsmith I'm guessing https://trio.readthedocs.io/en/latest/reference-io.html#low-level-networking-with-trio-socket is the specific section you're concerned about?

If you were to change that to refer to the "Socket API layer" and the "Stream API layer", you'd end up with the stack:

* `trio.streams` - connection oriented, mostly TCP
* `trio.socket` - everything else you might normally want to do with a network socket
* `trio.lowlevel` - low level plumbing and power tools

(Some currently top level APIs would need to be grouped under streams for that to work fully, but even without that code level structure, you could use the framing in the docs)