mbruzek / docs

Juju documentation, found at http://juju.ubuntu.com/docs
1 stars 1 forks source link

SoC vs. Juju Dev Docs #41

Open justcfx2u opened 8 years ago

justcfx2u commented 8 years ago

I was asked by @chuckbutler to give some feedback on: https://s3.amazonaws.com/juju-dev-files/docs/en/developer-getting-started.html

I trust this is the correct place. :)

With apologies up-front, this "issue" is a bit of a stream-of-consciousness. I fully expect to be told to break these into individual issues, but for this first iteration it might be reasonable to collect everything into one bin until it can be sorted.

_I feel as though the documentation isn't written for the ADHD-friendly._

What I mean by this is there are almost immediately in-line links. Following those leads to more and more and more documentation, which makes it easy to lose focus/direction/purpose.

I would like to experiment with having links be as footnotes, or perhaps not links at all! Perhaps some note to defer to 'Related Links' at the bottom.

A practical example is the very first link: it encourages you to have Juju installed. But the installation document has the user attempting to bootstrap an environment (which defaults to AWS), but information about using the local provider is a couple sentences later (in the dev doc, completely omitted from the installation doc). It's this sort of deep-distraction that makes things a bit hard to follow.

This might be solved by having the 'installed Juju' and 'tested your setup' links (this opinion applies to all other links in the document as well) be atomic. For example, if I clicked 'installed Juju' I might go and follow it to the point where Juju is installed... but not bootstrapped or configured. But, not knowing WHERE to stop as a newbie before returning to the "Getting Started Developing Charms" document has caused me some grief. A suggestion? Split off bootstrapping into a 'next step'... perhaps stopping even before the 'Configuring' section on the installation page, and creating a break: "After this step, Juju is installed! Continue to the Next Step: Configuring Juju", which itself might mention:

"Juju needs to be configured before it can be used. Juju defaults to using Amazon EC2, but for testing and development you may wish to use a Local Provider such as LXC if you are working on Linux, and [???] if on Mac or Windows." 

Something to that effect. Just enough information (and cross-references) to keep a newbie's attention focused without getting lost in endless documentation branches.

_Prerequisites and Tools_

It is not immediately obvious what the relation between Charm Tools and Charm Helpers is. I feel as though the subject is switched very abruptly, and -- because of similar names -- this section is really confusing.

For example, I might add something like: "After charm-tools is installed, the juju command-line utility will be extended with additional commands to make it easy to create, build, and fetch charms. See juju charm help for more information. (ex. juju charm proof -h)."

That would give a practical example of what installing the Charm Tools does for the user and where to get self-help. It also gives the implication that this step is done. The documentation can then move on to talking about Charm Helpers, which is an entirely different topic -- but currently does not have a separate (sub-)header. Maybe one should be added?

More documentation-tree madness linking to Charm Helpers and its associated documentation. I would almost rather not have any links here, particularly since Charm Helpers does not appear to be used directly anywhere else in the document (as it leads into using 'layers', instead of an example based on creating a charm from scratch using Charm Helpers).

Talking about 'Once the charm is complete....' seems off topic for "Prerequisites and Tools". Perhaps move that paragraph to the bottom of the document?

_Designing your charm_

If I have an idea of my first charm, the document seems to have me going to start drawing when I have no idea of the anatomy of a charm. While there is a fine diagram provided, at this point in reading I may not know enough about the components and terminology, nor workflow (e.g., hooks) to design anything effectively. Perhaps this is a simple understanding and a re-titling of "Designing your charm" -> "Design of our example charm" might keep the user in a mode of 'okay, follow the tutorial, worry about my OWN design later'. And with that, perhaps change references from 'your' to 'our', implying the document is building something mutual, that the reader is guided on a pre-set path, rather than encouraged to try applying unique designs immediately.

_Writing your Charm_

Again, as a guide... maybe "Writing the Charm".

An important consideration for me here is... what if I don't _want_ to use layers? At this point I am obsessed about the immediate payoff to make sure I'm doing things correctly before getting more complex. Before... adding layers, as it were.

I feel as though I need to be able to write a 'Hello World!' charm in like 3 lines of code. The documentation should provide me a minimal set of files and configurations (in both bash and Python formats), using as few tools and dependencies as possible (though perhaps with Charm Tools installed), something which will deploy a charm that creates a file on the filesystem /tmp/helloworld for example and can terminate. For a daemon example (since Juju appears to be service/daemon oriented), perhaps a Python-based tcp echo server. Something as bare-bones as possible.

In essence, I don't think I know enough anatomy about a charm, what set of minimum requirements it needs, and how it interacts with its environment before trying to get feet wet with actually creating one. If this is covered in other parts of the documentation, here a link WOULD be handy! But I suspect, since this is the 'developer-centric' portion of the documentation, it may be worthwhile discussing this from a developer's perspective.

There appears to be a typo, at minimum. And perhaps not quite enough has been 'export'ed.

export JUJU_REPOSITORY=$HOME/charms
LAYER_PATH=$JUJU_REPOSITORY/layers
INTERFACE_PATH=$JUJURE_REPOSITORY/interfaces
mkdir -p $JUJU_REPOSITORY/layers
cd $JUJU_REPOSITORY/layers

should maybe be:

export JUJU_REPOSITORY=$HOME/charms
export LAYER_PATH=$JUJU_REPOSITORY/layers
export INTERFACE_PATH=$JUJU_REPOSITORY/interfaces
mkdir -p $JUJU_REPOSITORY/layers
cd $JUJU_REPOSITORY/layers

_note in particular the INTERFACEPATH variable typo fix.

Interestingly, when trying to execute the demo deploy of vanilla, it fails pretty hard:

  vanilla:
    charm: local:trusty/vanilla-0
    exposed: true
    service-status:
      current: error
      message: 'hook failed: "install"'
      since: 08 Dec 2015 23:29:35Z

2015-12-08 23:29:28 INFO install apache2_switch_mpm Switch to prefork
2015-12-08 23:29:28 INFO install  * Restarting web server apache2
2015-12-08 23:29:29 INFO install AH00557: apache2: apr_sockaddr_info_get() failed for hjones-local-machine-2
2015-12-08 23:29:29 INFO install AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1. Set the 'ServerName' directive globally to suppress this message
2015-12-08 23:29:30 INFO install    ...done.
2015-12-08 23:29:30 INFO install apache2_invoke: Enable module php5
2015-12-08 23:29:30 INFO install  * Restarting web server apache2
2015-12-08 23:29:32 INFO install AH00557: apache2: apr_sockaddr_info_get() failed for hjones-local-machine-2
2015-12-08 23:29:32 INFO install AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1. Set the 'ServerName' directive globally to suppress this message
2015-12-08 23:29:33 INFO install    ...done.
2015-12-08 23:29:33 INFO install Processing triggers for libc-bin (2.19-0ubuntu6.6) ...
2015-12-08 23:29:33 INFO install  * Stopping web server apache2
2015-12-08 23:29:34 INFO install  *
2015-12-08 23:29:34 INFO install Site 000-default disabled.
2015-12-08 23:29:34 INFO install To activate the new configuration, you need to run:
2015-12-08 23:29:34 INFO install   service apache2 reload
2015-12-08 23:29:34 INFO juju-log FetchHandler charmhelpers.fetch.bzrurl.BzrUrlFetchHandler not found, skipping plugin
2015-12-08 23:29:34 INFO juju-log FetchHandler charmhelpers.fetch.giturl.GitUrlFetchHandler not found, skipping plugin
2015-12-08 23:29:34 INFO juju-log Making dir /var/lib/juju/agents/unit-vanilla-0/charm/fetched root:root 755
2015-12-08 23:29:35 INFO install Traceback (most recent call last):
2015-12-08 23:29:35 INFO install   File "/var/lib/juju/agents/unit-vanilla-0/charm/hooks/install", line 28, in <module>
2015-12-08 23:29:35 INFO install     main()
2015-12-08 23:29:35 INFO install   File "/usr/local/lib/python3.4/dist-packages/charms/reactive/__init__.py", line 64, in main
2015-12-08 23:29:35 INFO install     bus.dispatch()
2015-12-08 23:29:35 INFO install   File "/usr/local/lib/python3.4/dist-packages/charms/reactive/bus.py", line 397, in dispatch
2015-12-08 23:29:35 INFO install     _invoke(hook_handlers)
2015-12-08 23:29:35 INFO install   File "/usr/local/lib/python3.4/dist-packages/charms/reactive/bus.py", line 388, in _invoke
2015-12-08 23:29:35 INFO install     handler.invoke()
2015-12-08 23:29:35 INFO install   File "/usr/local/lib/python3.4/dist-packages/charms/reactive/bus.py", line 269, in invoke
2015-12-08 23:29:35 INFO install     self._action(*args)
2015-12-08 23:29:35 INFO install   File "/var/lib/juju/agents/unit-vanilla-0/charm/reactive/apache.py", line 24, in install
2015-12-08 23:29:35 INFO install     install_site(name, site)
2015-12-08 23:29:35 INFO install   File "/var/lib/juju/agents/unit-vanilla-0/charm/reactive/apache.py", line 59, in install_site
2015-12-08 23:29:35 INFO install     fetch.install_remote(dest=dest, **site['install_from'])
2015-12-08 23:29:35 INFO install   File "/usr/local/lib/python3.4/dist-packages/charmhelpers/fetch/__init__.py", line 388, in install_remote
2015-12-08 23:29:35 INFO install     installed_to = handler.install(source, *args, **kwargs)
2015-12-08 23:29:35 INFO install   File "/usr/local/lib/python3.4/dist-packages/charmhelpers/fetch/archiveurl.py", line 148, in install
2015-12-08 23:29:35 INFO install     self.download(source, dld_file)
2015-12-08 23:29:35 INFO install   File "/usr/local/lib/python3.4/dist-packages/charmhelpers/fetch/archiveurl.py", line 116, in download
2015-12-08 23:29:35 INFO install     raise e
2015-12-08 23:29:35 INFO install   File "/usr/local/lib/python3.4/dist-packages/charmhelpers/fetch/archiveurl.py", line 112, in download
2015-12-08 23:29:35 INFO install     dest_file.write(response.read())
2015-12-08 23:29:35 INFO install TypeError: must be str, not bytes
2015-12-08 23:29:35 ERROR juju.worker.uniter.operation runhook.go:107 hook "install" failed: exit status 1

I'll have more to contribute (read: fuss over) once this issue begins iterating.

mbruzek commented 8 years ago

@justcfx2u Thank you for the detailed review and feedback on the docs!

The over linking is my fault. When I write I err on the side of too much information. I now recognize that the links may not be helpful for many types of people. I will work on making this better, and like your idea of creating a link section at the end or something like that.

We did not have a hello world example charm (written in reactive). The vanilla example should not fail. We will fix that issue and look for a simpler example. We like the vanilla charm because it has one relation to a database, charms that have no relations are not really useful in the ecosystem. So for the balance of education and usefulness vanilla won, but to your point we would like to have a simple example that gives the readers something to accomplish.

Other authors already fixed the typos on the environment variables and we will work on making these docs better and look into all your suggestions here.

Thanks again for the time it takes to read and comment on the documentation. We want to improve and can only do so with impartial, and detailed comments such as this. I really appreciate the feed back and aim to do better in the future!

justcfx2u commented 8 years ago

Thank you for such positive feedback. As you seem to be genuinely interested in more, I tackled the next section. Standard disclaimer: These are impulsive thoughts, recorded at the time of initial reading by a newcomer. I may be wrong or misdirected or assume the wrong things. But it is my hope that they will be useful in this raw format. Please do with these suggestions as you will.

Event Cycle

Keep just the Charm Hooks link. :) Nix the distracting links as bullet points.

_Deployment Events by example_

"When deploying a charm..."

At this point in reading, I'm not sure what the Charm lifecycle stages are. I want to be able to look at a small flowchart. I don't even necessarily care what goes into it, or at what detail level. I just sort of want to visualize how a charm is deployed, lives, and is cleaned up.

Maybe have a corresponding 'development' lifecycle outline/flowchart, everything from git cloning a layer to basic steps common to all layers (e.g., incl. proofing a charm), adding it to a local/remote repository, through to (and referencing) the remainder of the lifecycle (deploy, live, cleaned up/undeployed).

"... basic series of hooks executed in the initial deployment cycle".

The word 'initial' implies that there can be a subsequent deployment cycle for the same charm in some way. But aren't the hooks listed for EACH charm deployment, no matter what? It is also not explicitly said if the listed hooks are GUARANTEED to run, and if they are guaranteed to be run in that order. It is easy to assume all of the above, but I don't want to guess. :)

_Configuration Events by example_

"vanilla charm", maybe make this a link to the previous section? The vaguarities of English might make 'vanilla charm' have alternate meanings unless it is crossreferenced.

"[...] Vanilla charm (per our previous example)" might shape up the sentence a bit, especially if it is linked.

Perhaps also give an example of what exactly would be needed to "change the configuration of the vanilla charm". When reading this I want to go try it right away, to trigger this config-changed hook. I also want to know how to observe that the hook has actually been triggered; where would I go, and what would I need to do to see the logs/ensure that this action occurred?

:%s/vanilla charm/Vanilla charm/g :)

"When a relation is added from the vanilla charm to a database charm, the first event is database-relation-joined [...]"

WHY is it specifically _database_-relation-joined? A reminder of exactly the syntax used to create the relation would be well-paired with a very terse explanation on how the system determines what a relation should be.

As of this reading, my understanding of the situation may be that:

charm add-relation mariadb vanilla

Hypothetical explanation/clarification follows:

This creates a 'relation' between the deployed MariaDB and Vanilla charms. The 'charm' command examines each charm provided and attempts to satisfy all relations defined within the charm. Vanilla requires a 'db:mysql' interface, and the MariaDB charm provides a 'db:mysql' interface, so the charm command will connect them together in what is known as a relation.

[Describe also how, if the relation is defined as 'db:mysql', why the hook name is called 'database-relation-joined'. This might dip into the 'anatomy of a charm'/filestructure which seems to be under consideration for being written.]

Also, is it possible to have multiple providers vs. required interfaces be satisified with a single command?

For example,

charm add relation [ Charm A ] [ Charm B ]

[ Charm A ] provides: [foo]
[ Charm A ] provides: [bar]
[ Charm A ] requires: [baz]
[ Charm A ] requires: [quux]

[ Charm B ] requires: [foo]
[ Charm B ] provides: [baz]

Will 'charm add-relation charm-a charm-b' satisfy both 'foo' and 'baz' connections at the same time? Is this even possible? What is the final state of each charm, knowing that the relations are not fully satisfied yet? It is easy to assume that a 'Charm C' will need to be deployed, one which ideally provides 'quux' and 'bar' in order to complete the mesh.

Tempted to follow the link 'creating a charm in the reactive framework' but I don't think I know enough at this juncture... :)

"We will preserve the install and config-changed hook(s) in most layers." -- what? Preserve? From what destructive force? We're talking about layers again, but now I'm thinking layers are potentially destructive to other layers in some way, and now I'm not sure how layers relates to Reactive, exactly.

"The relation hooks are generated when using an interface layer, during the charm build process."

Maybe:

[name]-relation-[event] scripts are created when using _(how? define 'using')_ an interface layer _(first time these words have been used, what is an interface layer compared to any other layer?)_ *during the charm build process ..."

It seems that Charm Layers and Interface Layers are covered _after_ this section, so baseline terminology has not yet been established. If the Table of Contents will not be rearranged, maybe a hover-over/dotted underline/tooltip, or sidebar to give a 1-sentence brief about these may be appropriate.

"The remainder of actions taken, will be directed with artificial states, set by the layers author." -- unnecessary commas?

_Handling reactive states_

"When charming..." has "charming" been defined? It is a casual term, which is fine. It seems to be the 2nd usage of the term, but a formal "using the Juju charm framework/ecosystem/writing Juju charms (commonly referred to as simply 'charming')" or something might do well here.

Is 'reactive framework' really 'Reactive Framework' or 'Reactive framework' in disguise? :)

"When charming with the reactive framework, it’s possible to use developer created states and boolean logic to run code when the states represent something meaningful to your deployment. Such as when a database is connected, and when it is available, or the primary workload becoming available for use. These are additional “events” that are abstracted from the hook event cycle outlined above."

might become:

"It is possible for a charm's developer to create custom states when using the Reactive Framework. These states can be triggered simply, as in the example below. Some examples of useful custom states might include:

  • when a database is connected
  • when a database is available for use
  • when a dependency has been installed
  • when the primary workload/function/daemon provided by the charm is available for use"

"When building a charm using layers, that implements an interface layer - these layers set states for the code to react to. This allows the charm author to focus solely on the states rather than the traditional method of putting all the code in the relationship hook sequence."

might become:

"Charm authors are encouraged to use states instead of relationship hooks whenever possible. [because why?]" Is it simpler? More reliable?

*"Interface layers commonly set states which can be reacted to by using the @when decorator. This allows the charm author to focus on [something good and nice goes here] as opposed to [contrasting statement]".

What are the relationships between 'traditional, hook sequences' versus states? This document is currently assuming a priori knowledge of how things were done without layers. This is not the case as a first-time reader trying to learn about 'Charming 2.0'.

"In the vanilla layers.yaml file, we include “interface:mysql"."

Code block snippet, please! I'm too lazy to alt-tab to my terminal, I just want to keep reading... For that matter, I would love an introduction to exactly what information can/should be included in the respective YAML files. (See also: "anatomy of a charm" I've been begging for...)

"This relates directly to the “database” relation defined in metadata.yaml - with this inclusion in layers.yaml your charm will pick up the mysql interface layer from the layers webservice."

I only sort-of understand this. Looking between this sentence and the .yml files...

requires:
  database:
    interface: mysql

"As 'interface. mysql' is found in metadata.yml, the parent key (named 'database') will thus trigger the relations beginning with "database", (e.g., database-relation-joined, database-relation-departed, and so-on.)"

"It’s important to note that the instructions for this particular interface ..." What instructions, exactly? Code? The code examples given? Or the code checked out by the git repo? Or the documentation I'm presently reading?

"Interfaces are unique to the author’s implementation, and any states set will vary from interface to interface."

Consider moving this sentence WAY up toward the front of the document. :)

"These states are documented in the interface layer repository README.md file."

might become:

"States implemented by a given interface layer are typically documented in the layer's README.md file."

justcfx2u commented 8 years ago

_Charm Layers_

What are layers?

Layers are collections of related files within a charm that are intended to be reusable. Layers are structured to work together in order to save time and increase consistency during the charm authoring process. Layers can be as broad as an entire charm, or as specific as a single interface or dependency. Charm authors may assemble their charms from these prewritten building blocks freely; using layers is considered best practice.

Why build from layers?

Layers provide many advantages over building a charm completely from scratch. Without layers, a charm author must manually create each hook and implement each side of the interface needed for each relation the charm requires or provides, and other such tasks. In contrast, the speed and consistency of charm development is improved by using layers, which are written to provide functionality to your charm in a clean and flexible way.

Types of Layers

  • Base/runtime
  • Interface
  • Charm layers

Each of these has a distinct role, and it’s important to understand how a charm should be broken up into these types of layers. Generally, a charm will contain one base layer, one charm layer, and one or more interface layers, but it is possible that a charm might include more than one base layer, as well."*

_Base, or Runtime, Layers_

"The most basic example is just that, layer-basic. It provides nothing more than the minimum needed to effectively use layered charms: charms.reactive, charmhelpers, and the skeleton hook implementations that call into the reactive framework. However, the most useful base layers are actually a type of runtime layer. For example, layer-apache-php provides Apache2 and mod-php, as well as mechanisms for fetching and installing a PHP project within that runtime."

Delete "However," and split into a new paragraph:

The most basic example is just that, layer-basic. It provides nothing more than the minimum needed to effectively use layered charms: charms.reactive, charmhelpers, and the skeleton hook implementations that call into the reactive framework.

The most useful base layers are actually a type of runtime layer. For example, layer-apache-php provides Apache2 and mod-php, as well as mechanisms for fetching and installing a PHP project within that runtime.

_NOTE:_ Dead link, 'layer-basic' yields GitHub 404. Was it renamed to reactive-base-layer?

"Base layers can be written in any language, but must at a minimum provide[1] the reactive framework that glues layers together, which is written in Python. This can be done trivially by building the base layer off of layer-basic."

[1] "Provide"? If I were attempting to write in bash, how would I "provide" the reactive framework?

"Interface layers are perhaps the most misunderstood type of layer, and are responsible for the communication that transpires over a relation between two services. This type of layer encapsulates a single “interface protocol” and is generally written and maintained by the author of the primary charm that provides that interface. However, it[1] does cover both sides (provides and requires) of the relation and turns the two-way key-value store that are Juju relations under-the-hood into a full-fledged API for interacting with charms supporting that interface."

[1] "It"? Maybe, "Interface layers do cover..."?

"It is important to note that interface layers _do not_ actually implement either side of the relation. Instead, they are solely responsible for the _communication_ that goes on over the relation, relying on charms on either end to decide what to do with the results of that communication."

It is important to note that interface layers do not actually implement either side of the relation. They are solely responsible for the communication that goes on over the relation, and instead rely on charms on either end of the interface to decide what to do with the results of that communication.

Add emphasis.

"Interface layers currently must be _written_ in Python and extend the ReactiveBase class, though they can then be _used_ by any language using the built-in CLI API.[1][2]"

[1] Tell me more! Link to a Python (preferred) example of using an interface. Link to a CLI API example.

[2] Maybe spell out CLI -> "command line" and mention which program/utility actually does this.

S3 Access Denied when attempting to click Developing Interface Layers: (https://s3.amazonaws.com/juju-dev-files/docs/en/developers-interface-layers.html)

Lowercase 'reactive' is as confusing as 'vanilla'. Change it to be a proper noun (or merely a disambiguated one) instead of regular-looking adjective?

"Building on base and interface layers, charm layers are what actually get turned into charms. This is where the core logic of the charm should go, the logic specific to that individual charm. This layer brings together all the pieces needed to create the charm. It is where most of the charm’s config options will be defined, and where the reactive (framework) handlers that do the specific work of the charm will go. It will need to contain the charm’s README, copyright, icon, and so on."

"Charm layers should be the most common type of layer, and is what most charm authors will be dealing with. However, the goal is to keep them[1] focused on just that specific charm’s logic and needs, and to push any commonality into an appropriate base layer. Charm layers should contain as little boilerplate as possible."

[1] "Them"? Charm authors or interface layers?

"Charm layers can be written in any language, and there are helpers for writing them in Bash."

Add link to examples? What if I'm not using bash or python? How do I get anything done?

"For languages other than python or bash, [do things] using [this utility/mechanism/command-line (and where/how to use it within a charm)]."

_"States are synthetic events that are defined by the layers author. States allow for the layer, or related layers to subscribe to these [events][1] and take action only when appropriate. Consider the example illustrated in the Getting Started[2] guide. [The apache.available state][3] is set from the apache-php layer. Any layers built on top of the apache-php layer can subscribe to this state with a @when decorator[4] to take action only after the Apache service has been started. subsequently the @whennot decorator has also been made to assist guarding against running code when a state has been set, which lends itself nicely to idempotent behavior."

[1] Edit for consistency.

[2] "Getting Started" link doesn't lead where I think it should lead.

[3] Clearer/more explicit.

[4] Include footnote/reference for other languages. How do they subscribe to states? Or say explicitly that they cannot.

"Charmers can set synthetic states:"

Maybe 'custom' states? Using the word 'synthetic' makes me want to go into a meta discussion about whether or not programming as a whole is synthetic vs. real. Or perhaps define the difference between a 'synthetic state' and a '... natural(?) state' if such things exist.

justcfx2u commented 8 years ago

_Writing a layer by example_

Is developer-layer-example.html linked to from the Table of Contents?

Is there a missing discussion on charm proofing and building? There also seem to be missing instructions on checking out a layer and getting set up with it before diving into editing files.

mbruzek commented 8 years ago

Hello @justcfx2u. Thank you for all the feedback! This is a treasure trove of data for us to work through to improve our docs. I took a shot at incorporating your feedback on the developer-getting-started document here: https://github.com/mbruzek/docs/commit/283d8fd073a5b2d5bbd438f7ae083264dff1a1d3

I hope this is an improvement and resolves some of your issues. Please take a look and let me know.

marcoceppi commented 8 years ago

@justcfx2u Thanks for this feedback, it's tremendous and extremely helpful! Over the next week I'll work on breaking down your feedback into actionable issues on the repo to help identify tasks that people can work on. This feedback will make sure when we land these docs onto jujucharms.com they'll be an awesome experience for everyone