See issue #17 first, for background reading. I'm moving forward on the assumption that we're going to use the technologies I suggest there - JSON for a config file, SQLite3 for a database engine, and Amazon S3 for blob storage and distribution - but I could be talked out of any of these.
Make a command-line tool to assist in the creation of a new mirror. I'm envisioning a multi-command executable that takes a subcommand name as its first argument, with the first subcommand to be implemented being "config", which simply asks questions, pokes at the systems it's ostensibly connecting to a little, and spits out a config file.
The config file should at the very least contain S3 credentials and bucket identifier. The credentials are two fields, an access key and a secret. (A "bucket" is the top-level container of stuff in S3.)
So I'm in a hurry now and want to get all these thoughts down, so I'm going to just describe the flow of the steps I envision "config" doing. I originally thought the command line might be suitable, but now that I see how many steps there are, I'm thinking something more like the "dialog" program, which is that great set of tools that Debian and the Linux kernel makefiles both use for graphical terminal-based configuration.
The reason I think something interactive of this nature is necessary is because I was trying to document these steps and they're fairly error-prone. Plus, Amazon's console is subject to change; its API is not.
"This command will assist you in configuring a new mirror of Codex, a Haskell software-distribution system. You probably don't need to run your own mirror, unless you have code which you wish to publish internally but not to the world at large. I'll assume since you haven't ^Ced out of the program that you wish to continue..."
"First, do you wish to set up the first server in a federation of servers, or a mirror of an existing federation?"
(User chooses first in a federation.)
"Okay. You will need to have an existing Amazon Web Services account. This tool can create the resources it needs therein, which consist of an S3 bucket, an IAM group, and an IAM user with an access key. The tool can also utilize existing resources, if you wish to create them manually. If you wish to go with the automated solution, you will need to supply an access key and secret which will not be stored, only used to create the credentials which will actually be used. Which would you like to do - automated, or manual?"
(User chooses automated.)
"I'm pleased to hear that." [Software should be polite! :D] "What is your access key?" (User does so.) "And your secret?" (User does.) "Checking - okay, these are valid. If there is an existing IAM group you wish to use for the machines in this federation, please select it now; otherwise, just choose "create" to create a new one. The following are the IAM groups extant: ..."
(User chooses "create".)
"Okay. Do you have a preference for the name of this group? If so, specify it now. If not, I will use "codex"."
(User chooses the default.)
"Good. I will use the group "codex" as the IAM group to create my user in. Or have you already created the user? There are no IAM users in the codex group, and fifteen users overall, as follows: ..."
(User chooses "create".)
"I notice that this computer's hostname is "silly-cat-joke". Would you like the user to be named that as well, or do you have a preference, or should it be set to something arbitrary?"
(User chooses "silly-cat-joke".)
"Good. Next, would you like to create an S3 bucket, or use an existing one? There are 3 extant buckets, as follows: ..."
(User chooses "create".)
"What should it be called? If you have no preference, I will use "codex"."
(User chooses the default.)
"All right. The bucket has been created." [Conveniently, we don't need to create directory structure; it doesn't really exist.] "I have also granted the "codex" group the appropriate permissions on it."
"We need to know where to keep our local database. The default is /var/lib/codex/database."
(User chooses the default.)
"Okay. The new config file is written to config.json in the current working directory; move it to wherever your init.d script will be able to find it. Note that this file contains precious information, so don't casually delete it to start over; doing so will leave inaccessible resources that require cleanup work by the federation administrator."
See issue #17 first, for background reading. I'm moving forward on the assumption that we're going to use the technologies I suggest there - JSON for a config file, SQLite3 for a database engine, and Amazon S3 for blob storage and distribution - but I could be talked out of any of these.
Make a command-line tool to assist in the creation of a new mirror. I'm envisioning a multi-command executable that takes a subcommand name as its first argument, with the first subcommand to be implemented being "config", which simply asks questions, pokes at the systems it's ostensibly connecting to a little, and spits out a config file.
The config file should at the very least contain S3 credentials and bucket identifier. The credentials are two fields, an access key and a secret. (A "bucket" is the top-level container of stuff in S3.)
So I'm in a hurry now and want to get all these thoughts down, so I'm going to just describe the flow of the steps I envision "config" doing. I originally thought the command line might be suitable, but now that I see how many steps there are, I'm thinking something more like the "dialog" program, which is that great set of tools that Debian and the Linux kernel makefiles both use for graphical terminal-based configuration.
The reason I think something interactive of this nature is necessary is because I was trying to document these steps and they're fairly error-prone. Plus, Amazon's console is subject to change; its API is not.
"This command will assist you in configuring a new mirror of Codex, a Haskell software-distribution system. You probably don't need to run your own mirror, unless you have code which you wish to publish internally but not to the world at large. I'll assume since you haven't ^Ced out of the program that you wish to continue..."
"First, do you wish to set up the first server in a federation of servers, or a mirror of an existing federation?"
(User chooses first in a federation.)
"Okay. You will need to have an existing Amazon Web Services account. This tool can create the resources it needs therein, which consist of an S3 bucket, an IAM group, and an IAM user with an access key. The tool can also utilize existing resources, if you wish to create them manually. If you wish to go with the automated solution, you will need to supply an access key and secret which will not be stored, only used to create the credentials which will actually be used. Which would you like to do - automated, or manual?"
(User chooses automated.)
"I'm pleased to hear that." [Software should be polite! :D] "What is your access key?" (User does so.) "And your secret?" (User does.) "Checking - okay, these are valid. If there is an existing IAM group you wish to use for the machines in this federation, please select it now; otherwise, just choose "create" to create a new one. The following are the IAM groups extant: ..."
(User chooses "create".)
"Okay. Do you have a preference for the name of this group? If so, specify it now. If not, I will use "codex"."
(User chooses the default.)
"Good. I will use the group "codex" as the IAM group to create my user in. Or have you already created the user? There are no IAM users in the codex group, and fifteen users overall, as follows: ..."
(User chooses "create".)
"I notice that this computer's hostname is "silly-cat-joke". Would you like the user to be named that as well, or do you have a preference, or should it be set to something arbitrary?"
(User chooses "silly-cat-joke".)
"Good. Next, would you like to create an S3 bucket, or use an existing one? There are 3 extant buckets, as follows: ..."
(User chooses "create".)
"What should it be called? If you have no preference, I will use "codex"."
(User chooses the default.)
"All right. The bucket has been created." [Conveniently, we don't need to create directory structure; it doesn't really exist.] "I have also granted the "codex" group the appropriate permissions on it."
"We need to know where to keep our local database. The default is /var/lib/codex/database."
(User chooses the default.)
"Okay. The new config file is written to config.json in the current working directory; move it to wherever your init.d script will be able to find it. Note that this file contains precious information, so don't casually delete it to start over; doing so will leave inaccessible resources that require cleanup work by the federation administrator."