Release: 'Erlvolt 0.3.3/beta'
Author: H. Diedrich
Production: Eonblast Corporation
Copyright: (c) 2013 VoltDB, Inc
Licence: MIT
Date: 4 Feb 2013
VoltDB: 2.8, 3.0 Erlang: R15B03, R16 RC
This is an Erlang VoltDB driver provided by Eonblast. It is [easy][Samples] to use but provides strong [connection pooling][Adding_a_Pool] and a broad array of [options][options]. It is optimized for a central node architecture and super high velocity OLTP.
While databases generally can be accessed via ODBC in Erlang, you should see better performance when using a driver like Erlvolt. For [samples][Samples], [API description][Usage] and fine tuning [options][Options] see below.
Erlvolt was initiated and [created][History] by Eonblast, in this second incarnation with sponsorship by VoltDB.
This is the master branch. Should you run into problems, please report them by opening an issue at github and try if they go away by checking out the 'stable' branch. Thank you.
Download: https://github.com/Eonblast/Erlvolt/archives/master
Issues: https://github.com/Eonblast/Erlvolt/issues
Repository: https://github.com/Eonblast/Erlvolt
$ git clone git://github.com/VoltDB/voltdb.git voltdb
For download via the VoltDB website, see below.
$ git clone git://github.com/VoltDB/voltdb-client-erlang.git erlvolt
For the git repository of the newest version, see below.
Makefile build rules
README.html this file
README.md or this
doc:
BENCHMARK-README.html Benchmark How To
BENCHMARK1.html Report of an Amazon EC2 benchmark
BENCHMARK1.md markdown source
BENCHMARK2.html Report of another Amazon EC2 benchmark
BENCHMARK2.md markdown source
CHANGES.html Changes between versions of this package
CHANGES.md markdown source
LICENSE LICENSE, ASCII
ebin:
empty git placeholder
etc:
ct_default.css styles for TC sample in README.html and md
grep grep batch over all erl, hrl, md sources
include.mk Makefile include
markdown.lua Lua markdown script to create html from md
markedoc.sed sed script to create edoc from md
replace sed in-place replace over all erl, hrl, md sources
bench:
Makefile build rules AND SAMPLE
README.html Benchmark How To
README.md markdown source
bench.config host config and start sync
bench.erl benchmark module
benchstart multi-vm bench start
test:
basics_SUITE.erl basic functionality suite
environment_SUITE.erl environment and setup tests
examples:
Makefile build rules
hello.erl barebones hello world
hello_plus.erl slightly more robust hello world
parallel.erl parallel hello word sample
voter.erl VoltDB staple voter sample
include:
erlvolt.hrl higher level driver include
erlvolt_wire.hrl wire protocol level driver include
priv:
empty git placeholder
src:
Makefile
erlvolt.erl main driver module
erlvolt_conn.erl socket connection
erlvolt_profiler.erl optional statistics
erlvolt_wire.erl protocol level bit wrangling
erlvolt.app.src template for app file
erlvolt_app.erl application behavior
erlvolt_conn_mgr.erl connection manager
erlvolt_sup.erl supervisorsThis is a hello world program. Follow the steps below to try it out.
-module(hello).
-export([run/0]).
-include("erlvolt.hrl").
run() ->
crypto:start(),
application:start(erlvolt),
erlvolt:add_pool(hello_pool, [{"localhost", 21212}]),
erlvolt:call_procedure(hello_pool, "Insert", ["Hej", "världen", "Swedish"]),
Result = erlvolt:call_procedure(hello_pool, "Select", ["Swedish"]),
Table = erlvolt:get_table(Result, 1),
Row = erlvolt:get_row(Table, 1),
Hello = erlvolt:get_string(Row, Table, "HELLO"),
World = erlvolt:get_string(Row, Table, "WORLD"),
io:format("~n~s ~s!~n~n", [Hello, World]),
erlvolt:close_pool(hello_pool).We'll be coming back to running this on your machine in a minute. Before we do, let's look at the basic building blocks first:
In VoltDB, in production, you are using stored procedures, written in Java. For more information about that please check out the [VoltDB docs][]. Such stored procedure is invoked like this using Erlvolt:
Result = erlvolt:call_procedure(hello_pool, "Select", ["Swedish"]).This is a snychronous, blocking call that returns the result data. The first parameter to the function is the pool ID atom, the second the name of the stored procedure that we want to invoke, as a string. The last a list of parameters we want to send to the SP.
Note that the stored procedure goes by the name 'Select' and that is the reason why 'Select' is the second parameter above. The query is SELECT HELLO, WORLD FROM HELLOWORLD WHERE DIALECT = ?; as you can see in voltdb/doc/tutorials/helloworld/Select.java.
For the exact spec, see below, [Usage][]. Regarding the 'pool', also see below.
In Erlang, you would usually be executing synchronous calls from many parallel worker processes, rather than asynchronous ones. Due to the architecture of Erlang this will amount to asynchronous action. But the driver can be used to make asynchronous calls, too. In which case the execute function returns as fast as possible, and the result is coming in via message passing, as soon as it becomes available from the VoltDB server:
erlvolt:call_procedure(hello_pool, "Select", ["Swedish"], [asynchronous]),
receive Result -> Result end.Regardless of how you called, you extract the actual values from the response data structure with getter functions. The response data structure contains the complete response; the getter functions point into it, there is no iteration or stream.
Table = erlvolt:get_table(Result, 1),
Row = erlvolt:get_row(Table, 1),
Hello = erlvolt:get_string(Row, Table, "HELLO"), Erlvolt uses a sophisticated connection pooling mechanism. You can have multiple connections in each pool, which will usually be one connection to each node in the VoltDB cluster. And you can have multiple pools which allows you to access multiple clusters. The pools have a queue each to cushion access peaks, protecting the server from overload.
erlvolt:add_pool(hello_pool, [{"localhost", 21212}]),Let's run the hello world sample from above. We'll need a VoltDB server for that. This walkthrough assumes a Unix or Linux OS.
You can clone the newest community edition from:
$ git clone git://github.com/VoltDB/voltdb.git voltdb Or download the newest VoltDB from http://voltdb.com/community/downloads.php and unpack, e.g.:
$ tar -zxvf voltdb-3.0.tar.gz
$ mv voltdb-3.0 voltdbThe Hello, World! tutorial example comes with every VoltDB distribution. It builds and runs out of the box, on localhost. (Note that it is NOT in the examples/ directory, but in doc/tutorials/):
$ cd voltdb/doc/tutorials/helloworld
$ ./run.shGet the official Erlvolt release from the VoltDB repository:
$ git clone git://github.com/VoltDB/voltdb-client-erlang.git erlvolt
Or get the newest version from from https://github.com/Eonblast/Erlvolt, e.g.:
$ git clone https://github.com/Eonblast/Erlvolt.git erlvolt
$ cd erlvolt
$ make$ make helloYou will see a simple
Hej, världen!which is Hello world! in Swedish, where Erlang was invented by Joe Armstrong, Robert Virding and Mike Williams.
There are more sample programs:
Erlang sample programs are in the driver root under ./examples.
To run the samples, do:
$ make hello-barebones
$ make hello-plus # same as 'make hello'
$ make parallel
$ make voteror (after building Erlvolt.app and the database, as explained above), start hello etc. manually along these lines:
$ make
$ cd examples
$ erlc -I ../include hello.erl
$ erl -pa ../ebin -s hello run -s init stop -noshellBe sure to run the right Volt-database for the respective samples.
The Volt sample databases are in voltdb/doc/tutorials/helloworld for all hello, world! examples and in voltdb/examples/voter for the voter example. Simply change into the respective directory of your VoltDB installation and build and start the database with ./run.sh.
The Erlvolt driver is an Erlang application. It also uses crypto. This is how you start the driver. If crypto is already working, which is likely in a more complex Erlang program, the first lines is obsolete.
crypto:start(),
application:start(Erlvolt).A pool consists of a number of connections, one to each server in the VoltDB cluster. In the future you will be able to have multiple pools to multiple clusters.
erlvolt:add_pool(hello_pool, [{"localhost", 21212}]),SQL statements are most of the times stored procedure calls in VoltDB. But you can also freely form 'ad hoc' queries. Calls can be synchronous or asynchronous.
Result = erlvolt:call_procedure(hello_pool, "Select", ["Swedish"]).
Result = erlvolt:call_procedure(PoolID, "@AdHoc", ["select COUNT(*) as cnt from contestants"]).
Result = erlvolt:call_procedure(hello_pool, "Select", ["Swedish"], [asynchronous]),
receive
Result -> Result
end.The Result of a query comes back in one piece. You extract tables, rows and fields out of it like this:
Table = erlvolt:get_table(Result, 1),
Row = erlvolt:get_row(Table, 1),
Hello = erlvolt:get_string(Row, Table, "HELLO"), The results arrive from the Java stored procedure that you would have complete freedom to program as simple or complex as you want. Knowledge of what you are returning from the SP is indispensable to handle the response. The meta structure is always a list of 'tables'. For example, the above Result comes from this Java procedure, which you find in voltdb/doc/tutorials/helloworld/Select.java:
public class Select extends VoltProcedure {
public final SQLStmt sql = new SQLStmt(
"SELECT HELLO, WORLD FROM HELLOWORLD " +
" WHERE DIALECT = ?;"
);
public VoltTable[] run( String language)
throws VoltAbortException {
voltQueueSQL( sql, language );
return voltExecuteSQL();
}
}Learn more about VoltDB in this hands-on tutorial.
Please add a Common Test suite if you are proposing a pull request.
Common Test suites (Unit Tests) can be found in the etc/test folder. They help to test the basic functionality of the driver. They might also help you find trip ups in your Erlang and VoltDB system set up (environment and basics suites).
If you are new to Erlang: Common Tests are the Erlang pendant to Unit Tests. They work in the same spirit but employ the Erlang way to assert, using the bind operator '=' (sic) and have a native focus on parallel execution. They are organized in suites and give you a summary of passed and failed tests in the end. But the test is centered around the idea of crashing or not, more than evaluating something to true. To verify results, you write things like ok = somefunc() where somefunc() is expected to return the atom ok. This will crash if somefunc() returns something other than ok and result in one fail count for the test suite, but not the abortion of the test suite.
For the tests you need to have the VoltDB Voter sample database running, which you find in the VoltDB installation:
$ cd voltdb/examples/voter
$ ./run.shTo run the tests, in a different terminal window, in the driver root folder, type:
$ make testThese tests currently check access to the database (environment suite) and basic functionality (basics suite).
The screen will look like the following but the actual results of Common Tests are stored in html.
make[1]: Nothing to be done for `all'.
make[1]: Nothing to be done for `all'.
(cd test; ct_run -suite environment_SUITE basics_SUITE -pa ../ebin /opt/local/var/macports/software/erlang/R14A_0/opt/local/lib/erlang/lib/crypto-2.0/ebin/)
Erlang R15B02 (erts-5.9.2) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false]
Converting "../ebin" to "/Users/hd/Erlvolt2/ebin" and re-inserting with add_patha/1
Eshell V5.9.2 (abort with ^G)
(ct@metal)1>
Common Test v1.6.2 starting (cwd is /Users/hd/Erlvolt2/test)
Common Test: Running make in test directories...
Recompile: basics_SUITE
CWD set to: "/Users/hd/Erlvolt2/test/ct_run.ct@metal.2013-02-02_13.17.55"
TEST INFO: 2 test(s), 9 case(s) in 2 suite(s)
Testing hd.Erlvolt2.environment_SUITE: Starting test, 6 test cases
Testing hd.Erlvolt2.environment_SUITE: TEST COMPLETE, 6 ok, 0 failed of 6 test cases
Testing hd.Erlvolt2.basics_SUITE: Starting test, 3 test cases
Testing hd.Erlvolt2.basics_SUITE: TEST COMPLETE, 3 ok, 0 failed of 3 test cases
Updating /Users/hd/Erlvolt2/test/index.html... done
Updating /Users/hd/Erlvolt2/test/all_runs.html... doneThe last lines give you the starting point for the detailed test results in html.
| Test Name | Label | Test Run Started | Ok | Failed | Skipped (User/Auto) |
Missing Suites |
Node | CT Log | Old Runs |
|---|---|---|---|---|---|---|---|---|---|
| hd.Erlvolt2.basics_SUITE | - | Sat Feb 02 2013 13:17:55 | 3 | 0 | 0 (0/0) | 0 | ct@metal | CT Log | Old Runs |
| hd.Erlvolt2.environment_SUITE | - | Sat Feb 02 2013 13:17:55 | 6 | 0 | 0 (0/0) | 0 | ct@metal | CT Log | Old Runs |
| Total | 9 | 0 | 0 (0/0) | 0 |