A DynamoDB based Eloquent model and Query builder for Laravel.
You can find an example implementation in kitar/simplechat.
Install the package via Composer:
$ composer require kitar/laravel-dynamodb
Add dynamodb configs to config/database.php
:
'connections' => [
'dynamodb' => [
'driver' => 'dynamodb',
'key' => env('AWS_ACCESS_KEY_ID'),
'secret' => env('AWS_SECRET_ACCESS_KEY'),
'region' => env('AWS_DEFAULT_REGION', 'us-east-1'),
'token' => env('AWS_SESSION_TOKEN', null),
'endpoint' => env('DYNAMODB_ENDPOINT', null),
'prefix' => '', // table prefix
],
...
],
Update the DB_CONNECTION
variable in your .env
file:
DB_CONNECTION=dynamodb
For usage outside Laravel, you can create the connection manually and start querying with Query Builder.
$connection = new Kitar\Dynamodb\Connection([
'key' => env('AWS_ACCESS_KEY_ID'),
'secret' => env('AWS_SECRET_ACCESS_KEY'),
'region' => env('AWS_DEFAULT_REGION', 'us-east-1'),
'token' => env('AWS_SESSION_TOKEN', null),
'endpoint' => env('DYNAMODB_ENDPOINT', null),
'prefix' => '', // table prefix
]);
$connection->table('your-table')->...
Many of the example codes in this document are querying to DynamoDB's official sample data. If you want to try these codes with actual DynamoDB tables, it's handy to load them to your tables before.
DynamoDB model extends Eloquent model so that we can use familiar features such as mutators, serialization, etc.
The main difference between Eloquent model and DynamoDB model is:
create
, createOrFirst
where
with
)getItem
, putItem
, scan
, filter
)Most of the attributes are the same as the original Eloquent model, but there are few DynamoDB-specific attributes.
Name | Required | Description |
---|---|---|
table | yes | Name of the Table. |
primaryKey | yes | Name of the Partition Key. |
sortKey | Name of the Sort Key. | |
sortKeyDefault | Default value for the Sort Key. |
For example, if our table has only partition key, the model will look like this:
use Kitar\Dynamodb\Model\Model;
class ProductCatalog extends Model
{
protected $table = 'ProductCatalog';
protected $primaryKey = 'Id';
protected $fillable = ['Id', 'Price', 'Title'];
}
If our table also has sort key:
use Kitar\Dynamodb\Model\Model;
class Thread extends Model
{
protected $table = 'Thread';
protected $primaryKey = 'ForumName';
protected $sortKey = 'Subject';
protected $fillable = ['ForumName', 'Subject'];
}
If we set sortKeyDefault
, it will be used when we instantiate or call find
without sort key.
use Kitar\Dynamodb\Model\Model;
use Illuminate\Auth\Authenticatable;
use Illuminate\Contracts\Auth\Authenticatable as AuthenticatableContract;
class User extends Model implements AuthenticatableContract
{
use Authenticatable;
protected $table = 'User';
protected $primaryKey = 'email';
protected $sortKey = 'type';
protected $sortKeyDefault = 'profile';
protected $fillable = [
'name', 'email', 'password', 'type',
];
}
Note that this model is implementing
Illuminate\Contracts\Auth\Authenticatable
and usingIlluminate\Auth\Authenticatable
. This is optional, but if we use them, we can use this model with authentication as well. For authentication, please refer to Authentication section) for more details.
$products = ProductCatalog::scan();
or alternatively,
$products = ProductCatalog::all();
You also can override the scan()
method to fit your needs, such as filtering models for single table design. For example:
public static function scan($exclusiveStartKey = null, $sort = 'asc', $limit = 50)
{
return static::index('GSI1')
->keyCondition('GSI1PK', '=', 'PRODUCT#')
->keyCondition('GSI1SK', 'begins_with', 'PRODUCT#')
->exclusiveStartKey($exclusiveStartKey)
->scanIndexForward($sort == 'desc' ? false : true)
->limit($limit)
->query();
}
DynamoDB can only handle result set up to 1MB per call, so we have to paginate if there are more results. see Paginating the Results for more details.
If the model has only partition key:
ProductCatalog::find(101);
If the model also has sort key:
Thread::find([
'ForumName' => 'Amazon DynamoDB', // Partition key
'Subject' => 'DynamoDB Thread 1' // Sort key
]);
If the model has sort key and sortKeyDefault
is defined:
User::find('foo@bar.com'); // Partition key. sortKeyDefault will be used for Sort key.
You also can modify the behavior of the find()
method to fit your needs. For example:
public static function find($userId)
{
return parent::find([
'PK' => str_starts_with($userId, 'USER#') ? $userId : 'USER#'.$userId,
'SK' => 'USER#',
]);
}
$user = User::create([
'email' => 'foo@bar.com',
'type' => 'profile' // Sort key. If we don't specify this, sortKeyDefault will be used.
]);
$user = new User([
'email' => 'foo@bar.com',
'type' => 'profile'
]);
$user->save();
$user->name = 'foo';
$user->save();
$user->update([
'name' => 'foobar'
]);
$user->delete();
When we call increment()
and decrement()
, the Atomic Counter will be used under the hood.
$user->increment('views', 1);
$user->decrement('views', 1);
We can also pass additional attributes to update.
$user->increment('views', 1, [
'last_viewed_at' => '...',
]);
We can use Query Builder functions through model such as query
scan
filter
condition
keyCondition
etc.
For example:
Thread::keyCondition('ForumName', '=', 'Amazon DynamoDB')
->keyCondition('Subject', 'begins_with', 'DynamoDB')
->filter('Views', '=', 0)
->query();
Please refer to Query Builder for the details.
We can create a Custom User Provider to authenticate with DynamoDB. For the detail, please refer to Laravel's official document.
To use authentication with the model, the model should implement Illuminate\Contracts\Auth\Authenticatable
contract. In this section, we'll use the example User
model above.
After we prepare authenticatable model, we need to make the custom user provider. We can make it own (it's simple), but we'll use Kitar\Dynamodb\Model\AuthUserProvider
in this section.
To register custom user provider, add codes below in App/Providers/AuthServiceProvider.php
.
use Kitar\Dynamodb\Model\AuthUserProvider;
...
public function boot()
{
$this->registerPolicies();
Auth::provider('dynamodb', function ($app, array $config) {
return new AuthUserProvider(
$app['hash'],
$config['model'],
$config['api_token_name'] ?? null,
$config['api_token_index'] ?? null
);
});
}
Then specify driver and model name for authentication in config/auth.php
.
'providers' => [
// Eloquent
// 'users' => [
// 'driver' => 'eloquent',
// 'model' => App\User::class,
// ],
// DynamoDB
'users' => [
'driver' => 'dynamodb',
'model' => App\User::class,
'api_token_name' => 'api_token',
'api_token_index' => 'api_token-index'
],
],
api_token_name
and api_token_index
are optional, but we need them if we use api token authentication.
You might need to modify the registration controller. For example, if we use Laravel Breeze, the modification looks like below.
class RegisteredUserController extends Controller
{
...
public function store(Request $request)
{
$request->validate([
'name' => 'required|string|max:255',
'email' => ['required', 'string', 'email', 'max:255', function ($attribute, $value, $fail) {
if (User::find($value)) {
$fail('The '.$attribute.' has already been taken.');
}
}],
'password' => 'required|string|confirmed|min:8',
]);
$user = new User([
'name' => $request->name,
'email' => $request->email,
'password' => Hash::make($request->password),
]);
$user->save();
Auth::login($user);
event(new Registered($user));
return redirect(RouteServiceProvider::HOME);
}
}
There are two modifications. The first one is adding the closure validator for email
instead of unique
validator. The second one is using the save()
method to create user instead of the create()
method.
We can use Query Builder without model.
$result = DB::table('Thread')->scan();
Or even outside Laravel.
$connection = new Kitar\Dynamodb\Connection([
'key' => env('AWS_ACCESS_KEY_ID'),
'secret' => env('AWS_SECRET_ACCESS_KEY'),
'region' => env('AWS_DEFAULT_REGION', 'us-east-1'),
'token' => env('AWS_SESSION_TOKEN', null),
'endpoint' => env('DYNAMODB_ENDPOINT', null),
'prefix' => '', // table prefix
]);
$result = $connection->table('Thread')->scan();
If we query through the model, we don't need to specify the table name, and the response will be the model instance(s).
$threads = Thread::scan();
$response = DB::table('ProductCatalog')
->getItem(['Id' => 101]);
Instead of marshaling manually, pass a plain array.
Kitar\Dynamodb\Query\Grammar
will automatically marshal them before querying.
DB::table('Thread')
->putItem([
'ForumName' => 'Amazon DynamoDB',
'Subject' => 'New discussion thread',
'Message' => 'First post in this thread',
'LastPostedBy' => 'fred@example.com',
'LastPostedDateTime' => '201603190422'
]);
DB::table('Thread')
->key([
'ForumName' => 'Laravel',
'Subject' => 'Laravel Thread 1'
])->updateItem([
'LastPostedBy' => null, // REMOVE
'Replies' => null, // REMOVE
'Message' => 'Updated' // SET
]);
Currently, we only support simple SET
and REMOVE
actions. If the attribute has value, it will be passed to SET
action. If the value is null, it will be passed to REMOVE
action.
DB::table('Thread')
->deleteItem([
'ForumName' => 'Amazon DynamoDB',
'Subject' => 'New discussion thread'
]);
A Projection Expression is a string that identifies the attributes that web want. (It's like select
statement for SQL)
We can specify Projection Expressions in the same manner as the original select
clause.
$response = DB::table('ProductCatalog')
->select('Price', 'Title')
->getItem(['Id' => 101]);
When we manipulate data in Amazon DynamoDB table, we use putItem
, updateItem
and DeleteItem
. We can use Condition Expressions to determine which items should be modified.
To specify Condition Expression, we use condition
clause. This works basically same as the original where
clause, but it's for Condition Expressions.
DB::table('ProductCatalog')
->condition('Id', 'attribute_not_exists')
->putItem([
'Id' => 101,
'ProductCategory' => 'Can I overwrite?'
]);
Note that we specify
attribute_not_exists
for the operator of condition. This is DynamoDB-specific operator which calledfunction
. See DynamoDB-specific operators for condition() and filter() for more details.
OR statements
DB::table('ProductCatalog')
->condition('Id', 'attribute_not_exists')
->orCondition('Price', 'attribute_not_exists)
->putItem([...]);
AND statements
DB::table('ProductCatalog')
->condition('Id', 'attribute_not_exists')
->condition('Price', 'attribute_not_exists)
->putItem([...]);
ProductCatalog::key(['Id' => 101])
->conditionIn('ProductCategory', ['Book', 'Bicycle'])
->updateItem([
'Description' => 'updated!'
]);
ProductCatalog::key(['Id' => 101])
->conditionBetween('Price', [0, 10])
->updateItem([
'Description' => 'updated!'
]);
The Query operation in Amazon DynamoDB finds items based on primary key values.
When we query
, we must specify keyCondition
as well.
We can use some comparison operators for sort key, but we must use the equality condition for the partition key.
$response = DB::table('Thread')
->keyCondition('ForumName', '=', 'Amazon DynamoDB')
->keyCondition('Subject', 'begins_with', 'DynamoDB')
->query();
$response = DB::table('Thread')
->keyCondition('ForumName', '=', 'Amazon DynamoDB')
->keyConditionBetween('Subject', ['DynamoDB Thread 1', 'DynamoDB Thread 2'])
->query();
query
results are always sorted by the sort key value. To reverse the order, set the ScanIndexForward
parameter to false
.
$response = DB::table('Thread')
->keyCondition('ForumName', '=', 'Amazon DynamoDB')
->scanIndexForward(false)
->query();
Note that DynamoDB's
ScanIndexForward
is a feature forquery
. It will not work withscan
.
$response = DB::table('Thread')->scan();
When we query
or scan
, we can filter results with Filter Expressions before it returned.
It can't reduce the amount of read capacity, but it can reduce the size of traffic data.
$response = DB::table('Thread')
->filter('LastPostedBy', '=', 'User A')
->scan();
OR statement
$response = DB::table('Thread')
->filter('LastPostedBy', '=', 'User A')
->orFilter('LastPostedBy', '=', 'User B')
->scan();
AND statement
$response = DB::table('Thread')
->filter('LastPostedBy', '=', 'User A')
->filter('Subject', 'begins_with', 'DynamoDB')
->scan();
$response = DB::table('Thread')
->filterIn('LastPostedBy', ['User A', 'User B'])
->scan();
$response = DB::table('ProductCatalog')
->filterBetween('Price', [0, 100])
->scan();
A single query
or scan
only returns a result set that fits within the 1 MB size limit. If there are more results, we need to paginate.
If there are more results, the response contains LastEvaluatedKey
.
$response = DB::table('ProductCatalog')
->limit(5)
->scan();
$response['LastEvaluatedKey']; // array
We can pass this key to exclusiveStartKey
to get next results.
$response = DB::table('ProductCatalog')
->exclusiveStartKey($response['LastEvaluatedKey'])
->limit(5)
->scan();
If you are using Query Builder through model, you can access to exclusiveStartKey
by:
$products = ProductCatalog::limit(5)->scan();
$products->getLastEvaluatedKey(); // array
Alternatively, you can achieve the same result using individual models; however, please be aware that this approach is planned to be deprecated in versions subsequent to v2.x.
$products->first()->meta()['LastEvaluatedKey']; // array
Some applications might need to perform many kinds of queries, using a variety of different attributes as query criteria. To support these requirements, you can create one or more global secondary indexes and issue query
requests against these indexes in Amazon DynamoDB.
Use index
clause to specify Global Secondary Index name.
$response = DB::table('Reply')
->index('PostedBy-Message-index')
->keyCondition('PostedBy', '=', 'User A')
->keyCondition('Message', '=', 'DynamoDB Thread 2 Reply 1 text')
->query();
DynamoDB supports Atomic Counter. When we call increment()
and decrement()
through Model or Query Builder, Atomic Counter will be used under the hood.
DB::('Thread')->key([
'ForumName' => 'Laravel',
'Subject' => 'Laravel Thread 1'
])->increment('Replies', 2);
We can also pass additional attributes to update.
DB::('Thread')->key([
'ForumName' => 'Laravel',
'Subject' => 'Laravel Thread 1'
])->increment('Replies', 2, [
'LastPostedBy' => 'User A',
]);
Batch operations can get, put or delete multiple items with a single call. There are some DynamoDB limitations (such as items count, payload size, etc), so please check the documentation in advance. (BatchGetItem, BatchWriteItem)
DB::table('Thread')
->batchGetItem([
[
'ForumName' => 'Amazon DynamoDB',
'Subject' => 'DynamoDB Thread 1'
],
[
'ForumName' => 'Amazon DynamoDB',
'Subject' => 'DynamoDB Thread 2'
]
]);
DB::table('Thread')
->batchPutItem([
[
'ForumName' => 'Amazon DynamoDB',
'Subject' => 'DynamoDB Thread 3'
],
[
'ForumName' => 'Amazon DynamoDB',
'Subject' => 'DynamoDB Thread 4'
]
]);
This is a handy method to batch-put items using
batchWriteItem
DB::table('Thread')
->batchDeleteItem([
[
'ForumName' => 'Amazon DynamoDB',
'Subject' => 'DynamoDB Thread 1'
],
[
'ForumName' => 'Amazon DynamoDB',
'Subject' => 'DynamoDB Thread 2'
]
]);
This is a handy method to batch-delete items using
batchWriteItem
DB::table('Thread')
->batchWriteItem([
[
'PutRequest' => [
'Item' => [
'ForumName' => 'Amazon DynamoDB',
'Subject' => 'DynamoDB Thread 3'
]
]
],
[
'DeleteRequest' => [
'Key' => [
'ForumName' => 'Amazon DynamoDB',
'Subject' => 'DynamoDB Thread 1'
]
]
]
]);
For condition
and filter
clauses, we can use DynamoDB's comparators and functions.
=
<>
<
<=
>
>=
can be used in the form of:
filter($key, $comparator, $value);
Available functions are:
filter($key, 'attribute_exists');
filter($key, 'attribute_not_exists');
filter($key, 'attribute_type', $type);
filter($key, 'begins_with', $value);
filter($key, 'contains', $value);
size
function is not supported at this time.
We can inspect what parameters (and which method) will actually send to DynamoDB by adding dryRun()
to our query. For example:
// via Model
$request = ProductCatalog::dryRun()->getItem(['Id' => 101]);
// via Query Builder
$request = DB::table('ProductCatalog')->dryRun()->getItem(['Id' => 101]);
dump($request);
Our PHPUnit tests also use this feature, without actually calling DynamoDB
$ ./vendor/bin/phpunit