Using vogels with a local DynamoDB without an internet connection?

[philipbailey]

I'm trying to develop against a local DynamoDB instance (http://localhost:8000), but without an internet connection. I can interact with the local instance through the browser http://localhost:8000/console, but I can't get vogels to work against it. As soon as there's an internet connection all is fine. The DynamoDB documentation states that it's possible to use a local instance without an internet connection, but does / should vogels support it?

[tielur]

For what it's worth I'm able to work with dynalite locally while offline just fine.

[ryanfitz]

vogels can be configured to work with DynamoDB local (our test suite runs against dynamdb local).

  var opts = { endpoint : 'http://localhost:8000', apiVersion: '2012-08-10' };
  vogels.dynamoDriver(new AWS.DynamoDB(opts));

Have you tried connecting to a local instance like this with versus without an internet connection?

[patoncrispy]

@tielur Nice one for pointing out dynalite. That is MUCH better than DynamoDB Local!

[damienleroux]

Someone could share information on how to have vogels working dynalite?

Here is what I do to instanciate my DB server:

import dynalite from 'dynalite';
import packageJSON from '../package.json'

// Returns a standard Node.js HTTP server
const dynaliteServer = dynalite({ path: '../mydb', createTableMs: 50 })

dynaliteServer.listen(packageJSON.config.endpointPort, function (err) {
    if (err) throw err
    console.log('Dynalite started on port ' + packageJSON.config.endpointPort);
});

Here is what I tried to init vogels:

import packageJSON from '../package.json';
import AWS from 'aws-sdk';
import vogels from 'vogels';

export default function initContext() {
  const AWSConfig = {
    accessKeyId: packageJSON.config.KEY,
    secretAccessKey: packageJSON.config.SECRET,
    region: packageJSON.config.region,
    endpoint: packageJSON.config.endpoint
  };

  //config AWD
  AWS.config.update(AWSConfig);

  //create dynamodb instance
  const dynamodb = new AWS.DynamoDB(AWSConfig);

  //config vogels AWS
  vogels.AWS.config.update(AWSConfig);
  vogels.dynamoDriver(dynamodb);

  return {
    dynamodb,
    vogels
  };
}

I have defined a model and try to create it in initDB.js:

initContext();

myVogelsModel.create({
    name: 'foo',
    email: 'foo@example.com',
}, function (err, model) {
    console.log(err);
});

But it only returned me the error:

{ [ResourceNotFoundException: Requested resource not found]
  message: 'Requested resource not found',
  code: 'ResourceNotFoundException',
  time: Thu Feb 09 2017 22:53:46 GMT+0100 (Paris, Madrid),
  requestId: 'CATZQFWSMVJKOLP0YTIMU9IEVSLJBVAB7NJB8L3EIHXCDXLT30AQ',
  statusCode: 400,
  retryable: false,
  retryDelay: 0 }
C:\Users\damien\Documents\workspace\myProject\node_modules\aws-sdk\lib\request.js:31
            throw err;
            ^

TypeError: Cannot read property 'get' of undefined
    at C:/Users/damien/Documents/workspace/myProject/src/initDB.js:23:34
    at C:\Users\damien\Documents\workspace\myProject\node_modules\vogels\lib\table.js:193:16
    at Response.<anonymous> (C:\Users\damien\Documents\workspace\myProject\node_modules\vogels\lib\table.js:69:14)
    at Request.<anonymous> (C:\Users\damien\Documents\workspace\myProject\node_modules\aws-sdk\lib\request.js:355:18)
    at Request.callListeners (C:\Users\damien\Documents\workspace\myProject\node_modules\aws-sdk\lib\sequential_executor.js:105:20)
    at Request.emit (C:\Users\damien\Documents\workspace\myProject\node_modules\aws-sdk\lib\sequential_executor.js:77:10)
    at Request.emit (C:\Users\damien\Documents\workspace\myProject\node_modules\aws-sdk\lib\request.js:668:14)
    at Request.transition (C:\Users\damien\Documents\workspace\myProject\node_modules\aws-sdk\lib\request.js:22:10)
    at AcceptorStateMachine.runTo (C:\Users\damien\Documents\workspace\myProject\node_modules\aws-sdk\lib\state_machine.js:14:12)
    at C:\Users\damien\Documents\workspace\myProject\node_modules\aws-sdk\lib\state_machine.js:26:10

Any hint will be greatly appreciated :)

[damienleroux]

I have posted my question on stackoverflow: On Vogels iwth dynalite, Requested resource not found. Thanks

[set-killer]

Can you actually try out the suggestion of @ryanfitz? You are clearly missing the apiVersion option...

Also its not clear how do you define your schemas and tables. Take a look at this example. Also there are another examples which you may find usefull.

The couse of this error is:

The operation tried to access a nonexistent table or index. The resource might not be specified correctly, or its status might not be ACTIVE.

Source

So maybe try to create the tables first... Like this example maybe...

[damienleroux]

Actually, it didn't work either. But I fixed my issue:

The problem seem to be that I created the table using aws-sdk-js dynamodb.createTable function. When trying to add items in the table with vogels: it didn't work: It was confusing because command aws dynamodb list-tables --endpoint-url http://localhost:4567: I got a correct list of Tables. So I tried to create the table with vogels vogels.createTables function. After that, calling myVogelsModel.create function worked perfectly fine.

I don't know why I cannot create a table a with aws-sdk and then use vogels only for feeding my table. If anyone has an idea, here is how I created my table:

const tableXDefinition = {
    TableName: "TableX",
    KeySchema: [
        { AttributeName: "id", KeyType: "HASH" }
    ],
    AttributeDefinitions: [
        { AttributeName: "id", AttributeType: "N" } 
    ],
    ProvisionedThroughput: {
        ReadCapacityUnits: 10,
        WriteCapacityUnits: 10
    }
};

and here is how I defined my model:

const tableModel = vogels.define('TableX', {
  hashKey : 'id',

  timestamps : true,

  schema : {
    id : vogels.types.uuid(),
    name: Joi.string(),
    email: Joi.string().email(),
    passwordHash: Joi.string(),
    passwordSalt: Joi.string()
  }
});

[set-killer]

    AttributeDefinitions: [
        { AttributeName: "id", AttributeType: "N" } 

First of all do you know what that N means ?

S - the attribute is of type String
N - the attribute is of type Number
B - the attribute is of type Binary

But in the schema you define your id as UUID which surprisingly contains charecters 🤔

The second thing is that when you create a model with:

myVogelsModel.create({
    name: 'foo',
    email: 'foo@example.com',
}, ...);

you don't specify an ID for your new redord. How do you want to create a record without unique identifier (hash key)?

Also you may try to add the rest of the properties to your aws table definition... And also you may try to strictly specify the name of table in your vogels schema with additional property.

---

Dude? Just execute: vogels.createTables(function (err) {...}); on every start of the nodejs. It will not truncate your tables. It will create the tables only if they do not exists.

I suggest you to open a separate issue, but do your homework before that.

[damienleroux]

Yes I know all that Dude...

I was thinking that specifying a key as UUID generates a unique number identifier that increment automatically the key id when inserting a new item. Vogels document doesn't say if it is String or Number at the end. Or at least, I didn't find it

Concerning the item creation, I think it is correct to create an item without unrequired properties.

For the Table creation part, It is called only once using a separate node script. I'm mounting a mock env with random auto-generated fake data. I don't think having mentioned that I called it at every start.

I began testing dynamoDB last week so sorry if my question has obvious answers. I never handled UUID before... Any way, I don't think that the error ResourceNotFoundException: Requested resource not found give any relevant hint about a vogels model that doesn't fit an existing table definition. More appropriate error messages could help improve the vogels learning curve.

Thank you for your time and for pointing the defect in my implementation. I'll do better :) :)

[set-killer]

It is good that the problem is resolved. Yes, the error messages of DynamoDB sometimes are really misleading, but hey - we cannot teach them how to make web services ;P