Automate s3 prefix naming

[rozhok]

I want to use single lambda for backing up of many tables. Therefore, I want to distinguish s3 prefixes so I could easily restore only necessary tables. Now I'm doing following

let eventSourceArn = event.Records[0].eventSourceARN;
let tableName = eventSourceArn.substring(
    eventSourceArn.indexOf('table/') + 6, 
    eventSourceArn.indexOf('stream/') - 1
);
let config = {
    S3Bucket: bucket,
    S3Region: 'eu-west-1',
    S3Prefix: tableName
};

Inside my lambda, but it looks as a hack. It would be nice to have such function bundled into backup-er.

[keen99]

I haven't dug into this project yet but - using a single lambda function to back up many tables can run into the lambda execution time limits pretty easily. no lambda job can run longer than 300 seconds. so keep that in mind...

[rozhok]

@keen99 as far as I understand, several triggers in lambda just execute several lambdas simultaneously or execute them one by one. I've already configured single lambda to handle 30's streams and so far everything goes smooth.

[yonahforst]

@rozhok - i like your approach for dynamic prefixing. One concern I have is deriving the tablename for all updates from the eventSourceARN of the first record.

Do we know that amazon will never bundle updates from different tables in same stream invocation?