support for encrypted PEM keys

[kunkku]

Please let me know if further changes are required to this patch set.

[daurnimator]

So this PR works, but I've found it very odd to use.

e.g. here is a valid invocation:

pk=require "openssl.pkey"
a=pk.new()
k=a:getPrivateKey("aes-256-cbc", "bar")
b = pk.new(k, "PEM", "private", "bar")

But the following invocations fail (with mostly hard to understand/debug error messages)

Passing "public" rather than "private":

$ lua -e 'pk=require "openssl.pkey";  a=pk.new(); k=a:getPrivateKey("aes-256-cbc", "bar") pk.new(k, "PEM", "public", "bar")'
lua: pkey.new: pem_lib.c:691:error:0906D06C:PEM routines:PEM_read_bio:no start line
stack traceback:
    [C]: in function 'openssl.pkey.new'
    (command line):1: in main chunk
    [C]: in ?

Passing wrong password (this is probably fine):

$ lua -e 'pk=require "openssl.pkey";  a=pk.new(); k=a:getPrivateKey("aes-256-cbc", "bar") pk.new(k, "PEM", "private", "foo")'
lua: pkey.new: evp_enc.c:536:error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt
stack traceback:
    [C]: in function 'openssl.pkey.new'
    (command line):1: in main chunk
    [C]: in ?

Passing wrong password and also no public vs private:

$ lua -e 'pk=require "openssl.pkey";  a=pk.new(); k=a:getPrivateKey("aes-256-cbc", "bar") pk.new(k, "PEM", nil, "foo")'
lua: pkey.new: pem_lib.c:691:error:0906D06C:PEM routines:PEM_read_bio:no start line
stack traceback:
    [C]: in function 'openssl.pkey.new'
    (command line):1: in main chunk
    [C]: in ?

Passing wrong password and no "PEM" choice:

$ lua -e 'pk=require "openssl.pkey";  a=pk.new(); k=a:getPrivateKey("aes-256-cbc", "bar") pk.new(k, nil, "private", "foo")'
lua: pkey.new: tasn_dec.c:1129:error:0D0680A8:asn1 encoding routines:asn1_check_tlen:wrong tag
stack traceback:
    [C]: in function 'openssl.pkey.new'
    (command line):1: in main chunk
    [C]: in ?

Otherwise, I found it troublesome that not passing a cipher would result in no password being used. Perhaps throw an error if a password is specified and a cipher isn't?

[kunkku]

I was thinking if the API could be something like this:

pk = require "openssl.pkey"
a = pk.new()
k = a:toPEM{type="private", cipher="aes-256-cbc", password="bar"}
b = pk.new(k, {format="PEM", type="private", password="bar"})

This would make the code more readable and there would be no need for a separate getPrivateKey method.

[kunkku]

The new patch set implements passing options via a table as suggested above, avoiding the need for a new getPrivateKey method.

[vinayakhulawale]

@kunkku - Is this patch safe to use ? are there any further changes expected ?

[kunkku]

Honestly, I do not know. I would like to get feedback from the maintainers.