Crash in `stub_main_genkeys`

[HarryR]

This was causing crashes in ethsnarks-miximus and other projects, but only with GCC on Linux, it doesn't seem to be a problem with clang++ on OSX.

The pointer to pb.constraint_system.constraints seems to differ across calls.

template<class GadgetT>
int stub_genkeys( const char *pk_file, const char *vk_file )
{
    ppT::init_public_params();

    ProtoboardT pb;
    GadgetT mod(pb, "module");
    mod.generate_r1cs_constraints();

    return stub_genkeys_from_pb(pb, pk_file, vk_file);
}

Then in stub_genkeys_from_pb it crashes in the copy constructor for libsnark::r1cs_constraint_system during call to get_constraint_system.

int stub_genkeys_from_pb( ProtoboardT& bt
    pb, const char *pk_file, const char *vk_file )
{
    const auto constraints = pb.get_constraint_system();

Backtrace from GDB:

gdb) p &pb
$7 = (ethsnarks::ProtoboardT *) 0x7fffffffdd00
(gdb) down
#16 0x000000000043b510 in ethsnarks::stub_genkeys_from_pb (pb=..., pk_file=0x7fffffffe27e ".keys/miximus.pk.raw", vk_file=0x7fffffffe293 ".keys/miximus.vk.json")
    at /home/user/github.com/HarryR/ethsnarks-miximus/ethsnarks/src/stubs.cpp:46
46      const auto constraints = pb.get_constraint_system();
(gdb) p &pb
$8 = (ethsnarks::ProtoboardT *) 0x7fffffffdd00
(gdb) p *pb
No symbol "operator*" in current context.
(gdb) p pb.constraint_system.constraints.size()
$9 = 1756833606647864122
(gdb) p &pb.constraint_system 
$10 = (libsnark::r1cs_constraint_system<libff::Fp_model<4, (libff::bigint<4> const&)(&libff::alt_bn128_modulus_r)> > *) 0x7fffffffdda0
(gdb) p &pb.constraint_system.constraints
$11 = (std::__debug::vector<libsnark::r1cs_constraint<libff::Fp_model<4, (libff::bigint<4> const&)(&libff::alt_bn128_modulus_r)> >, std::allocator<libsnark::r1cs_constraint<libff::Fp_model<4, (libff::bigint<4> const&)(&libff::alt_bn128_modulus_r)> > > > *) 0x7fffffffddb0
(gdb) up
#17 0x000000000040fcb9 in ethsnarks::stub_genkeys<ethsnarks::mod_miximus> (pk_file=0x7fffffffe27e ".keys/miximus.pk.raw", vk_file=0x7fffffffe293 ".keys/miximus.vk.json")
    at /home/user/github.com/HarryR/ethsnarks-miximus/ethsnarks/src/stubs.hpp:29
29      return stub_genkeys_from_pb(pb, pk_file, vk_file);
(gdb) p &pb.constraint_system.constraints
$12 = (std::vector<libsnark::r1cs_constraint<libff::Fp_model<4, (libff::bigint<4> const&)(&libff::alt_bn128_modulus_r)> >, std::allocator<libsnark::r1cs_constraint<libff::Fp_model<4, (libff::bigint<4> const&)(&libff::alt_bn128_modulus_r)> > > > *) 0x7fffffffdd70

GDB shows &pb.constraint_system.constraints in stub_genkeys_from_pb differs from the address in stub_genkeys. In stub_genkeys the address is correct (0x7fffffffdd70), but one level down in stub_genkeys_from_pb the address is incorrect (0x7fffffffddb0).

[HarryR]

This seems to originate from different build flags being used to compile ethsnarks versus the host program.

For example:

In stub_genkeys

(gdb) ptype pb
type = class libsnark::protoboard<libff::Fp_model<4, (libff::bigint<4> const&)(&libff::alt_bn128_modulus_r)> > 
    [with FieldT = libff::Fp_model<4, (libff::bigint<4> const&)(&libff::alt_bn128_modulus_r)>] {
  private:
    FieldT constant_term;
    libsnark::r1cs_variable_assignment values;
    libsnark::var_index_t next_free_var;
    libsnark::lc_index_t next_free_lc;
    std::__debug::vector<FieldT, std::allocator<FieldT> > lc_values;
    libsnark::r1cs_constraint_system<FieldT> constraint_system;

vs In stub_genkeys_from_pb:

(gdb) ptype pb
type = class libsnark::protoboard<libff::Fp_model<4, (libff::bigint<4> const&)(&libff::alt_bn128_modulus_r)> > 
    [with FieldT = libff::Fp_model<4, (libff::bigint<4> const&)(&libff::alt_bn128_modulus_r)>] {
  private:
    FieldT constant_term;
    libsnark::r1cs_variable_assignment values;
    libsnark::var_index_t next_free_var;
    libsnark::lc_index_t next_free_lc;
    std::vector<FieldT, std::allocator<FieldT> > lc_values;
    libsnark::r1cs_constraint_system<FieldT> constraint_system;

This explains the difference in offsets, in stub_genkeys (a template, defined in a header) the std::vector type has debug information, meaning ProtoboardT has different sizes, hence the different offsets and the crash.

This is probably caused by the -D_GLIBCXX_DEBUG -D_GLIBCXX_DEBUG_PEDANTIC flags on compile:

1. Being enabled by default
2. Not being propagated up via compile flags.

The fix should be to:

1. Ensure the compile flags are propagated up
2. Disable GLIBCXX_DEBUG by default