Batched-Insert is causing SSTable Tombstones.

ramtg commented 6 years ago

Please answer these questions before submitting your issue. Thanks!

What version of Cassandra are you using?

Cassandra Server Release v3.0.9

What version of Gocql are you using?

CQLVersion: 3.0.0 (as of Jan 1, 2018)

What did you do?

Batch inserted prepared statements.

insertCQL:= fmt.Sprintf("INSERT INTO %v (lsid, f1, f2) VALUES (?, ?, ?)", "tbl_sparse_values")

// Execute test
batch := session.NewBatch(gocql.LoggedBatch)

//batch.Query(insertCQL, 999, float32(99.99), nil) // Results in SSTable Tombstone.

batch.Query(insertCQL, 999, float32(99.99)) // Skipping a parameter would solve the tombstone problem, but gocql would not throw run-time error: gocql: batch statement 0 expected 3 values send got 2

What did you expect to see?

gocql is not allow me to omit bind a parameters.
If specify a nil for the paramter I want to skip, it results in an SSTable tombstone.

(as indicated in the sstabledump JSON export).

What did you see instead?

A Runtime error: "run-time error: gocql: batch statement 0 expected 3 values send got 2"
Or an SSTable tombstone.
Suggestion: Please supported named optional parameters when binding. It'd be nice to add this feature supported in the Java Driver from Datastax .

Please see this Java code for reference:

https://raw.githubusercontent.com/ramtg/java-cql-batch/master/CQLBatchInsertingJavaClientNoTombs.java

If you are having connectivy related issues please share the following additional information

Describe your Cassandra cluster

please provide the following information

output of nodetool status Datacenter: datacenter1

Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- Address Load Tokens Owns (effective) Host ID Rack UN 127.0.0.1 507.86 KB 256 100.0% 560d9bfa-7139-44d5-82bb-f984e6a8603f rack1
output of SELECT peer, rpc_address FROM system.peers

select cql_version, native_protocol_version, release_version from system.local; -- cql_version: 3.4.0, -- native_protocol_version: 4 -- release_version: 3.0.9

rebuild your application with the gocql_debug tag and post the output 2018/01/01 16:10:31 gocql: Session.handleNodeUp: 127.0.0.1:9042 2018/01/01 16:10:31 &{cons:4 pageSize:5000 prefetch:0.25 routingKeyInfoCache:{lru:0xc042002300 mu:{state:0 sema:0}} schemaDescriber:0xc0420022a0 trace: hostSource:0xc0420362c0 stmtsLRU:0xc042034180 connCfg:0xc042038840 executor:0xc042002340 pool:0xc0420387e0 po licy:0xc042036300 ring:{endpoints:[0xc0421ce000 0xc0421ce160] mu:{w:{state:0 sema:0} writerSem:0 readerSem:0 readerCount:0 readerWait:0} hosts:map[127.0.0.1:0xc0421ce420] hostList:[0xc0421ce420] pos:1} metadata:{mu:{w:{state:0 sema:0} writerSem:0 readerSem:0 readerC ount:0 readerWait:0} partitioner:} mu:{w:{state:0 sema:0} writerSem:0 readerSem:0 readerCount:0 readerWait:0} control:0xc042036340 nodeEvents:0xc0421ac0f0 schemaEvents:0xc0421ac140 hosts:[] useSystemSchema:true cfg:{Hosts:[127.0.0.1 127.0.0.1] CQLVersion:3.0.0 Proto Version:4 Timeout:600000000 ConnectTimeout:600000000 Port:9042 Keyspace:weatherlink NumConns:2 Consistency:4 Compressor: Authenticator:{Username: Password:} RetryPolicy: SocketKeepalive:0 MaxPreparedStmts:1000 MaxRoutingKeyInfo:1000 PageSize:5000 SerialCon sistency:0 SslOpts: DefaultTimestamp:true PoolConfig:{HostSelectionPolicy:} ReconnectInterval:60000000000 MaxWaitSchemaAgreement:60000000000 HostFilter: AddressTranslator: IgnorePeerAddr:false DisableInitialHostLookup:false Events:{DisableNodeSta tusEvents:false DisableTopologyEvents:false DisableSchemaEvents:false} DisableSkipMetadata:false disableControlConn:false} quit:0xc042172120 closeMu:{w:{state:0 sema:0} writerSem:0 readerSem:0 readerCount:0 readerWait:0} isClosed:false} 2018/01/01 16:10:31 gocql: batch statement 0 expected 3 values send got 2

Zariel commented 6 years ago

For this can you use an unset column via UnsetValue ?

ramtg commented 6 years ago

@Zariel Thank you for the suggestion.

I'm not sure what I'm doing wrong, but I get the below error when I tried specifying UnsetValue in Batch.Query():

"java.lang.IndexOutOfBoundsException: index: 28, length: 158052 (expected: range(0, 84))" .

Here's the code I tried:

   // CREATE TABLE tbl_sparse_values(id int, sortkey int, v1 int, v2 int, v3 int, PRIMARY KEY (id, sortkey));
insertStmt := `INSERT INTO tbl_sparse_values(id, sortkey, v1, v2) VALUES (:id, :sortkey, :v1, :v2)`
batch.Query(insertStmt,
    gocql.NamedValue("id", 333),
    gocql.NamedValue("sortkey", 9),
    gocql.NamedValue("v1", gocql.UnsetValue),
    gocql.NamedValue("v2", 2))

if err := session.ExecuteBatch(batch); err != nil {
    log.Println(err)
    // Results in the below error:
    // java.lang.IndexOutOfBoundsException: index: 28, length: 158052 (expected: range(0, 84))
}

ramtg commented 6 years ago

@Zariel It appears Batch.Query() doesn't currently accept named values. Could you please confirm ?

Zariel commented 6 years ago

Reading the docs for the protocol it says this about named values in batches

        0x40: With names for values. If set, then all values for all <query_i> must be
              preceded by a [string] <name_i> that have the same meaning as in QUERY
              requests [IMPORTANT NOTE: this feature does not work and should not be
              used. It is specified in a way that makes it impossible for the server
              to implement. This will be fixed in a future version of the native
              protocol. See https://issues.apache.org/jira/browse/CASSANDRA-10246 for
              more details].

apache / cassandra-gocql-driver