Function to verify data persisted in storage

influxdata / Litmus

testing framework

0 stars 0 forks source link

Function to verify data persisted in storage #109

Closed gshif closed 5 years ago

gshif commented 5 years ago

After Data gets to kafka from gateway, the storage service is pulling it (I am guessing, every 10 sec) from kafka. Need to have a way of verifying data persisted on disk

gshif commented 5 years ago

Use kubectl to execute a command on storage node. By default we have only one storage - storage-0 - but --storage-pods flag can be passed to the fenvctl to have more than one storage pod (will deal with this later...)

kubectl --kubeconfig=%s --context=%s exec storage-0 -c storage -n %s -- bash -c "cd /data;find . -name \"*.tsm\" -print"

gshif commented 5 years ago

Or :

kubectl --kubeconfig=%s --context=%s exec storage-0 -c storage -n %s -- bash -c "cd /data;find . -name "*.tsm" -print" | xargs grep -a

gshif commented 5 years ago

Since we can have more than 1 storage pod (use --storage-pods ) then function needs to be able to search all of the storage containers in all of the storage pods and as such storage pods should be a parameter to a function, type list or tuple BY default we have 64 partitions on one storage node, if we have two storage nodes, then each node will have 32 partitions.