search

datastax / dsbulk

DataStax Bulk Loader (DSBulk) is an open-source, Apache-licensed, unified tool for loading into and unloading from Apache Cassandra(R), DataStax Astra and DataStax Enterprise (DSE)
Apache License 2.0
83 stars 30 forks source link

Allow file input for dsbulk unload #491

Open bhisey opened 7 months ago

bhisey commented 7 months ago

We have different scenarios where we needed to use file as source to put it in where clause but its not allowed.

[solr2:0] automaton@ip-172-31-19-246:~$ cat test/output-000001.csv
str_num,day_no,transaction_uid,trans_typ,sequence_no
006,7,14d6f78c-c468-4b71-84ce-0a8009bf3dc1,SALE,seq3
006,7,14d6f78c-c468-4b71-84ce-0a8009bf3dc1,SALE,seq4

Error The provided statement (schema.query) contains unrecognized WHERE restrictions; the WHERE clause is only allowed to contain one token range restriction of the form: WHERE token(...) > ? AND

[solr2:0] automaton@ip-172-31-19-246:~$ cat test/output-000001.csv  | dsbulk-1.11.0/bin/dsbulk unload -query "SELECT str_num, day_no, transaction_uid, trans_typ, sequence_no from test.live_mom_transaction WHERE str_num=:str_num and day_no=:day_num and transaction_uid=:transaction_uid and trans_typ=:trans_typ and  sequence_no=:sequence_no"
Operation directory: /home/automaton/logs/UNLOAD_20240312-213555-414352
Operation UNLOAD_20240312-213555-414352 failed: The provided statement (schema.query) contains unrecognized WHERE restrictions; the WHERE clause is only allowed to contain one token range restriction of the form: WHERE token(...) > ? AND token(...) <= ?.