search

brimdata / super

A novel data lake based on super-structured data
https://zed.brimdata.io/
BSD 3-Clause "New" or "Revised" License
1.39k stars 64 forks source link

Failed attempt to write to S3 bucket looks like success #900

Closed philrz closed 4 years ago

philrz commented 4 years ago

Found while verifying https://github.com/brimsec/zq/issues/771. With zq commit 4e6680f, if I attempt to send the output to an S3 bucket that doesn't exist, zq behaves as if it worked.

$ echo '{"foo": "bar"}' | zq -t -
#0:record[foo:string]
0:[bar;]

$ echo '{"foo": "bar"}' | zq -t -o s3://this-bucket-does-not-exist/foo.zng -
$ echo $?
0
mattnibs commented 4 years ago

Thanks @philrz on it

philrz commented 4 years ago

Verified in zq commit 8820a88. Now we get a proper error message and a non-zero return code.

$ echo '{"foo": "bar"}' | zq -t -o s3://this-bucket-does-not-exist/foo.zng -
NoSuchBucket: The specified bucket does not exist
    status code: 404, request id: 29FD1AEEC0312124, host id: agzgnI0JVKL56po5zSQf6iwpO7DYEqhYkkLjK9zoA6pNHFmI6AxualIKya+0Si3S772JG2odQzo=

$ echo $?
1

Thanks @mattnibs!