search

parseablehq / parseable

Open Source ElasticSearch Alternative. Parseable helps you search and get insights from your logs in the most simple way possible.
https://parseable.com
GNU Affero General Public License v3.0
1.91k stars 116 forks source link

Stream names should be less restrictive #917

Closed sebzimmermann closed 2 months ago

sebzimmermann commented 2 months ago

As the stream names are coupled with s3 prefixes, they should also be aligned in terms of naming conventions (at the moment only lower case alphanumeric characters are allowed).

The target should be to also allow "Safe characters" listed in the aws docs: https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html

nikhilsinhaparseable commented 2 months ago

Hi @sebzimmermann as per the doc, below is the list of allowed special characters -

Exclamation point (!) Hyphen (-) Underscore (_) Period (.) Asterisk (*) Single quote (') Open parenthesis (() Close parenthesis ())

Ingestion works well to S3 but in the query api, we have to wrap the table name in double quotes for datafusion to understand these prefixes in S3 to find data, say, i create a stream with a name a.b, we will have to send this query to datafusion - "select * from \"a.b\""

IMO other than below special chars - Hyphen (-) Underscore (_) nothing else make sense in the stream/table name

please provide your opinion on this.

Thanks!

sebzimmermann commented 2 months ago

Hi @nikhilsinhaparseable, sure - and _ would already be beneficial!