vector upgrade breaks superbase (docker)

[stefanstidlffg]

the line to_timestamp is not working anymore: https://github.com/supabase/supabase/blob/e559600d76ab93db111e265225a92079b9b66708/docker/volumes/logs/vector.yml#L104

is not working anymore. it's a deprecation https://vector.dev/highlights/2023-08-15-0-32-0-upgrade-guide/#deprecated-to-timestamp

as i dont know the timeformat here, I dont know which formatstring is needed for parse_timestamp()

Any hint would be appreciated.

[cyluss]

The format string would be %d/%b/%Y:%T %z.

1. I assume that log source is PostgREST https://github.com/supabase/supabase/blob/e559600d76ab93db111e265225a92079b9b66708/docker/volumes/logs/vector.yml#L95

2. Sample timestamp would be 06/May/2024:08:16:11 -0500 according to PostgREST observability documentation

3. parse_timestamp function accepts strptime format string and is implemented in Rust chrono crate.

Here is a reproducible example for timestamp parsing:

use chrono::DateTime;
use chrono::offset::FixedOffset;
use chrono::format::ParseError;

fn parse_date(date_str: &str) -> Result<DateTime<FixedOffset>, ParseError> {
    DateTime::parse_from_str(date_str, "%d/%b/%Y:%T %z")
}

fn main() {
    let timestamp = "06/May/2024:08:16:11 -0500";
    println!("Original timestamp: {}", timestamp);

    match parse_date(timestamp) {
        Ok(date) => println!("Parsed date: {}", date),
        Err(e) => println!("Failed to parse date: {}", e),
    }
}

which outputs

$ cargo run
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.02s
     Running `target/debug/parse-pgrest-timestamp`
Original timestamp: 06/May/2024:08:16:11 -0500
Parsed date: 2024-05-06 08:16:11 -05:00

[stefanstidlffg]

Thanks for the help.

I forgot to mention that the version of vector I was trying to use is 0.38 while in the master branch we use 0.28.1 here.

Are there any reasons not to upgrade to 0.38? 0.2x version is quite old now.

[cyluss]

@stefanstidlffg Could you change the issue so it matches the issue template? https://github.com/supabase/supabase/issues/new?assignees=&labels=bug&projects=&template=1.Bug_report.md

[stefanstidlffg]

I don't think it's a bug in the current code. It's just a breaking change if you ever go to vector 0.3x.

I could do a PR with the update to vector 0.38 if you prefer that. But I don't know if anything else is affected by the update.