influxdata / influxdb

Scalable datastore for metrics, events, and real-time analytics
https://influxdata.com
Apache License 2.0
28.92k stars 3.55k forks source link

[feature request] Support month and year as duration unit #3991

Closed ghost closed 4 years ago

ghost commented 9 years ago

For example:

Its useful for me when i want to query the data last month and year...

sanderson commented 4 years ago

To clarify, year and month duration units are supported in the version of Flux that will ship with 1.8. They will not be available in InfluxQL.

timhallinflux commented 4 years ago

InfluxDB 1.8 is available now. You must use Flux to leverage this capability. https://docs.influxdata.com/flux/v0.65/guides/windowing-aggregating/#windowing-data

RedShift1 commented 4 years ago

What about performance? Is it the same as CQL?

mabnz commented 4 years ago

To clarify, year and month duration units are supported in the version of Flux that will ship with 1.8. They will not be available in InfluxQL.

Will aggregateWindow support 'offset'?

polcape commented 4 years ago

Hi, I try with aggregate function but what id the duration for month? 1M? Seems not work...

sanderson commented 4 years ago

1mo is for month. 1m is for minute.

gaardiolor commented 4 years ago

@RedShift1

What about performance? Is it the same as CQL?

192176 total records, both queries have the same output, piped to /dev/null to suppress.

time ./influx.sh 'SELECT sum("TEST") FROM "test" WHERE time >= 1512345600000000000 GROUP BY time(4w,4d)' > /dev/null

real    0m0.132s
user    0m0.009s
sys     0m0.004s

time ./flux.sh 'from(bucket: "test") |> range(start: 2017-12-03T00:00:00.000000001Z) |> filter(fn: (r) => r._measurement == "test" and r._field == "TEST") |> group() |> window(every: 4w, period: 4w) |> sum()' > /dev/null
real    0m7.149s
user    0m5.763s
sys     0m0.080s

Really hammering the CPU. So, I only need 54x more hardware to do the same thing in flux. Hooray.

castillo92 commented 4 years ago

+1 GROUP BY time(1M) GROUP BY time(1Y)

In this moment Flux is not the solution. If we have to change everything to have 1M we should move our data to another database

hiei17 commented 4 years ago

+1

vbenhur commented 4 years ago

+1

nanhao1986 commented 4 years ago

+1

aolaria commented 4 years ago

+1

adiantek commented 4 years ago

+1

tonioa commented 4 years ago

+1

dFohlen commented 4 years ago

+1

miky-boy commented 4 years ago

+1

nanicpc commented 4 years ago

this shoudn't have being close since is not available in influxql

timhallinflux commented 4 years ago

We don't plan to offer this functionality in InfluxQL. The functionality is available in Flux and performance optimizations will be introduced shortly to address the concerns raised above.

castillo92 commented 4 years ago

We don't plan to offer this functionality in InfluxQL. The functionality is available in Flux and performance optimizations will be introduced shortly to address the concerns raised above.

Flux is InfluxDB? We will be able to query data grouping by time "month" or "year" ?

timhallinflux commented 4 years ago

We introduced Flux in InfluxDB 1.8. It exists side-by-side with InfluxQL. https://docs.influxdata.com/influxdb/v1.8/flux/installation/

Yes, if you fundamentally need these capability today, it is there to take advantage of. In many cases, using 30d as a mechanism is good enough, but there are use cases where the strict month and year boundaries are desired.

https://docs.influxdata.com/influxdb/v1.8/flux/guides/window-aggregate/#windowing-data

castillo92 commented 4 years ago

We introduced Flux in InfluxDB 1.8. It exists side-by-side with InfluxQL. https://docs.influxdata.com/influxdb/v1.8/flux/installation/

Yes, if you fundamentally need these capability today, it is there to take advantage of. In many cases, using 30d as a mechanism is good enough, but there are use cases where the strict month and year boundaries are desired.

https://docs.influxdata.com/influxdb/v1.8/flux/guides/window-aggregate/#windowing-data

Great functionality!

So... Flux is a new way of querying data using it's new language?

timhallinflux commented 4 years ago

Yes. Think of it as a domain specific language which provides the power you need to move beyond simple dashboarding.

As we release InfluxDB 2.0 OSS, planned for end of Oct/early Nov, we have both Flux and InfluxQL present as languages for accessing data. 1.x READ and WRITE compatibility APIs will be present (starting with the 1st release candidate -- coming soon) allowing for ease of upgrade from 1.x to 2.x.

Flux can be used to build interactive queries such as dashboard cells AND it also powers a task system. You can write Flux-based tasks which execute on a schedule. We'll also look to introduce "triggers" for task execution in subsequent releases.

If you want to learn more about Flux, I'd highly suggest diving into some primers like: https://www.influxdata.com/resources/introduction-to-flux-and-functional-data-scripting/

kptkip commented 3 years ago

I am using Grafana to read data from influxDB.

I am wondering, how to use the feature with grouping by month.

pportelaf commented 3 years ago

We don't plan to offer this functionality in InfluxQL. The functionality is available in Flux and performance optimizations will be introduced shortly to address the concerns raised above.

But no timezone feature :(

sauvant commented 9 months ago

...and now Flux, the long awaited solution for the ever wanted calendar grouping feature, gets deprecated. Hard to believe. So please reopen this issue, we need the functionality in InfluxQL as it seems.

See my comment in the influx forum:

https://community.influxdata.com/t/i-am-lost-which-query-language-to-use/32866

RedShift1 commented 9 months ago

...and now Flux, the long awaited solution for the ever wanted calendar grouping feature, gets deprecated. Hard to believe. So please reopen this issue, we need the functionality in InfluxQL as it seems.

See my comment in the influx forum:

https://community.influxdata.com/t/i-am-lost-which-query-language-to-use/32866

Don't count on this getting fixed. Instead, put your summaries in a different database like MySQL or PostgreSQL. Yes it's annoying having to use two different databases but we are limited by the technology of our time.

castillo92 commented 9 months ago

For me Influx started like a rocket.... but pass 2 or 3 years and see the same problems...

I do not understand how a time series database do not have good GROUP BY (time) filters

cesaroni commented 6 months ago

how to calculate how long is a month:

isLeapYear = (currentYear) => {
  leapYear = if  currentYear % 4 == 0 and ( currentYear % 100 != 0 or currentYear % 400 == 0)  then 
    true
  else 
    false

  return leapYear
}

daysOfMonth = (currentYear,currentMonth) => {
  days = if  contains(value: currentMonth, set: [4,6,9,11]) then 30 
  else if currentMonth == 2 and isLeapYear(currentYear) == true then 29
  else if currentMonth == 2 and isLeapYear(currentYear) == false then 28
  else 31

  return days
}

daysOfMonth(currentYear: 2024, currentMonth: 2)  // 29
Endocipher7 commented 6 months ago

Does the model apply for suspension of durability in terms of contraction in cycling practice in 1000 year loops?

krtschmr commented 6 months ago

@cesaroni

isLeapYear = (currentYear) => {
  return currentYear % 4 == 0 && ( currentYear % 100 != 0 || currentYear % 400 == 0) 
}

had to fix it :P

cesaroni commented 6 months ago

@krtschmr

flux operators...

isLeapYear = (currentYear) => {
  return currentYear % 4 == 0 and ( currentYear % 100 != 0 or currentYear % 400 == 0)
}
krtschmr commented 6 months ago

Haaa, terrible. you're right :P

regnaio commented 5 months ago

+1