Open strNewBee opened 1 year ago
Now i can only query 3 select in a batch, and I'm worrying about the extra I/O may slow it down. When I set batch up to more than 3, it still gets this problem by missing some queries.
public static QueryResult queryMetricsByIpRegexV2(String start, String end,
Map<QueryMapEntity, List<String>> tableMap,
Map<String, String> rs2ip,
Map<String, String> ip2regex)
{
/* basic where */
StringBuilder whereBase = new StringBuilder();
StringBuilder objBase = new StringBuilder();
StringBuilder signBase = new StringBuilder();
createTimeBaseRFC(start, end, whereBase);
/* create a filter */
objBase.append(" AND (").append(createSignFilterRegex(rs2ip, ip2regex, false)).append(")");
signBase.append(" AND (").append(createSignFilterRegex(rs2ip, ip2regex, true)).append(")");
/* generate queries */
StringBuilder queryCmd = new StringBuilder();
long queryCount = 0L;
List<String> queryCmdList = new ArrayList<>();
for(Map.Entry<QueryMapEntity, List<String>> pair: tableMap.entrySet()) {
QueryMapEntity target = pair.getKey();
Boolean is_sign = target.getIs_sign();
SelectQueryImpl query = QueryBuilder.select(pair.getValue().toArray())
.from(target.getDbName(), target.getTableName())
.where(whereBase.toString() + (is_sign ? signBase.toString() : objBase.toString()))
.groupBy((Object[]) (is_sign ? groupBySign : groupByUniq))
.tz("Etc/GMT-8");
queryCmd.append(query.getCommand());
queryCount += 1;
if(queryCount == CommonConstant.queryBatch){ // 3 in a batch by default
queryCmdList.add(queryCmd.toString());
queryCmd.setLength(0);
queryCount = 0L;
}
}
if(queryCount != 0)
queryCmdList.add(queryCmd.toString());
/* perform queries */
QueryResult result = null;
for(String queryStr: queryCmdList){
log.info("influxdb query: {}", queryStr);
if(result == null)
result = InfluxDBUtil.query(queryStr);
else{
for(QueryResult.Result res: InfluxDBUtil.query(queryStr).getResults()){
result.getResults().add(res);
}
}
}
return result;
}
Grateful for any help. (T T)
Environment
Platform
Question
My influxdb database stores the data from May 2nd to May 5th, and the query string is as below, which contains 5 select:
I'm calling InfluxDB.query() to execute it, but only the first 3 select query returns QueryResult. The Json format of QueryResult is:
When I execute each select query seperately, they can all return QueryResults. And there is more, which is when I narrowed down the time range to { 2023-05-02T10:10:30.78+08:00 ---- 2023-05-04T10:10:30.78+08:00 } instead of { 2023-05-02T10:10:30.78+08:00 ---- 2023-05-20T08:20:30.78+08:00 }, it works just fine and gives all the results for 5 queries.
I had no clue why this happened. Is there any constrants for the number of select query in one query() call?