ci(tests): add mini dataset to stateful test

[ZeaLoVe]

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

Add mini dataset to stateful test, include ontime and hits

Fixes #issue

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated
databend	✅ Ready (Inspect)	Visit Preview	Sep 1, 2022 at 4:00AM (UTC)

[Xuanwo]

IMHO, this test should be added as a separate github action because we can't run them locally.

[Xuanwo]

I fixed the permission denied problem, hoping this will help you.

[Xuanwo]

repo.databend.rs is a public-read bucket, I think we don't need to set ak/sk here.

Should work after https://github.com/datafuselabs/databend/pull/6623

:) curl -I https://repo.databend.rs/dataset/stateful/ontime_2006_100000.csv
HTTP/1.1 200 Connection established

HTTP/2 200
content-type: text/csv
content-length: 43870043
date: Thu, 14 Jul 2022 06:12:00 GMT
last-modified: Mon, 11 Jul 2022 07:05:03 GMT
etag: "59ac7677d8c51c2c8a7ce3137947b5d8-6"
accept-ranges: bytes
server: AmazonS3
x-cache: Hit from cloudfront
via: 1.1 90a66f433bb445328c58ce9cb2a6fa80.cloudfront.net (CloudFront)
x-amz-cf-pop: LAX50-C4
x-amz-cf-id: O-g_FFbC1fopFH20jMBsSGEIh6WdSqFRm65_cdpIDa9K33U3T3WVrA==
age: 825

[Xuanwo]

@mergify update

[mergify[bot]]

update

✅ Branch has been successfully updated

[Xuanwo]

This PR is blocked by https://github.com/datafuselabs/databend/pull/6624

[Xuanwo]

@mergify update

[mergify[bot]]

update

✅ Branch has been successfully updated

[Xuanwo]

I'm fighting with GitHub's actions.

[Xuanwo]

Oh, I figure out the reason. This PR is from a fork, so we can't access github secrets.

[Xuanwo]

After https://github.com/datafuselabs/opendal/issues/442, we can download data from CDN instead.

As in this PR, let's fall back to curl and COPY from the internal stage.

[Xuanwo]

Blocked by https://github.com/datafuselabs/databend/pull/6691

[Xuanwo]

@ZeaLoVe The test data can load now. Can you help fix the test results?

[ZeaLoVe]

@ZeaLoVe The test data can load now. Can you help fix the test results?

OK, I will turn to this tomorrow.

[Xuanwo]

@mergify update

[mergify[bot]]

update

✅ Branch has been successfully updated

[ZeaLoVe]

cc @sundy-li seems hits.sql could not create table, I run it local with error.

set enable_planner_v2 = 1; still error.

[ZeaLoVe]

Continue after these PRs merged. #6880 #6899

[Xuanwo]

@mergify update

[mergify[bot]]

update

✅ Branch has been successfully updated

[Xuanwo]

Continue after these PRs merged. #6880 #6899

PTAL

[ZeaLoVe]

@mergify update

[mergify[bot]]

update

✅ Branch has been successfully updated

[ZeaLoVe]

Maybe split this pr is a good choice, one for ontime, the other for hits. Then we can track problem clearly. Now the hits always timeout , any idea? cc @Xuanwo

[Xuanwo]

Maybe split this pr is a good choice, one for ontime, the other for hits. Then we can track problem clearly. Now the hits always timeout , any idea? cc @Xuanwo

LGTM.

Let's merge the test case for ontime first.

[ZeaLoVe]

Already split into 2 pr, close here.