tinybirdco / ci

Collection of configuration files that enable CI flows for Tinybird projects
MIT License
3 stars 3 forks source link

Out of the box regression tests failing #42

Closed lukasburns closed 8 months ago

lukasburns commented 9 months ago

Im running the out of the box cI workflow in github and its currently logging

** Environment regression tests job url ***/v0/jobs/0aee3aa8-5c12-4f58-9204-50c60c0b1f60
Regression tests 

🚨 No requests found for regression testing Pipe deduplicated_events - coverage.
πŸ’‘ Hint: Use `export TB_SKIP_REGRESSION=1` or add `--skip-regression-tests` to the command to skip all regression tests or `skip: True` in the config file to skip this assert for this specific Pipe. See this guide for more info about the regression tests configuration file => https://www.tinybird.co/docs/guides/continuous-integration.html#testing-strategies
🚨 No requests found for regression testing Pipe analytics_hits - coverage.
πŸ’‘ Hint: Use `export TB_SKIP_REGRESSION=1` or add `--skip-regression-tests` to the command to skip all regression tests or `skip: True` in the config file to skip this assert for this specific Pipe. See this guide for more info about the regression tests configuration file => https://www.tinybird.co/docs/guides/continuous-integration.html#testing-strategies

With no tests failing, but the step of the action is failing.

Dont know why

==== Results Summary ====


-------------------------------------------------------------------------------------------------------------
| Endpoint                        | Test     | Run | Passed | Failed | Mean response time | Mean read bytes |
-------------------------------------------------------------------------------------------------------------
| deduplicated_events             | coverage |   0 |      0 |      0 | +0.0 %             | +0.0 %          |
| analytics_hits                  | coverage |   0 |      0 |      0 | +0.0 %             | +0.0 %          |
| deduplicated_event_ticket_types | coverage |   0 |      0 |      0 | +0.0 %             | +0.0 %          |
| trend                           | coverage |   0 |      0 |      0 | +0.0 %             | +0.0 %          |
| event_ticket_types_report       | coverage |   1 |      1 |      0 | +142.86 %          | +0.0 %          |
| top_browsers                    | coverage |   0 |      0 |      0 | +0.0 %             | +0.0 %          |
| top_pages                       | coverage |   0 |      0 |      0 | +0.0 %             | +0.0 %          |
| kpis                            | coverage |   0 |      0 |      0 | +0.0 %             | +0.0 %          |
| kpis_event                      | coverage |   1 |      1 |      0 | +3.78 %            | +0.0 %          |
| top_locations                   | coverage |   0 |      0 |      0 | +0.0 %             | +0.0 %          |
| top_devices                     | coverage |   0 |      0 |      0 | +0.0 %             | +0.0 %          |
| top_sources                     | coverage |   0 |      0 |      0 | +0.0 %             | +0.0 %          |
| monthly_producer_tickets        | coverage |   1 |      1 |      0 | +[160](https://github.com/evently-cl/evently/actions/runs/7402176309/job/20139469353#step:18:161).64 %          | +0.0 %          |
| monthly_producer_sales          | coverage |   1 |      1 |      0 | +23.61 %           | +0.0 %          |
| event_sales                     | coverage |   1 |      1 |      0 | -6.53 %            | +0.0 %          |
| event_tickets                   | coverage |   1 |      1 |      0 | +18.25 %           | +0.0 %          |
-------------------------------------------------------------------------------------------------------------

Error: Process completed with exit code 1.

alrocar commented 8 months ago

Thanks for reporting.

We were reporting as failed pipe endpoints with no requests, we'll change it to warning in the upcoming 3.0.0 version of tinybird-cli

alexon1234 commented 8 months ago

Two days ago, we released a developer edition of the CLI 2.2.1.dev3 that contains a change of the behavior of the regression-test to only show a warning instead of throwing an error.

You can switch to use this version or wait we realize the 3.0.0 stable version

alrocar commented 8 months ago

CLI 3.0.0 has been released = > https://pypi.org/project/tinybird-cli/

Closing the issue. Thanks!