Can one run `fal test`?

[CavalcanteLucas]

I'm trying to make something like this:

fal test --profiles-dir . --target dev --selector my_data --vars '{my_var: "my_var_value"}'

Is it currently supported?

[chamini2]

Hello @CavalcanteLucas, thanks for reaching out. We currently don't have a test command. I am curious what would you expect a fal test command to do? Specifically, what would it do different than dbt test, for example?

[CavalcanteLucas]

Thank you for your quick answer @chamini2

I wondered if the user could associate fal with DBT tests. I assumed we could establish let's say a relationship test like:

version 2

models:
    - name: my_model
    - name: my_other_model
        columns:
            - name: my_model_id
                tests:
                    - relationships:
                        to: ref('my_model')
                        field: id
        meta:
            fal:
                scripts:
                    - fal_script.py

And, in case the DBT test failed, fal would trigger the script fal_script.py that would send a slack message.

Is there maybe an alternative way to accomplish this?

[chamini2]

There is a way to accomplish this today.

You can run dbt test followed by fal run. In this script we have an example of checking for tests: https://github.com/fal-ai/fal/blob/2b9247eba84dab88225bc106c11681e477f1e6b7/projects/fal/integration_tests/projects/005_functions_and_variables/fal_scripts/write_to_source_twice.py#L25-L36

This is the output of running it:

❯ dbt test
00:04:27  Running with dbt=1.4.5
00:04:27  Found 3 models, 4 tests, 0 snapshots, 0 analyses, 299 macros, 0 operations, 0 seed files, 2 sources, 0 exposures, 0 metrics
00:04:27
00:04:28  Concurrency: 3 threads (target='fal_pg')
00:04:28
00:04:28  1 of 4 START test not_null_some_model_my_int ................................... [RUN]
00:04:28  2 of 4 START test not_null_some_model_my_text .................................. [RUN]
00:04:28  3 of 4 START test source_not_null_results_ns____ns__some_source_my_int ......... [RUN]
00:04:28  1 of 4 FAIL 1 not_null_some_model_my_int ....................................... [FAIL 1 in 0.04s]
00:04:28  3 of 4 FAIL 2 source_not_null_results_ns____ns__some_source_my_int ............. [FAIL 2 in 0.04s]
00:04:28  2 of 4 PASS not_null_some_model_my_text ........................................ [PASS in 0.04s]
00:04:28  4 of 4 START test source_not_null_results_ns____ns__some_source_my_text ........ [RUN]
00:04:28  4 of 4 PASS source_not_null_results_ns____ns__some_source_my_text .............. [PASS in 0.01s]
00:04:28
00:04:28  Finished running 4 tests in 0 hours 0 minutes and 0.34 seconds (0.34s).
00:04:28
00:04:28  Completed with 2 errors and 0 warnings:
00:04:28
00:04:28  Failure in test not_null_some_model_my_int (models/schema.yml)
00:04:28    Got 1 result, configured to fail if != 0
00:04:28
00:04:28    compiled Code at ./target/compiled/fal_005/models/schema.yml/not_null_some_model_my_int.sql
00:04:28
00:04:28  Failure in test source_not_null_results_ns____ns__some_source_my_int (models/schema.yml)
00:04:28    Got 2 results, configured to fail if != 0
00:04:28
00:04:28    compiled Code at ./target/compiled/fal_005/models/schema.yml/source_not_null_results_ns____ns__some_source_my_int.sql
00:04:28
00:04:28  Done. PASS=2 WARN=0 ERROR=2 SKIP=0 TOTAL=4

❯ fal run
00:04:36  Found 3 models, 4 tests, 0 snapshots, 0 analyses, 299 macros, 0 operations, 0 seed files, 2 sources, 0 exposures, 0 metrics
00:04:36 [fal]: [WARNING]: Could not read dbt sources artifact
00:04:36 [fal]: Starting fal run for following models and scripts:
(some_model, fal_scripts/write_to_source_twice.py)

# this file is created during the fal run
❯ cat some_model.write_to_source_twice.txt
my_float 1.2
source size 1
source size 2
source results.other_source has 0 tests, source status is skipped
source results.some_source has 2 tests, source status is tested
model some_model has 2 tests, model status is tested
model third_model has 0 tests, model status is skipped
model other_model has 0 tests, model status is skipped

[chamini2]

You can check more information in https://docs.fal.ai/fal/fal-dbt/fal-with-dbt-test

[chamini2]

Please feel free to re open if this does not satisfy a use case you have! I will be closing for now.

Thanks for reaching out!