Run tests with fb v2 v3 and v4

juarezr commented 4 years ago

Changes

Continuous integration in Github now tests:

building, clippy, code formatting
FirebirdSQL versions 2.5-ss 3.0 4.0
linking, dynamic_loading
client, embedded
coverage

This is a follow up to #25.

juarezr commented 4 years ago

Follow up:

testing (v2, dynamic_loading) looks like a orange factor because testing (v2, linking) succeeded on same test.
testing (v4, linking) and testing (v4, dynamic_loading) are failing but the tests can fixed in further commits/PR.

jairinhohw commented 4 years ago

The V4 versions are failing as the installed client library is version 3.0.2, but the firebird 4.0 needs the Srp256 authentication implemented only in client versions 3.0.4 and above

juarezr commented 4 years ago

On my local computer:

I do not fear computers. I fear the lack of them.
    -- Isaac Asimov
 23:07:21  ~  🙂 
👉 apt search firebird-dev
Sorting... Done
Full Text Search... Done
firebird-dev/focal,now 3.0.5.33220.ds4-1build2 amd64 [installed]
  Development files for Firebird

 23:07:55  ~  🙂 
👉 uname -a
Linux D2019599 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

But ubuntu-latest still defaults to ubuntu-18-04 according to the github docs:

Note: The Ubuntu 20.04 virtual environment is currently provided as a preview only. The ubuntu-latest YAML workflow label still uses the Ubuntu 18.04 virtual environment.

Options for now:

Disable v4 until github upgrades to ubuntu-20-04
Fix ubuntu-20-04 for testing and change it later when github upgrades.

jairinhohw commented 4 years ago

Let's test with fixing ubuntu-20-04. If it doesn't work, we disable the V4 for now.

juarezr commented 4 years ago

Upgrading from ubuntu-latest to ubuntu-20-04 installed version 3.0.5.

Most of the errors gone except this error.

Any clues?

(Also testing (v2, linking) still fails and testing (v2, dynamic_loading) don't for some unknown reason.)

jairinhohw commented 4 years ago

I think the cargo tarpaulin can be removed from all jobs except the coverage one to reduce this kind of problem.

About the V2 dynamic, i don't know either.

juarezr commented 4 years ago

I think the cargo tarpaulin can be removed from all jobs except the coverage one to reduce this kind of problem.

@jairinhohw

According to the Coveralls Github action documentation there is 2 modes of operation: Standard and Parallel.
1. The mode that used in the job testing is the parallel: true
  1. Step Run cargo-tarpaulin generates the Lcov coverage file
  2. Step Coveralls Parallel sends the partial coverage to coveralls.io
2. The mode that used in the job coverage is the parallel-finish:true
  1. This jobs runs after the job testing
  2. Step Coveralls Finished finishes the coverage in coveralls.io
3. By doing this way, we collect coverage for each configuration defined in the job testing:
  1. This covers linking, dynamic_loading, v2, v3 and v4
  2. As further improvement we can include embeddedlater.
4. I may have misunderstood something, off course.
There is just one remaining error:
1. This error is triggered by cargo tarpaulin on firebird=v4
2. The test failing is: test statement::linking::old_api_select ... FAILED. See the backtrace bellow.
3. The test fails both in linking and dynamic_loading.
Some possible options are:
1. Proceed with the test failing and fix later.
2. Exclude coverage from v4 and enable later after the test is fixed. (If someone would remember...)
3. Wait for the fix.
4. Get rid of coverage completely.

About the V2 dynamic, i don't know either.

This failue is **Weird*** !!!
The steps succeeds with dynamic_loading but fails with linking
The logs on the failure are shorter also.
It needs further investigation.

LOGS

Failing test on V4

---- statement::linking::old_api_select stdout ----
thread 'statement::linking::old_api_select' panicked at 'Error in the transaction: FbError { msg: "deadlock\nupdate conflicts with concurrent update\nconcurrent transaction number is 93", code: -913 }', src/statement.rs:433:9
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/libunwind.rs:86
   1: backtrace::backtrace::trace_unsynchronized
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print_fmt
             at src/libstd/sys_common/backtrace.rs:78
   3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
             at src/libstd/sys_common/backtrace.rs:59
   4: core::fmt::write
             at src/libcore/fmt/mod.rs:1076
   5: std::io::Write::write_fmt
             at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54/src/libstd/io/mod.rs:1537
   6: std::io::impls::<impl std::io::Write for alloc::boxed::Box<W>>::write_fmt
             at src/libstd/io/impls.rs:176
   7: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:62
   8: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:49
   9: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:198
  10: std::panicking::default_hook
             at src/libstd/panicking.rs:215
  11: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:486
  12: rust_begin_unwind
             at src/libstd/panicking.rs:388
  13: core::panicking::panic_fmt
             at src/libcore/panicking.rs:101
  14: core::option::expect_none_failed
             at src/libcore/option.rs:1272
  15: core::result::Result<T,E>::expect
             at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54/src/libcore/result.rs:963
  16: rsfbclient::statement::linking::setup
             at src/statement.rs:433
  17: rsfbclient::statement::linking::old_api_select
             at src/statement.rs:286
  18: rsfbclient::statement::linking::old_api_select::{{closure}}
             at src/statement.rs:285
  19: core::ops::function::FnOnce::call_once
             at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54/src/libcore/ops/function.rs:232
  20: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once
             at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54/src/liballoc/boxed.rs:1076
  21: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54/src/libstd/panic.rs:318
  22: std::panicking::try::do_call
             at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54/src/libstd/panicking.rs:297
  23: std::panicking::try
             at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54/src/libstd/panicking.rs:274
  24: std::panic::catch_unwind
             at /rustc/d3fb005a39e62501b8b0b356166e515ae24e2e54/src/libstd/panic.rs:394
  25: test::run_test_in_process
             at src/libtest/lib.rs:541
  26: test::run_test::run_test_inner::{{closure}}
             at src/libtest/lib.rs:450
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

failures:
    statement::linking::old_api_select

test result: FAILED. 17 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out

Error: "Test failed during run"
##[error]The process '/usr/share/rust/.cargo/bin/cargo' failed with exit code 1

Failing test on V2

Status: Downloaded newer image for jacobalberty/firebird:2.5-ss
2ff45fd5168766b766fdab6e105c97d3972c0c372fb8b96bedb6209387661ab1
##[group]Run docker exec firebirdsql /bin/bash '/usr/local/firebird/bin/createAliasDB.sh' 'test.fdb' '/firebird/data/test.fdb'
[36;1mdocker exec firebirdsql /bin/bash '/usr/local/firebird/bin/createAliasDB.sh' 'test.fdb' '/firebird/data/test.fdb'[0m
[36;1mdocker container restart firebirdsql[0m
[36;1msleep 10[0m
shell: /bin/bash -e {0}
##[endgroup]
Use CONNECT or CREATE DATABASE to specify a database
Statement failed, SQLSTATE = 28000
Your user name and password are not defined. Ask your database administrator to set up a Firebird login.
##[error]Process completed with exit code 1.

Successful test on V2

Status: Downloaded newer image for jacobalberty/firebird:2.5-ss
9fc96ffd6f3c036ba5d2188264714ffa6002d391cd3e9f4b57309d98e99dcb23
##[group]Run docker exec firebirdsql /bin/bash '/usr/local/firebird/bin/createAliasDB.sh' 'test.fdb' '/firebird/data/test.fdb'
[36;1mdocker exec firebirdsql /bin/bash '/usr/local/firebird/bin/createAliasDB.sh' 'test.fdb' '/firebird/data/test.fdb'[0m
[36;1mdocker container restart firebirdsql[0m
[36;1msleep 10[0m
shell: /bin/bash -e {0}
##[endgroup]
firebirdsql
##[group]Run SQL='select RDB$CHARACTER_SET_NAME as charset_fbv2 from rdb$database;' 
[36;1mSQL='select RDB$CHARACTER_SET_NAME as charset_fbv2 from rdb$database;' [0m
[36;1mecho $SQL | isql-fb -bail -quiet -z -user SYSDBA -password masterkey 'localhost:/firebird/data/test.fdb'[0m
shell: /bin/bash -e {0}
##[endgroup]
ISQL Version: LI-V3.0.2.32703 Firebird 3.0
Server version:
LI-V2.5.9.27139 Firebird 2.5
LI-V2.5.9.27139 Firebird 2.5/tcp (9fc96ffd6f3c)/P12
LI-V3.0.2.32703 Firebird 3.0/tcp (fv-az76)/P12

CHARSET_FBV2                    
=============================== 
UTF8

coveralls commented 4 years ago

Pull Request Test Coverage Report for Build 207619332

0 of 0 changed or added relevant lines in 0 files are covered.
37 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-8.7%) to 67.843%

Files with Coverage Reduction	New Missed Lines	%
src/connection/mod.rs	18	49.7%
rsfbclient-native/src/connection.rs	19	59.51%
<!--	Total:	37	-->

Totals
Change from base Build 202332703:	-8.7%
Covered Lines:	846
Relevant Lines:	1247

💛 - Coveralls

juarezr commented 4 years ago

I done with this PR.

Anybody has any considerations about these changes?

jairinhohw commented 4 years ago

I think that it will be better to change the embedded feature to embedded_tests or something. I'm approving the changes

juarezr commented 4 years ago

I think that it will be better to change the embedded feature to embedded_tests or something. I'm approving the changes

I'm unsure about what is the best approach related to conditional compilation and this features.

Certainly we can improve further this.

fernandobatels commented 4 years ago

I think that it will be better to change the embedded feature to embedded_tests or something. I'm approving the changes

I'm unsure about what is the best approach related to conditional compilation and this features.

Certainly we can improve further this.

I agree with the @jairinhohw. We can use prefixes like 'tests_' for features only used in tests.

coveralls commented 1 month ago

Pull Request Test Coverage Report for Build 206141939

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

0 of 0 changed or added relevant lines in 0 files are covered.
53 unchanged lines in 3 files lost coverage.
Overall coverage decreased (-10.0%) to 66.56%

Files with Coverage Reduction	New Missed Lines	%
rsfbclient-native/src/ibase.rs	3	25.0%
rsfbclient-native/src/connection.rs	25	56.59%
src/connection/mod.rs	25	45.45%
<!--	Total:	53	-->

Totals
Change from base Build 202332703:	-10.0%
Covered Lines:	830
Relevant Lines:	1247

fernandobatels / rsfbclient