search

google / webcrypto.dart

Cross-platform implementation of Web Cryptography APIs
https://pub.dev/packages/webcrypto
Apache License 2.0
81 stars 48 forks source link

Add test coverage collection with coveralls #134

Closed HamdaanAliQuatil closed 3 weeks ago

HamdaanAliQuatil commented 5 months ago

Closes #133

I've used the release v2 in this collection (https://github.com/coverallsapp/github-action/releases). It'd be interesting to know why dart-lang uses the specific commit and if we need to do the same.

jonasfj commented 5 months ago

I enabled coveralls.io for this repository.

We can add the following badge to the README too. Though it's not much fun until we've figured out how to get coverage numbers out of the various tests, combined them and submit them.

[![Coverage Status](https://coveralls.io/repos/github/google/webcrypto.dart/badge.svg)](https://coveralls.io/github/google/webcrypto.dart)

It would also be cool to collect coverage numbers from integration tests, but that's probably hard to do, and I'm not sure it'll increase coverage much. Because it's the same tests we run :D

Probably just having coverage numbers for normal tests on Linux, Chrome and maybe Windows and Mac would be fine.

jonasfj commented 5 months ago

Weird, that the tests seem to fail:

🎉 1433 tests passed.
Error: Process completed with exit code 1.

Do they also fail locally when running with --coverage? Like what is the exit code locally?

It seems like the test actually pass, but that the program still exits non-zero. Maybe there is a bug somewhere. Maybe this is specific to --platform chrome, or?

Worth investigating for sure.

HamdaanAliQuatil commented 4 months ago

Weird, that the tests seem to fail:

🎉 1433 tests passed.
Error: Process completed with exit code 1.

Do they also fail locally when running with --coverage? Like what is the exit code locally?

It seems like the test actually pass, but that the program still exits non-zero. Maybe there is a bug somewhere. Maybe this is specific to --platform chrome, or?

Worth investigating for sure.

--platform chrome gives me a timeout error everytime. All tests passed in flutter test --coverage : https://gist.github.com/HamdaanAliQuatil/77f9b09bba4929171c9fa11b3949250a

jonasfj commented 4 months ago

--platform chrome gives me a timeout error everytime. All tests passed in flutter test --coverage :

But it works on Github Actions. This might be windows specific, it might be specific to your setup. Feel free to debug it, you can try to see if flutter pub run test --platform chrome works, and you can try to do verbose mode, etc.

But it doesn't seem related to this issue. Notice that all tests are passing on the master branch.

HamdaanAliQuatil commented 3 weeks ago

@jonasfj perhaps we don't need this now XD. Should I close this?

jonasfj commented 3 weeks ago

Yeah, let's close this.

I don't think we'll get much improvement by collecting coverage from iphone and android emulators, maybe a little, but I'm guessing not much.

We can explore it at some point in the future, but I think our immediate focus should be finishing the refactoring and write more tests :D