Open janpio opened 5 years ago
The plugin tests (run on CI via paramedic
) themselves would probably have caught that, as they create a project and then see if functionality of the plugins work. But those tests are done with the released Cordova CLI and platforms.
We have the mobilespec project, but I am not exactly sure what that is and how it works.
I just raised https://github.com/apache/cordova-mobile-spec/pull/147 to document a few known quirks & issues:
npm install
insidecordova-js
is needed to resolve a local grunt issue
- A Gradle file in
platforms/android
subdirectory tries to read non-existing debug keys. Workaround is to comment the lines out of the Gradle file.- It may be required to uninstall the
cordova-plugin-compat
plugin due to multiple "BuildHelper" classes defined in dex files (https://stackoverflow.com/questions/46562289/multiple-dex-files-define-lorg-apache-cordova-buildhelper/46562523#46562523)
and not all of the automatic tests are passing for me. It seems to help to select 1 plugin at a time to run the tests on.
So mobilespec
is one thing.
For Android I just discovered this: https://github.com/apache/cordova-android/tree/master/test (Was linked at https://github.com/apache/cordova-coho/blob/master/docs/platforms-release-process.md#android-extras, under https://github.com/apache/cordova-coho/blob/master/docs/platforms-release-process.md#ios-extras mentions something similar for iOS)
So
mobilespec
is one thing.For Android I just discovered this: [...] ([...] mentions something similar for iOS)
Here are the updated Android & iOS unit test links for the sake of clarity, in response to https://github.com/apache/cordova/issues/54#issuecomment-441290852 above:
NOTE that these unit tests are included by npm test
and are run by CI, according to https://github.com/apache/cordova/issues/54#issuecomment-442457744 below.
How is this related to plugins @brodybits?
(All those tests are include in npm test
and thus run by CI, the Android ones for example are run via npm run java-unit-tests
)
How is this related to plugins @brodybits?
Here is my response, along with a motion below.
I provided updated information in response to some information with an outdated link in https://github.com/apache/cordova/issues/54#issuecomment-441290852. I edited my response to clarify.
(All those tests are include in
npm test
and thus run by CI, the Android ones for example are run vianpm run java-unit-tests
)
Thanks for the info, which triggered https://github.com/apache/cordova-coho/pull/213. I added the info to my response above.
I would like to motion the following:
My motion is amended as follows:
And I am doing one more thing as a collaborator: mark the comments hidden so far as "resolved". My rationale is that the first hidden comment I wrote was in response to a comment by someone else (fair discussion) before we decided to apply the strikeout.
So
mobilespec
is one thing.
This and https://github.com/apache/cordova-mobile-spec/issues/184#issuecomment-442827057 imply that there are some other ways to test platform releases and maybe some other package releases that I don't know about. Can you enlighten me and others?
So mobilespec is one thing.
This
That was the conclusion of us finding and understanding mobilespec
as a way to test plugins compatibility on platforms in the comments before (which this topic is about).
apache/cordova-mobile-spec#184 (comment)
The comment you link to is about plugin testing, not platform releases.
Generally plugin testing is done via all the plugin tests, that are run via cordova-paramedic
in CI. But that doesn't offer any (simple) options to control the platform being used.
But if you have such questions, please ask them in a new issue, Slack or dev mailing list thread and don't abuse one of my issues that has a clear topic or question - which is not about enlightening you.
Step back:
What would help to "test platform releases regarding plugin compatibility"? How can we find out if a new platform release breaks anything with plugins?
cdvtests/index.html
via cordova-plugin-test-framework
) still works on platform with plugin installedcordova-paramedic
) still work on platform with plugin installedOther ideas?
Could we maybe somehow compare what a plugin installation does between two platform versions? That could catch if changes in our "copy files from a to b" logic (that caused problems in the past) will break a plugin without executing its code.
I will need some time to evaluate these ideas. I think more comprehensive testing would be good on major releases and occasional minor releases. I would really favor a more straightforward testing procedure for hotfixes and other patch releases.
This exists:
https://github.com/alsorokin/cordova-periodic-build-android https://github.com/alsorokin/cordova-periodic-build-ios
(Take a look at the Travis build history)
Broken right now, but could be a blueprint on how to create a repo that can test platforms with all plugins.
The 7.1.3 release of cordova-android had a breaking bug with multiple plugins that included
app
in their name, including our owncordova-plugin-inappbrowser
.The cordova-android tests did not catch that.
We should figure out a way to better test our platforms with plugins.