flutter / website

Flutter documentation web site
https://docs.flutter.dev
Other
2.81k stars 3.22k forks source link

'Integration testing' page issue - Update iOS instructions #5645

Closed neiljaywarner closed 3 years ago

neiljaywarner commented 3 years ago

Page URL: https://flutter.dev/docs/testing/integration-tests/ Page source: https://github.com/flutter/website/tree/master/src/docs/testing/integration-tests/index.md

1) iOS is sparse compared to Android in the screenshots and explanations 2) If you go to the part that says "iOS Setup Follow the instructions in the iOS Device Testing section of the README." it takes you to pub.dev - which i believe is no longer correct, pub.dev listing is deprecated for this plugin since it has been moved and should be installed with flutter root instead of "1.0.2" 3) the instructions end with "Now you can start RunnerTests to kick-off integration tests!" which makes no sense to me or an ios dev.

i did eventually find https://github.com/flutter/flutter/tree/master/packages/integration_test which looks more promising

Description of issue:

johnpryan commented 3 years ago

Follow the instructions in the iOS Device Testing section of the README." it takes you to pub.dev - which i believe is no longer correct

Yep, it's included in the SDK now(https://github.com/flutter/flutter/issues/80607). We updated the integration_test docs to reflect this, but not the pub package. (We need to mark this package as discontinued: https://github.com/flutter/flutter/issues/80607)

dependencies:
  integration_test:
    sdk: flutter

the instructions end with "Now you can start RunnerTests to kick-off integration tests!" which makes no sense to me or an ios dev.

Can you add more detail here? Should we change this to "Now you can start the RunnerTests target" instead?

neiljaywarner commented 3 years ago

@johnpryan thanks for asking 1) many devs will come from Android or somewhere else and have no Xcode or iOS knowledge at all so instructions as specific as possible would be great. 2) the zip file uploaded to firebase test lab includes neither an IPA or an xctest file if I recall. It's understandable that firebase test lab came first but it would be awesome if you could help us out with instructions that will work for other services...i.e. aws device farm in particular.. especially since sylph was first but integration_test plugin is much better.

Going back to the old way for integration tests just to get iOS seems awful...

neiljaywarner commented 3 years ago

@johnpryan @dnfield flutter.dev says "The integration_test package is now the recommended way to write integration tests. " - which leads me to believe that going back to sylph https://pub.dev/packages/sylph is not a good idea because it uses old flutter_driver appium/other type config i think rather than espresso/test-apk type tests

neiljaywarner commented 3 years ago

In other words. What do we do for aws device farm to get xctest instead of xctestrun

neiljaywarner commented 3 years ago

@johnpryan any update or further thoughts at this point please? We are thinking of making a ticket with AWS as well.

johnpryan commented 3 years ago

The Integration testing on flutter.dev points to the correct integration_test README in the SDK now, which is the source of truth for building the iOS test target and uploading to Firebase TestLab. We don't have plans to copy this text into flutter.dev at the moment.

We could add some information on uploading the ios_tests.zip file to Firebase Testlab. They should be the same as the official Firebase instructions though, so I'd rather link to them for now.

https://github.com/flutter/website/pull/5954 adds links to the README and Firebase TestLab documentation for information on uploading the ZIP file.