gwaredd / unium

Automation for Unity games
MIT License
227 stars 44 forks source link

Running unium at AWS Device Farm #36

Open daemon4d opened 5 years ago

daemon4d commented 5 years ago

Hi. Thanks for fantastic framework. One of my big wishes will be to run this on public device farms such as AWS Device Farm. Do you have any ideas how to approach implementation of this feature. As a programmer I can certainly contribute to code this feature. My first idea is that we can implement special appium driver to forward our messages via appium server.

gwaredd commented 5 years ago

Hey, I have been giving this some thought and it should be possible as-is without any modification.

Appium and Unium can already work together to give you the benefits of both.

My understanding is that if you create a custom test environment, you can enable port forwarding with adb within the device farm just as you would for an emulator. I am not sure how you would do this with iOS though.

When I have time I shall give it a go.

daemon4d commented 5 years ago

Hi. Hm, one problem with Device Farm devices is that you don't have complete control over devices running in the Farm. Basically you can't touch device configuration parameters or open ports on devices or appium hosts in the cloud. So my idea is to use cloud native services like message queues in the cloud. In my solution I just use two message queues, one for commands stream, one for results stream. I pushed commands to commands stream just using AWS Web Console as client. And then I observed execution result in the same console but results queue. It's really very simple approach. Much simpler then using a lot of networking configurations and forwarders, port openings, etc. Amazon SQS is standard web services, it also has AWS Unity SDK client. I used that to consume messages from queues on client side and push execution results into results queue.

ср, 24 апр. 2019 г. в 07:54, Gwaredd Mountain notifications@github.com:

Hey, I have been giving this some thought and it should be possible as-is without any modification.

Appium and Unium can already work together https://youtu.be/UbPk2VljW78 to give you the benefits of both.

My understanding is that if you create a custom test environment, you can enable port forwarding with adb https://forums.aws.amazon.com/thread.jspa?messageID=882269&#882269 within the device farm just as you would for an emulator. I am not sure how you would do this with iOS though.

When I have time I shall give it a go.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gwaredd/unium/issues/36#issuecomment-486067360, or mute the thread https://github.com/notifications/unsubscribe-auth/AAUWWNWCUCBVETYSDS3J5PDPR7RZ5ANCNFSM4G6EVURQ .

-- Дмитрий Сидоренко Mobile: +79267943540 mailto: daemon4d@gmail.com Skype: daemon4d

gwaredd commented 5 years ago

It's a good approach, and would also work with iOS. It would make tests less portable though.

Port forwarding on android is not complicated via adb. I haven't looked into iOS.

I don't know enough about AWS to have an informed opinion here.

daemon4d commented 5 years ago

We can make it protable if needed. Just use some RabbitMQ message brocker instead of Amazon SQS. With RabbitMQ you can launch it at any internet or LAN connected computer. But I wonder if you need server part to be portable. Because AWS SQS is public web service. Which means basically that you can access it from any device having Internet connection. It also works with standard SSL 443 port. This means that you will not hit any issues with firewall configuration of your Internet provider. I have a lot of experience building and running services in AWS Cloud. Building and running service that provides you commands queue and then allows you to push execution results into another queue is what we need here. I don't want to open ports or configure proxies. Because it's just another layer of networking configuration that I should do to use my simple queues. But when I'll make this all configurations(if possible) I'll realize that it's just fancy way of implementing the same two queues.

ср, 24 апр. 2019 г. в 11:19, Gwaredd Mountain notifications@github.com:

It's a good approach, and would also work with iOS. It would make tests less portable though.

Port forwarding on android is not complicated via adb. I haven't looked into iOS.

I don't know enough about AWS to have an informed opinion here.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gwaredd/unium/issues/36#issuecomment-486119971, or mute the thread https://github.com/notifications/unsubscribe-auth/AAUWWNUQCP5OOMWEH2J6HKLPSAJ2LANCNFSM4G6EVURQ .

-- Дмитрий Сидоренко Mobile: +79267943540 mailto: daemon4d@gmail.com Skype: daemon4d

gwaredd commented 5 years ago

I don't think it will hurt to add.

Unium has always been created from the view of a game developer first, tester second and everyone else ignored - so some architectural decisions have been influenced by that (like that lack of POST requests). So I still want to figure out a unified approach if possible where tests just work regardless of where they are running.

ldrly-roger commented 4 years ago

I'm working on this starting ... right about now. I believe all one really needs is a very lightweight Appium driver. AWS Device Farm will happily run Appium tests, so it should be feasible to upload a compiled application (with Unium enabled) along with an appium test suite zipped up with this driver. If I understand it correctly, writing some appium tests in python and bundling it with the Unium appium driver also written in python, device farm should be able to execute the tests against the application.

It's a parallel approach to what's been done with AltUnityTester + AltUnityRunner. For an example see https://gitlab.com/altom/altunity/altunitytester/-/tree/master/aws-device-farm-example for the tests and https://pypi.org/project/altunityrunner/ for the Appium driver that it will bundle up.

gwaredd commented 4 years ago

Hey, sounds pretty cool! I don't know enough about the inner workings of appium or it's drivers to comment but sounds feasible to me 😀 Let me know how you get on.

drapostolos commented 4 years ago

Hey, I have been giving this some thought and it should be possible as-is without any modification.

as said above, Unium should work as-is in AWS Device farm custom environment.