Closed olehmisar closed 6 years ago
Yeah pls remove captcha from headless :d
You can do this using chrome's headless remote debugging features and ssh.
On the server run chrome using these arguments:
chrome --headless --remote-debugging-port=9222 https://www.haxball.com/headless
Then on your computer on a new terminal open an ssh tunnel to your server using this command:
ssh username@yourserver -L localhost:9222:localhost:9222
Without closing the ssh connection you just created open localhost:9222 in chrome, you will be able to remotely control the instance of chrome running on your vps from there. You can pass the recaptcha just fine with it, although it's a bit slow.
I downloaded chromium-browser because chrome is not available on my instance. Then I started it in headless mode with remote debugging port 9222. Then connected to it via ssh and opened localhost:9222 on my local machine. But when I click "HaxBall headless host" link, chrome says that page not found. I tried with "www.google.com" link and it still doesn't work.
Commands used:
$ chromium-browser --remote-debugging-port=9222 --headless https://www.haxball.com/headless
$ ssh -i mykeypair.pem ubuntu@********.compute.amazonaws.com -L localhost:9222:localhost:9222
@basro, could you remove captcha, so I will be able to load my bot using node.js or Python on my AWS instance. Or make trusted IPs where captcha does not appear. Today I hosted my "autohost" bot on my computer and everything was great except ping. I spent about 4 days to write this bot and now I can't use it.
I'll probably do something so that the captcha can be solved in a different computer in the future, you'll be able to obtain a special token and use that to start the room.
Headless chrome has worked for me on a vps, you should try solving the problem you are having since at the moment it's the best way to do it.
@basro, I tried to solve this problem but can't find anything useful in Google because remote debugging is not very popular topic. I tried to run chrome with remote debug port on my local machine and it worked. When I try to connect to AWS instance I get 404 error page. Then I tried to make WebSocket directly: new WebSocket(chromeDevToolsUrl)
and got 500 (!) response status. So, what can this mean?
Do you get 404 when you try to connect to localhost:9222 or when trying to follow the link of open tab?
Make sure you are running the same version of chrome in both the AWS instance and your computer.
I get it when trying to follow the link. Oh, maybe versions are different. I will check.
I tried to do this using chromium browser instead of google chrome browser and replicated your problem (it errors 404 when following the link to the tab)
Try using google chrome instead, that should make it work.
Also, remember to allow all UDP ports in the security group or webrtc will fail.
To install chrome browser you can follow the instructions in this SO answer: https://askubuntu.com/questions/79280/how-to-install-chrome-browser-properly-via-command-line
One thing I haven't tried is opening the remote debugging page using chromium browser on my desktop as well as on the VPS, you might want to try that.
@basro, thank you! It works! The problem was incompatibility of chrome and chromium.
Good to hear!
offtopic @basro, in headless host documentation you should mention that all API calls are asynchronous. And make some examples how to start because you use not standard way of binding callbacks. For the first time I tried to write something like this:
onHBLoaded(function() {
const room = HBInit(roomConfig)
room.onPlayerChat(player => {
return false
}
}
You are right about the asynchronous functions, I've added a note to the docs.
@olegmisar @basro Can you clarify how you upload or link your Javascript API code to chrome when you run it in headless mode ? after i open haxball.com/headless using the headless mode, what should i do to create the room using my JS code?
On my computer I use violentmonkey plugin with following code:
const loader = document.createElement('script')
loader.src = 'path/to/script'
document.head.appendChild(loader)
Notice that Firefox, and maybe Chrome, blocks local resources which are loaded using file protocol. This means you can't load scripts from this path 'file:///c:/myscript.js'.
Also if 'haxball.com/headless' uses https protocol then you can't load scripts using not secure http. This is called mixed resources. You can allow mixed resources in your browser config.
You MUST be sure that you load your script BEFORE all haxball scripts are loaded. You must wrap your code with following wrapper function:
window.onHBLoaded = function () {
// Your code here
}
It's because when you load your script you can't be sure that function HBInit()
is initialized. AFTER HBInit()
is created window.onHBLoaded()
is called automatically.
@basro, you should make query string parameter for script url. So, we would be able to load scripts without hacks. Url example: https://www.haxball.com/headless/?myscripturl=https://myawesomebotjs.com/
@olegmisar You mean that i should surround my whole js code with :
window.onHBLoaded = function () {
//all my code
}
This means that i will define functions inside a function ! is this regular in javascript ?
And can you provide a simple walkthrough in something like a GIF or a small video for using the violentmonkey plugin and uploading the code as i didn't get the may so clearly.
Yes, you can define functions inside functions. It's very common in JavaScript. You should use greasemonkey instead of violentmonkey because it is more popular. There are enough resources in google to get started.
P.S. I use Python http.server
to open local server. Using terminal, navigate to folder where your script is. Then enter this command:
$ python -m http.server 8080
Let's assume your name of script is 'bot.js'.
Then go to localhost:8080/bot.js
in your browser. You will see source of your bot.
So, greasemonkey/violentmonkey script will look something like this:
const loader = document.createElement('script')
loader.src = 'http://localhost:8080/bot.js'
document.head.appendChild(loader)
where should the location of the JS file so i can access it using localhost:8080/bot.js
?
It can be located anywhere. For example path to script is 'c:/botfolder/botfile.js'. Then open terminal and enter two following commands:
$ cd c:/botfolder
$ python -m http.server 8080
Note that dollar sign ($) isn't a part of command, it shows that this is terminal command but not code.
So after this folder 'c:/botfolder' "becomes" localhost. And you can access files in this folder through localhost: enter in browser 'http://localhost:8080/botfile.js'.
Thanks @olegmisar i made it !
But currently i'm stuck on the CAPTCHA test, any idea how to pass it?
I need more details.
Can i pass the I'm not a robot test using code? as i want when i open haxbal.com/headless the room is created automatically.
No, you cannot. This is Google's recapcha and it is very secure. Hope for tokens in future.
so how did you create the room using headless chrome ?
@Raamyy, you can connect to headless chrome using the remote debugging port. It lets you solve the captcha and you can input the bot script using the console like you would normally do using chrome's inspector.
@bastro help I am using CentOs on my vps , installed chrome. run it with debbuging port and opened ssh tunnel. localhost:9222 on my pc doesn't work :(
@basro I menaged to make it work somehow but idc what's wrong with my fonts/characters: Also my console is spammed with this warrning (i think it has something to do with webrtc):
Also how can I host 2 rooms ? How to open 2 debbuging tabs?
@basro same problem with console.
@basro you should add new wiki page: "Uploading bot on vps"
I ended up installing gnome gui on my centos which is not "light weight"... but that way I can host 2 rooms and interact with browser directly...
@olegmisar There has been added token for captcha passing in headless browser, did it solve the problem with having to connect to debugging port to solve captcha?
@lmnoup yes. Solve captcha on https://haxball.com/headlesstoken, get that token and use this code to init the room:
const room = HBInit({
roomName: 'room name',
token: 'paste token here'
})
Could someone explain me step by step what should be done to upload the bot to a vps? (note: Vps Used, Commands Used)
I have js host bot. On my computer I use violentmonkey plugin to load bot. Now I want to host room on ubuntu server on AWS. Firefox browser is already installed but i can't figure out how to run host bot. Help me.
UPD: there is recaptcha on haxball.com/headless. Is it impossible to pass it only with command line? If it is, so what is the point of that headless server?