Could we automate this?

Glavic commented 6 years ago

Hi. I would like to make a proposition to automate finding version, timestamp and secret. For the tool I have chosen Docker, but I'm open for other tools.

All you need to do is run Docker command: docker run --rm glavich/foe-decryption

Expected output would be:

Currently I only added fetch for secret. I would still like to implement:

~~fetch version number (can you explain where can I find that)~~ tnx: @VividWombat
~~make some Docker modification~~
~~try to minimize docker image size (don't know how, since Java8 is so big)~~ tnx: @m3talstorm
~~make some nicer print-out of variables fetched~~

Why say u? Please don't merge this yet. First lets fix all of the above.

m3talstorm commented 6 years ago

Hey @Glavic, very cool idea :) I'm liking the looks of this. I'll try and have a better look when i'm home tonight.

VividWombat commented 6 years ago

yes, nice idea.

if you use scripts/de/innogames/strategycity/Version you get nice access to the version number and date. we only need the secret, but its good to confirm what version it is. then any code using the secret can check for version update, and automatically pull the new secret when needed.

m3talstorm commented 6 years ago

You can probably use the alpine based docker image to cut down its size:

https://hub.docker.com/_/openjdk/

Glavic commented 6 years ago

@m3talstorm & @VividWombat : tnx 👍

Another question: why is VERSION_SECRET same as SECRET? One is found in de.innogames.shared.networking.providers.JSONConnectionProvider and other in de.innogames.strategycity.Version.

m3talstorm commented 6 years ago

@Glavic No idea, maybe they just duplicate it :)

Glavic commented 6 years ago

Any more ideas?

@m3talstorm before merge (if there will be any):

you should add project to your https://hub.docker.com
replace https://github.com/Glavic/foe-decryption/blob/7fd88f1cd2f4ed5864898b2e8a8b05fcc78aec47/Dockerfile#L16 with your repo
remove this line https://github.com/Glavic/foe-decryption/blob/7fd88f1cd2f4ed5864898b2e8a8b05fcc78aec47/Dockerfile#L17
replace origin/Docker with origin/master in https://github.com/Glavic/foe-decryption/blob/7fd88f1cd2f4ed5864898b2e8a8b05fcc78aec47/script.sh#L7
fix instruction in README how to run docker

I think that is it...? :)

mwsupra commented 6 years ago

I also like the idea. I tried running it but it didn't work on my system (armhf instead of amd64). :(

Glavic commented 6 years ago

@mwsupra: do yo have docker installed? If yes, what kind of error do you receive?

update1: I misread that you wrote armhf :/

The automated build feature on the official Docker Hub only runs the normal x86_64 docker implementation. Since arm code is not something an x86_64 CPU can execute, that’s why you get the ‘exec format failure’ on the automated build system. the normal ‘node’ image is an x86_64 image, so it works.

Based on that, it is not currently possible to run this image on ARM :(

update2: on other hand, you can build docker image yourself, on your arm machine. All you need to do is:

copy this file to your machine
run command docker build -t test .
after build is done, just run docker run --rm test

mwsupra commented 6 years ago

Your solution worked beautifully. Tool gives the info needed. Thanks for the tip!

paskyorg commented 6 years ago

Is it possible to extract the information with Python? I see that ffdec is used for that extraction, but i would like automate this only with Python to include it in other script.

Glavic commented 6 years ago

@paskyorg: of course you can extract all the info with python. All I did is throw some basic commands together that can also be called from python, but for me it is easier to run one command, than to have python, java jre and ffdec (+bunch of bash commands I don't have on my Windows) installed on my PC. On other hand, in docker image there is always alpine linux os, no matter what os does user have, and all commands will run the same; where in python script you would need to detect users os and correspond accordingly (more work).

paskyorg commented 6 years ago

Innogames is going to change the Flash technology by HTML5, so we will no longer have to decompile swf files. In the beta version, everything is in a javascript file, in the _createKey function:

https://zz1.forgeofempires.com/cache/ForgeHX-b1794b96.js

_createKey:function(a,b){
   return va.substr(
      wia.hash(
         a +
         "skuRD0TZbEyVRF7L6XWy9oWh2OOp0InC2mrQmJaK/FIeSAstQB61dRMlVX2dMM7aJtHHGtYnRS4+39P20/1h6g=="
         + b
      ),0,10
   )
}

m3talstorm commented 6 years ago

@paskyorg Haha funny, whats even the point :)

VividWombat commented 6 years ago

I checked this a few days ago. you dont need to create the key send a GET request to https://zz1.forgeofempires.com/game/index?ref=

response includes -

'string_gatewayUrl': 'https://zz1.forgeofempires.com/game/json?h=hPs6HJxs4UPEmkL3lrZtYTr-',

paskyorg commented 6 years ago

@VividWombat how do you generate the Signature header?

VividWombat commented 6 years ago

I was commenting on the user key. you can use that for auto-login. the signature is done using the function you found, presumably

Glavic commented 6 years ago

VERSION = 1.128
VERSION_SECRET = UwcqrG5ECK1Amne9nE4iHY7HPQhY9qaVSDxfBSuT67Mixv4jSASPzuqKr8lAYFjtsllmgZmH18ryDaHplaUFfw==
BUILD_TIME = 11.06.2018 10:38
BUILD_NUMBER = 4ea9d0a68d

Glavic commented 5 years ago

After half an year, this is still required :)

VERSION = 1.133
SECRET = BPmkpdLgpRePZjT4bAHnfI+IqyTv3tkoZNBo9ZtUzjHaZCYiBxWTo9Nap4IZZmEDm66g9+NalSsNQbeqeieKCg==
BUILD_TIME = 22.08.2018 10:44
BUILD_NUMBER = cc2d5474f9

m3talstorm commented 5 years ago

@Glavic I don't play FoE anymore, thus this isn't maintained.

kaf3in0 commented 5 years ago

Hello! I am new to python and I would need someone to help me.

My goal is to scrape the Guild Treasury Contribution window from FOE for better management for my guild, transforming it into a spreadsheet and using that spreadsheet to keep track of minimum donations and so on. I would prefer if I could make it based on HTML5.

I have tried to look through @m3talstorm's code for both, the decryption tool and the BOT itself to see how I can achieve this, but failed to understand most of it. I then tried scraping by grabbing screenshots of the area and then using OCR (pytesseract) to transform it to text, which was easier for me to understand how to use, however the results were inconsistent.

I don't expect you to feed me the code for that, but I would appreciate it if you could guide me to what I should look into for achieving my goal.

I am posting this here because it seems to be the most active discussion.

Thanks!

diogofacin commented 5 years ago

@kaf3in0 ,

OCR for gaming is not that safe, eventually GUI will change, you will have resolution problems, or will have the need to maintain the active game window.

FOE's web services are pretty straight forward to understand, you could have your own account (or a bot account inside your guild) only to keep track of those donations at real-time (integrate that with an online spreadsheet like Google's and it would be awesome), and your bot could do much more like: auto-donate its production, support guild members with trading, etc...

And the most important, you could manage dozens of accounts at the same time using just a Raspberry.

Besides improving your coding skills, download a traffic monitor like "Fiddler 4" (or just use your browser console F12 for Chrome/Firefox), you will see how clear is every request/response made and how easy is to fetch the needed data with 100% accuracy (not possible to achieve with an OCR)

I made from scratch a whole bot to manage the guild/account using Java, but the game become so much boring after that as I didn't even need to play anymore...

Once learned, this kind of skill will allow you to make bots for any online based game, think in long term... :+1:

m3talstorm commented 5 years ago

@alucardeck Nice summary

m3talstorm / foe-decryption

Could we automate this? #17