Proposition: the external agent project structure

[GoogleCodeExporter]

I appended an illustration of the structure change I have in mind. From my 
understanding of this project, here are the pros, cons and aspects sorted 
by devastation:

#1- The AIModule.dll is loaded into Starcraft, is given full memory access 
with no way to detect cheating. This lowers BWAPI's credibility as a 
competition platform. Since only "legal" data is copied to the shared 
memory, no undetected cheating is possible from an AIModule.exe.

#2- Debugging is currently not possible, but even if it will be, the 
AIModule.dll will be debugged in the starcraft memory space. The 
AIModule.exe can obviously be debugged and stepped-through quickly and 
independently.

#3- A contra. Dynamic information classes like UnitImpl will be hidden in 
BWAPI.dll. Unit will have part of that functionality exported from the 
BWAPIAgent.dll. Essentially code will have to be splitted. This means no 
gradual transition from one structure to another, but a major revision.

#4- Since AIModule.exe's runtime is independent of Broodwars, you could 
close AIModule.exe, recompile and start again without even pausing the 
game in starcraft. Also, menu controling AIs become more managable.

#5- Starcraft multithreads and is forcing BWAPI to do the same, including 
synchronisation and instruction queueing, instead of simple calls to the 
draw functions (ideal case). There is no drawback in instruction caching 
over the bridge. In fact, if implemented properly performance will improve.

#6- A contra. AIModule's event handlers are usually called directly, but 
will now have to be queued over the bridge. Considering that the event 
handlers are not called very often, I just noticed that this is not much 
of a performance issue =D

In case you accept, I would like to join the team.

beeep ;)

Original issue reported on code.google.com by goo...@teabix.com on 23 Nov 2009 at 5:50

Attachments:

• plan.png

[GoogleCodeExporter]

Sorry I'm not posting much explanations now, using 70% of my day to make a 
clean 
BWAPI port to this new system, 30% for eat and sleep ;). No really there are 
not 
lots of days left where I could just sit in front of the PC all day. Just 
feeling 
kind of bad about jamming the work of my fellow commiters... But I can tell you 
now 
I'm leaving practically all the BWAPI2 interface porting to you =P

PS.: The dependencies of this project were a MESS! It's really lots of work...

Original comment by goo...@teabix.com on 7 Dec 2009 at 1:29

[GoogleCodeExporter]

Original comment by lowerlo...@gmail.com on 7 Dec 2009 at 6:08

• Added labels: Type-Enhancement

[GoogleCodeExporter]

I built a Java bridge using strong ties w/ the headers and JNI, so I am worried 
about
mass overhauls. I have seen statements in this issue that somewhat confirms 
this, but
I would like a definitive answer on a couple of questions to provide myself and
others some peace of mind:

1. Will this 100% preserve backwards compatibility with 2.4 as far as method 
names?
Of course, any method name changes or method additions as part of the "3.0" 
release
but not specifically part of this issue notwithstanding.

2. Will the previous method (i.e. not using the agent or the shared memory 
solution)
of a single DLL compiled off of the common libs still work? Will it be 
"supported" in
future revisions or die off?

I would also love to know any ballpark guess of the release date or completion
percentage, but I know that's hard on a personal project like this.

Sorry for all the annoying questions...thanks in advance for the info.

Original comment by chad.r...@gmail.com on 7 Dec 2009 at 11:14

[GoogleCodeExporter]

-"Sorry for all the annoying questions..."
Not at all, you chose the right place to ask.

-"strong ties w/ the headers"
We now use an export header generator, the header files will be all be 
differently 
named. 

1. Yes. The BWAPI2 interface is planned to be as compatible with the 2.4 
interface 
in naming and functionality as the new system allows. There is only one major 
change: Currently you have to use the thread provided by BWAPI. Afterwards you 
will 
have to provide your own thread by e.g. calling a blocking function.

2. No. We want BWAPI to become an approved platform for AI battles, and the 
current 
implementation has no means to prevent cheating, so it will be not supported. 
But 
transition of existing bots and hopefully your wrapper will be easy thanks to 
BWAPI2. This bridge provides full transparency, you will not see any of the 
shared 
memory solution, neither by using the new BWAPI1 interface nor BWAPI2.

Since you are using JNI you might consider switching to the new BWAPI1 
interface if, 
and it most probably will be native.

completion percentage is indeed hard in this early stage, but if you're asking 
about 
proportions, then it's rather a week than a day or a month. We will start 
publishing 
a documentation for BWAPI1 when major functionality is implemented.

Original comment by goo...@teabix.com on 8 Dec 2009 at 12:07

[GoogleCodeExporter]

It looks like functionality keeps being removed with no end in sight. I would 
have
never agreed to this proposition if I knew Teabix intended to rewrite all of 
BWAPI
from scratch. I think it might be worth considering reverting back to a previous
revision. Our best choice may be to revert back to r1762, since at that stage 
we had
most of the Unit, lag compensation, and Order functionality still in place and a
pretty straightforward road map of how to complete the transition to the new 
design,
which is a lot better than the current state of affairs which has virtually no 
unit
functionality.

Original comment by lowerlo...@gmail.com on 11 Dec 2009 at 3:22

[GoogleCodeExporter]

The uprooting of the functionality has already ended. And I'm not actually 
doing it 
from scratch, I use your code as a template. The change is actually less 
drastic 
than it might seem.

If you decide to revert, I'd like to keep working on the current setup. I've 
heared 
SVN has some kind of a "branch" feature where you can develop 2 branches of the 
project at the same time in the same SVN.

As this is already in the right thread, here is what I've got so far: I've 
developed 
2 approaches to simplify pipe and memory communication to make the BridgeCliend 
and 
BridgeServer files more maintainable. The first one turned out to be too 
general for 
this purpose, I had to revert it. In such cases my second one usually turns out 
to 
be a win. I've not got the nerve for that right now, so I'm making the unit and 
command functionalities, and then afterwards will go back to that 
maintainability 
issue.

Original comment by goo...@teabix.com on 11 Dec 2009 at 5:03

[GoogleCodeExporter]

for the BWAPI1 interface doc, is there a way to make a wiki subfolder, or 
should I 
just cram everyting into one page named BWAPI1?

Original comment by goo...@teabix.com on 11 Dec 2009 at 8:19

[GoogleCodeExporter]

adding a folder seems not to work. I will just prepend all BWAPI1 wiki files 
with 
some prefix...

Original comment by goo...@teabix.com on 14 Dec 2009 at 2:01

[GoogleCodeExporter]

With the shared memory solution, is it possible that the core BWAPI DLL in the
process space remain 32 bit while the agent DLL be 64 bit? I know you can via 
COM. I
ask because I have run into an ugly issue
(http://code.google.com/p/bwapi-jbridge/issues/detail?id=3) and I was just 
curious
before I write a COM "agent" of sorts. Thanks.

Original comment by chad.r...@gmail.com on 23 Dec 2009 at 11:54

[GoogleCodeExporter]

Yes. Ofcourse first some adjustments would have to be made throughout the BWAPI 
project for BWAgent.dll to become correcly compilable as a 64bit .dll, all 
minor 
changes though. Please fork this to a separate issue.

PS: will be back in about 2 days. no worries, i'll have BWAPIv3 ready till new 
year 
(for legal purposes new year may be 2011 ;D )

Original comment by goo...@teabix.com on 24 Dec 2009 at 12:52

[GoogleCodeExporter]

The BWAPI1 interface will not be native. Sorry Python =\

details:
It became too sticky to keep all the native interface C++-free. And according 
to my 
estimation over the past weeks, the usage of this feature (accessing BWAPI1 
from non-
C++ languages) is trifling. Partially my fault that I didn't see it earlier, 
could've saved precious time.

Original comment by goo...@teabix.com on 1 Jan 2010 at 7:04

[GoogleCodeExporter]

Seems like all the chain reaction remodeling is done. If lowerlogic will keep 
online 
the next days all the rest should get done fast.

Original comment by goo...@teabix.com on 2 Jan 2010 at 9:04

[GoogleCodeExporter]

just remembered: I'll be away from sunday evening till monday evening 
(hopefully 
midday)...

Original comment by goo...@teabix.com on 2 Jan 2010 at 9:07

[GoogleCodeExporter]

When I debug&run ConsoleAI from VC++, it runs correct but when it's time to end 
and 
close the console window, it hangs itself. I suspect a problem with my VC++ so 
please check if you experience the same problem.

PS.: I'm out-of-town for 1 day

Original comment by goo...@teabix.com on 3 Jan 2010 at 5:26

[GoogleCodeExporter]

most of BWAPI1 is done (except structuring code and make it more 
readable/maintainable, but that I relay untill after the beta). And it's good 
timing, because starting tomorrow I'll have less, chunkier time for programming 
and 
won't be in IRC on demand. I'll try to act upon issues ASAP.

lowerlogic: I've looked into Game.cpp and implemented all I could find.. and 
all 
that belongs into BWAPI1. remember, BWAPI1 just presets data, it does/will not 
present any reduntant information nor features. This is very similar to the 
original 
Broodwar memory (which also only provides you with one version of the data), so 
the 
structure of all your BWAPI2 code stays the same, you just query your 
information 
from the BWAPI instead of the BW namespace.
Example of reduntant information: PlayerId array in Force structure. This is 
redundant because Players have already a ForceId member, so technically this is 
a 
cross-reference, so I didn't implement it.
Now it's up to you to make BWAPI2 run again, post requests here, then we can 
publish 
our first v3 beta ;)

Original comment by goo...@teabix.com on 11 Jan 2010 at 12:40

[GoogleCodeExporter]

BWAPI1 is looking great and indeed appears to be almost done, however I believe 
it is
lacking the functionality needed to implement the following functions in BWAPI2:

Game::self() (determine which player is self)
Game::setScreenPosition()
Game::drawTextMouse()
Game::drawBoxMouse()
Game::drawTriangleMouse()
Game::drawCircleMouse()
Game::drawEllipseMouse()
Game::drawDotMouse()
Game::drawLineMouse()

Unit::getAngle()
Unit::getVelocityX()
Unit::getVelocityY()
Unit::getTarget()
Unit::getTargetPosition()
Unit::getOrder()
Unit::getOrderTarget()
Unit::getOrderTimer()
Unit::getSecondaryOrder()
Unit::getBuildUnit()
Unit::getBuildType()
Unit::getRemainingBuildTime()
Unit::getRemainingTrainTime()
Unit::getChild()
Unit::getTrainingQueue()
Unit::getTransport()
Unit::getLoadedUnits()
Unit::getInterceptorCount()
Unit::getScarabCount()
Unit::getSpiderMineCount()
Unit::getTech()
Unit::getUpgrade()
Unit::getRemainingResearchTime()
Unit::getRemainingUpgradeTime()
Unit::getRallyPosition()
Unit::getRallyUnit()
Unit::getAddon()
Unit::getHatchery()
Unit::getLarva()
Unit::getUpgradeLevel(UpgradeType upgrade)

Original comment by lowerlo...@gmail.com on 15 Jan 2010 at 3:55

[GoogleCodeExporter]

thx for the workplan, last hour was very productive =)

for the first block: r2033

Original comment by goo...@teabix.com on 15 Jan 2010 at 2:27

[GoogleCodeExporter]

Everything should be implementable now except

Unit:: getSecondaryOrder ()
Unit:: getChild ()
Unit:: getUpgradeLevel (UpgradeType upgrade)

Original comment by goo...@teabix.com on 16 Jan 2010 at 4:06

[GoogleCodeExporter]

I implemented an adaptive resource freer algorithm into the Tracer in r2046. It 
totally works, see the screenshot.

Purpose: Ever noticed that Broodwar always takes 100% of PCs resources? 
Broodwar 
uses this power to ensure a fluent user interface. The mouse actually gets 
redrawn 
several times per frame. (By the way, that's the hook we use, we also redraw 
our 
shapes several times per frame, we really should find another hook). I do not 
know 
if to actually implement the algorithm into the Engine, because it will not 
give AIs 
an advantage. The only real advantage is that you can do stuff simultaniously 
now 
like compiling. On my PC compiling took VERY long when broodwar was running... 
But 
the draw back is that the UI is not so fluent anymore, especially on slower 
speed 
settings.

Original comment by goo...@teabix.com on 16 Jan 2010 at 4:08

Attachments:

• recfree.png

[GoogleCodeExporter]

Exams have started. I'll be back in about 2.5 maybe 3 months (yeah they are 
spread). 
But looks like lowerlogic was not able to make any progress in BWSL in the 
BWAPI 
v3.0 branch anyway, I thought he'd be ready by now, now I see my time 
estimations 
were by far too optimistic. Also work is being done on the BWAPI v2.6 branch, 
which 
will make merging stuff a headache. So just to be clear here is my angle: I 
have 
nothing against the v3.0 branch being terminated. I had my share of fun thank 
you 
=), but looks like developers can't cope with the new structure. Also if you do 
not 
terminate it and need something in the new BWAPI interface, just ask and I'll 
try to 
find some time, but you are free (and encouraged ;) ) to implement it yourself.

Original comment by goo...@teabix.com on 30 Jan 2010 at 5:47

[GoogleCodeExporter]

Update: the earliest I might be getting some time is the 10.04. my suggestion: 
don't 
wait up. I'm really loosing touch anyway...

Also (just remembered, seeing that BWAPI2 is making great progress): when/if 
you 
remove the BWAPI3 branch, don't forget to remove or rewrite the wikipedia 
articles 
and stubs. Else people might get confused. You might also want to adopt the 
drawDot 
and drawScanLine functions I made from BWAPI3 to BWAPI2, it'll boost drawing 
speed.

Original comment by goo...@teabix.com on 7 Mar 2010 at 11:01

[GoogleCodeExporter]

Does this mean BWAPI 3.0 has been put off for a while? I'm mostly concerned 
about 
running the AI Module as a separate process to make debugging easier.

Original comment by thisisnt...@gmail.com on 19 Mar 2010 at 4:10

[GoogleCodeExporter]

@thisisntreallymine: That depends on whether BWAPI3 will be developed further 
without my help
@all_committers (which probably rounds down to just lowerlogic): I got like.. 
short 
unpredictable time spans I might dedicate to BWAPI iff you are still interested 
in 
BWAPI3. I completely lost touch though so please enumerate the reasons BWAPI3 
isn't 
out there BY THIS TIME already, and what I shall do to be of help.

Original comment by goo...@teabix.com on 10 Apr 2010 at 9:56

[GoogleCodeExporter]

For a quick summary read only the last couple of words in each passage:

I am delighted to say that lowerlogic obviously decided to patch BWAPI 2 with 
pipe 
features rather than finish BWAPI 3. Although the latter will give performance, 
the 
former will hopefully be supported by more developers. It does not require 
porting 
all the buxfixes, made to BWAPI 2 over the period when BWAPI 3 was developed, 
to 
BWAPI 3. It also does not require my participation, all 3 things that will 
probably 
contribute to its quicker completion. And since I only joined to help out with 
the 
restructuring and shared data techniques and had only so much time, now I 
resign.

PS.: The initial cause why I joined was my bot becoming too complicated because 
it 
acted on the basis of all the knoledge it could get and ignored the 
forced "features" of BWAPI 2 like "units on tile". The data first had to be 
collected through all the BWAPI 2 interfacing and stored in a rather faster-
accessible manner which reduced the lag my bot caused significantly, but the 
collecting took significant time itself. Since "eliminate the overhead of 
presenting 
the game data in a overloaded interface and then reducing it back to pure data" 
and "shared memory" are technically similar, they could be dealt with in 1 
move. 
Maybe in a year or two when the situation becomes less strained I'll create a 
separate project, to not to mess up BWAPI again, to resemble those ideas. For 
now, 
have fun with BWAPI =)

Original comment by goo...@teabix.com on 7 May 2010 at 10:53

[GoogleCodeExporter]

I really liked the concept of BWAPI 3, but it was trying to change too much too 
fast
which made it hard figure out how to make it all work correctly with existing
2.x-based AIs. With time I plan to make incremental changes to incorporate the 
good
ideas gathered from BWAPI 3 and use them to optimize BWAPI 2.x to get the sort 
of
performance BWAPI 3 aimed to achieve, though first I plan to make a much-needed
testing framework to ensure nothing breaks along the way :).

Original comment by lowerlo...@gmail.com on 20 May 2010 at 11:04

• Changed state: WontFix
• Removed labels: Priority-Critical, Milestone-Release1.0