interfaces and portability

[GoogleCodeExporter]

forked issue 134, collecting facts:

1.
fact:
current interface is the "abstract interface". Read more under "C++ Mature 
Approach: Using an Abstract Interface" at 
http://www.codeproject.com/KB/cpp/howto_export_cpp_classes.aspx . This 
interface is only portable between C++ compilers. More on that under "Why 
This Works With Other Compilers". 

plan:
remove the abstract interface and let agents compile the receiver code. 
This option became available since the bridge from issue 134.
Note: "receiver" is coined as "agent-side wrapper classes around bridge's 
protocoll"

2.
fact:
Many nonC++ classes beg to use BWAPI. This feature is best implemented on 
the agent's side. Since the bridge, the are no drawbacks this way.

suggestion:
igalvelis has a promising project using swig to port the C++ receiver into 
any target language. Technically this is a better way than using sockets. 
I suggest making a wiki acticle for nonC++ users about portability and 
link to igalvelis's project page, where they hopefully, soon will be able 
to download an adapter library for their target language.
Or we could make igalvelis's project part of BWAPIs and publish the 
adapter libraries with our releases.

Original issue reported on code.google.com by goo...@teabix.com on 28 Nov 2009 at 12:58

[GoogleCodeExporter]

Just note that the page is still TBD as well as the Wrapper(Parser/Generator), 
and it
will not use SWIG.

Original comment by igalve...@gmail.com on 28 Nov 2009 at 1:30

[GoogleCodeExporter]

Author of the Java ProxyBot here. If we were able to launch the AIModule as a
separate process, I would not use SWIG to implement the interface with Java. 

I would use a JNI approach, by creating a Java process that calls the main on 
the c++
code and have it issue a callback to Java on each frame. JNI calls could then be
implemented for all methods available to the AI Module.   

In the long run, this seems like the best setup in terms of performance. 
However, I
think it is more important to focus on stability before optimization. Using the
socket based approach, I am capable of sending state every frame on the fastest
setting without slowing down the game, and around 4-10x speed when using the 
speed hack.

Original comment by bgwe...@gmail.com on 28 Nov 2009 at 6:30

[GoogleCodeExporter]

-"I would use a JNI approach"

swig is the JNI approach. swig exports all classes as C-style native functions, 
and 
generates a java class structure wrapping those natives.

further, consider this callback system:
a function that blocks untill something happens, waitSomethingHappen(). It 
returns a 
code that is 1 for onStart, 2 for onUnitCreate etc. This way we do not need 
technical callbacks for nonC++ languages, just simple native calls. (because 
callbacks are different from language to language, native calls however are 
fully 
portable)

Original comment by goo...@teabix.com on 28 Nov 2009 at 8:34

[GoogleCodeExporter]

Weber: I did send you some emails earlier, asking about sockets vs JNI. I am 
currently implementing JNI to work with the bwapi api. So far it seems very 
possible. 
However, I take another approach, where I implement the JNI-code directly into 
the 
MyExampleAi C++ class and from there create or hook onto the JVM. Then I can 
easily 
obtain the "main" object which i call JNIManager.class and call methods such as 
onStart, onFrame etc.. Here onStart returns the init settings, such as flags 
and 
onFrame can return the commands which is currently in the queue. Another 
approach to 
asynchronous commands, could be implemented by having a c++ method running as a 
thread and updating some global queue. But, since only the main bwapi thread 
can 
issue the commands this wouldn't serve a big purpose.

Original comment by jahes...@gmail.com on 28 Nov 2009 at 2:46

[GoogleCodeExporter]

-"since only the main bwapi thread can issue the commands"

Please note that this fact will get outdated in some days. With the bridge, you 
will 
have to provide BWAPI a thread, usage will be either (exact names not binding 
;) ):
- a call to AIModule::takeover() which blocks. AIModule::onSomething() will be 
called from that thread. 
- a call to AIModule::waitForNextFrame(), which blocks untill next frame. Then 
you 
can query all the data you need with methods like AIModule::getNextCallback() 
and 
immitate callbacks in your java wrapper, to eliminate those nasty C++-java 
callbacks.

Original comment by goo...@teabix.com on 28 Nov 2009 at 3:43

[GoogleCodeExporter]

I believe this generator could also help produce the code for the bridge, 
unless I am
mistaken and you would be able to use existing header files to deal with shared 
memory.

I am done with the parser (I think), now I am working on the generator part. 
Here is
some progress (I am not sure if it compiles or works yet):

Original comment by igalve...@gmail.com on 2 Dec 2009 at 1:31

Attachments:

• ExampleAIModule.zip

[GoogleCodeExporter]

The bridge itself is hidden from the AI programmer, you need only produce code 
to 
wrap the interface BWAgent.dll provides.

Original comment by goo...@teabix.com on 2 Dec 2009 at 1:38

[GoogleCodeExporter]

I dont think I need to wrap any interface that BWAgent.dll (when done) will 
provide -
swig should do it automatically.

I am talking about generating code for the BWAgent itself. To automate writing 
of all
the methods that communicate between BWAPI.dll and BWApiAgent.dll. Would that 
not be
better than writing them all by hand? I expect you will need at least one 
method for
each method in existing classes.

Currently I am able to parse all the h files in the include directory. 
Previously
attached ExampleAIModule.zip shows how I already can generate some helper 
methods for
all the classes. So changing this code to generate BWAPI/BWAgent communication 
code
should be a snap. 

I would just need to understand exactly what kind of C++ needs to be generated.

The parser/generator source is available:
http://breakablec.redirectme.net/svn/repos/trunk/BWParser
Unfortunately it is absolutely documented and messy, i will try to add some 
Readme today.

Original comment by igalve...@gmail.com on 2 Dec 2009 at 1:58

[GoogleCodeExporter]

Unfortunately it is absolutely documented ->I meant undocumented

Original comment by igalve...@gmail.com on 2 Dec 2009 at 1:59

[GoogleCodeExporter]

Going to sleep. I will probably finish generating code for my .net port of 
ProxyBot
tomorrow, and well see where to go from there.
Please email any questions about BWParser (if any).

Original comment by igalve...@gmail.com on 2 Dec 2009 at 2:34

[GoogleCodeExporter]

Generating CPP is not as easy as I thought. At least compiling making it 
compile.
I was able to make the toString methods compile with some changes in the 
generator as
well as few small manual modifications.
Now i just need to make the C# client side modifications and I should have
BWAPI->.net communication (after fixing the bugs).
What will be left to make generators for BWAPI<-.net communication.

Original comment by igalve...@gmail.com on 4 Dec 2009 at 12:50

Attachments:

• ExampleAIModule.zip

[GoogleCodeExporter]

I updated the parser/generator to be able to generate CSharp classes and 
converters
for those classes from string data, sent by bwapi server.
So basically the BWAPI->.NET communication is feature copleate (if not tested, 
and
not working yet). After finishing with that what will be left is the .NET->BWAPI
communication.
The current version of parser is available at the same svn.

Original comment by igalve...@gmail.com on 7 Dec 2009 at 1:13

Attachments:

• ExampleAIModule.zip

[GoogleCodeExporter]

Original comment by lowerlo...@gmail.com on 7 Dec 2009 at 6:16

• Added labels: Type-Enhancement, Priority-Medium

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

update:
for further compatibility with BWTA and BWAI, BWAPI2 interface will not be 
compiled 
by the AI module. In other words: no change.

Original comment by goo...@teabix.com on 8 Dec 2009 at 1:45

[GoogleCodeExporter]

While I don't understand the complexities of changing the interface, "no change"
seems good to me.

Original comment by igalve...@gmail.com on 10 Dec 2009 at 1:27

[GoogleCodeExporter]

I am no longer working on parser/generator for bwapi due to another project
addressing most of issues for .net
http://code.google.com/p/bwapi-mono-bridge/
and BWAPI v3 interface that will address the rest.
The sourcecode should be still reachable trough svn, and if you think i should
reconsider - drop me a line.
I got pretty far btw, was able to parse all the h files, and generate cpp side
(untested), but not the C# side yet, so maybe it could be used for other 
language/ipc
method.

Original comment by igalve...@gmail.com on 23 Dec 2009 at 3:49

[GoogleCodeExporter]

I approve, a specialized project is superior to a generic one, and .net covers 
most 
of the non-C++ audience. The name of that project is kinda fuzzy tho.

Original comment by goo...@teabix.com on 23 Dec 2009 at 4:58

[GoogleCodeExporter]

I tried making this as understandable as possible, still plz read carefully
------------------------------------------------------------------------

I once suggested to make BWAPI2 interface virtual and thus the 
BWAgent.dll loadable from any C++ compiler. By now I realized 3 ugly problems:
- A huge amount of functions would have to be rewritten, because of (*). 
Probably 
redefining all templated classes which are used as parameters or return values 
e.g. 
std::set<Unit*> (used in getAllUnits()) redefine as BWAPI::UnitSet. This is a 
slap 
in the face for anyone using BWAPIv2.5 (, a stab in the gut performance-wise, 
and a 
kick in the groin maintenance-wise ;) ).
- Alternatives would be needed for variable export (**), where all e.g. 
UnitType 
information now is exported with the "extern" statement. Another slap for 
BWAPIv2.5 
users.
- people developing BWAPI2 would need to understand the do's, dont's of virtual 
interfaces or COM development to avoid writing (*) or (**) code. This limits 
our C++ 
possibilities. But we wanted to make a C++ portable interface, so why should we 
limit C++? This is ridiculous!

(*)BWAPI v2.5 provides functions with templated parameters and return values 
like 
std::set<something>. Even std::string is a template (template base class). This 
is a 
problem for pure virtual interfaces which aim for portability, because 
templates are 
not portable.

(**)BWAPI v2.5 provides variables by "extern" syntax. E.g. 
Units::Terran_Command_Center. This is not a portable way of exporting.
-----------------------------------------------------------------------

the above, in a nutshell:

BWAPI2 would with time morph to something not even nearly similar (and worse 
than) 
to BWAPIv2.5. It's not worth it for only C++ portability. Instead there is a 
better 
solution in issue 159
-----------------------------------------------------------------------

A virtual interface is in fact a COM interface. I created a COM discussion in 
issue 
174

Original comment by goo...@teabix.com on 26 Dec 2009 at 2:31

• Removed labels: Type-Enhancement, Priority-Medium

[GoogleCodeExporter]

decisions made, ends are tied

Original comment by goo...@teabix.com on 1 Jan 2010 at 9:37

• Changed state: Invalid