Question about games wrapper for Ludii

[alexminnaar]

I saw that a games wrapper for the Ludii General Game System was something that you were looking to add in contributing.md. It looks like Ludii is written in Java. Was the idea to be able to call the Ludii jar from C++ in order to interact with it?

[lanctot]

I meant a direct C++ interpreter for the game descriptions (i.e. http://ludii.games/lud/games/BreakThrough.lud) avoiding the cost of having to externally load/communicate with a JVM which introduces a dependency and (maybe) an artificial performance hit.

The other thing I am not sure about is what kind of API the Java interpreter provides to query things like observations that the games would need to expose.

But maybe that is (too) ambitious. Wrapping the standard java interpreter would be a great place to start, so I would welcome this contributiion. Would you be happy to do it, even if the longer term goal was to interpret it in C++ directly?

Either way, we should loop the Ludii team in the discussion (I have contacted them). They recently told me Ludii is in pre-alpha and their plan is to release in January. So I would love their feedback on this too.

[alexminnaar]

@lanctot possibly. I am actually working on the laser tag gridworld environment right now but after that I might be able to. I'd first have to learn more about the Ludii API.

[Ludeme]

Our recommendation would be to have OpenSpiel wrap around Ludii's JAR file, and call its Java code.

This is because the grammar that defines legal .lud game descriptions is not restricted to a relatively small set of instructions. It is automatically derived from all the "Ludeme" classes in the JAR file. These would all have to be re-implemented in C++ in order to avoid a wrapper around Ludii's java code, which would be a large and ever-growing amount of work as we are likely to continue updating Ludii.jar with new Ludemes quite regularly for a while still.

I've been having a look through OpenSpiel's API, and I think the latest version of Ludii.jar at this time of writing (v0.3.0) does not yet support all the methods that we'd want to be calling into from an OpenSpiel wrapper. In particular the representation of game state observations is currently a bit different. I do not suspect any major problems with updating Ludii.jar to provide all the methods that we'd want to have, but we will have to find some time to sit down and do that first.

We'll post here once we've updated Ludii to include all the methods that we expect to be required, and keep an eye out here in general in case any questions or suggestions pop up about OpenSpiel + Ludii.

[lanctot]

Ok, thanks for tuning in. That all sounds good to me. I am happy, then, to support the @alexminnaar's original suggestion of loading and communicating with the Java Ludii interpreter.

The game state observations could be converted from e.g. ASCII text in some consistent way. But having a one-hot concatenated representation of bit fields would also be good too. Whatever makes the most sense as a first cut and could then be revised.

Feel free to continue using this thread for discussion on this and thanks again!

[alexminnaar]

@lanctot I did some experiments and I was able to call the Ludii jar through C++ using JNI (for example here is a gist that calls the Ludii GameLoader.listGames() method). I was thinking perhaps a good first step would be to create a utils/helper file that does some Ludii API calls and encapsulates the JNI logic. I'd be happy to work on this. Or maybe we should hold off until @Ludeme has made the updates? I'm fine either way. What do you think?

[lanctot]

That's awesome, @alexminnaar!

Yes that plan sounds good to me. Feel free to start putting all of the Ludii-related code under a subdirectory ludii (under games) with a README on what needs installing to get it to work. I will test it on my end too.

I noticed the command in the gist code uses openjdk? Does this mean on Debian / Ubuntu you can get the Java dependencies (other than the Ludii .jar of course) with a one-line apt-get command? Been out of the Java game for a while..

I think these first steps are straight forward / experimental and I doubt anything here is too controversial so I think it's ok to go forward for now. But also I encourage @Ludeme to subcribe to this thread so we can continue to use it for design discussions.

[Ludeme]

Yes I'm already subscribed to this thread.

Looking good so far! @alexminnaar I imagine that figuring out which Ludii methods to call is going to be difficult in Ludii's current state, because for the current pre-release version we only focussed on documentation on how to implement new games / implement AIs for Ludii. There is no public documentation currently on some of the "inner workings" of Ludii, which you'd likely have to call into. And some of the methods probably do not exist yet.

If you're going to be playing around with Ludii already anyway and give it a try, please don't hesitate to contact us with questions about the API.

[alexminnaar]

@lanctot yes I think that would be all you need to do. The JNI headers and libjvm.so should live in openjdk. Just to clarify though, for the time being, we would be building everything in the ludii directory separately from the rest of open_spiel i.e. we won't include any of it in cmake just yet. Is this correct?

[lanctot]

Yep that is what I was thinking. In any case we would likely always treat Ludii as an optional dependency (enabling a Ludii wrapper would be something people can enable and if so they would install Ludii on their own) so that it doesn't require everyone using OpenSpiel to install Ludii.

[lanctot]

(I only meant the OpenSpiel code you write specifically for that subdir, and the README instructions could detail how to modify the necessary make files to add Ludii's build dir)

[alexminnaar]

@lanctot sounds good, I'll post my updates here

[alexminnaar]

I wanted to post an update. So I have created a ludii directory in my fork. In my previous update I was able to call GameLoader.listGames() via JNI so next I wanted to do a proof-of-concept for actually creating a Ludii game object and calling one of its methods - i.e. do something like

Game game = GameLoader.loadGameFromName("board/space/blocking/Amazons.lud");
String gameName = game.name();

I was able to implement this with JNI here and it appears to be working. Also, in this Ludii class I've created, the jvm is initialized and the Ludii jar is put on the classpath in the constructor and then destroyed in the destructor. Clearly it's all pretty rough right now and there are probably some much better ways to abstract this but my thinking is perhaps I should continue implementing some of these Ludii methods in this class and then worry about the abstractions after that. Let me know if there are any thoughts.

[DennisSoemers]

@lanctot Any thoughts on how crucial it would be to have an accurate implementation for Game::NumDistinctActions() and/or State::NumDistinctActions()? I assume this is primarily used for policy networks to construct an output layer of the correct size?

Given a general game description file (in e.g. Ludii's format), it is usually not straightforward to make this number explicit. Exactly this is one of the reasons why I've been sticking to simpler linear policy function approximators which I apply separately to every available action in a given state in most of my research using Ludii so far, instead of one large function approximator that handles all actions at once.

If it's crucial to get this right... I'll have to look into it, but I think we should be able to at least get an upper bound working (which may in some cases turn out to be excessively large).

[lanctot]

Hmm, yeah that's exactly what it is used for. A first cut could simply exclude that function as we figure out how/whether it is possible. It is too bad because general learning using neural nets was one of aims but I can see how hard that one can be with simpler game description languages.

Is that the only one though? I assume the observations functions are tough too.

It makes an interesting research problem, though! Like, ca n an agent can keep an ever growing list of global legal moves and add outputs to its network dynamically as it discovers them via inspecting legal actions (and how to ensure keeping the current policy with the other actions intact when it does). This would make the agents event more general and is an interesting challenge and research direction.

(Is this even possible? Are the actions in Ludii globally identifiable? E.g. in chess is e2e4 represented the same way to the agent whether it's turn 1 or turn 10?)

[Ludeme]

Hi,

It makes an interesting research problem, though! Like, ca n an agent can keep an ever growing list of global legal moves and add outputs to its network dynamically as it discovers them via inspecting legal actions (and how to ensure keeping the current policy with the other actions intact when it does). This would make the agents event more general and is an interesting challenge and research direction.

That is an interesting problem! This would be nice to inerstigate.

Regards, Cameron

On 23 Sep 2019, at 16:09, lanctot notifications@github.com wrote:

Hmm, yeah that's exactly what it is used for. A first cut could simply exclude that function as we figure out how/whether it is possible. It is too bad because general learning using neural nets was one of aims but I can see how hard that one can be with simpler game description languages.

Is that the only one though? I assume the observations functions are tough too.

It makes an interesting research problem, though! Like, ca n an agent can keep an ever growing list of global legal moves and add outputs to its network dynamically as it discovers them via inspecting legal actions (and how to ensure keeping the current policy with the other actions intact when it does). This would make the agents event more general and is an interesting challenge and research direction.

(Is this even possible? Are the actions in Ludii globally identifiable? E.g. in chess is e2e4 represented the same way to the agent whether it's turn 1 or turn 10?)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/deepmind/open_spiel/issues/39?email_source=notifications&email_token=AM2LGBENQZXLZ2BH5OM6QNDQLDE3NA5CNFSM4IU55UZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7K7PVI#issuecomment-534116309, or mute the thread https://github.com/notifications/unsubscribe-auth/AM2LGBESUHPPV43XXV46ODDQLDE3NANCNFSM4IU55UZQ.

[DennisSoemers]

Is that the only one though? I assume the observations functions are tough too.

It's probably not the only tricky one, just the first one I ran into that seems tricky :D

Yes, the action representation in Ludii is independent of the state, such an e2e4 move would look the same regardless of turn number. Sometimes our actions will contain even more data than that (like, I think it would specifically also contain data about the piece being removed if there is an opposing piece on the destination), but we should be able to relatively easily map from less data (like just from- and to-position) to a more complex action object as long as it's sufficient data for the move to be uniquely identifiable (given list of legal moves in current game state).

We'll continue thinking about how to get a reasonable upper bound on the number of distinct actions...

[DennisSoemers]

I think we've got fairly reasonable approximations for the number of distinct actions in Ludii now (in our dev version, not yet online). Of course not perfect, usually at least a bit of an overestimation, but usually doesn't seem to be too bad.

Internally, we do this based on "from" and "to" positions in Ludii, plus (in rare cases) an index for things like pieces (e.g. to choose piece to promote to). This tends to lead to numbers that are not excessively large.

The only type of games in which this is likely to fail is "stacking" games, i.e. games where there can be multiple pieces stacked on top of each other in a single location. If players are free to make choices for the level within a stack at which they manipulate it (e.g. if they can choose how many pieces to take off of a stack and move onto a different position), such moves would all get mapped to the same "unique" (but not really unique) index. What we can do in Ludii if an OpenSpiel agent decides to play such an action, is to simply select an action at random from all the legal actions that map to that same index. Implementing a proper, unique mapping in these cases would lead to excessive large numbers. Multiplying all from-positions with all to-positions tends to still be tractable, but multiplying that again by the maximum size of a stack (in theory unbounded, in practice just a sufficiently large number)... not really tractable anymore.

---

Different issue: when REGISTER_SPIEL_GAME() is used to register a game, it expects a GameType object with data about stochastic vs deterministic, simultaneous vs sequential, etc. I think this information has to be fixed at compile-time? How would we register a hypothetical "LudiiGame" class that can load any arbitrary Ludii game based on a supplied String name?

For any given Ludii game, we can easily have it provide all that information... but only once we know which game we're going to be loading, which is not yet known at compile-time if we want a generic "LudiiGame". I don't suppose we'd want a separate C++ class for every game implemented in Ludii either...

[alexminnaar]

Update here. I was able to wrap most of the functionality from the Ludii tutorial in my fork. Some of the wrappings are tested here. It's definitely not a complete wrapper but I figure we can add functions later as needed. I think it would probably be more efficient if I pre-fetched jclass's and methodid's in the constructors so maybe I'll make that change next. Let me know if there are any thoughts.

[DennisSoemers]

@alexminnaar I have been spending some time going through the OpenSpiel API, and taking notes on which Ludii functions would have to be called. You can see this here: https://www.overleaf.com/read/rkyfprfdqmky

Whilst doing this, I've also been adding a bunch of new methods because some of the required functionality did not yet exist in the online version of Ludii (or was only accessible through deep nested calls, which is very inconvenient from C++). I've also created two new wrapper classes to make all this functionality easily accessible without deep calls.

It's still very much work in progress on our end though, also got other things to focus on... and we're not ready yet to put this current version of Ludii with these new methods online.

[alexminnaar]

Sounds good. @DennisSoemers @lanctot I'll leave you to it. Feel free to ping me if you need me.

[lanctot]

Great progress guys!

@alexminnaar I'd really love to see some of that early stuff in the repos; there's no detriment to having it in there and actually there's at least one other suggestion (https://github.com/deepmind/open_spiel/issues/65) where it would have been useful to have an exampel of using JNI to wrap the .jar. So even just the first thing you did to call gameLoader.ListGames or load a specific game given a path would be great to have in the repos now. (Including steps to run it in the comments)

Personally I don't see why not add the rest you have already. It's clearly a work in progress but people could already start looking at it and trying stuff out. It's something we could be clear is a work in progress and link to this issue. Might even help @DennisSoemers . Anyway, just my opinion!

[DennisSoemers]

@lanctot Some other parts of Ludii (mostly GUI for some classes of games) are currently (in our dev version) broken, due to some things there being overhauled. So we don't really want to put a half-broken Ludii.jar out there. But sure, when those other things are patched up and ready to go live, it will also include whatever we have ready at that time in terms of OpenSpiel support.

[lanctot]

Ok but very simple API calls (ListGames, Loadgame) to show how C++ can hook up to Java through JNI would be fine, right? I'm suggesting a directory called ludii that only contains the loading of the jar an calls those two functions as an example, and instructions on how to set it up. That would be fine, no? Or are you saying there exists no stable Ludii .jar file yet?

I'll let you both decide but I'd love to see even the early stuff start to make it into the repos. There's no downside, and only benefit, from what i can tell.

[DennisSoemers]

Oh sorry I misread that previous post. Yes, absolutely, everything that @alexminnaar managed to get running so far already was tested with Ludii-0.3.0.jar, which is a stable version that we've currently got up on our website.

[alexminnaar]

@lanctot That sounds good too. I can create a PR for the ListGames and LoadGame functionality.

[DennisSoemers]

@lanctot We're currently working on adding the "swap" or "pie" rule to some games (like Hex), where players sometimes have the option of choosing to swap roles.

Are there any plans for adding such rules to existing OpenSpiel games? If not, we're going to have to be very careful about what data we return from Ludii to OpenSpiel when it's requesting things like "who is the current mover" or "what are the rewards/returns for player X". I guess they'd usually have to be from the "perspective" of the original role assignments rather than the swapped assignments for AI correctness, but that might get confusing sometimes...

[lanctot]

@DennisSoemers, this came up when were adding Hex and Havannah. @locked-deepmind suggested we implement a general game transform that can change any game into one with the pie rule (wrapping the underlying game I guess). Looks like we did not implement it in the end but it seems like the easiest way to support it for all games.

[lanctot]

Hi @alexminnaar and @DennisSoemers, looking through the issues tidying things up. AFAIU this is still on the todo list but there are no unresolved outstanding "issues" or things you're waiting on, right? Can we close this for now then?

[DennisSoemers]

@lanctot That's some nice timing, it just so happens I've been working a lot on implementing some of the things on the Java side of Ludii that were still missing that were required / convenient for a bridge between the frameworks.

No, on the OpenSpiel side there are no unresolved outstanding issues here, so ar far as I'm concerned you could close this issue. Hopefully within the next few weeks (where "few" is undefined :D), I'll find time somewhere to also set up the C++ code that would be required in my fork of OpenSpiel. When that's done, I'll send a pull request.

[lanctot]

Cool! No rush of course, I know how these things go. It's cool to have the community involved.

Someone at our AAAI RL in Games workshop (Jakub Kowalski) mentioned integration with his general framework so I mentioned your ongoing efforts with Ludii. He has a paper titled "Regular Boardgames", thought you might be interested. Can send it to you if you don't find it. I will also point him to this thread.

Also the work @alexminnaar did for this on loading Jave through JNI turned out to be relevant at the workshop too: Simon Lucas told me has has a mini RTS engine in Java that is discrete (turn based, simultaneous moves) and is interested in wrapping it as an OpenSpiel game. Now we can point him to the work you guys started here.

Anyway, looking forward to more collaboration on this and no time pressure. Just want to keep the open issues for outstanding problems that remain unresolved