johnramsden commented 6 years ago

Diablo 3 Flatpack

This pull request is for Blizzard's Diablo 3, and is packaged much like Starcraft 2, WoW and OverWatch.

The flatpak is currently runnable, and plays smoothly for me when in 32-bit mode. The last few things are being figured out related to configuration during installation.

I'll continue to update the pull request as things progress.

Iolaum commented 6 years ago

The nvidia problem might be fixable with this PR as mentioned in the comments. Don't know enough to test it locally myself so I 'm just waiting for it to be merged.

julianrichen commented 6 years ago

Sorry about the delay, I was on vacation and got back a few days ago.

Thanks for the contribution, added some review comments. I'm building the GL32 fix from winepak/winepak-sdk-images#15 and I'll see if I can push to the repo tonight so you can test. I'll comment when it's added.

julianrichen commented 6 years ago

We'll also need the appdata & desktop stuff added before pulling.

johnramsden commented 6 years ago

Thanks, I'll push an update once I've tested if it works.

julianrichen commented 6 years ago

@johnramsden winepak/winepak-sdk-images#15 was pulled, built, & pushed. Go ahead and update and see if it improves things. Hopefully it works.

johnramsden commented 6 years ago

Update, I now have it running and the changes seem to have worked. I'll push a commit once I've gotten all the last few details sorted out.

johnramsden commented 6 years ago

So here is a working version c450d67. Unfortunately, at least for me it's completely unplayable as its quite laggy, i'm not sure if that's just my computer, or if it will be a general problem. I'm hoping someone else has some ideas on what might be causing the bad performance.

Let me know if you have any other change requests.

jrock2004 commented 6 years ago

@johnramsden is there anything I can do to test? I have not messed with winepak before but would love to get D3 to work

johnramsden commented 6 years ago

@jrock2004 First add the flatpak winepak repo and install winepak.

flatpak remote-add --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepo
flatpak remote-add --if-not-exists winepak https://dl.winepak.org/repo/winepak.flatpakrepo

Install org.winepak.Sdk and org.winepak.Platform.

Clone my repo and checkout.

git clone https://github.com/johnramsden/winepak-applications
cd winepak-applications
git checkout com.blizzard.Diablo3

Now build:

flatpak-builder --user --arch=x86_64 --force-clean --install builds com.blizzard.Diablo3.yml

Then run

flatpak run com.blizzard.Diablo3

I think that's more or less correct.

julianrichen commented 6 years ago

The manifest visually looks good (haven't tested yet) but because of recent changes to the com.blizzard.BattleNet.BaseApp you'll need to add org.winepak.Platform.Extension.vcrun2015 as an inherit-extensions. See the Overwatch manifest for an example

inherit-extensions:
  - org.winepak.Platform.Compat32
  - org.winepak.Platform.Extension.corefonts
  - org.winepak.Platform.Extension.vcrun2015

@jrock2004 If you have an update version of the BattleNet baseapp you'll need to patch what I mentioned above before testing. Or it'll fail since it won't auto-download the extension. Or just manually install org.winepak.Platform.Extension.vcrun2015 from the repo before installing Diablo3.

jrock2004 commented 6 years ago

@julianrichen I just install winepak right now and have nothing. I ran this command flatpak-builder --user --arch=x86_64 --force-clean --install builds com.blizzard.Diablo3.yml Can't load 'com.blizzard.Diablo3.yml': Failed to open file “/home/jcostanzo/Development/winepak-applications/com.blizzard.Diablo3.yml”: No such file or directory

I changed into the diablo3 directory and got a different error

jrock2004 commented 6 years ago

Here is that other error

Can't parse 'com.blizzard.Diablo3.yml': :1:6: Parse error: unexpected identifier `app-id', expected value

jrock2004 commented 6 years ago

@johnramsden maybe I am missing a library to be install to build this?

julianrichen commented 6 years ago

@jrock2004 Building fine on my end. What version of flatpak/flatpak-builder do you have? app-id is a valid key type. I'm wondering if maybe you have an older version of flatpak-builder that doesn't support yml and only supports json?

flatpak --version
flatpak-builder --version

jrock2004 commented 6 years ago

@julianrichen here is what I have. Installed both via apt-get. Using Ubuntu 18.04

julianrichen commented 6 years ago

You need flatpak-builder 0.10.10 :(

https://github.com/flatpak/flatpak-builder/releases/tag/0.10.10

Major changes in 0.10.10

We now support yaml manifest as well as json.

Something you can do temporarly is convert the yml to json and run that manifest. Assuming you can't update flatpak-builder, or it would be to much work.

jrock2004 commented 6 years ago

@julianrichen trying to install that release now. Its been a while to install the old fashion way

julianrichen commented 6 years ago

I got in game but as @johnramsden mentioned it's really, really laggy. The Wine DB has one test listed:

https://appdb.winehq.org/objectManager.php?sClass=version&iId=29952

What does not

64-bit client has poor performance;

With High texture quality settings, items icons in inventory/stash take a long time to load (esp. in act V);

Sometimes when starting the game, it crashes just after displaying the town;

When Battle.net voice chat is enabled, some of the mouse clicks register as mouse downs (bug #44032);

With dual-monitor setup setting Windowed Fullscreen mode makes the game have doubled resolution (3840x1080 shrinked to 1920x1080).

Workarounds

Use 32-bit game client;

Set texture quality to Low;

Kill and restart the game;

Don't use voice chat, or compile Wine Staging without "server-send_hardware_message" patch'

Use Fullscreen mode, or use Wine Virtual Desktop, or set the resolution manually in D3Prefs.txt (change "DisplayModeWidth" to proper value).

It's from wine-staging 3.12 so it's recent.

If the 32bit client really is better, and we can't find a fix for the 64bit client, then maybe we can just push a 32bit build for people to use. Then in the future when the 64bit version is better we offer that.

johnramsden commented 6 years ago

@julianrichen I'll add a 32bit build then when I get the chance. Thanks for testing.

julianrichen commented 6 years ago

In the Battle.net app their is a setting to toggle the 32bit client

Options -> Game Settngs (Diablo 3) -> Launch 32bit client instead of 64bit client

and performance is significantly better in the menus and in game. But on my Ryzen 7 1700x / RX 580 the FPS feels like it's between 45 ~ 50 fps.

We can change client options in the config file located at ${WINEPREFIX}/dosdevices/c:/users/${USER}/Application Data/Battle.net/Battle.net.config. We do it in the Batte.net installer to make the client work already with accelerated graphics: https://github.com/winepak/applications/blob/master/com.blizzard.BattleNet.BaseApp/com.blizzard.BattleNet.BaseApp.yml#L96

And we're discussing how to add options cleaning with #88 after modifying it.

jrock2004 commented 6 years ago

When I build I will change x86_64 to i386

jrock2004 commented 6 years ago

Well that did not work because it stated there is no i386 version of the blizzard app

jrock2004 commented 6 years ago

Also after I login, all my windows are black

johnramsden commented 6 years ago

So I take it there's no reason to set up a 32-bit version then? Just run the 64-bit version in 32-bit mode?

johnramsden commented 6 years ago

@julianrichen I looked at the OverWatch pr to see how you're changing the json, I wonder if better option would be to use something like jsed which lets you edit json objects in a programmatic away. That way you could parse to see what the user has set if need be at runtime, and you can also edit it at a later date if an update means meeting to add something new.

I'll enable 32 bit mode for now as done in com.blizzard.BattleNet.BaseApp.yml#L96.

@julianrichen Oh, i've been wondering where you find the official artwork. I've been having trouble finding an official icon.

julianrichen commented 6 years ago

I proper json editor would be good since "sed-ing" JSON is generally bad and bound to cause issues (I agree with you @TLATER).

jsed looks interesting but it's written in go and go isn't native to the fd.o runtime, and I'm not sure what kind of issues arise from inheriting an extension from the fd.o runtime when we use the same extension point for our extensions.

I don't know if jshon is any good but it's written in C which we can build out of the box. Or I also found jq which is still being update (May 9 last commit) and is also written in C.

Since all Battle.net apps can benefit from this I think building one of these programs in com.blizzard.BattleNet.BaseApp would be good so every can use them. Then OSTree will de-duplicate the code since they'll have the same checksum.

I'll be gone this weekend but I'll try them out when I get back and hopefully get a better, not hacky solution to this.

This will also effect pr #88 and @TLATER concerns over the same issue.

Also all the icons (as @LeandroStanger mentioned) can be found on this page: https://us.shop.battle.net/en-us

https://us.shop.battle.net/static/3.1.1/images/family-icons/call-of-duty.svg https://us.shop.battle.net/static/3.1.1/images/family-icons/destiny-2.svg

johnramsden commented 6 years ago

@julianrichen I assume you mean't to respond to me.

jsed was just the first one I found, using a c based one would probably be better. Thanks for the links.

johnramsden commented 6 years ago

After some experimentation I found echoing configuration in at install time just gets overwritten. I assume because if the echo into the configuration is done while installer is open, and the battle net installer wants to change something, it probably already has the configuration in memory and just changes the old configuration it had. At least that's the experience I've had, I can't get the configuration to hold if it's done after the battle net installer is running. Advice?

I don't want to do it at runtime because it will then overwrite any configuration the user makes.

johnramsden commented 6 years ago

So I just tested and the game is very playable on the 32-bit setting. I guess for now users will just have to set it themselves?

TLATER commented 6 years ago

@johnramsden The configuration is persistent after the installer has run the first time. That means that you will need to run the installer, get the user to close battle.net without starting the game, and then write the configuration.

Similar problems in #88 - I'll see about emailing blizzard support, as suggested, perhaps there's a magic set of CLI flags that will solve our problems.

johnramsden commented 6 years ago

@TLATER That's not very plausible, how do you get the user to close Battlenet without starting the game? Not all users will be running it from the command line. The only way I can see doing that would be opening up a dialog box.

TLATER commented 6 years ago

@johnramsden I agree, it's annoying, but so far the only way I've found to get persistent configuration to work.

Alternatively, perhaps we could launch battle.net in a subshell and test whether it's created the configuration file in a loop, applying our patch as soon as we detect a change.

johnramsden commented 6 years ago

@TLATER I agree, that's probably the only way to get it to work, seems like it could be fraught with issues though. Considering we don't know how the battle.net launcher works, for all we know it's reading the configuration from memory if it hasn't made a change itself. So if the user installs an application, doesn't close battle.net, and then starts the application, the changes might not be used anyway.

With the current method we are over writing the users configuration too, not patching it. That could be irritating to the user if they want to change the settings themselves.

A way I can see it working, would be doing it at runtime before starting battle.net at all, but using a cli json editor that just manipulates the current state as opposed to rewriting it all together. If we wanted to make sure we only do this the first time, we could touch a file in the ${WINEPREFIX} directory, and then not do it again if that file exists. Doing it this way we could even use a type of 'versioning' where we name the file, say ${WINEPREFIX}/.winepak_battlenet_config.0, then in the future if we wanted to add a configuration option we could create ${WINEPREFIX}/.winepak_battlenet_config.n.

I'll see if I can mock something up.

johnramsden commented 6 years ago

Not sure if anyone else has seen this problem before, it's really strange. When I add a new module for jq it appears to break something related to the linker when I attempt to run the flatpak.

Before adding the module I have no issues installing, and after I get the following message during source /app/bin/battlenet-installer "battlenet://D3":

Setting-up wine prefix...
/lib/ld-linux.so.2: could not open
wine: configuration in '/var/data/wine' has been updated.

Here's the module, it compiles and runs fine:

  - name: jq
    config-opts:
      - --disable-maintainer-mode
    sources:
      - type: archive
        url: https://github.com/stedolan/jq/releases/download/jq-1.5/jq-1.5.tar.gz
        sha256: c4d2bfec6436341113419debf479d833692cc5cdab7eb0326b5a4d4fbe9f493c

julianrichen commented 5 years ago

@johnramsden Do you have org.winepak.Platform.Wine.Compat32//3.10-staging installed?

I've been pushing more work to https://github.com/winepak/winepak-sdk-images/tree/redesign and hope to close winepak/winepak-sdk-images#12 soon-ish. Should hopefully make it less likely that something like that will happen in the future. I need to test some applications first.

julianrichen commented 5 years ago

Scratch what I said above, I'm also getting the /lib/ld-linux.so.2: could not open & yet I have the packages. Will look into.

julianrichen commented 5 years ago

Yup, you're right @johnramsden jq seems to delete the ld-linux.so.2 file. Since the setup-compat32 runs before and returns OK, jq must be doing something.

julianrichen commented 5 years ago

Solution for the ld-linux.so.2 is in https://github.com/winepak/applications/pull/101#issuecomment-410796339

jq seems to be working well for com.blizzard.BattleNet.BaseApp. Regarding:

The configuration is persistent after the installer has run the first time. That means that you will need to run the installer, get the user to close battle.net without starting the game, and then write the configuration.

Writing the config before the application starts installing means that the Battle.net.exe will pick-up the configs, however, the additional changes we are doing in each installer isn't being picked-up. This is the jq I have for Diablo3

echo "Update Battle.net.config"
battlenet_config="${WINEPREFIX}/dosdevices/c:/users/${USER}/Application Data/Battle.net/Battle.net.config"
battlenet_config_temp=$(mktemp)

mkdir -p "${WINEPREFIX}/dosdevices/c:/users/${USER}/Application Data/Battle.net"
jq --null-input '.Games.diablo3.Run64Bit = "false" | .Games.diablo3.Run64BitSet = "true"' "$battlenet_config" > "$battlenet_config_temp"
mv -f "$battlenet_config_temp" "$battlenet_config"

A solution could be to pass the additional edits into the /app/bin/battlenet-installer.

Regarding closing and restarting the client. This seems to sorta work:

/app/bin/GAME_NAME-installer; wineserver -k && wineserver -w && wine64 "/var/data/wine/dosdevices/c:/Program Files (x86)/Battle.net/Battle.net Launcher.exe" --game INSTALLER_UUID --install

Basically we kill the wineserver then relaunch battlenet, but I don't know what kind of side-effects that might have. I believe the battlenet-installer.exe won't continue until you get to the login screen so we might be safe to just kill the process and re-launch it.

julianrichen commented 5 years ago

Something else I observed is we can always launch the game with --game diablo3 --install:

wine64 "${WINEPREFIX}/dosdevices/c:/Program Files (x86)/Battle.net/Battle.net Launcher.exe" --game diablo3 --install

and Battle.net will ignore the --install flag if it's already installed or being installed. Just need to kill the existing Battle.net installer nicely.

johnramsden commented 5 years ago

@julianrichen Passing instructions to /app/bin/battlenet-installer seems ideal to me. Something like

/app/bin/battlenet-installer -c \
    '.Games.diablo3.Run64Bit = "false" | .Games.diablo3.Run64BitSet = "true"'

I implemented this in #101, let me know what you think.

winepak / applications

[WIP] [com.blizzard.Diablo3] Initial Diablo III flatpak #73

Diablo 3 Flatpack