View Bugs (1.9.0.0.10535)

[Cernelius]

Intel Core i5, 7th generation. 8 GB RAM. Windows 10. TripleA 1.9.0.0.10535 installed on SSD (TripleA.vmoptions: -Xmx2G -Xms2G). world_at_war-master.zip on HDD.

File Video (thanks to @RoiEXLab for uploading it on the RoiEX channel): https://youtu.be/PpgGGC3d5ks

00:25. I try to change the zoom from 10 to 100, but it doesn't work (eventually, I zoom using the mouse wheel, instead). 01:06. Since the borders are drawn medium when loading under 100 zoom (due to this issue: https://github.com/triplea-game/triplea/issues/3525), I try to change to low, partially failing, and getting an error. 01:59. I start the game at 100 zoom, but the map is not readily drawn in all cases outside the starting view. 02:35. I set the zoom at 10, to have all the map on view, but it will take until 10:51 to complete drawing (so more than 8 minutes to fully draw the map). 11:38. I move some units to attack some territory, but they are not redrawn in the target territory (I believe this happened only after the changes eliminating the flickering issues when moving units).

[RoiEXLab]

@Cernelius Well the most obvious choice is to upload it to youtube and make it unlisted ^^

[Cernelius]

I'll add that my engine was frying trying to draw the whole World At War map, in the about 10 minutes it took. You can see that the GIF in Notes is being displayed badly, since the engine was so focused on trying to get the map drawn that there was not much left for other tasks (you can see the GIF becomes smoothly displayed some seconds after the map is fully drawn, beside the fact that it is not rendered very well because the GIF is 50 fps while my display is 60 fps).

[RoiEXLab]

@Cernelius Uggh, the limited download speed makes the download take ages. I'd really appreciate if you could upload it to a video streaming platform so it get's auto-compressed and I don't have to download ~2Gigs (at 200KB/s which is 2 and a half hours) in order to see a ~12 minute video.

[Cernelius]

The only streaming platform that doesn't require an account I found had a limit of 1 GB. I'm converting it to low quality MP4 (that one was high quality MP4) and see what I can do. I assumed it was not a big deal to leave it open downloading for 2 hours, while doing something else.

[RoiEXLab]

@Cernelius No problem if you don't have youtube account or similar. I can download the file in the background at some time, but definitely not today. It'll just take longer for me to actually watch it unless someone else wants to jump the gun and download it and upload it to a video platform ^^

[Cernelius]

I've converted it to low quality MP4 (622 MB) and I'm uploading it in a GitHub repository of mine, but it seems it is very slow going.

Edit: Nevermind. The overall limit might be 1 GB, but it says the files are restricted to 100 MB.

[asvitkine]

I think the issue is basically "World at War" map is really slow to draw the tiles and scrolling e.g. results in a lot of blank tiles for a long time. It should be easy to repro - just start a World at War game and try moving around and see how slow redrawing is!

[RoiEXLab]

@Cernelius a new pre-release is out. Could you check if this issue you're having still exists? My hopes are that you'll get an error message this time/or don't get one and I can eliminate another potential cause for this.

[Cernelius]

I try to change the zoom from 10 to 100, but it doesn't work (eventually, I zoom using the mouse wheel, instead). Since the borders are drawn medium when loading under 100 zoom (due to this issue: #3525), I try to change to low, partially failing, and getting an error. I start the game at 100 zoom, but the map is not readily drawn in all cases outside the starting view. I set the zoom at 10, to have all the map on view, but it will take until 10:51 to complete drawing (so more than 8 minutes to fully draw the map). I move some units to attack some territory, but they are not redrawn in the target territory (I believe this happened only after the changes eliminating the flickering issues when moving units).

Tested right now with TripleA engine version 1.9.0.0.10640. All these issues are still there except:

• I'm not getting anymore an error when I change the borders from medium to low (but it still partially fails, the same way as before, with part of the map having it medium and the rest having it low).

My hopes are that you'll get an error message this time/or don't get one and I can eliminate another potential cause for this.

No error messages at all.

[RoiEXLab]

@Cernelius I found another spot where errors might get suppressed. Could you try again to see if you get an error this time? I doubt it, but you never know.

[Cernelius]

TripleA engine version 1.9.0.0.10660: Same story, no errors.

[RoiEXLab]

@Cernelius I got access to a fast internet connection today and uploaded your bug report to youtube. It's available here: https://youtu.be/PpgGGC3d5ks

[Cernelius]

I gave you moral allowance to update my first post with such a thing. Doing it. I guess I should consider having a youtube account, but, as long as the files are 100 MB or less, I should be able to upload them here in a GitHub repository. That was an unusually long one, as I had to wait for the drawing completion.

[RoiEXLab]

@Cernelius Finally had some time to have a look at your video and I think I know why didn't happen previously to my changes (because rendering events of the currently visible frames were being fired over and over again until the tile was completely drawn), but I still don't get why this is happening at all. Definitely seems like an issue tied to your system. Unless you have a java version with a bug on your specific system and updating to the latest version of Java 8 fixes your problem (unlikely, but who knows) I need you to help me debug your TripleA:

1. Create a Text file named logging.properties (or some other name of your choice) on your desktop (or some other folder on your system you can easily access) and insert the content I'll post below.
2. Go to your TripleA installation folder and press Shift+Right Click and click on open command prompt/powershell (depends on system setting).
3. Enter java "-Djava.util.logging.config.file=<config_file>" -jar .\bin\triplea-<version>-all.jar and replace <config_file> with the full path to your config file you just created, also replace <version> with the currently installed TripleA version, alternatively you can just enter .\bin\triplea and press Tab to auto-complete the name. If this command doesn't work and tells you something along "invalid command", you may need to replace java with the full installation path to your java installation. In my case: C:\Program Files\Java\jre1.8.0_172\bin\java.exe
4. Press Enter to run TripleA and reproduce the error again.
5. After you've finished, there should be a lot of timing logs in your console. Create a gist (https://gist.github.com) and post a link to it here. This should reveal which part of the rendering takes so long to execute.

logging.properties

handlers=games.strategy.debug.ConsoleHandler

games.strategy.triplea.ui.screen.Tile.level=FINEST
games.strategy.triplea.ui.MapPanel.level=FINER

[asvitkine]

Doing a simple profile of the rendering code reveals that we're reading tile images from disk every time we draw. So we're doing IO just to draw the tiles. This is of course expensive - and I can imagine being very slow if the user has a slow spinning disk. Can we read the images once into memory instead of every time we redraw?

[RoiEXLab]

@asvitkine Actually the IO operations run (or at least should be) in the background. Once the image is available in memory a repaint is triggered and the screen gets updated. If you're not convinced have a look at the MapPanel class and checkout the paint method.

[asvitkine]

I agree that it's being done in the background. Still, it seems to be continuously re-read from file - can that be avoided - i.e. cache the read image in memory (in a SoftReference) to avoid the IO?

On Tue, Jul 24, 2018, 1:51 AM RoiEX notifications@github.com wrote:

@asvitkine https://github.com/asvitkine Actually the IO operations run (or at least should be) in the background. Once the image is available in memory a repaint is triggered and the screen gets updated. If you're not convinced have a look at the MapPanel class and checkout the paint method.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/triplea-game/triplea/issues/3539#issuecomment-407289092, or mute the thread https://github.com/notifications/unsubscribe-auth/AABE8CKtmMH5Q0XMLZ5xAbgIrLfuPDy7ks5uJrXhgaJpZM4VJph0 .

[RoiEXLab]

@asvitkine We already should be avoiding this entirely. The Tiles should be read from memory when they're needed and then be stored in a BufferedImage to be redrawn from there. So if you can spot a place where we continously read from the File System I'd be happy to have it fixed.

[RoiEXLab]

Oh and @Cernelius Bump. Have a look at my instructions above.

[Cernelius]

With 1.9.0.0.10660, when I reproduce mainly what I did at the linked video, but following the instructions above, I have the issue of not being able to set the zoom from 10 to 100, but none of all the other issues is evident (the borders are set low correctly, reloading at 100% zoom the map is redrawn in less than 1 second, all moved units are redrawn correctly). I still have all those issues if I start TripleA normally, instead.

For what is worth, here it is the logging: https://gist.github.com/Cernelius/204af7169c3259d314fcc7dbcf58e310

[RoiEXLab]

@Cernelius Well that makes it super weird. By 'not being able to set the zoom' you mean you can set the zoom, but nothing happens, right? Maybe trying to scroll around a bit causes the Tiles to be redrawn correctly? In any case: My options should not alter the behaviour at all. So try running triplea again using just java -jar ./bin/triplea-<version>-all.jar (like you did earlier), without any additional settings. If this does affect behaviour, you might want to either check if there's any old (or additional/different) java version on your machine and remove that or if there's stuff in your TripleA.vmoptions file that makes everything behave weirdly.

[RoiEXLab]

Slightly Off-Topic but from the drawing logs I was reminded that the current code still enqueues too many drawing operations: While in theory tiles could be drawn twice due to the fact that the drawing started boolean check and set is non atomic, it's unlikely this happens often enough to make a difference in terms of performance. However it's still uneccessary to enqueue the Tile loading more than once, but I haven't figured out an elegant way to achieve this by now

[Cernelius]

By 'not being able to set the zoom' you mean you can set the zoom, but nothing happens, right?

Just like at 00:25 in the video.

you might want to either check if there's any old (or additional/different) java version on your machine and remove that

Where should I look?

or if there's stuff in your TripleA.vmoptions file that makes everything behave weirdly.

It's default.

I did it twice so far. No problems doing it your way, except for the zoom issue (maybe just a long delay), while the same problems as in the video, except no errors log, running it normally.

@asvitkine What of the stuff at the video you are getting too, if any?

[Cernelius]

Really, I think that if the map would be fully drawn when you start the game, instead of waiting scrolling around to load the tiles in view, I think this may be fixed. I think that would be the best behaviour anyways, so you don't see the tiles loading when you scroll around for the first time.

[Cernelius]

So, I'm doing further testing and, nevermind, now I see that either way usually, but not always, there are the issues, especially the slow drawing, and now it usually also happens that the game gets totally stuck I cannot even drag around the boardview anymore or click on any items of the menu.

Now I'm just waiting if it stops being stuck, because I cannot access the console, otherwise, and I see that PowerShell cuts the old entries. Is there a way to impede PowerShell doing that or increase its logging limit?

[RoiEXLab]

@Cernelius Not sure if this is 100% the same for powershell, but if you right click the title bar and go to settings, there should be an option somewhere for setting the "height" of the window aka the stored lines.

[Cernelius]

No problem, it got unstuck; so now I can access the TripleA console. I'm documenting the stuff and will make a post with the gist in the following minutes.

[Cernelius]

Also, how do I select the whole console? I had to stay stoned with my fingers on the uppercase button and the page down for like 5 minutes to select all; to then copy it.

[RoiEXLab]

@Cernelius Normally you wouldn't, you would redirect the output to a file, however powershell uses a weird encoding, so that's why copying is safer.

[Cernelius]

So, let's do one thing at a time, shall we?

In this one, I just loaded World At War at 100% zoom, and did nothing else but scrolling the map around, waiting for it to be fully drawn.

I started the game at 1:39:20. The initial view got loaded instantly, but, as soon as I dragged the board around, I encountered the massive delays I already documented in my video, tiles being drawn seldom and very slowly, without any clear pattern.

Around 1:40:30 the game got almost completely stuck, I was almost unable to drag the board around (holding right click). Moreover, for about a minute before that I already had huge delays, in doing so.

1:41:42 is the moment in which, suddenly at least what I see on the screen is fully drawn, and the logging stops, but the game now is completely stuck. I cannot drag the board view anymore at all for more than 1 minute.

Then, at 1:43:24 the board is suddenly fully loaded (I notice it immediately since my drag movement I tried to do previously was finally made), and now I can drag around freely, the tiles in the zones I didn't move the view yet are getting drawn in a fraction of a second, as they should.

However, that was speaking too soon, since after some minutes the game got totally stuck again (not exactly sure when, here, as I assumed I was out of the swamp, and stopped paying attention).

After something around 10 minutes, at 1:56:40, the game ends being totally stuck, again, and I can freely move around and play, also the units images getting redrawn correctly after moving them. This time is definitive.

Here it is the logging (from the TripleA console):

https://gist.github.com/Cernelius/493dfd4ad1c6cb40d8029c6b0444c147

[Cernelius]

I'll add that I've the vmoptions set at default, that is 2 GB, and World At War was a map that you could have loaded, back in 1.8.0.9, with no more than 0.5 GB (it's big, but the graphic is pretty light, actually).

[asvitkine]

On my side, the main symptoms I've seen is sometimes very slow repaint times - i.e. noticeable white regions for a noticeable amount of time while scrolling the map. One time it was really bad where it would last 10s of seconds before redraw.

[asvitkine]

In terms of file IO that I mentioned in my previous post, it seems that my concerns are invalid. I added a print call to the place that reads the file and I can confirm that each image is only read once. I guess there's just so many of them that it takes a long time for a map like WAW to get all of its tile images read as you scroll around...

I do wonder whether we should try to load them more eagerly in the background if the user will eventually scroll to those parts of the map anyway. This way, when they do scroll, there's less chance of a delay to read the image.

[panther2]

@RoiEXLab @Cernelius

I just want to add here that I am experiencing the same/a similar issue. Map drawing/rendering appears to be extremely slow since some versions. I am still seeing it with 1.9.0.0.10784:

I am on Java 8 Update 181 at the moment.

[Cernelius]

1.9.0.0.10784 gives the same issues documented at the first post of this ticket, except no error (so, practically, same situation as with 1.9.0.0.10660).

[Cernelius]

I guess you can bring back whatever you changed, as that didn't make any difference.

[Cernelius]

I've played 270BC 40% (a game of the 270bc_variants map) right now, for more than 8 hours nonstop, with 3 different players, for a total of 4 games (some finished, some saved). That map is very small, so, on full HD you load almost the half of it on the starting screen. If you are on a 4K, the entire map is loaded at 100%, and you also have a big unused space of about 1/3 of the bottom view. Consequently, the slow downs, on that one, are bearable. At the very first game I started soon after hosting it, I hod the slow downs in loading tiles out of the starting view that I described (but nowhere as slow as World At War) and, doing my first Carthage turn, I also had the not-redrawing units issues. The game would have been unplayable, if not that I can truly play the Carthage 1 blind, on that one. After I ended Carthage, all the units got correctly redrawn in the positions I put them, and, from them on, I had no further issues. And all the games I started after that, without closing my host, I never had any issues, the board getting loaded just fine at any new games. So it's like if the system would take too much time to get there, but once you get it running, it keeps running fine (you can keep reloading and restarting, and you don't have any such problems anymore, as long as it is the same process).

I hope you can get around finding a fix, because your quality and speed enhancements of the zoom are great, if they work.

[Cernelius]

1.9.0.0.10793 gives the same issues documented at the first post of this ticket, except no error (so, practically, same situation as with 1.9.0.0.10660).

[RoiEXLab]

@DanVanAtta @ron-murhammer @ssoloff I'm running out of ideas why this would happen. I can't believe the only reason this happens is because we no longer send way to much loading requests. So if you have any ideas on this topic, feel free to post them here. As a last option we can of course just load them all for once and hope that works for everyone.

[Cernelius]

As a last option we can of course just load them all for once and hope that works for everyone.

Really, I think this is how it should work, by now. We are not anymore in the nineties, so you can expect almost all machines to be able to have all the board view loaded up. Once Global Dominance is finished, that might be an issue, tho.

[FrostionAAA]

I hope this can get fixed. I have not been able to play with a pre-release since circa 1.9.0.0.10500. I never fool around with zoom on any map. I just use standard settings. But I have still experienced very very slow loading of tiles, just like others have described all ready. I have never waited 10 minutes to see if the tiles finally load, instead just assumed, after looking at un-loaded tiles for a long time, that the engine just hanged. If needed, I can try to make a video of how the issue looks on my PC? Otherwise I can just say that the issue is like others have described it all ready.

[Cernelius]

Yeah, this issue is making very hard to use the prerelease since a while, so who knows what other issues we might be overlooking. Myself, I'm back to the stable till this gets solved.

[ron-murhammer]

@RoiEXLab Not many ideas on this. Only things I can think of is checking thread priority of worker vs EDT and reviewing how much computation is being done on the EDT.

[ssoloff]

@RoiEXLab Unfortunately, I am unable to reproduce this issue on my dev VM, so it's quite difficult for me to gain any insight into what's going wrong.

Given that the tile rendering changes have spanned multiple PRs over the past month or so, what would be the realistic chance to successfully revert to the original code without introducing other regressions due to conflict resolution?

[RoiEXLab]

@ssoloff I have a 4 step recovery plan ^^

1. Try simply loading all tiles once they need to be regenerated, i.e. whenever we reset them. This is probably the most elegant solution of all, if it works.
2. Theory of quick access by minimum throughput: We could fall back to simply creating a lot of tasks for the same tile whenever we redraw and hope this is the root cause for this issue. It could work too.
3. Maybe we don't need to revert all of the changes, maybe it'd be sufficient to revert the changes in MapPanel.java the changes that actually affect the rendering behaviour.
4. As a last resort we could attempt a full revert. Would be unfortunate because this would add back lots of weird code and make zooming require a complete redraw, however although this type of change is split over multiple PRs, the rendering code wasn't touched by too many other commits, so we shouldn't run in too many conflicts.

However I can't promise I'll have the time to do it next week, I'll try though.

[Cernelius]

The zoom improvements were really a great advancement, so I hope you can find a way to fix this without going back to the previous zoom.

[Cernelius]

Also, if you can link all the various prereleases that progressively modified the zoom, I can test them all, if that might be useful.

[prastle]

@RoiEXLab @ssoloff
Just as I side note I noticed this just before we crashed today and got booted out of the game using stable / Newest. TripleA engine version 1.9.0.0.10850 WW2v341 The autosave did catch the save. Yes I noticed a very slow load of the allies. I am not sure if there is any relevance just adding here.

Loading map: world_war_ii_v3, from: C:\Users\prast\triplea\downloadedMaps\world_war_ii_v3-master.zip
Loading resources from the following paths: [C:\Users\prast\triplea\downloadedMaps\world_war_ii_v3-master.zip, G:\10850\TripleA\assets]
Loading map: world_war_ii_v3, from: C:\Users\prast\triplea\downloadedMaps\world_war_ii_v3-master.zip
Loading resources from the following paths: [C:\Users\prast\triplea\downloadedMaps\world_war_ii_v3-master.zip, G:\10850\TripleA\assets]
org.pushingpixels.substance.api.UiThreadingViolationException: Component creation must be done on Event Dispatch Thread
    at org.pushingpixels.substance.internal.utils.SubstanceCoreUtilities.testComponentCreationThreadingViolation(SubstanceCoreUtilities.java:1832)
    at org.pushingpixels.substance.internal.ui.SubstanceLabelUI.createUI(SubstanceLabelUI.java:76)
    at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at sun.reflect.misc.Trampoline.invoke(Unknown Source)
    at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at sun.reflect.misc.MethodUtil.invoke(Unknown Source)
    at javax.swing.UIDefaults.getUI(Unknown Source)
    at javax.swing.UIManager.getUI(Unknown Source)
    at javax.swing.JLabel.updateUI(Unknown Source)
    at javax.swing.JLabel.<init>(Unknown Source)
    at javax.swing.JLabel.<init>(Unknown Source)
    at games.strategy.triplea.image.ResourceImageFactory.getLabel(ResourceImageFactory.java:40)
    at games.strategy.triplea.image.ResourceImageFactory.getLabel(ResourceImageFactory.java:33)
    at games.strategy.triplea.ui.TripleAFrame$5.refresh(TripleAFrame.java:687)
    at games.strategy.triplea.ui.TripleAFrame$5.mouseEntered(TripleAFrame.java:629)
    at games.strategy.triplea.ui.MapPanel.notifyMouseEntered(MapPanel.java:346)
    at games.strategy.triplea.ui.MapPanel.updateMouseHoverState(MapPanel.java:223)
    at games.strategy.triplea.ui.MapPanel.lambda$new$0(MapPanel.java:202)
    at java.util.Observable.notifyObservers(Unknown Source)
    at java.util.Observable.notifyObservers(Unknown Source)
    at games.strategy.ui.ImageScrollModel.updateListeners(ImageScrollModel.java:39)
    at games.strategy.ui.ImageScrollModel.set(ImageScrollModel.java:137)
    at games.strategy.ui.ImageScrollerLargeView.setTopLeft(ImageScrollerLargeView.java:237)
    at games.strategy.triplea.ui.MapPanel.centerOn(MapPanel.java:292)
    at games.strategy.triplea.ui.TripleAFrame.getOkToLetAirDie(TripleAFrame.java:852)
    at games.strategy.triplea.TripleAPlayer.canAirLand(TripleAPlayer.java:346)
    at games.strategy.triplea.TripleAPlayer.move(TripleAPlayer.java:310)
    at games.strategy.triplea.TripleAPlayer.move(TripleAPlayer.java:327)
    at games.strategy.triplea.TripleAPlayer.move(TripleAPlayer.java:327)
    at games.strategy.triplea.TripleAPlayer.move(TripleAPlayer.java:327)
    at games.strategy.triplea.TripleAPlayer.move(TripleAPlayer.java:327)
    at games.strategy.triplea.TripleAPlayer.move(TripleAPlayer.java:327)
    at games.strategy.triplea.TripleAPlayer.move(TripleAPlayer.java:327)
    at games.strategy.triplea.TripleAPlayer.start(TripleAPlayer.java:122)
    at games.strategy.engine.framework.ClientGame.lambda$new$0(ClientGame.java:155)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at games.strategy.engine.message.unifiedmessenger.EndPoint.invokeSingle(EndPoint.java:149)
    at games.strategy.engine.message.unifiedmessenger.EndPoint.invokeMultiple(EndPoint.java:132)
    at games.strategy.engine.message.unifiedmessenger.EndPoint.invokeLocal(EndPoint.java:118)
    at games.strategy.engine.message.unifiedmessenger.UnifiedMessenger.lambda$messageReceived$1(UnifiedMessenger.java:269)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

[ron-murhammer]

@prastle That is a different issue already reported here #3639

[prastle]

@ron-murhammer cool I was just trying to help