BVE5: Known Issues

[leezer3]

Known Issues:

Nagoya route has a black tunnel animation, which doesn't work correctly.

---

ViewingDistance command is not supported

---

Scripted trains do not currently have sounds implemented.

---

Tozai has a couple of misplaced rails (repeaters?) causing duplicate track on Rail0 in a couple of places.

---

JR Kobe line has various misplaced repeaters.

---

Files using ATS-EX are currently unsupported.

---

The default color for BVE5 when no background is present is pure black. Yurachuko uses as a side-effect to hide below the tunnels.

---

Backgrounds use (generally) cylindrical objects. BVE5 does not allow free camera rotation, so most of these do not have a cap.

It may be useful to auto-generate a cap- This should be possible relatively simply (find top points and center of circle), but it may glitch out if using a non-standard cube or something. Think about this....

[ginga81]

I think there will be a lot of incorrect displays due to subtle differences in interpretation of Irony for each route, so I think it would be better to create a dedicated reporting thread or issue.

[ginga81]

For testing: Hankyu https://sites.google.com/view/fwchbve/top/bve%E3%83%87%E3%83%BC%E3%82%BF%E3%83%BC%E5%85%AC%E9%96%8B/bve%E9%98%AA%E6%80%A5%E7%B7%9A%E4%BB%A3%E7%90%86%E5%85%AC%E9%96%8B?authuser=0 BVE5Hankyu_new_ver3.4.zip

This data is very heavy, and some of the variables only use numbers. I remember that this caused the S520 code to become unreadable, so I'm reporting it. There should also have been some places where the cant was reversed, but that was several years ago and Slack made the data paid and hid it, so I don't know what the corresponding distances were.

Some routes returned errors and could not be loaded. The selected route is corrupt: No stations defined. HK京都本線N - 準急11035.txt HK京都本線N - 準急3061.txt HK京都本線N - 普通727.txt HK千里線N - 普通1053.txt HK千里線N - 普通751.txt HK嵐山線N - 普通1285.txt

JR Kobe line This data consumes a lot of memory, and even if we load it in Bve5, it may not load on some PCs. https://moffbarrel.stars.ne.jp/JRKobe/Top.htm JR神戸線_20200426.zip This data requires Nagoya_Common for the object. Place it in: Scenarios/Nagoya_Common https://moffbarrel.stars.ne.jp/Nagoya_Common.html Nagoya_Common_20231225.zip

Most of the data cannot be read due to the following error. Object reference not set to an instance of an object. Mc223kobe_3449a.txt Mc223kobe_757t.txt Mc223kobe3405m.txt Mc223kobe9506m.txt Mc223kobe9607.txt

[leezer3]

The Kobe crash should actually already be fixed as of the most recent daily, caused by an empty time in a scripted train.

They're somewhat broken at the minute though, something to do with repeaters not right in a lot of places.

[leezer3]

Hankyu: BVE5_Parsing is using ReadJEnc ( https://github.com/hnx8/ReadJEnc ) to determine the charset encoding of each file. For some reason, it thinks the file \Scenarios\hankyu_new\Map11030_common.txt is actually a non-text file, and so returns no text for it. As this is the main map file, nothing works with it missing.

[leezer3]

OK, build from today replaces the charset detection library in BVE5_Parsing.

This sorts out the initial problems with Hankyu, which seems to work reasonably well, other than FPS.

FPS issues are due to some trees it's using. These are found under the folder hankyu_new\Structures\L53\HK\trees 8,000 alpha faces for a 4 tree cluster (repeated multiple times) is why it's not at all happy.

I'll take a look and see if I can't get these to optimise a little better, but this may be a problem.

[leezer3]

Optimisation wasn't as bad as I thought actually.

Latest build will also force-optimise objects where there are 500 times more faces than materials. This gets the trees down to a single face apice, and gets Hankyu from ~5fps to a much more sensible 60fps.

[ginga81]

I was surprised because I didn't think RouteViewer would support it. I guess the loading time is unavoidable.

So I looked into why the curve display was strange on the Hankyu line. HK京都本線N - 快速急行6005.txt This is Map6005_paramxxx.txt, which includes Map6005-Fine.txt or Map6005-Rain.txt, which in and includes Include'Map6005_Track.txt'; Include'Map6005_common.txt'; When viewed in RouteViewer, the distance is 3330 to R-4207.56. At 3350 it becomes R-402. This is extreme and the track display is also strange. Line 709 of Map6005_Track.txt says 3325; Track['LOC'].Cant.BeginTransition(); but this is a cant so it is probably unrelated. Line 896 of Map6005_common.txt says

3325; Curve.BeginTransition(); 3350; Curve.BeginCircular(-402,-0.1);

and I think that the parser may be misunderstanding this.

[hotdamndel]

Some route files on Yurakucho files don't load: https://utatanesuya.web.fc2.com/bve/yurakucho.html On others there's grey background visible and there are some missing objects

[hotdamndel]

Gakunan Railway (haven't found an available link now) has the start station clearing departure and finishing boarding before a TFO occupying the track arrives. New Rinkai line (http://miyakoji.bugyo.tk/BVE/twr/twr3.html) has rollercoaster cants

[ginga81]

@hotdamndel https://github.com/leezer3/OpenBVE/pull/1041#issuecomment-2127146782 Gakunan here. It is officially closed to the public currently, it showld be used only for discussion as much as possible. I did not propose Gakunan as a sample because I do not think it had any special use like the Fujikyu Line's starting crossing. And also it is closed officially same Fujikyu.

[hotdamndel]

I see. I've noticed a similar issue on Tobu Tojo line (signal clears before a train pulls into the station) and also misplaced TFOs in a few locations. There are also missing track sounds and misplaced track bits in some places.

https://sites.google.com/view/bvey98mtojochiai/bve-trainsim/%E3%83%87%E3%83%BC%E3%82%BF%E3%81%AE%E5%8F%96%E3%82%8A%E6%89%B1%E3%81%84%E3%81%AA%E3%81%A9%E3%81%AB%E9%96%A2%E3%81%97%E3%81%A6/%E3%83%80%E3%82%A6%E3%83%B3%E3%83%AD%E3%83%BC%E3%83%89%E3%83%9A%E3%83%BC%E3%82%B8?authuser=0

[ginga81]

We can display the route using RouteViewer and get the distance, so I think it would be extremely difficult to verify unless we could write down which command in which file and for which distance is wrong. So I researched as much as I could for Hankyu before writing this. I have omitted the distance for Fujikyu because there is no need to move at the starting station.

[hotdamndel]

Announcements play sequentially in each car instead of simultaneously (especially noticeable on trains with bogies)

[leezer3]

Hankyu glitch: It's the cant causing this, although I haven't quite figured out the interaction yet. Basically, the whole station should be on a quite steep curve with cant.

[leezer3]

So the issue here is actually a little complex :/

Hankyu mixes repeater lengths. Our glitch occurs in a block where a transition takes place. The trouble is that our repeater here is of 25m length, whereas the block is of 5m length (that being the shortest repeater)

Essentially, it takes into account the orientation of the next block, that being of 5m length, not the block in 25m, hence our object is glitched.

Might take a little thinking about.....

[leezer3]

Build from today will make these a lot better, but they're not perfect.

I can't quite get it to work as I think it mathematically should, which probably suggests to me that it's something to do with our interpolation implementation differing from that of BVE5.

[leezer3]

Yurakucho: Some map files for this use the ATS-EX Map plugin. This isn't supported by either us or the parser library, and is unfortunately unlikely to be at present. I'll see what I can do about adding a more sensible error message at some point.

I'll need to investigate the missing objects a little more, but I suspect they may well be related to the above, at least tangentially.

[ginga81]

At first there was no connection, but now 'おーとま', the creator of AtsEX, has been allowed by mackoy to directly view, modify, and release the source code of Bve56 as Bve. AtsEX will be linked to this, and will be closely related from now on. As a result of this, there may be route data that is difficult to unreadable from now on.

[leezer3]

Latest build fixes the cant issues and has a lot of other minor improvements.

Some routes are unfortunately still not quite right (Kobe now looks a lot better, but isn't quite there yet), but I think we're getting to the stage where this can be pushed onto the main site as a starting point.

[ginga81]

I think that from now on, we will be reporting abnormalities for individual dataes, but since the number of datas is huge, the number of reports will be proportionally huge. Is it okay to use this topic to efficiently report and request corrections in the future? I think it would be better to prepare a report format like an issues and report the information.

[leezer3]

That's fine. I'm just trying to keep together a single list of broken stuff in BVE5 content, as otherwise things end up getting forgotten. We're also still at the stage where fixing one issue tends to make lots of stuff work better, as opposed to trying to work around glitches :)

Today's build implements the new crack algorithm. This is still to be improved (it'll still break if the object does not use square faces), but that makes most stuff previously broken with this work correctly.

[marcriera]

I've found an issue when parsing floating-point numbers, which currently seems to be affected by system locale. My PC uses the ca_ES locale, which has commas as the decimal separator. This is the exact same route (Tokyo Metro Tozai Line):

mono OpenBve.exe

LC_ALL=C mono OpenBve.exe

[leezer3]

Should be fixed when things rebuild.

It's the classic decimal separator error. (Not helped by the format parser library storing everything internally in strings)

FWIW, I'm also seeing the missing track in the second screenshot. I think at this point it's normal (basically to do with BVE5 being restricted to the cab and the route starting point....)

[Kenny-Hui]

Apologies if I missed anything I should have known about, but it seems that for Tozai Line, non-local route (i.e. 2012_B1373S_rapid_Map) errored out midway during the parsing, which resulted in an incomplete route that only contains the starting section:

Error       [650:16] Error: The input string '8' is not valid for the expected Map Format ARG_END in /media/ModAndGames/BveTs5/Scenarios/TokyoMetroTozai_Line/2012_B1373S_rapid_Map.txt

Seems that there is also some form of weird cylinder taking shape at around 111609m. I've read something about Tozai using Bve bugs to create rain earlier, is that related or no?

[leezer3]

Are you on the build from today? IIRC I fixed that one a few days ago.

With the cylinder thing, it's an attempt to create dark tunnels. Basically, some sort of variant on the BVE2 / BVE4 block based visibility trick. (With that, you basically build an object at a good distance entirely in front of the centerpoint. Legacy visibility then disposes of it when the camera passes the placement position) I've not yet got as far as figuring out exactly how it works. It's probably got something to do with the ViewingDistance statements.

[Kenny-Hui]

Ah, was missing a week of bug fixes...

Now hopefully actually on the latest nightly build: A couple of misplaced tracks of the following route (1575m): http://bve-shikokuke.geo.jp/download.html

It also appears that the parsing is case-sensitive at least on Linux (I believe that's not the case for CSV/RW), and some content with mismatched case would not load the full section, such as the above.

[leezer3]

I can reproduce that track glitch. Need to look into it.

However, I don't immediately see anything that's case sensitive path wise. Which exact routefile is that please?

[ginga81]

Similar symptoms occur around 25219m on the Fuji-Q line. Also, I think it is necessary to recreate animations that use impossible numerical settings, such as east-west rain and the railroad crossing at the starting station of the Fuji-Q line.

[Kenny-Hui]

I can reproduce that track glitch. Need to look into it.

However, I don't immediately see anything that's case sensitive path wise. Which exact routefile is that please?

In the aforementioned route, in Scenarios/瀬戸大橋・予讃・土讃線/Scenarios/3115M.txt Line 80: include '../Map\Map_movetrains\3115M_movetrains.txt'; The path seems to be Map\Map_Movetrains\3115M_Movetrains.txt

Error       [80:9] Error: The specified file ../Map\Map_movetrains\3115M_movetrains.txt does not exist. in /media/ModAndGames/BveTs5/Scenarios/瀬戸大橋・予讃・土讃線/Scenarios/3115M.txt

[leezer3]

All the linked file does there is control some of the TFOs- It's definitely not causing the track to end unexpectedly.

As far as I can see, Scenarios/瀬戸大橋・予讃・土讃線/Scenarios/3115M.txt is OK on Windows. Will see about trying Linux in a bit.

[Kenny-Hui]

Track just doesn't load anymore at around 40000m Disregard, wrong routefile

I just saw the error message and thought that might be relevant, although I don't really have that much of a chance to test it now due to the load time so I am not sure

[hotdamndel]

A bit too obvious but there should be an official BVE5 route documentation somewhere which contains all the info on how exactly BVE5 behaves. Following it is more reliable than fishing out bugs via specific scenarios (this is also useful, but not for general cases, more like when a route author does something smart and undocumented) I will look if there are any, and there's train data support lacking too

[ginga81]

Bve5/6 uses the parser generator Irony to interpret grammar. https://github.com/IronyProject/Irony And since the Bve source code is not public, the only way to know what will be displayed depending on what is written is to look at the results.

[hotdamndel]

Bve5/6 uses the parser generator Irony to interpret grammar. https://github.com/IronyProject/Irony And since the Bve source code is not public, the only way to know what will be displayed depending on what is written is to look at the results.

But route developers have to know how to make the data. Similar how there's documentation for Openbve

[leezer3]

Mackoy has never been the best with documentation.

There's a Japanese (more complete) and English version on bvets.net but I don't think either are fully complete, and they make a bunch of assumptions. IIRC there's also some stuff on his blog, and probably other stuff scattered around unofficially too.

[ginga81]

As we are discussioning about, the data creators has been created the data based on mackoy's grammar manual and released the data based on the results that were correctly displayed in the interpretation of Irony and Bve, but in reality I think the only way to bridge the discrepancies between these and the parser's interpretation and the rendering of the OpenBVE engine is to interpret the display results of each individual piece of data.

[hotdamndel]

Many misplaced track bits. Route is https://shtr-m.net/bve/hrd.html

[leezer3]

Build from today will help considerably with some glitches (I've replaced the original averages algorithm with actually calculating the block positions), but I'm afraid there are still quite a few minor ones in places. With the route linked above (Tokyo Rapid Transit), this now appears mostly evident in terms of small gaps in repeaters (track) in some places.

Most seem to be gradient / pitch of object related, along with some interesting repeater length choices, which are confusing things somwhere along the line. Looking back at the BVETS development blog, I suspect that BVE5 may well only internally place repeaters at 1m intervals: https://mackoy-exblog-jp.translate.goog/9595544/?_x_tr_sl=ja&_x_tr_tl=en&_x_tr_hl=en

Need to do some more experimentation on this front, which may well end up being quite a slow process.

[marcriera]

The Hokuhoku line (https://hkhkbve.web.fc2.com/index.html) has improved (there were missing track segments all over the place, now they are rare), but a new issue has appeared. Some repeaters (tracks and bridges mostly) now make "steps" when placed on what I assume are sections with pitch. I see in the BVETS documentation that the Repeater command has a tilt argument which defines whether the object is affected by pitch, cant, both or neither (related to the link in the previous post). Maybe related to that.

Latest build:

Reference video: https://www.youtube.com/watch?v=M7A6aLVOMg8

[leezer3]

Build from today I think fixes most of the slope issues.

(I'm not entirely sure why, but the pitch for the primary track seems to need to be inverted when calculating the position for secondary tracks..... I suspect a typo somewhere, but if it works as a starting point....). This is better again in most cases.

Still need to investigate exact repeater behaviour with regards to block lengths with a copy of BVE5, as I suspect reproducing this correctly will fix some of the minor gaps.

[hotdamndel]

A lot less track mismatches now, but some still remain (tested on Tobu Tojo line) There are also TFOs oddly cutting thru some curves. Btw I think it's a good idea to test BVE5 compat using only routes natively built for it and not upconverted in the past (Hankyu, most of Tokyo Metro including Tozai line, etc), these tend to show more of the current bugs.

[adfriz]

Tested on daily builds 2024-07-05.zip

there's many miss placed object.

I think this route is more suitable as a test for improving the route parser because its natively made for bve5 not converted from bve4.

link: https://x.com/rts_trainsim/status/1028686370242322433

Error logs:

Error       Plugin Object.DirectX.dll returned unsuccessfully at LoadObject
Error       Plugin Object.DirectX.dll returned unsuccessfully at LoadObject
Error       Plugin Object.DirectX.dll returned unsuccessfully at LoadObject
Error       Plugin Object.DirectX.dll returned unsuccessfully at LoadObject
Error       Plugin Object.DirectX.dll returned unsuccessfully at LoadObject
Error       Plugin Object.DirectX.dll returned unsuccessfully at LoadObject
Error       Plugin Object.DirectX.dll returned unsuccessfully at LoadObject
Warning     The structure p42_04 has been declared twice: The most recent declaration will be used.
Error       InvalidData : Invalid or unsupported BitDepth in PNG file KCIAolineBVE5\YkKl_ckrAoline\Structures\obj\haus\h1.png
Error       InvalidData : Invalid or unsupported BitDepth in PNG fileKCIAolineBVE5\YkKl_ckrAoline\Structures\obj\haus\h2.png
Error       InvalidData : Invalid or unsupported BitDepth in PNG file KCIAolineBVE5\YkKl_ckrAoline\Structures\obj\haus\h3.png
Error       InvalidData : Invalid or unsupported BitDepth in PNG file KCIAolineBVE5\YkKl_ckrAoline\Structures\obj\haus\h4.png
Error       InvalidData : Invalid or unsupported BitDepth in PNG file KCIAolineBVE5\YkKl_ckrAoline\Structures\obj\haus\h1.png
Error       InvalidData : Invalid or unsupported BitDepth in PNG file KCIAolineBVE5\YkKl_ckrAoline\Structures\obj\haus\h2.png
Error       InvalidData : Invalid or unsupported BitDepth in PNG file KCIAolineBVE5\YkKl_ckrAoline\Structures\obj\haus\h3.png
Error       InvalidData : Invalid or unsupported BitDepth in PNG file KCIAolineBVE5\YkKl_ckrAoline\Structures\obj\haus\h4.png

[adfriz]

Another test, daily builds 2024-07-05.zip.

90% usable route, only small numbers of objects miss placed and a bit of odd draw ordering bug (z index related maybe?).

link: https://10spanrail.web.fc2.com/ShinmeiDownload.html

Error logs:

Error       Texure Scenarios\IIDA\深名線\Structures\2020-05-21.png was not found in file É[û╝ÉⁿÿHÉⁿâfü[â^\Scenarios\IIDA\深名線\Structures\map.x
Warning     An invalid sound radius was specified for key 90, 90.wav,,4

[leezer3]

OK, so the first of your two test cases uses some slightly more unusual binary X files, which we weren't handling correctly. (reference based materials in binary format, along with a dual frame global transform) I've made various fixes to the X parsers which means these actually load properly.

Unfortunately, it's still broken, but the objects aren't glitched anymore. I'll try and investigate this one more fully next week.

Not got as far as looking into the second yet, at some point next week.

[hotdamndel]

Kintetsu Nara line at Shin-Omiya (second station)

[leezer3]

OK, I've found the issue with Fukamata. The first glitch is much reduced (it now matches Quick Map Viewer from @zbx1425 ), and it sorts the second nicely.

TLDR: BVE's rotation notation is in DirectX style, whereas OpenBVE uses OpenGL. What this means is that pitch has to be reversed internally. My initial slope fix assumed that it was something to do with the math involving calculating the positions and slope of secondary tracks, but it turned out that all the calculated pitch values just needed reversing. (As a general rule, very little in BVE5 content is placed on the notional track 0)

This is likely to apply to a lot of other content, basically anything using a non-zero pitch value on an object. It was actually very helpful that this object was so large, much easier to see the effects.

[leezer3]

With regards to the trees / transparency issues on Fukamata: Essentially, this is a current engine issue, and I'm not sure how easily fixable at the minute. The trees in question get squashed into a single face per texture for performance reasons, but in this case it's not so good for Z-sorting (not helped by the fact that they use alpha channel transparency)

I'm investigating alternative transparency methods, but dunno how long this will take if at all. If this happens, GL1.2 (the 'old' renderer) will likely be removed.

[hotdamndel]

My more knowledgeable in graphics friend who also does Openbve developing suggests considering Vulkan as a long-term render bug/performance improvement solution

[leezer3]

The underlying tech (Vulkan, OpenGL, DirectX etc.) isn't really the issue, although it might help marginally.

Basically, we use a simplistic version of the painters algorithm to handle transparencies. This requires sorting the visible faces by depth, which is the slow part, and also why the glitches occur.

There are various techinques for doing transparency in an alternative manner, but it took a lot of effort to get GL3 to exactly replicate the behaviour of GL1.2. Implementing these is probably possible, but I don't know at this point what else is likely to break if we do so.

[zbx1425]

OK, I've found the issue with Fukamata. The first glitch is much reduced (it now matches Quick Map Viewer from @zbx1425 ), and it sorts the second nicely.

Please refer to BVE5 for behaviour validation, as I'm not confident that QuickMapViewer's behaviour matches BVE5 perfectly, and it most likely doesn't xD

---

Geometry-wise, one thing is that rotation angles are converted as -rx and -ry, and another is that BVE5 uses X>Z>Y rotation order.