BVE5 Route Parser

[leezer3]

This PR contains a much cleaned up and fixed version of S520's BVE5 route parser.

It relies upon a fork of the BVE5_Parsing library by aiosupersix, source for which which may be found here, which I've been updating as I go along (this is basically the latest branch from aiosupersix, some unmerged stuff from S520 and a bunch of minor tweaks from me): https://github.com/leezer3/Bve5_Parsing/

Compatability appears good, although there are occasional slight issues with object alignment on curves (see for example the level crossings on Keisei)

Loading speed however is somewhat slow. This is a structural issue with BVE5_Parsing, which makes heavy (over?) use of reflection.

Test Build:

BVE5_Routes.zip

I think this is now in a reasonable state for considering merging, and then fixing further issues in the main builds once other people have had a chance to break it.

Known TODO:

• Remove the CSV parsing library. This isn't doing much, and we should be able to handle this ourselves for a slight improvement.
• Check TFOs, There's an option to enable them which isn't hooked up. Not sure yet whether they were not fully implemented by S520, or whether this just needs hooking in.
• Need to think about blocking text files better, as this now displays too much stuff in the main menu.

@ginga81

[ginga81]

Sorry. I don't know why, but when I run it from mono on Ubuntu22.04, the latest 1.10.1.1 doesn't progress even after waiting an hour from 0%. I'm thinking of updating to 24.04, waiting for the Japanese Team version and trying it after cleaning it up, but it hasn't been released yet. It loads when I start it from wine, but when I test this build, a dialog box appears saying 'A fatal error has occurred' and it fails to load. The crash log is here, but I've omitted it because my D51 folder is deep. Sorry.

2024-05-21 01:51:29
OpenBVE 1.7.1665.1037-leezer3 Crash Log
Program is running on the Windows backend
Current screen resolution is: Windowed 960px X 600px 
Current routefile is: C:\users\user\Desktop\Scenarios へのリンク\東京メトロ東西線(TokyoMetroTozai_Line)-2004_B0969S_各駅停車中野行.txt
Current train is: Z:\media\user\…\D51516\Passengers_5cars
The current routefile caused the following exception: 
System.NullReferenceException: Object reference not set to an instance of an object.
   at OpenBve.Loading.LoadEverythingThreaded() in B:\SourceCode\source\OpenBVE\System\Loading.cs:line 346
   at OpenBve.Loading.LoadThreaded() in B:\SourceCode\source\OpenBVE\System\Loading.cs:line 273   at System.Environment.GetStackTrace(Exception e, Boolean needFileInfo)
   at System.Environment.get_StackTrace()
   at OpenBve.Loading.LoadThreaded() in B:\SourceCode\source\OpenBVE\System\Loading.cs:line 273
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
Current program memory usage: 0mb
Unable to determine the current used / free memory figures.
This may indicate a wider system issue.

http://tozai.s77.xrea.com/BVE/ETozai.html The reason there is no TFO is that when the Bve5 parser was first made, we asked S520 to push the version without TFO to github first, and they released it.

After that, there is also a version with bug fixes and TFO.

I think when we discussed it, it was BSD, but we released it on Slack.

Since there is no clear record of license exchange, we limited distribution to Slack. However, before I could commit a fixed version, the S520's disease worsened, and more recently, Slack cut off the free version for its small community and made it a paid version, making it impossible to download.

[leezer3]

An hour is definitely too long.

I'll see what I can find on Ubuntu, but I can't think of anything that should affect things.

I haven't tried Tozai either, mostly a bunch of converted BVE2 / BVE4 stuff, Keisei and some older BVE5 content I had about.

(For what it's worth, his parser is mostly based upon the original CSV / RW parser, with most of the work being done by BVE5_Parsing)

[ginga81]

Even people using Windows environments, including the Keisei Line, are unable to load the data at all, and a dialog box appears saying `A fatal error has occurred' and they are returned to the menu. They reported to me. At least, we want to load Mackoy's keisei line.

[leezer3]

BVE5_Routes_2.zip

Second attempt.

I've found the issue with the fatal error (caused by the preview accidentally duplicating the station list), and fixed another issue in the parser found with Tozai.

Tozai works reasonably well with this, but there's a misplaced track I haven't yet tracked down the cause of.

(I've not yet got as far as trying to test under Linux. I'll do that tomorrow)

[ginga81]

The situation created by S520 has changed considerably from the current OpenBVE, so I think it would be quicker to complete this build than to wait for him to recover. I think it's almost done by incorporating S520's work. All that's left is to report and fix the issues on each individual route one by one, add TFO, and I think it will be complete. Most of all, I'm happy that the fps is higher than Wine+Bve. If possible, if it were possible to go in the opposite direction to PrayerPath here as well, I think we'll be able to enjoy more routes, since this is not currently possible with Bve.

[leezer3]

BVE5_Routes_3.zip

This build fixes a bunch more stuff, and enables scripted trains.

The one on Keisei (only one I've tested properly at the minute) is placed and moves correctly, but glitches back to the starting point as you pass. I've not yet investigated this, and it's likely to be a common issue with other scripted trains.

Biggest thing that appears broken at the minute is something to do with interpolation of curves / positioning on secondary tracks. Most routes don't seem to use this, but there are a few that do.

[ginga81]

https://web.archive.org/web/20200408092415/https://root5-bve.jimdofree.com/%E3%83%80%E3%82%A6%E3%83%B3%E3%83%AD%E3%83%BC%E3%83%89/ √5's created Fujikyu Railway uses 'other trains' to represent the railroad crossing at the starting station, but it forcibly uses values ​​that normally wouldn't be set. On the Tozai Line, rain is represented using impossible settings. Since THE Animated Object is not implemented in Bve, there are quite a few expressions that take advantage of Bve's bugs. Also, the routine to stop 'other trains' does not seem to be working, so the train passes through without stopping.

[leezer3]

I've now merged this into the main builds.

Some datapoints:

• Fujikyu Railway- working reasonably well. Some missing grounds to investigate.
• Keisei- I believe fully correct.
• Nagoya- Generally good, has some issues in tunnels with an animation effect.
• Tozai- Generally good, but I believe a couple of misplaced tracks popping up.
• TRTA- The main tracks are good, and nothing is spectacularly broken, but we appear to be missing some stuff (I believe related to using interpolation on secondary track positions. To investigate.....)
• Converted content from BVE2 / BVE4 appears fully correct.

Complex animations using scripted trains or exploiting BVE5 bugs may or may not work as designed. Passing trains generally are good.

Please note that the loading speed is very slow. This is a structural issue with BVE5_Parsing, which makes heavy use of reflection.

[ginga81]

What, it's been integrated into the main build?! So that means everyone can use it in the daily builds, right? Amazing! It seems like the order in which the triangular faces are taken in Putbetween is wrong, so it ends up becoming a triangle instead of a square. Please fix this. This is Fujikyu. As far as I can tell from Fujikyu, the loading speed wasn't an issue. Other Train's Stopping is still not seems not works. Please jump Fujikyu's Third Station, Tanokura(田野倉).

[leezer3]

Correct, it's now in the daily builds.

Otherwise, it ends up getting out of sync with changes in the main builds and gets stuck.....

PutBetween: It's an issue with the face windings in those platform pieces. They don't follow either of the two supported windings here https://openbve-project.net/documentation_hugo/en/routes/formroofcrack.html (this must have changed with BVE5)

Will have to consider writing an entirely new deformation algorithm :/

[ginga81]

I received a report that begintransition doesn't work and the vehicle tilts suddenly. Could you take a look just to be sure?

[leezer3]

The code appears sensible, and I can't find anything immediately wrong. If this is gradient related, it's noted (by S520) as a provisional implementation. I haven't really done a lot to the underlying algorithms other than fix obvious issues.

I need a route and approximate location so I can compare with BVE5 really- There's a lot of things that could be going wrong here.

[leezer3]

The build from today contains a change that may help, in that it fixes an issue where interpolation was allowed with zero-distance (not a good idea- just gives a broken transform).

This definitely fixes the missing objects on secondary tracks on the TRTA route, and is likely to solve some other glitches, but without the route to play with I'm not really sure :)

Haven't yet investigated the trains on Fujikyu, this is on the list of things to look at properly.

[ginga81]

begintransition seems to refer to the transition curve. The interpolation of the radius from when the curve begins to when it reaches the actual radius, for example, from a straight line to R600: straight from, R1000, R900, R800, R700, R600 --- R600, R700, R800, R900, R1000, back to a straight line It seems to refer to this gradual curve and return part.

[leezer3]

Curve / gradient transition is definitely working on Fujikyu, which means it's likely a route related issue somehow.

I've also checked the train in the third station against BVE5 behaviour, and this appears reasonably consistant. (It doesn't stop, just runs through quite slowly)

The trains on Tozai, which makes pretty heavy use of them also seem to be working reasonably consistantly with BVE5.

I think any differences are just down to the slightly more random nature of OpenBVE, and probably the fact that being able to see from outside the cab highlights things.