Error when getting any departure board

[livetracks]

Describe the bug When getting a departure board for any station, it returns a 500 error.

To Reproduce Any request to the public endpoint seems to be affected.

Additional context Only some stations were returning errors starting earlier this week, from yesterday it appears now all show errors. I think it's related again to the format of service IDs from what I can see? I don't know if the format has been changed again.

Log:

Category: Huxley2.Controllers.DeparturesController
EventId: 0
SpanId: 27be07a0100bf079
TraceId: dab9e9a35c04ae0cc31f4232efb35206
ParentId: 0000000000000000
RequestId: 80000f73-0000-fb00-b63f-84710c7967bb
RequestPath: /departures/BMH
ActionId: 76dee548-cb72-4871-aa09-47fe2c55dec0
ActionName: Huxley2.Controllers.DeparturesController.Get (Huxley2)

Open LDB DepartureBoard API call failed

Exception: 
System.ArgumentOutOfRangeException: Index and length must refer to a location within the string. (Parameter 'length')
   at System.String.Substring(Int32 startIndex, Int32 length)
   at OpenLDBWS.BaseServiceItem.ToGuid(String serviceID) in D:\a\Huxley2\Huxley2\Huxley2\Connected Services\OpenLDBWS\BaseServiceItem.cs:line 39
   at ServiceIdGuidGetter(Object )
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.GetMemberAndWriteJson(Object obj, WriteStack& state, Utf8JsonWriter writer)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryWrite(Utf8JsonWriter writer, T value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.Converters.ArrayConverter`2.OnWriteResume(Utf8JsonWriter writer, TElement[] array, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryWrite(Utf8JsonWriter writer, TCollection value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.GetMemberAndWriteJson(Object obj, WriteStack& state, Utf8JsonWriter writer)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryWrite(Utf8JsonWriter writer, T value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.JsonConverter`1.WriteCore(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.JsonConverter`1.WriteCoreAsObject(Utf8JsonWriter writer, Object value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.JsonSerializer.WriteCore[TValue](JsonConverter jsonConverter, Utf8JsonWriter writer, TValue& value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.JsonSerializer.WriteStream[TValue](Stream utf8Json, TValue& value, JsonTypeInfo jsonTypeInfo)
   at System.Text.Json.JsonSerializer.Serialize(Stream utf8Json, Object value, Type inputType, JsonSerializerOptions options)
   at Huxley2.Services.ChecksumGenerator.GenerateChecksumObj(Object obj) in D:\a\Huxley2\Huxley2\Huxley2\Services\ChecksumGenerator.cs:line 57
   at Huxley2.Services.ChecksumGenerator.GenerateChecksumImpl(Object response) in D:\a\Huxley2\Huxley2\Huxley2\Services\ChecksumGenerator.cs:line 43
   at Huxley2.Services.ChecksumGenerator.GenerateChecksum(BaseStationBoard board) in D:\a\Huxley2\Huxley2\Huxley2\Services\ChecksumGenerator.cs:line 14
   at Huxley2.Services.StationBoardService.GenerateChecksum(BaseStationBoard board) in D:\a\Huxley2\Huxley2\Huxley2\Services\StationBoardService.cs:line 103
   at Huxley2.Controllers.DeparturesController.Get(StationBoardRequest request) in D:\a\Huxley2\Huxley2\Huxley2\Controllers\DeparturesController.cs:line 50

[jasonkapadia]

Did you find a solution to this?

[jalada]

I'm seeing service IDs like 46002CLPHMJM_ which are definitely not new-format.

[livetracks]

Did you find a solution to this?

Sorry I can't be of assistance, I don't know any C# , I only do front-end stuff

[jalada]

Here's my really crude patch whereby I don't understand:

1. C#
2. What these service ID formats actually are
3. Why we convert them to GUIDs
4. What indentation this code is meant to be

It works enough to bring https://traintrack.io back online for now until someone does a proper job 😊 I'll update here if this causes any more problems.

Basically, I'm handling a dynamic number of numbers in the service ID. I feel like there's a regex waiting to be written here but I am already stretching my knowledge of C# with this patch.

diff --git a/Huxley2/Connected Services/OpenLDBWS/BaseServiceItem.cs b/Huxley2/Connected Services/OpenLDBWS/BaseServiceItem.cs
index aedaef5..b27ba0b 100644
--- a/Huxley2/Connected Services/OpenLDBWS/BaseServiceItem.cs
+++ b/Huxley2/Connected Services/OpenLDBWS/BaseServiceItem.cs
@@ -31,15 +31,29 @@ namespace OpenLDBWS

         public static Guid ToGuid(string serviceID)
         {
+           // Count the number of...numbers.
+           int numerics = 0;
+           for (int i=0; i<serviceID.Length; i++) {
+               if ((serviceID[i] >= '0') && (serviceID[i] <= '9')) {
+                       numerics++;
+               }
+           }
+
+           // Pad so we have 16 chars including our number
+           string pad = "";
+           for (int i=16-numerics; i>0; i--) {
+                   pad += "0";
+           }
+
             // because we have 128 bits to use in the GUID (and only ~56 bits
             // from the service ID), we can afford to encode the numeric
             // characters directly and the 8 alpha characters as ASCII instead
             // of finding a more efficient solution, and zero-pad the centre
-            string num = serviceID.Substring(0, 7); // digits
-            string str = serviceID.Substring(7, 8); // letters
+            string num = serviceID.Substring(0, numerics); // digits
+            string str = serviceID.Substring(numerics, 8); // letters
             byte[] bytes = System.Text.Encoding.UTF8.GetBytes(str);
             string hexString = Convert.ToHexString(bytes);
-            string guidString = num + "000000000" + hexString.Substring(0, 16);
+            string guidString = num + pad + hexString.Substring(0, 16);
             return new Guid(guidString);
         }

[davwheat]

This is my own fix for the problem. It replaces the current method's splitting of numbers and strings into just converting the entire string to a GUID. This works fine with the current 15-byte service IDs, but could start producing duplicates if they go above 16 bytes. The code should hold up in terms of not causing errors if this is the case, though.

I have assumed nothing else needs to be changed to let this work, though.

Old patch -- see below

```patch diff --git a/Huxley2/Connected Services/OpenLDBWS/BaseServiceItem.cs b/Huxley2/Connected Services/OpenLDBWS/BaseServiceItem.cs index aedaef5..10fb783 100644 --- a/Huxley2/Connected Services/OpenLDBWS/BaseServiceItem.cs +++ b/Huxley2/Connected Services/OpenLDBWS/BaseServiceItem.cs @@ -31,16 +31,27 @@ namespace OpenLDBWS public static Guid ToGuid(string serviceID) { - // because we have 128 bits to use in the GUID (and only ~56 bits - // from the service ID), we can afford to encode the 7 numeric - // characters directly and the 8 alpha characters as ASCII instead - // of finding a more efficient solution, and zero-pad the centre - string num = serviceID.Substring(0, 7); // digits - string str = serviceID.Substring(7, 8); // letters - byte[] bytes = System.Text.Encoding.UTF8.GetBytes(str); + byte[] bytes = System.Text.Encoding.UTF8.GetBytes(serviceID); + string hexString = Convert.ToHexString(bytes); - string guidString = num + "000000000" + hexString.Substring(0, 16); - return new Guid(guidString); + + // Need 8 chars for first group of GUID, plus one extra for the "rest" of the GUID. + // In reality, this should always be the case, but let's try to make this method as + // robust as possible. + if (hexString.Length < 9) + { + hexString = hexString.PadRight(9, '0'); + } + + // First group of GUID + string guidStart = hexString[0..8]; + + // Rest of the hex string to form the rest of the GUID, padding the left until it + // makes up a full GUID string when combined with the first group, and ensuring + // it's no longer than 24 chars in the unlikely case it is. + string guidRest = hexString[8..].PadLeft(24, '0').Substring(0, 24); + + return new Guid(guidStart + guidRest); } public static string FromGuid(Guid serviceGuid) ```

[davwheat]

Oops, I did just totally ignore FromGuid below it... here's a fixed fix!

diff --git a/Huxley2/Connected Services/OpenLDBWS/BaseServiceItem.cs b/Huxley2/Connected Services/OpenLDBWS/BaseServiceItem.cs
index aedaef5..b1426be 100644
--- a/Huxley2/Connected Services/OpenLDBWS/BaseServiceItem.cs
+++ b/Huxley2/Connected Services/OpenLDBWS/BaseServiceItem.cs
@@ -31,27 +31,40 @@ namespace OpenLDBWS

         public static Guid ToGuid(string serviceID)
         {
-            // because we have 128 bits to use in the GUID (and only ~56 bits
-            // from the service ID), we can afford to encode the 7 numeric
-            // characters directly and the 8 alpha characters as ASCII instead
-            // of finding a more efficient solution, and zero-pad the centre
-            string num = serviceID.Substring(0, 7); // digits
-            string str = serviceID.Substring(7, 8); // letters
-            byte[] bytes = System.Text.Encoding.UTF8.GetBytes(str);
+            byte[] bytes = System.Text.Encoding.ASCII.GetBytes(serviceID);
+
             string hexString = Convert.ToHexString(bytes);
-            string guidString = num + "000000000" + hexString.Substring(0, 16);
-            return new Guid(guidString);
+
+            // Need 8 chars for first group of GUID, plus one extra for the "rest" of the GUID.
+            // In reality, this should always be the case, but let's try to make this method as
+            // robust as possible.
+            if (hexString.Length < 9)
+            {
+                hexString = hexString.PadRight(9, '0');
+            }
+
+            // First group of GUID
+            string guidStart = hexString[0..8];
+
+            // Rest of the hex string to form the rest of the GUID, padding the left until it
+            // makes up a full GUID string when combined with the first group, and ensuring
+            // it's no longer than 24 chars in the unlikely case it is.
+            string guidRest = hexString[8..].PadLeft(24, '0').Substring(0, 24);
+
+            return new Guid(guidStart + guidRest);
         }

         public static string FromGuid(Guid serviceGuid)
         {
-            // reverse of ToGuid above
+            // Opposite of the above
+
             string guidString = serviceGuid.ToString("N", CultureInfo.InvariantCulture);
-            string num = guidString.Substring(0, 7); // get digits
-            string hexString = guidString.Substring(16, 16); // get letters
-            byte[] bytes = Convert.FromHexString(hexString);
-            string str = System.Text.Encoding.UTF8.GetString(bytes);
-            return num + str;
+
+            // Gets the raw byte array from the GUID, filtering out padded null bytes (0x00)
+            byte[] bytes = Array.FindAll(Convert.FromHexString(guidString), b => b != 0x00);
+
+            // Convert bytes back to string
+            return System.Text.Encoding.ASCII.GetString(bytes);
         }
     }
 }

[jasonkapadia]

Hi @davwheat, do you know what could be causing the /service/{service ID} to error 500?