search

philvessey / NextDepartures

NextDepartures is a .NET Library that queries GTFS (General Transit Feed Specification) data sets stored locally or in an Azure SQL Database. The library will work with any well formed GTFS data set.
MIT License
1 stars 1 forks source link

Initial loading #15

Closed hypervtechnics closed 4 years ago

hypervtechnics commented 4 years ago

Initial loading of all required data.

Two ways:

  1. Add a new parameter (preload or cache) to the load value which creates an inner cache proxy for the IDataStorage.
  2. Or simply enhance the current solution. But code is not at the right place.

I prefer 1.

philvessey commented 4 years ago

What do we need to do to put this in place - if 1 is the best option we can go with that.

hypervtechnics commented 4 years ago

I'd go with 1. Will implement it now. Without changing functionality. Caching in between will be some more advanced as the query filters are applied in the sql.

philvessey commented 4 years ago

Is the intention being less hits on the database as the data already been preloaded / cached?

hypervtechnics commented 4 years ago

Yeah and in 95% of the case the data is static which may make querying via LINQ more easier for us and guarantuees.

hypervtechnics commented 4 years ago

Implemented and pushed. But unfortunaley the parameter list grew a lot...

philvessey commented 4 years ago

I don't think that's a problem as its the inner library making those calls - the end user doesn't really have to specify many parameters at all.

philvessey commented 4 years ago

I will do some testing tomorrow evening to make sure results are still as expected - some classes are reporting that _dataStorageProperties is never used - Feed.cs and PreloadDataStorage.cs - is that needed?

hypervtechnics commented 4 years ago

Yeah for the Feed.cs the private field should be left. I removed the other one.

What I don't like that much for now is that there are still calls to the provider required at the start of the GetServices methods... I will look for a better solution. Maybe in parallel with restructuring the loops.

philvessey commented 4 years ago

OK - I will be doing some more testing on Sunday to ensure results are as expected. If you do anymore commits before then I can test those too.

philvessey commented 4 years ago

Everything looks good testing wise - data returned by all the endpoints is as expected. Is work on this issue now complete or is there more work needed?

hypervtechnics commented 4 years ago

I will do some refactoring with the other issues. Can be closed.

hypervtechnics commented 4 years ago

So what's up with preloading? Also are there any other tasks?

philvessey commented 4 years ago

Seems ok to me - been testing the 2.0.0 version over the last couple of days and don't think there are any loading issues.