eqasim-org / ile-de-france

An open synthetic population of Île-de-France for agent-based transport simulation
GNU General Public License v2.0
47 stars 69 forks source link

matsim.simulation.prepare : ScenarioValidator rejects everything #20

Closed Nitnelav closed 4 years ago

Nitnelav commented 4 years ago

I adapted the ile_de_france pipeline for another city in France. (you can have a look here : https://github.com/Nitnelav/eqasim-nantes)

I didn't change much apart from being able to set a list of requested commune_id instead of a list of departments...

Everything is working all right until the ScenarioValidator step. I get a "Person X has car leg without a route" for every single person in my population file... (see attached log file)

I'm a bit confused about where to look in order to fix this ...

Any idea ?

out.log

balacmi commented 4 years ago

Most probably in the routing stage something went wrong (in simulation/prepare there is a RunPopulationRouting stage before the validator). Usually, this problem occurs when the SwissRailRaptor fails to load the schedule because it is empty or there are errors in it. Do you maybe have a log from this RunPopulationRouting stage?

Nitnelav commented 4 years ago

You are right there is a few errors at the end...

2020-06-25 22:08:50,588  INFO SwissRailRaptorData:69 Preparing data for SwissRailRaptor...
2020-06-25 22:08:50,713  INFO SwissRailRaptorData:69 Preparing data for SwissRailRaptor...

Exception: java.lang.RuntimeException thrown from the UncaughtExceptionHandler in thread "Thread-1"
2020-06-25 22:08:50,793  INFO SwissRailRaptorData:69 Preparing data for SwissRailRaptor...

Exception: java.lang.RuntimeException thrown from the UncaughtExceptionHandler in thread "Thread-4"
2020-06-25 22:08:50,878  INFO SwissRailRaptorData:69 Preparing data for SwissRailRaptor...

Exception: java.lang.RuntimeException thrown from the UncaughtExceptionHandler in thread "Thread-2"

Exception: java.lang.RuntimeException thrown from the UncaughtExceptionHandler in thread "Thread-3"

They don't say much. I'll try to debug the java code tomorrow, but if you have any clues they are welcome !

Here is the full log : out.log

Nitnelav commented 4 years ago

The "RuntimeException" are actually :

com.google.inject.ProvisionException: Unable to provision, see the following errors:

1) Error in custom provider, java.lang.NullPointerException
  while locating ch.sbb.matsim.routing.pt.raptor.SwissRailRaptorFactory
  while locating ch.sbb.matsim.routing.pt.raptor.SwissRailRaptor
  while locating org.matsim.pt.router.TransitRouter
    for the 1st parameter of org.eqasim.core.components.transit.EqasimTransitModule.provideEnrichedTransitRouter(EqasimTransitModule.java:35)
  while locating org.eqasim.core.components.transit.routing.EnrichedTransitRouter
    for the 1st parameter of org.eqasim.core.components.transit.routing.EnrichedTransitRoutingModule.<init>(EnrichedTransitRoutingModule.java:30)
  while locating org.eqasim.core.components.transit.routing.EnrichedTransitRoutingModule
  while locating org.matsim.core.router.RoutingModule annotated with @com.google.inject.name.Named(value=pt)
  while locating org.matsim.core.router.RoutingModule annotated with @com.google.inject.multibindings.Element(setName=,uniqueId=182, type=MAPBINDER, keyType=java.lang.String)
  at org.matsim.core.router.TripRouter.<init>(TripRouter.java:108)
  while locating org.matsim.core.router.TripRouter
    for the 2nd parameter of org.eqasim.core.scenario.routing.PlanRouter.<init>(PlanRouter.java:28)
  while locating org.eqasim.core.scenario.routing.PlanRouter

It is thrown in PopulationRouter.java, line 71 I'm not familiar with the use of Injection in java ... will look into it ...

balacmi commented 4 years ago

In my experience it has nothing to do with the injection, but with the transit schedule. Usual problems are an empty transit line or a transit route without any stops. This sometimes happens when GTFS is converted to the MATSim transit schedule. Do you maybe have a complete log file of the scenario.prepare stage (not sure there will be something meaningful, but just in case)?

Nitnelav commented 4 years ago

Ok thank you for your input. It would be a relief not to go down the java dependency injection rabbit hole ! :)

I looked at the previous logs in simulation.prepare. The only "WARN" are :

2020-06-26 10:35:19,508  WARN LinkImpl:147 length=0.0 of link id 102575 may cause problems
2020-06-26 10:35:19,508  WARN LinkImpl:149  Future occurences of this logging statement are suppressed.
2020-06-26 10:35:19,509  WARN LinkImpl:107 fromNode=toNode=[id=315009105][coord=[x=359631.0334324084][y=6691366.068738026]][type=null][nof_inlinks=0][nof_outlinks=0]: link is a loop
2020-06-26 10:35:19,510  WARN LinkImpl:109  Future occurences of this logging statement are suppressed.
2020-06-26 10:35:19,511  WARN NodeImpl:87 [id=315009105][coord=[x=359631.0334324084][y=6691366.068738026]][type=null][nof_inlinks=0][nof_outlinks=1]: inlink_id=102575 is now in- and out-link
2020-06-26 10:35:19,511  WARN NodeImpl:88  This message given only once.

I'll try to remove the faulty nodes and links to see if that's the problem ...

Here is the full log containing RunPreparation, RunGenerateConfig and RunAdaptConfig, before RunPopulationRouting : out.log

balacmi commented 4 years ago

These warnings are probably not causing the problem, even though you might want to fix them in any case. Most definitely the problem is int he transit schedule generated by pt2matsim, so i would start looking there. And the complete RunPopulationRouting log would also help

Nitnelav commented 4 years ago

Ok thank you i'll keep looking. The full RunPopulationRouting log is here : https://github.com/eqasim-org/ile-de-france/issues/20#issuecomment-649795491

Nitnelav commented 4 years ago

Ok I found the problem.

I forgot I forked the pt2matsim repo to force a test to pass. https://github.com/Nitnelav/pt2matsim/commit/dac5ce1896c2e64b52dcfb2120fb68888cd77d57

The problem was in the input data, but you know I'm a developer, I try to fix the code before questioning anything else... 😁

Anyway, I fixed the input, fell back to the matsim-org pt2matsim code and the pipeline got to the matsim.run and finished successfully.

Thank you very much for your help !!