Error: Could not remove parked vehicle

[sashigurram]

I am getting the following error when I tried a MATSim simulation that includes road pricing and public transit. The error occurs about 44 iterations into the simulation and I have never seen this error before so I am a bit out of my depth here. Any suggestions that address this will be helpful. I am using matsim 0.9.0

Below is the short form of the error:

Could not remove parked vehicle with id 24267902 on the link id 23440. Maybe it is currently used by someone else? (In which case ignoring this exception would lead to duplication of this vehicle.) Maybe was never placed onto a link?

Here is the complete error from the log file:

2019-06-09 15:44:30,826 ERROR QSim:314 got exception while cleaning up java.lang.NullPointerException at org.matsim.core.mobsim.qsim.qnetsimengine.QueueWithBuffer.clearVehicles(QueueWithBuffer.java:674) at org.matsim.core.mobsim.qsim.qnetsimengine.QLinkImpl.clearVehicles(QLinkImpl.java:105) at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngine.afterSim(QNetsimEngine.java:272) at org.matsim.core.mobsim.qsim.QSim.cleanupSim(QSim.java:311) at org.matsim.core.mobsim.qsim.QSim.run(QSim.java:243) at org.matsim.core.controler.NewControler.runMobSim(NewControler.java:120) at org.matsim.core.controler.AbstractController$7.run(AbstractController.java:199) at org.matsim.core.controler.AbstractController.iterationStep(AbstractController.java:231) at org.matsim.core.controler.AbstractController.mobsim(AbstractController.java:195) at org.matsim.core.controler.AbstractController.iteration(AbstractController.java:154) at org.matsim.core.controler.AbstractController.doIterations(AbstractController.java:121) at org.matsim.core.controler.AbstractController.access$000(AbstractController.java:29) at org.matsim.core.controler.AbstractController$1.run(AbstractController.java:91) at org.matsim.core.controler.MatsimRuntimeModifications.run(MatsimRuntimeModifications.java:48) at org.matsim.core.controler.MatsimRuntimeModifications.run(MatsimRuntimeModifications.java:31) at org.matsim.core.controler.AbstractController.run(AbstractController.java:99) at org.matsim.core.controler.NewControler.run(NewControler.java:79) at org.matsim.core.controler.Controler.run(Controler.java:205) at org.matsim.roadpricing.RunRoadPricingExample.main(RunRoadPricingExample.java:51) 2019-06-09 15:44:33,097 ERROR AbstractController:210 Mobsim did not complete normally! afterMobsimListeners will be called anyway. java.lang.RuntimeException: java.lang.RuntimeException: Could not remove parked vehicle with id 24267902 on the link id 23440. Maybe it is currently used by someone else? (In which case ignoring this exception would lead to duplication of this vehicle.) Maybe was never placed onto a link? at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngine.run(QNetsimEngine.java:344) at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngine.doSimStep(QNetsimEngine.java:282) at org.matsim.core.mobsim.qsim.QSim.doSimStep(QSim.java:361) at org.matsim.core.mobsim.qsim.QSim.run(QSim.java:237) at org.matsim.core.controler.NewControler.runMobSim(NewControler.java:120) at org.matsim.core.controler.AbstractController$7.run(AbstractController.java:199) at org.matsim.core.controler.AbstractController.iterationStep(AbstractController.java:231) at org.matsim.core.controler.AbstractController.mobsim(AbstractController.java:195) at org.matsim.core.controler.AbstractController.iteration(AbstractController.java:154) at org.matsim.core.controler.AbstractController.doIterations(AbstractController.java:121) at org.matsim.core.controler.AbstractController.access$000(AbstractController.java:29) at org.matsim.core.controler.AbstractController$1.run(AbstractController.java:91) at org.matsim.core.controler.MatsimRuntimeModifications.run(MatsimRuntimeModifications.java:48) at org.matsim.core.controler.MatsimRuntimeModifications.run(MatsimRuntimeModifications.java:31) at org.matsim.core.controler.AbstractController.run(AbstractController.java:99) at org.matsim.core.controler.NewControler.run(NewControler.java:79) at org.matsim.core.controler.Controler.run(Controler.java:205) at org.matsim.roadpricing.RunRoadPricingExample.main(RunRoadPricingExample.java:51) Caused by: java.lang.RuntimeException: Could not remove parked vehicle with id 24267902 on the link id 23440. Maybe it is currently used by someone else? (In which case ignoring this exception would lead to duplication of this vehicle.) Maybe was never placed onto a link? at org.matsim.core.mobsim.qsim.qnetsimengine.VehicularDepartureHandler.teleportVehicleTo(VehicularDepartureHandler.java:118) at org.matsim.core.mobsim.qsim.qnetsimengine.VehicularDepartureHandler.handleCarDeparture(VehicularDepartureHandler.java:85) at org.matsim.core.mobsim.qsim.qnetsimengine.VehicularDepartureHandler.handleDeparture(VehicularDepartureHandler.java:56) at org.matsim.core.mobsim.qsim.QSim.arrangeAgentDeparture(QSim.java:441) at org.matsim.core.mobsim.qsim.QSim.arrangeNextAgentAction(QSim.java:406) at org.matsim.core.mobsim.qsim.QSim.access$000(QSim.java:98) at org.matsim.core.mobsim.qsim.QSim$1.arrangeNextAgentState(QSim.java:151) at org.matsim.core.mobsim.qsim.ActivityEngine.handleActivity(ActivityEngine.java:163) at org.matsim.core.mobsim.qsim.QSim.arrangeAgentActivity(QSim.java:422) at org.matsim.core.mobsim.qsim.QSim.arrangeNextAgentAction(QSim.java:403) at org.matsim.core.mobsim.qsim.QSim.access$000(QSim.java:98) at org.matsim.core.mobsim.qsim.QSim$1.arrangeNextAgentState(QSim.java:151) at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngine.letVehicleArrive(QNetsimEngine.java:426) at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngine.access$200(QNetsimEngine.java:74) at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngine$1.letVehicleArrive(QNetsimEngine.java:97) at org.matsim.core.mobsim.qsim.qnetsimengine.AbstractQLink.letVehicleArrive(AbstractQLink.java:155) at org.matsim.core.mobsim.qsim.qnetsimengine.QueueWithBuffer.letVehicleArrive(QueueWithBuffer.java:515) at org.matsim.core.mobsim.qsim.qnetsimengine.QueueWithBuffer.moveQueueToBuffer(QueueWithBuffer.java:446) at org.matsim.core.mobsim.qsim.qnetsimengine.QueueWithBuffer.doSimStep(QueueWithBuffer.java:399) at org.matsim.core.mobsim.qsim.qnetsimengine.QLinkImpl.doSimStep(QLinkImpl.java:119) at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngineRunner.moveLinks(QNetsimEngineRunner.java:187) at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngineRunner.call(QNetsimEngineRunner.java:122) at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngineRunner.call(QNetsimEngineRunner.java:37) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 2019-06-09 15:44:33,326 WARN CharyparNagelActivityScoring:252 The first and the last activity do not have the same type. 2019-06-09 15:44:33,326 WARN CharyparNagelActivityScoring:253 Will score the first activity from midnight to its end, and the last activity from its start to midnight. 2019-06-09 15:44:33,326 WARN CharyparNagelActivityScoring:254 Because of the nonlinear function, this is not the same as scoring from start to end. 2019-06-09 15:44:33,326 WARN CharyparNagelActivityScoring:255 first activity: [type=home][coord=null][linkId=30881][startTime=undefined][endTime=06:40:23][duration=undefined][facilityId=null] 2019-06-09 15:44:33,326 WARN CharyparNagelActivityScoring:256 last activity: [type=school][coord=null][linkId=31176][startTime=06:57:31][endTime=undefined][duration=undefined][facilityId=null] 2019-06-09 15:44:33,326 WARN CharyparNagelActivityScoring:257 This may also happen when plans are not completed when the simulation ends. 2019-06-09 15:44:33,326 WARN CharyparNagelActivityScoring:259 Additional warnings of this type are suppressed. 2019-06-09 15:44:38,127 ERROR MatsimRuntimeModifications:54 Getting uncaught Exception in Thread main java.lang.RuntimeException: java.lang.RuntimeException: Could not remove parked vehicle with id 24267902 on the link id 23440. Maybe it is currently used by someone else? (In which case ignoring this exception would lead to duplication of this vehicle.) Maybe was never placed onto a link? at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngine.run(QNetsimEngine.java:344) at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngine.doSimStep(QNetsimEngine.java:282) at org.matsim.core.mobsim.qsim.QSim.doSimStep(QSim.java:361) at org.matsim.core.mobsim.qsim.QSim.run(QSim.java:237) at org.matsim.core.controler.NewControler.runMobSim(NewControler.java:120) at org.matsim.core.controler.AbstractController$7.run(AbstractController.java:199) at org.matsim.core.controler.AbstractController.iterationStep(AbstractController.java:231) at org.matsim.core.controler.AbstractController.mobsim(AbstractController.java:195) at org.matsim.core.controler.AbstractController.iteration(AbstractController.java:154) at org.matsim.core.controler.AbstractController.doIterations(AbstractController.java:121) at org.matsim.core.controler.AbstractController.access$000(AbstractController.java:29) at org.matsim.core.controler.AbstractController$1.run(AbstractController.java:91) at org.matsim.core.controler.MatsimRuntimeModifications.run(MatsimRuntimeModifications.java:48) at org.matsim.core.controler.MatsimRuntimeModifications.run(MatsimRuntimeModifications.java:31) at org.matsim.core.controler.AbstractController.run(AbstractController.java:99) at org.matsim.core.controler.NewControler.run(NewControler.java:79) at org.matsim.core.controler.Controler.run(Controler.java:205) at org.matsim.roadpricing.RunRoadPricingExample.main(RunRoadPricingExample.java:51) Caused by: java.lang.RuntimeException: Could not remove parked vehicle with id 24267902 on the link id 23440. Maybe it is currently used by someone else? (In which case ignoring this exception would lead to duplication of this vehicle.) Maybe was never placed onto a link? at org.matsim.core.mobsim.qsim.qnetsimengine.VehicularDepartureHandler.teleportVehicleTo(VehicularDepartureHandler.java:118) at org.matsim.core.mobsim.qsim.qnetsimengine.VehicularDepartureHandler.handleCarDeparture(VehicularDepartureHandler.java:85) at org.matsim.core.mobsim.qsim.qnetsimengine.VehicularDepartureHandler.handleDeparture(VehicularDepartureHandler.java:56) at org.matsim.core.mobsim.qsim.QSim.arrangeAgentDeparture(QSim.java:441) at org.matsim.core.mobsim.qsim.QSim.arrangeNextAgentAction(QSim.java:406) at org.matsim.core.mobsim.qsim.QSim.access$000(QSim.java:98) at org.matsim.core.mobsim.qsim.QSim$1.arrangeNextAgentState(QSim.java:151) at org.matsim.core.mobsim.qsim.ActivityEngine.handleActivity(ActivityEngine.java:163) at org.matsim.core.mobsim.qsim.QSim.arrangeAgentActivity(QSim.java:422) at org.matsim.core.mobsim.qsim.QSim.arrangeNextAgentAction(QSim.java:403) at org.matsim.core.mobsim.qsim.QSim.access$000(QSim.java:98) at org.matsim.core.mobsim.qsim.QSim$1.arrangeNextAgentState(QSim.java:151) at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngine.letVehicleArrive(QNetsimEngine.java:426) at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngine.access$200(QNetsimEngine.java:74) at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngine$1.letVehicleArrive(QNetsimEngine.java:97) at org.matsim.core.mobsim.qsim.qnetsimengine.AbstractQLink.letVehicleArrive(AbstractQLink.java:155) at org.matsim.core.mobsim.qsim.qnetsimengine.QueueWithBuffer.letVehicleArrive(QueueWithBuffer.java:515) at org.matsim.core.mobsim.qsim.qnetsimengine.QueueWithBuffer.moveQueueToBuffer(QueueWithBuffer.java:446) at org.matsim.core.mobsim.qsim.qnetsimengine.QueueWithBuffer.doSimStep(QueueWithBuffer.java:399) at org.matsim.core.mobsim.qsim.qnetsimengine.QLinkImpl.doSimStep(QLinkImpl.java:119) at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngineRunner.moveLinks(QNetsimEngineRunner.java:187) at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngineRunner.call(QNetsimEngineRunner.java:122) at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngineRunner.call(QNetsimEngineRunner.java:37) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 2019-06-09 15:44:38,128 ERROR MatsimRuntimeModifications:60 ERROR --- This is an unexpected shutdown! 2019-06-09 15:44:38,128 ERROR MatsimRuntimeModifications:63 Shutdown possibly caused by the following Exception: java.lang.RuntimeException: java.lang.RuntimeException: Could not remove parked vehicle with id 24267902 on the link id 23440. Maybe it is currently used by someone else? (In which case ignoring this exception would lead to duplication of this vehicle.) Maybe was never placed onto a link? at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngine.run(QNetsimEngine.java:344) at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngine.doSimStep(QNetsimEngine.java:282) at org.matsim.core.mobsim.qsim.QSim.doSimStep(QSim.java:361) at org.matsim.core.mobsim.qsim.QSim.run(QSim.java:237) at org.matsim.core.controler.NewControler.runMobSim(NewControler.java:120) at org.matsim.core.controler.AbstractController$7.run(AbstractController.java:199) at org.matsim.core.controler.AbstractController.iterationStep(AbstractController.java:231) at org.matsim.core.controler.AbstractController.mobsim(AbstractController.java:195) at org.matsim.core.controler.AbstractController.iteration(AbstractController.java:154) at org.matsim.core.controler.AbstractController.doIterations(AbstractController.java:121) at org.matsim.core.controler.AbstractController.access$000(AbstractController.java:29) at org.matsim.core.controler.AbstractController$1.run(AbstractController.java:91) at org.matsim.core.controler.MatsimRuntimeModifications.run(MatsimRuntimeModifications.java:48) at org.matsim.core.controler.MatsimRuntimeModifications.run(MatsimRuntimeModifications.java:31) at org.matsim.core.controler.AbstractController.run(AbstractController.java:99) at org.matsim.core.controler.NewControler.run(NewControler.java:79) at org.matsim.core.controler.Controler.run(Controler.java:205) at org.matsim.roadpricing.RunRoadPricingExample.main(RunRoadPricingExample.java:51) Caused by: java.lang.RuntimeException: Could not remove parked vehicle with id 24267902 on the link id 23440. Maybe it is currently used by someone else? (In which case ignoring this exception would lead to duplication of this vehicle.) Maybe was never placed onto a link? at org.matsim.core.mobsim.qsim.qnetsimengine.VehicularDepartureHandler.teleportVehicleTo(VehicularDepartureHandler.java:118) at org.matsim.core.mobsim.qsim.qnetsimengine.VehicularDepartureHandler.handleCarDeparture(VehicularDepartureHandler.java:85) at org.matsim.core.mobsim.qsim.qnetsimengine.VehicularDepartureHandler.handleDeparture(VehicularDepartureHandler.java:56) at org.matsim.core.mobsim.qsim.QSim.arrangeAgentDeparture(QSim.java:441) at org.matsim.core.mobsim.qsim.QSim.arrangeNextAgentAction(QSim.java:406) at org.matsim.core.mobsim.qsim.QSim.access$000(QSim.java:98) at org.matsim.core.mobsim.qsim.QSim$1.arrangeNextAgentState(QSim.java:151) at org.matsim.core.mobsim.qsim.ActivityEngine.handleActivity(ActivityEngine.java:163) at org.matsim.core.mobsim.qsim.QSim.arrangeAgentActivity(QSim.java:422) at org.matsim.core.mobsim.qsim.QSim.arrangeNextAgentAction(QSim.java:403) at org.matsim.core.mobsim.qsim.QSim.access$000(QSim.java:98) at org.matsim.core.mobsim.qsim.QSim$1.arrangeNextAgentState(QSim.java:151) at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngine.letVehicleArrive(QNetsimEngine.java:426) at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngine.access$200(QNetsimEngine.java:74) at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngine$1.letVehicleArrive(QNetsimEngine.java:97) at org.matsim.core.mobsim.qsim.qnetsimengine.AbstractQLink.letVehicleArrive(AbstractQLink.java:155) at org.matsim.core.mobsim.qsim.qnetsimengine.QueueWithBuffer.letVehicleArrive(QueueWithBuffer.java:515) at org.matsim.core.mobsim.qsim.qnetsimengine.QueueWithBuffer.moveQueueToBuffer(QueueWithBuffer.java:446) at org.matsim.core.mobsim.qsim.qnetsimengine.QueueWithBuffer.doSimStep(QueueWithBuffer.java:399) at org.matsim.core.mobsim.qsim.qnetsimengine.QLinkImpl.doSimStep(QLinkImpl.java:119) at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngineRunner.moveLinks(QNetsimEngineRunner.java:187) at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngineRunner.call(QNetsimEngineRunner.java:122) at org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEngineRunner.call(QNetsimEngineRunner.java:37) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 2019-06-09 15:44:38,149 ERROR MatsimRuntimeModifications:78 ERROR --- MATSim unexpectedly terminated. Please check the output or the logfile with warnings and errors for hints. 2019-06-09 15:44:38,150 ERROR MatsimRuntimeModifications:79 ERROR --- results should not be used for further analysis. 2019-06-09 15:44:38,150 ERROR MatsimRuntimeModifications:83 ERROR --- This was an unexpected shutdown! See the log file for a possible reason.

[kainagel]

Ouch.

Could you please say which type of vehicle this is, to which type of agent it belongs, etc.

Also, if possible, could you please make the full output directory available somewhere? Dropbox or google drive or some large file provider? If this is too large, remove all large files in the iteration directories except for the last version of each file type (i.e. if you are crashing in iteration 44 and you are writing events every 10 iterations, then keep the events file from iteration40).

[sashigurram]

Thanks for your response Dr. Nagel.

I will have to guess the vehicle type here because I did not write output in this iteration. I think this vehicle type is car because I just see a number for the vehicle id (if it is a ride vehicle type it normally would show up as 24267902_ride). I see the error for person ID 24267902.

I uploaded all the data onto a University storage server but unfortunately you may have to download in multiple attempts due to download size restrictions. I was writing outputs every 25 iterations, so the last complete output folder will be it.25 (download from here). If you just want the last full output, then downloading this single folder may be enough. Full output folder including all iterations is here.

Here are the input files including config, original input plans, network, toll links, transit vehicles, transit schedule, and vehicles.

Finally, here is the entire error folder (which includes output and input files) if you want to browse through before downloading.

Please let me know if you need further details or want me to rerun the simulation so that I get a closer output to the error (by writing outputs every 10 iterations).

[kainagel]

Dear Sashi,

thanks.

Unfortunately, the git hash of your run does not correspond to anything in our repo. So I cannot stare at the code because I don't know which version it is. Isn't there a way you can run this with a release?

However, very unfortunately such things are terribly difficult to debug. Maybe the first attempt would be to re-run with writing events every 5 iterations so that there will be output in the 45th iteration where it crashes, and we can check if the same car was picked up before.

If you have enough computational power, I would additionally start another run with usingThreadpool=false (in the qsim config group), based on the wild speculation that this is related to multithreading.

And finally, if you have even more computational power, start with numberOfThreads=1 (also in qsim config group). (I don't think that it will be 20 times slower.)

(All of these runs with writeevents every 5iterations.)

Brrrrrr ....

[sashigurram]

Thanks Dr. Nagel.

I still am not sure why the runs fail. The error appears to popup randomly; some runs fail in the 87th iteration while some failed in the 92nd iteration. In any case, I started a few runs simultaneously with no settings changed and one of them finished properly. I also started a single thread run with the setting numberOfThreads=1 and this too finished properly. The single thread run took almost 70% more time (about 8.5 days in total) compared to the multi-threaded run.

[kainagel]

Annoying. It may indeed be a concurrency issue. I changed the parking queue data structure from LinkedHashMap to ConcurrentHashMap and will run the tests over it to see if that fails anything else (since it changes the sort order).

[shahriarzame]

Dear Kai Nagel, faced the same issue. Running UAM extension. Worked well in scenarios with different UAM Station coordinates, just in one specific network the issue occured twice in 4th and once in 6th iteration. I was using numberOfThreads = 12.

logfileWarningsErrors.log

Is there solution yet? Not using Parallel threading is very slow. Thanks