Installation of SUMO (Simulation of Urban Mobility) on Remote HPC for Improved Performance

[anaferraz]

I have been using SUMO (Simulation of Urban Mobility) for simulating urban traffic scenarios using AI on my local computer. However, the simulations are quite time-consuming, and I am looking to explore the possibility of installing SUMO on the remote HPC system for better performance. I would like to know if it is feasible to install SUMO on the remote HPC and run simulations there using Jupyter Lab Installed App.

[anaferraz]

https://sumo.dlr.de/docs/Downloads.php#linux This is the reference page for sumo and download

[asubah]

Hi, I looked in the link you provided. Getting it installed on the cluster looks straight forward. However, I didn't understand how are you planning to use the application? From the website I see that it is a GUI application, but from your message I understand that you want to use it from Jupyter Lab. Normally, we use VNC to launch GUI apps if the application supports it. And we use Jupyter Lab for python notebooks and CLI applications. Also, Slurm can be used to submit batch scripts to the compute nodes. Could you please clarify your workflow? How do you intend to use the package on the cluster?

[anaferraz]

Thank you for your questions. Let me clarify my approach to using SUMO and the TraCI interface on the cluster for my traffic simulations and reinforcement learning experiments. You are correct that SUMO is primarily a GUI-based application. However, I am running SUMO in a headless mode, without the GUI. This allows me to run the simulations more efficiently. In Jupyter notebook, I set up the traffic simulation scenario, open SUMO in a headless mode, run the simulation using the TraCI interface, save the simulation outputs, and then close the SUMO instance. This workflow does not require a GUI, as I am solely interacting with SUMO programmatically through the TraCI API. Utilizing a cluster environment can greatly accelerate the reinforcement learning workflow by running a large number of traffic simulations, enabling efficient exploration of different reward scenarios.

[asubah]

Thanks for the clarification! Just to confirm, you need us to install the sumo binary, then you are going to use it with some other tools like TraCI. I assume you will get TraCI installed using pip and python venv, correct? If that is the case, please confirm.

[anaferraz]

Yes, that is the case

[asubah]

Hi again, I installed the package on the nodes. Please test the installation and send us your feedback.

[anaferraz]

Thank you. I am getting the following error : SystemExit: please declare environment variable 'SUMO_HOME' Could you please let me know how to set the SUMO_HOME environment variable in this system?

[https://sumo.dlr.de/docs/Basics/Basic_Computer_Skills.html#:~:text=and%20press%20enter-,SUMO_HOME,the%20folders%20bin%20and%20tools.]

[asubah]

I just checked and the SUMO_HOME is set properly on the compute nodes. Could you run the following commands and send post the output:

hostname

echo $SUMO_HOME/

Could you please give me steps to reproduce this error? I think you are running your code on the master node, not on the compute nodes.

[asubah]

I just remembered that you said you are going to use Jupyter Lab. If that is the case, try this in one of the cells before you use SUMO:

import os
os.environ["SUMO_HOME"] = "/usr/share/sumo/" 

[anaferraz]

Thank you, it seems to be working now. Can you please confirm the SUMO version installed in the nodes?

[asubah]

Running sumo -V gives the following output:

Eclipse SUMO sumo Version v1_13_0+0457-439799d12ee
 Build features: Linux-5.14.21-150400.22-default x86_64 GNU 8.5.0 Release FMI Proj GUI SWIG GDAL GL2PS
 Copyright (C) 2001-2022 German Aerospace Center (DLR) and others; https://sumo.dlr.de

Eclipse SUMO sumo Version v1_13_0+0457-439799d12ee is part of SUMO.
This program and the accompanying materials
are made available under the terms of the Eclipse Public License v2.0
which accompanies this distribution, and is available at
http://www.eclipse.org/legal/epl-v20.html
This program may also be made available under the following Secondary
Licenses when the conditions for such availability set forth in the Eclipse
Public License 2.0 are satisfied: GNU General Public License, version 2
or later which is available at
https://www.gnu.org/licenses/old-licenses/gpl-2.0-standalone.html
SPDX-License-Identifier: EPL-2.0 OR GPL-2.0-or-later

[anaferraz]

Thank you for checking the version. Is this the last version available for Linux? I am having some compatibility issues, and they told me could be from the version installed: https://github.com/eclipse-sumo/sumo/issues/14908

[asubah]

It seems that the sumo pre-compiled binaries for CentOS are outdated, as the latest version available is 1.13.0. Therefore, I had to compile the sumo version 1.20.0 from source.

We use spack to manage package compilation for the cluster, so from the CLI you will have to run the following on a compute node to get sumo:

source /data/software/spack/env.sh
spack load sumo@1.20.0

From our OnDemand portal, you can use the modules field to make the Jupyter Lab instance starts with the sumo packages loaded.

[anaferraz]

Thank you, that's great, I was able to start with the SUMO 1.20.0 module. Should I keep the SUMO_HOME as the previous?

import os
os.environ["SUMO_HOME"] = "/usr/share/sumo/" 

[asubah]

No, you should remove this line. Spack should handle exporting the correct path. You can test it by running the following in a notebook cell:

! echo $SUMO_HOME

[anaferraz]

Thank you, all is working now