JetteSchumann commented 4 years ago

Since the basic functionality of Method D, I and J (and the code) are the same, we should consider an integration of Method I and J into Method D. The basic function of Method D should remain the same. By defining additional input parameters Method I and J could be realized in Method D. Options could be:

use_blind_points = "True" to enable the calculation of vornoi cells for less than 3 people as in Method I
calc_IFD_only = "True" to enable the calculation as provided in Method I
velocity_calculation = "XYZ" to use velocity calculation which is implemented in Method J They should be False by default. Any suggestions for input parameters?

In order to solve this issue we need to:

clarify input parameters (available options for the methods; Should Method I and J still be available or just be realized via Method D?)
clarify how to restructure the code
update documentation
update tests
update tutorials

Affected issues are: JuPedSim/jpscore#707 Scripts/notebooks for analysis JuPedSim/jpscore#720 Testing JuPedSim/jpscore#775 Testing

Bugs that could be solved when restructuring: JuPedSim/jpscore#706 MA out of bounds JuPedSim/jpscore#772 Profile Option JuPedSim/jpscore#776 Bug Method I

chraibi commented 4 years ago

Maybe use_blind_points=True for all?

JetteSchumann commented 4 years ago

@chraibi you mean using blind points by default for all methods?

chraibi commented 4 years ago

Yes, exactly. Even with Method D, you sometimes want to measure the FD of a, say 1D experiment. In low densities, you don't have always 3 people in the measurement area at once.

gjaeger commented 4 years ago

At first: I like Jette's suggestion. 👍

For a proposal for the combination of methods I would like to start with a small study.

Starting from the Methods test suite in jpsreport/Utest, I have performed the following test. I have taken into account the result of issue JuPedSim/jpscore#772 that the profiles are created for the entire geometry.

geometry with three subrooms (geo_Methods_subrooms.xml.zip)
inifile for geometry with thee subrooms (ini_Methods_subrooms.xml.zip)

My expectations at the start:

Profiles over the entire geometry across all subrooms.
Individual data (IFD) for Method I over the entire geometry across all subrooms.

The considered measuring areas:

  <measurement_areas unit="m">

    <area_B id="1" type="BoundingBox" zPos="None">
    </area_B>

    <area_B id="5" type="BoundingBox" zPos="None">
      <vertex x="20" y="12" />
      <vertex x="24" y="12" />
      <vertex x="24" y="8" />
      <vertex x="20" y="8" />
    </area_B>

    <area_B id="9" type="BoundingBox" zPos="None">
      <vertex x="16" y="12" />
      <vertex x="20" y="12" />
      <vertex x="20" y="8" />
      <vertex x="16" y="8" />
    </area_B>

    <area_B id="10" type="BoundingBox" zPos="None">
      <vertex x="28" y="12" />
      <vertex x="32" y="12" />
      <vertex x="32" y="8" />
      <vertex x="28" y="8" />
    </area_B>

  </measurement_areas>

An excerpt of the input file:

  <method_D enabled="true">
    <measurement_area id="1" start_frame="None" stop_frame="None" get_individual_FD="true"/>
    <measurement_area id="5" start_frame="None" stop_frame="None" get_individual_FD="true"/>
    <measurement_area id="8" start_frame="None" stop_frame="None" get_individual_FD="true"/>
    <measurement_area id="9" start_frame="None" stop_frame="None" get_individual_FD="true"/>
    <measurement_area id="10" start_frame="None" stop_frame="None" get_individual_FD="true"/>
    <one_dimensional enabled="false"/>
    <cut_by_circle enabled="true" radius="1.0" edges="10"/>
    <profiles enabled="true" grid_size_x="0.20" grid_size_y="0.20"/>
  </method_D>

  <method_I enabled="true">
    <measurement_area id="1" start_frame="None" stop_frame="None" get_individual_FD="true"/>
    <cut_by_circle enabled="true" radius="1.0" edges="10"/>
    </method_I>

  <method_J enabled="true">
    <measurement_area id="1" start_frame="None" stop_frame="None" get_individual_FD="true"/>
    <measurement_area id="5" start_frame="None" stop_frame="None" get_individual_FD="true"/>
    <one_dimensional enabled="false"/>
    <cut_by_circle enabled="true" radius="1.0" edges="10"/>
    <profiles enabled="true" grid_size_x="0.20" grid_size_y="0.20"/>
  </method_J>

In the following comments I give the results.

gjaeger commented 4 years ago

profiles - density

The profiles are independent of the measurement area (see issue JuPedSim/jpscore#772). They extend over the entire geometry across all subrooms.

Edit: The following plots show the averaged density over the entire time.

measurement area 1

profile_id_1_density

measurement area 5

profile_id_5_density

measurement area 9

profile_id_9_density

⚠️ Warning ⚠️

If measuring areas are used for different methods, the profile is created anew for each method. Existing files are overwritten.

gjaeger commented 4 years ago

profiles - velocity

The methods D and J differ in the calculation of the mean velocity.

In the current version, the profiles for the different methods are overwritten. This should be noted. The profiles for methods D and J are as follows.

Edit: The following plots show the averaged velocity over the entire time.

measurement area 9 - method D

profile_v_id_9

measurement area 5 - method J

profile_v_id_5

summary

There are differences. ⚠️ We should take this knowledge into account in our further work.

gjaeger commented 4 years ago

trajectories based on individual data

method I

Method I can be used for time-series analysis of individual data. Across the boundaries of the subrooms, the individual data is determined and stored in the file IFD_I.

traj_I_1

I had expected this result.

method D

The individual data for agents that have an intersection (with their voronoi cell) with the measurement area are stored in the file IFD_D. The data is limited to the area of the measuring surface and depends on the setting.

measurement area 5

traj_D_5

measurement area 9

traj_D_9

measurement area 10

traj_D_10

I had expected these result. We should take these differences into account.

gjaeger commented 4 years ago

When comparing the velocity profiles of methods D and J, you can think to recognize small differences. The detailed view for one frame (e.g. frame 21-22) illustrates the differences.

profile based on method J - measurement area 5

vel_J_id_5_fr_21-22

profile based on method D - measurement area 9

vel_D_id_9_fr_21-22

The velocity profile based on method D shows blurred edges. This becomes clear in a detail view (first: method J - measurement area 5; second: method D - measurement area 9):

vel_J_id_5_fr_21-22_detail

vel_D_id_9_fr_21-22_detail

The difference is due to the method being called in the GetProfiles function (method D: line 364; method J: line 368). When calculating the profiles, the grid cells are used as measurement areas and the different velocity functions GetVoronoiDensityVelocity (method D: line 328; method J: line 326) are called for the methods. @JetteSchumann sees great potential for optimisation in these functions.

By the way, the profiles for the density are the same, since the methods of calculation do not differ.

I ask myself the basic questions:

Is the calculation of the profiles correct?
Is there a faster calculation algorithm?

During the implementation we have to document the calculation method.

How can we define the requirements for an implementation and create a work plan?

gjaeger commented 4 years ago

To my question "Is the calculation of the profiles correct?" I was asked why I ask this. I would like to clarify my question:

With the methods for a measurement area we calculate the mean density and velocity for each grid cell. Afterwards we use an interpolation method when plotting with Matplotlib. Do we need this accuracy when we interpolate at the end? Is there perhaps a faster algorithm if we use the individual data (IFD_I)? The calculation of individual data with jpsreport is fast. Maybe a python script for the profiles is better? Does anyone know a good solution?

chraibi commented 4 years ago

Afterwards we use an interpolation method when plotting with Matplotlib

which interpolation?

Maybe a python script for the profiles is better?

depends on your definition of "profiles". Could you give more details about what you suggest?

gjaeger commented 4 years ago

Afterwards we use an interpolation method when plotting with Matplotlib

which interpolation?

see /jpsreport/scripts/_Plot_profiles.py, line 81: interpolation = 'nearest'

   im = plt.imshow(density, cmap=cm.jet,
                    interpolation='nearest', origin='lower',
                    vmin=0, vmax=np.amax(density),
                    extent=[geominX, geomaxX, geominY, geomaxY])

Maybe a python script for the profiles is better?

depends on your definition of "profiles". Could you give more details about what you suggest?

In the following I refer to the individual data (IFD_I) and the default setting of cell size dx=dy=0.2 Assuming that an agent can occupy only one cell, one could iterate over the agents for each frame and assign the individual velocity or density to the cell. When creating the profile, the intermediate values between the central cells could be set via the interpolation setting of imshow. One would have to look at the possible difference.

This is a first thought.

chraibi commented 4 years ago

If I understand correctly you propose an alternative script, not only to plot the profiles but to calculate them in a different way. If the methodology is correct and the script is fast enough, we could remove the calculation in jpsreport and outsource it to the script.

Wanna give it a try?

gjaeger commented 4 years ago

Using the example of the demo "bottleneck", I made an analysis with the profile files and the IFD_I file. The results can be found in the notebook (ipynb) on gists for issue JuPedSim/jpscore#576.

gjaeger commented 4 years ago

velocity_calculation = "XYZ" to use velocity calculation which is implemented in Method J

They should be False by default. Any suggestions for input parameters?

I suggest the following parameters for veleocity calculation:

"voronoi" for Method D
"arithmetic" for Method J

gjaeger commented 4 years ago

in addition to my last comment ... I propose to change the documentation of the velocity calculation based on the formulas describing the previous methods C and J as follows:

method C

method J

new description:

$\langle v \rangle_{A}=\frac{1}{N_{(x,y) \in A}}\sum_{i=1}^{N_{(x,y) \in A}}{v_i(t)}$

gjaeger commented 4 years ago

After dicussion with Jette, our proposal for the realisation of issue JuPedSim/jpsreport#193, JuPedSim/jpscore#809 and JuPedSim/jpscore#810:

<measurement_method_voronoi enabled="true">
   <measurement_area id="1" start_frame="None" stop_frame="None" local_individual="true"/>
   <measurement_area id="2" start_frame="None" stop_frame="None" local_individual="true"/>
   <cut_by_circle enabled="false" radius="1.0" edges="10"/>
   <global_individual enabled="true" zPos="None"/>
   <profiles enabled="false" zPos="None" grid_size_x="0.20" grid_size_y="0.20"/>
 </measurement_method_voronoi>

This includes default settings:

   <vel_calculation="Voronoi"/>
   <blind_points enabled="true"/>

Current alternatives:

   <vel_calculation="Arithmetic"/>
   <blind_points enabled="false"/>

gjaeger commented 4 years ago

As a reminder, when restructuring, we should also review the need for the jpsreport/Utest folder.

gjaeger commented 3 years ago

After dicussion with Jette, our proposal for the realisation of issue JuPedSim/jpsreport#193, JuPedSim/jpscore#809 and JuPedSim/jpscore#810:

<measurement_method_voronoi enabled="true">
   <measurement_area id="1" start_frame="None" stop_frame="None" local_individual="true"/>
   <measurement_area id="2" start_frame="None" stop_frame="None" local_individual="true"/>
   <cut_by_circle enabled="false" radius="1.0" edges="10"/>
   <global_individual enabled="true" zPos="None"/>
   <profiles enabled="false" zPos="None" grid_size_x="0.20" grid_size_y="0.20"/>
 </measurement_method_voronoi>

This includes default settings:

   <vel_calculation="Voronoi"/>
   <blind_points enabled="true"/>

Current alternatives:

   <vel_calculation="Arithmetic"/>
   <blind_points enabled="false"/>

The following changes were made and are to be considered in the documentation (PR JuPedSim/jpscore#840):

old: local_individual="true" to new: local_IFD="true"
old: <global_individual enabled="true" zPos="None"/>to new: <global_IFD enabled="true"/>
old: <blind_points enabled="false"/> to new: <use_blind_points enabled="false"/>

The parameter zPos must be checked after refactoring.

chraibi commented 3 years ago

thanks Gregor and Jette. That happens when global_IFD enabled="true" and local_IFD enabled="true"?

JetteSchumann commented 3 years ago

@chraibi data are calculated twice. Once for the whole geometry (global) and once for the specified measurement area (local)

gjaeger commented 3 years ago

thanks Gregor and Jette. That happens when global_IFD enabled="true" and local_IFD enabled="true"?

With global_IFD enabled="true" the data for an IFD_file for the entire geometry is created. With local_IFD enabled="true", the corresponding measurement area an IFD file related to the measurement area is created with the information about the intersection polygon.

chraibi commented 3 years ago

Okay, thanks. Is there any scenario where the following setup does make sense?

<measurement_area id="1" start_frame="None" stop_frame="None" local_individual="false"/>

JetteSchumann commented 3 years ago

yes. that makes sense if you are only interested in voronoi density and velocity but not the individual data

JuPedSim / jpsreport

Refactoring of Method D (I and J) #194

profiles - density

measurement area 1

measurement area 5

measurement area 9

⚠️ Warning ⚠️

profiles - velocity

measurement area 9 - method D

measurement area 5 - method J

summary

trajectories based on individual data

method I

method D

measurement area 5

measurement area 9

measurement area 10

profile based on method J - measurement area 5

profile based on method D - measurement area 9