search

LBNL-UCB-STI / beam

The Framework for Modeling Behavior, Energy, Autonomy, and Mobility in Transportation Systems
https://transportation.lbl.gov/beam
Other
147 stars 57 forks source link

Replacing errors with warnings #3391

Open nikolayilyin opened 3 years ago

nikolayilyin commented 3 years ago

Some of errors in the simulation output log are actually warnings because they are usually ignored. The goal of this issue is:

  1. to figure out what errors among currently presented in the output files are actually warnings.
  2. to replace those errors by warnings in the code.

In order to achieve the first goal one needs to analyze actual output logs, I think the good idea is to use existing simulation output from our spreadsheet , especially the runs with big numbers in the Error column.

For example: https://s3.us-east-2.amazonaws.com/beam-outputs/index.html#output/newyork/nyc-200k-baseline-04__2021-11-28_18-59-38_wvi https://s3.us-east-2.amazonaws.com/beam-outputs/index.html#output/newyork/nyc-200k-baseline-04__2021-11-28_17-32-45_ooc https://s3.us-east-2.amazonaws.com/beam-outputs/index.html#output/sfbay/smart-baseline-15k-FREIGHT__2021-11-29_14-33-42_lcm https://s3.us-east-2.amazonaws.com/beam-outputs/index.html#output/austin/austin-prod-200k-flowCap-0.2-speedScaling-1.1__2021-11-23_14-21-33_auf

or other recent runs

norberto108 commented 2 years ago

@nikolayilyin this file is an LibreOffice sheet with errors count, let me know your thoughts

Errors.ods

Xuan-1998 commented 2 years ago

@norberto108 which beambranch and databranch are those based on? Did you see any errors as follows: 16:40:04.445 [main] ERROR com.conveyal.r5.streets.IntHashGrid - Visiting too many spatial index cells.

norberto108 commented 2 years ago

@Xuan-1998 I think it comes from master. And i didn't get any errors like that. Do you have any log files with this kind of errors ?

Xuan-1998 commented 2 years ago

Check out this file beamLog.txt

norberto108 commented 2 years ago

Check out this file beamLog.txt

it is a candidate to be a WARN indeed

Xuan-1998 commented 2 years ago

Check out this file beamLog.txt

it is a candidate to be a WARN indeed

cool, that sounds good

norberto108 commented 2 years ago

@nikolayilyin the new set of errors created from the recent runs Errors-1.ods

nikolayilyin commented 2 years ago

Great! Now let's keep more information about each error (the name of the class is not enough to judge) and let's not group the errors by simulation name (or branch name) - let's just have the errors and the amount of them

norberto108 commented 2 years ago

I joined all ERROR messages removing the numeric values from the files bellow in an unique result sheet:

./new-york-600k-june2022-calib-4_zneedellATgmail_com/beamLog.out
./0.1pop-no-car400_j503440616ATberkeley_edu/beamLog.out
./0.1pop-cheaper-bikes_j503440616ATberkeley_edu/beamLog.out
./0.1pop-nocar1600-twice-bikes_j503440616ATberkeley_edu/beamLog.out
./0.1pop-nocar1600-twice-bikes_j503440616ATberkeley_edu/beamLog_2.out
./0.1pop-more-bikes_j503440616ATberkeley_edu/beamLog.out
./0.1pop-fewer-bikes_j503440616ATberkeley_edu/beamLog.out
./0.1pop-fewer-docks_j503440616ATberkeley_edu/beamLog.out
./0.1pop-baseline-twice-bikes_j503440616ATberkeley_edu/beamLog.out
./new-york-600k-august2020-calib-4_zneedellATgmail_com/beamLog.out
./0.1pop-free-bikes_j503440616ATberkeley_edu/beamLog.out
./gemini-scenario-8-MaxEV-10p-TEST_10712736+htmlrbATusers_noreply_github_com/beamLog.out
./0.1pop-baseline_j503440616ATberkeley_edu/beamLog.out
./0.1pop-more-docks_j503440616ATberkeley_edu/beamLog.out
./new-york-600k-jan2022-calib-4_zneedellATgmail_com/beamLog.out
./0.1pop-no-shared-bikes_j503440616ATberkeley_edu/beamLog.out
./0.1pop-nocar1600_j503440616ATberkeley_edu/beamLog.out
./0.1pop-nocar1600_j503440616ATberkeley_edu/beamLog_1.out
./0.1pop-nocar1600_j503440616ATberkeley_edu/beamLog_2.out
./0.1pop-nocar400-twice-bikes_j503440616ATberkeley_edu/beamLog.out

Errors-2.ods

nikolayilyin commented 2 years ago

It seems there is something wrong with grouping of errors, the following errors looks the same to me: image There are other errors that looks the same as well.

Please, check if the grouping work

norberto108 commented 2 years ago

The fixed errors list errors-3.ods

norberto108 commented 2 years ago

Last review in 19-sep-2022 errors-3.ods

norberto108 commented 2 years ago

To try to find the error messages that could be replaced with warnings, i grouped the error messages removing the timestamp and process numbers to generic messages. Some of them are triggered from the frameworks out of scope. Attached is the errors sheet. errors-3.ods