The entity-class 'Entities\TrafficDailyPhysInt' mapping is invalid

[rfc1036]

ISSUE TYPE

Bug Report

VERSION

define( 'APPLICATION_VERSION', '5.5.0' );
define( 'APPLICATION_VERDATE', '2020032100' );

SUMMARY

I get this error on a current checkout of release-v5:

root@manager:/srv/ixpmanager# ./artisan doctrine:schema:update

Checking if database connected to default entity manager needs updating...
Nothing to update - your database is already in sync with the current entity metadata.                                                                          
root@manager:/srv/ixpmanager# ./artisan doctrine:schema:validate

Validating for default entity manager...
[Mapping]  FAIL - The entity-class 'Entities\TrafficDailyPhysInt' mapping is invalid:                                                                           

* The association Entities\TrafficDailyPhysInt#PhysicalInterface refers to the inverse side field Entities\PhysicalInterface#TrafficDailiesPhysInt which does not exist.
[Database] OK - The database schema is in sync with the mapping files.
[Exit 1]
root@manager:/srv/ixpmanager#

Apparently everything still works.

[rfc1036]

Actually the In and Out columns in the Utilisation page have the same values for all members, but I do not understand if this is related or not.

[barryo]

Hi @rfc1036 / Marco,

hope you folks are doing okay.

root@manager:/srv/ixpmanager# ./artisan doctrine:schema:validate

There was an (unused) reverse mapping missing. But we do like the validation to work so fixed in 3bd4b64895f45b2ec5c02d4d914e5b9cfe1cf00f

Apparently everything still works.

Yes, just schema definition issues more than anything else and unused also.

[barryo]

Actually the In and Out columns in the Utilisation page have the same values for all members, but I do not understand if this is related or not.

I don't see how it would be related.

And we don't have this issue:

What do you see - same figures for every member? Is your MRTG graphing generally working otherwise?

[rfc1036]

MRTG works, both the graphs and the league table are correct. The utilisation page apparently uses the Out value also for In:

[barryo]

What's the output of these commands for you:

git log | head -1

And:

git status

[rfc1036]

root@manager:/srv/ixpmanager# git log | head -1
commit 45629b2f5533c9e33da71442ea1201951a76a71e
root@manager:/srv/ixpmanager# git status
On branch release-v5
Your branch is up to date with 'origin/release-v5'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   public/robots.txt

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        public/weathermap/

no changes added to commit (use "git add" and/or "git commit -a")

[barryo]

That's a head-scratched then Marco. I can't imagine how you have the same values for in/out when you have the exact same revision as us.

I guess the next question is whether or not the data is identical for in/out in the database too?

If it helps, this is the SQL I used before I had the UI done to get an idea of the data and to play with presentation:

select
    c.name as cname,
    ANY_VALUE( v.name) as vname,
    ANY_VALUE( s.name ) as switch,
    round( max( tdpi.week_max_in )/1000000000, 2) as max_in_gbps,
    round( max( tdpi.week_max_out )/1000000000, 2) as max_out_gbps,
    count( pi.id ) as num_ports_in_lag,
    cast( sum( pi.speed )/1000 as unsigned ) as vi_speed_gb,

    round( GREATEST( (max( tdpi.week_max_in )/1000000/max( pi.speed ))*100, (max( tdpi.week_max_out )/1000000/max( pi.speed ))*100 ), 2) as util

    from traffic_daily_phys_ints as tdpi

    left join physicalinterface as pi on tdpi.physicalinterface_id = pi.id
    left join virtualinterface as vi on pi.virtualinterfaceid = vi.id
    left join cust as c on vi.custid = c.id

    left join vlaninterface as vli on vli.virtualinterfaceid = vi.id
    left join vlan as v on vli.vlanid = v.id

    left JOIN switchport as sp on pi.switchportid = sp.id
    left join switch as s on sp.switchid = s.id

    where tdpi.`day` = '2020-03-15' and tdpi.category = 'bits'

    group by vi.id

    order by util desc

If your database in/out are not equal then how does the data look from the SQL above (after changing the date)?

[rfc1036]

The values are the same in the database:

mysql> select id,physicalinterface_id,day,day_max_in,day_max_out,day_max_in_at,day_max_out_at from traffic_daily_phys_ints limit 22;
+----+----------------------+------------+------------+-------------+---------------------+---------------------+
| id | physicalinterface_id | day        | day_max_in | day_max_out | day_max_in_at       | day_max_out_at      |
+----+----------------------+------------+------------+-------------+---------------------+---------------------+
|  1 |                   96 | 2020-03-20 |   76185136 |    76185136 | 2020-03-19 20:45:00 | 2020-03-19 20:45:00 |
|  2 |                   96 | 2020-03-20 |      42870 |       42870 | 2020-03-19 20:30:00 | 2020-03-19 20:30:00 |
|  3 |                   96 | 2020-03-20 |          0 |           0 | NULL                | NULL                |
|  4 |                   96 | 2020-03-20 |          0 |           0 | NULL                | NULL                |
|  5 |                   96 | 2020-03-20 |          0 |           0 | NULL                | NULL                |
|  6 |                   51 | 2020-03-20 |  108341792 |   108341792 | 2020-03-20 00:35:00 | 2020-03-20 00:35:00 |
|  7 |                   51 | 2020-03-20 |      17743 |       17743 | 2020-03-18 23:00:00 | 2020-03-18 23:00:00 |
|  8 |                   51 | 2020-03-20 |          0 |           0 | NULL                | NULL                |
|  9 |                   51 | 2020-03-20 |          0 |           0 | NULL                | NULL                |
| 10 |                   51 | 2020-03-20 |          0 |           0 | NULL                | NULL                |
| 11 |                   50 | 2020-03-20 | 8158270712 |  8158270712 | 2020-03-19 18:35:00 | 2020-03-19 18:35:00 |
| 12 |                   50 | 2020-03-20 |     736337 |      736337 | 2020-03-19 18:35:00 | 2020-03-19 18:35:00 |
| 13 |                   50 | 2020-03-20 |          0 |           0 | NULL                | NULL                |
| 14 |                   50 | 2020-03-20 |          0 |           0 | NULL                | NULL                |
| 15 |                   50 | 2020-03-20 |          0 |           0 | NULL                | NULL                |
| 16 |                   65 | 2020-03-20 | 8367184048 |  8367184048 | 2020-03-18 21:15:00 | 2020-03-18 21:15:00 |
| 17 |                   65 | 2020-03-20 |     746408 |      746408 | 2020-03-18 20:50:00 | 2020-03-18 20:50:00 |
| 18 |                   65 | 2020-03-20 |          0 |           0 | NULL                | NULL                |
| 19 |                   65 | 2020-03-20 |          0 |           0 | NULL                | NULL                |
| 20 |                   65 | 2020-03-20 |          0 |           0 | NULL                | NULL                |
| 21 |                   71 | 2020-03-20 |   12989248 |    12989248 | 2020-03-20 00:00:00 | 2020-03-20 00:00:00 |
| 22 |                   71 | 2020-03-20 |       1121 |        1121 | 2020-03-20 00:20:00 | 2020-03-20 00:20:00 |
+----+----------------------+------------+------------+-------------+---------------------+---------------------+
22 rows in set (0.00 sec)

mysql> select id,physicalinterface_id,day,day_max_in,day_max_out,day_max_in_at,day_max_out_at from traffic_daily_phys_ints where category='bits' and day='2020-03-23' limit 22;
+------+----------------------+------------+------------+-------------+---------------------+---------------------+
| id   | physicalinterface_id | day        | day_max_in | day_max_out | day_max_in_at       | day_max_out_at      |
+------+----------------------+------------+------------+-------------+---------------------+---------------------+
| 1216 |                   96 | 2020-03-23 |   82451848 |    82451848 | 2020-03-22 14:35:00 | 2020-03-22 14:35:00 |
| 1221 |                   51 | 2020-03-23 |  103225304 |   103225304 | 2020-03-21 23:05:00 | 2020-03-21 23:05:00 |
| 1226 |                   50 | 2020-03-23 | 8024370800 |  8024370800 | 2020-03-21 21:55:00 | 2020-03-21 21:55:00 |
| 1231 |                   65 | 2020-03-23 | 8110178504 |  8110178504 | 2020-03-21 22:20:00 | 2020-03-21 22:20:00 |
| 1236 |                   71 | 2020-03-23 |   18440880 |    18440880 | 2020-03-22 00:05:00 | 2020-03-22 00:05:00 |
| 1241 |                   14 | 2020-03-23 |  121519896 |   121519896 | 2020-03-22 16:05:00 | 2020-03-22 16:05:00 |
| 1246 |                   76 | 2020-03-23 |  149152552 |   149152552 | 2020-03-21 20:25:00 | 2020-03-21 20:25:00 |
| 1251 |                   92 | 2020-03-23 |  470461008 |   470461008 | 2020-03-21 16:25:00 | 2020-03-21 16:25:00 |
| 1256 |                    3 | 2020-03-23 |  774764784 |   774764784 | 2020-03-21 18:00:00 | 2020-03-21 18:00:00 |
| 1261 |                   25 | 2020-03-23 | 1938700600 |  1938700600 | 2020-03-22 14:55:00 | 2020-03-22 14:55:00 |
| 1266 |                   26 | 2020-03-23 | 1900764256 |  1900764256 | 2020-03-22 14:50:00 | 2020-03-22 14:50:00 |
| 1271 |                   69 | 2020-03-23 |   37407944 |    37407944 | 2020-03-21 17:00:00 | 2020-03-21 17:00:00 |
| 1276 |                    7 | 2020-03-23 |   27402968 |    27402968 | 2020-03-21 18:05:00 | 2020-03-21 18:05:00 |
| 1281 |                   46 | 2020-03-23 |  165799768 |   165799768 | 2020-03-22 04:15:00 | 2020-03-22 04:15:00 |
| 1286 |                   19 | 2020-03-23 |   14072608 |    14072608 | 2020-03-21 17:25:00 | 2020-03-21 17:25:00 |
| 1291 |                   56 | 2020-03-23 |  170421592 |   170421592 | 2020-03-22 12:50:00 | 2020-03-22 12:50:00 |
| 1296 |                   57 | 2020-03-23 |  151357320 |   151357320 | 2020-03-22 14:25:00 | 2020-03-22 14:25:00 |
| 1301 |                   34 | 2020-03-23 | 4605125400 |  4605125400 | 2020-03-22 20:10:00 | 2020-03-22 20:10:00 |
| 1306 |                   35 | 2020-03-23 | 4701056144 |  4701056144 | 2020-03-22 20:10:00 | 2020-03-22 20:10:00 |
| 1311 |                   36 | 2020-03-23 | 4295913976 |  4295913976 | 2020-03-21 20:05:00 | 2020-03-21 20:05:00 |
| 1316 |                   37 | 2020-03-23 | 4282146288 |  4282146288 | 2020-03-21 20:15:00 | 2020-03-21 20:15:00 |
| 1321 |                   91 | 2020-03-23 |   75129968 |    75129968 | 2020-03-22 07:00:00 | 2020-03-22 07:00:00 |
+------+----------------------+------------+------------+-------------+---------------------+---------------------+
22 rows in set (0.00 sec)

[barryo]

Thanks for that Marco. Are you using MRTG log or rrd files?

[rfc1036]

RRD.

[barryo]

@rfc1036 - still on my todo list, not forgotten!

[rfc1036]

I have also noticed an interesting units issue: a maxed (Cisco) 10GE interface is reported by MRTG a 9.88 Gbps, hence the utilisation page will list it as 98.59% instead of 100%. This is only a cosmetic problem, but maybe it could be solved with a corrective factor.

[nickhilliard]

I have also noticed an interesting units issue: a maxed (Cisco) 10GE interface is reported by MRTG a 9.88 Gbps, hence the utilisation page will list it as 98.59% instead of 100%. This is only a cosmetic problem, but maybe it could be solved with a corrective factor.

not in general, no. traffic monitoring is a dark art full of black magic. Some devices will report L3 throughput; others will include everything right down to a calculation for the IPG. You can see immediately that the actual numbers can depend on the packets-per second on the interface, and you can further work out that if the hardware/software is only reporting frame contents, that if your port is saturated at layer 2 with 64-byte l3 packets, then the throughput on the interface might reported as low as 60% utilisation (worst case scenario).

There is no a-priori way of determining what any particular device + software combination is actually reporting. Quite often the vendors don't know themselves, as the figures are delivered by HAL interfaces and they're just interfacing with a chipset SDK. IXP Manager is not going to attempt to correct any of this.