search

RexxLA / NetRexx

Other
6 stars 0 forks source link

The calculation of clauses per second made by rexxcps.nrx is not aligned with rexxcps.rex #64

Closed jlfaucher closed 1 week ago

jlfaucher commented 1 week ago

Tests made after applying the workaround for #63 (using the file rexxcps_fixed.nrx which is a copy of rexxcps.nrx where I renamed the variables 'source' and 'version')

I get these numbers on my laptop:

18_407_194_702_387 NetRexx clauses per second        -- netrexx compiled: nrc rexxcps_fixed; java rexxcps_fixed
   302_886_536_330 NetRexx clauses per second        -- netrexx interpreted: nrc -exec rexxcps_fixed.nrx
        23_736_393 REXX clauses per second           -- oorexx: rexx rexxcps.rex

The difference with oorexx is stunning... After comparing the sources, I think there is an error in rexxcps.nrx

rexxcps.nrx:
  fempty=empty/averaging
  ffull=full/averaging
  looptime=(ffull-fempty)/(count*1000000000) -- 1 000 000 000 for nanosecs->secs
  cps = (1000000000/looptime+0.5)            -- error? should be 1000/looptime+0.5

rexxcps.rex:
empty=empty/averaging
full=full/averaging
looptime=(full-empty)/count
say'     Performance:' format(1000/looptime,,0) 'REXX clauses per second'

If I use 1000/looptime, I get:

Performance: 19_580_824 NetRexx clauses per second        -- netrexx compiled
Performance:    301_796 NetRexx clauses per second        -- netrexx interpreted

which is of the same order of magnitude as rexxcps.rex

Detailed output for rexxcps.nrx compiled

sh> nrc rexxcps_fixed
NetRexx portable processor 4.06-GA build 152-20240304-0612
Copyright (c) RexxLA, 2011,2024.   All rights reserved.
Parts Copyright (c) IBM Corporation, 1995,2008.
Program rexxcps_fixed.nrx
    function subroutine(Rexx,Rexx)
    function formatThousands(Rexx)
Compilation of 'rexxcps_fixed.nrx' successful
Duration: 0.609579

sh> java rexxcps_fixed
----- REXXCPS 2.1n -- Measuring NetRexx clauses/second -----
 NetRexx version is: NetRexx 4.06 03 Mar 2024
          System is: Java
          Averaging: 100 measures of 100 iterations

Total (full DO): 0.00510703743 secs (average of 100 measures of 100 iterations)
Time for one iteration (1000 clauses) was: 0.0000510703743 seconds

     Performance: 19_580_824 NetRexx clauses per second
                ( 19.581E+6 cps)

Duration: 0.588406

Detailed output for rexxcps.nrx interpreted


sh> nrc -exec rexxcps_fixed.nrx 
NetRexx portable processor 4.06-GA build 152-20240304-0612
Copyright (c) RexxLA, 2011,2024.   All rights reserved.
Parts Copyright (c) IBM Corporation, 1995,2008.
Program rexxcps_fixed.nrx
    function subroutine(Rexx,Rexx)
    function formatThousands(Rexx)
===== Exec: rexxcps_fixed =====
----- REXXCPS 2.1n -- Measuring NetRexx clauses/second -----
 NetRexx version is: NetRexx 4.06 03 Mar 2024
          System is: Java
          Averaging: 100 measures of 100 iterations

Total (full DO): 0.33134968507 secs (average of 100 measures of 100 iterations)
Time for one iteration (1000 clauses) was: 0.0033134968507 seconds

     Performance: 301_796 NetRexx clauses per second
                ( 301.796E+3 cps)

Processing of 'rexxcps_fixed.nrx' complete
Duration: 33.517014
rvjansen commented 1 week ago

ok, rexxcps.nrx

but then you are dividing microseconds to seconds, not nanoseconds to seconds. A nanosecond is still 10E-9 seconds. Can you check in your fixed version? We might have to ask Mike, who never published the NetRexx version himself because it felt odd to have such a large discrepancy.

best regards,

René.

On 6 Jul 2024, at 11:47, jlfaucher @.***> wrote:

Tests made after applying the workaround for #63 https://github.com/RexxLA/NetRexx/issues/63 (using the file rexxcps_fixed.nrx which is a copy of rexxcps.nrx where I renamed the variables 'source' and 'version')

I get these numbers on my laptop:

18_407_194_702_387 NetRexx clauses per second -- netrexx compiled: nrc rexxcps_fixed; java rexxcps_fixed 302_886_536_330 NetRexx clauses per second -- netrexx interpreted: nrc -exec rexxcps_fixed.nrx 23_736_393 REXX clauses per second -- oorexx: rexx rexxcps.rex The difference with oorexx is stunning... After comparing the sources, I think there is an error in rexxcps.nrx

rexxcps.nrx: fempty=empty/averaging ffull=full/averaging looptime=(ffull-fempty)/(count*1000000000) -- 1 000 000 000 for nanosecs->secs cps = (1000000000/looptime+0.5) -- error? should be 1000/looptime+0.5

rexxcps.rex: empty=empty/averaging full=full/averaging looptime=(full-empty)/count say' Performance:' format(1000/looptime,,0) 'REXX clauses per second' If I use 1000/looptime, I get:

Performance: 19_580_824 NetRexx clauses per second -- netrexx compiled Performance: 301_796 NetRexx clauses per second -- netrexx interpreted which is of the same order of magnitude as rexxcps.rex

Detailed output for rexxcps.nrx compiled

sh> nrc rexxcps_fixed NetRexx portable processor 4.06-GA build 152-20240304-0612 Copyright (c) RexxLA, 2011,2024. All rights reserved. Parts Copyright (c) IBM Corporation, 1995,2008. Program rexxcps_fixed.nrx function subroutine(Rexx,Rexx) function formatThousands(Rexx) Compilation of 'rexxcps_fixed.nrx' successful Duration: 0.609579

sh> java rexxcps_fixed ----- REXXCPS 2.1n -- Measuring NetRexx clauses/second ----- NetRexx version is: NetRexx 4.06 03 Mar 2024 System is: Java Averaging: 100 measures of 100 iterations

Total (full DO): 0.00510703743 secs (average of 100 measures of 100 iterations) Time for one iteration (1000 clauses) was: 0.0000510703743 seconds

 Performance: 19_580_824 NetRexx clauses per second
            ( 19.581E+6 cps)

Duration: 0.588406 Detailed output for rexxcps.nrx interpreted

sh> nrc -exec rexxcps_fixed.nrx NetRexx portable processor 4.06-GA build 152-20240304-0612 Copyright (c) RexxLA, 2011,2024. All rights reserved. Parts Copyright (c) IBM Corporation, 1995,2008. Program rexxcps_fixed.nrx function subroutine(Rexx,Rexx) function formatThousands(Rexx) ===== Exec: rexxcps_fixed ===== ----- REXXCPS 2.1n -- Measuring NetRexx clauses/second ----- NetRexx version is: NetRexx 4.06 03 Mar 2024 System is: Java Averaging: 100 measures of 100 iterations

Total (full DO): 0.33134968507 secs (average of 100 measures of 100 iterations) Time for one iteration (1000 clauses) was: 0.0033134968507 seconds

 Performance: 301_796 NetRexx clauses per second
            ( 301.796E+3 cps)

Processing of 'rexxcps_fixed.nrx' complete Duration: 33.517014

— Reply to this email directly, view it on GitHub https://github.com/RexxLA/NetRexx/issues/64, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC3WJQO6SXQTEVWQOUD333LZK64LZAVCNFSM6AAAAABKOJFVHOVHI2DSMVQWIX3LMV43ASLTON2WKOZSGM4TGNBZGYYTINA. You are receiving this because you are subscribed to this thread.

rvjansen commented 1 week ago

ah I see what you mean - because we measure a loop of 1000 iterations. Would be good if more people had a look at this.

René

On 6 Jul 2024, at 11:47, jlfaucher @.***> wrote:

Tests made after applying the workaround for #63 https://github.com/RexxLA/NetRexx/issues/63 (using the file rexxcps_fixed.nrx which is a copy of rexxcps.nrx where I renamed the variables 'source' and 'version')

I get these numbers on my laptop:

18_407_194_702_387 NetRexx clauses per second -- netrexx compiled: nrc rexxcps_fixed; java rexxcps_fixed 302_886_536_330 NetRexx clauses per second -- netrexx interpreted: nrc -exec rexxcps_fixed.nrx 23_736_393 REXX clauses per second -- oorexx: rexx rexxcps.rex The difference with oorexx is stunning... After comparing the sources, I think there is an error in rexxcps.nrx

rexxcps.nrx: fempty=empty/averaging ffull=full/averaging looptime=(ffull-fempty)/(count*1000000000) -- 1 000 000 000 for nanosecs->secs cps = (1000000000/looptime+0.5) -- error? should be 1000/looptime+0.5

jlfaucher commented 1 week ago

Hi René

Please find attached the file rexxcps_fixed.nrx that I use for testing. For me, the performance displayed by the official rexxcps.nrx looks wrong, if we assume that "the number of clauses per seconds" has the same meaning fo netrexx, rexx and oorexx.

The interpreted rexxcps.nrx takes about 32 seconds, but it has 12,760 times the number of clauses compared to oorexx which runs in about 0.5 seconds. 302886536330 / 23736393 = 12760.42810422

The ratio for the compiled rexxcps.nrx versus oorexx is 775484. The duration is approximately the same for netrexx and oorexx, around 0.5s on my laptop.

Not asking to modify rexxcps.nrx, I wanted just to share my astonishment.

rexxcps_fixed.nrx.txt

rvjansen commented 1 week ago

When I said rexxcps cannot be changed, I did not mean the NetRexx version. Will change the repository version to form the base of further research. Thank you very much for your effort!