search

antoinecarme / pyaf

PyAF is an Open Source Python library for Automatic Time Series Forecasting built on top of popular pydata modules.
BSD 3-Clause "New" or "Revised" License
457 stars 73 forks source link

PyAF support for ARM64 Architecture #187

Closed antoinecarme closed 2 years ago

antoinecarme commented 2 years ago

Check PyAF ARM64 support. This can be seen as a software robustness/portability test for PyAF.

Need to run all PyAF tests on a Debian Linux ARM64 on an AWS machine and/or an SBC.

  1. Check that all needed data packages are available (point missing packages).
  2. Run build tests (make -f tests/Makefilebuild-tests), circle-ci equivalent
  3. Run extended tests (make -f tests/Makefile all)
  4. Analyze numerical differences if any.
  5. Reconfigure Cicle-CI to run tests for this hardware architecture on every commit.
  6. Parallel hardware performance ?

Target Release : 2022-07-14

antoinecarme commented 2 years ago

https://aws.amazon.com/fr/ec2/instance-types/c7g/

Amazon EC2 C7g instances, powered by the latest generation AWS Graviton3 processors, provide the best price performance in Amazon EC2 for compute-intensive workloads. C7g instances are ideal for high performance computing (HPC), batch processing, electronic design automation (EDA), gaming, video encoding, scientific modeling, distributed analytics, CPU-based machine learning (ML) inference, and ad-serving. They offer up to 25% better performance over the sixth generation AWS Graviton2-based C6g instances. C7g instances are the first in the cloud to feature DDR5 memory, which provides 50% higher memory bandwidth compared to DDR4 memory to enable high-speed access to data in memory.

antoinecarme commented 2 years ago

AWS missed our Rendez-vous. ARM64 instances are in preview mode only (a lot of web forms ;).

Using Oracle Cloud with Ampere A1 compute instance (Free Tier). Ubuntu instances. No Debian, but will install/force debian packages anyway.

image

antoinecarme commented 2 years ago

Oracle cloud has ARM64 ubuntu instances (some danger here, I use debian packages ;)

root@instance-20220320-0117:/home/ubuntu/dev/pyaf# uname -a
Linux instance-20220320-0117 5.13.0-1018-oracle #22~20.04.1-Ubuntu SMP Wed Feb 16 09:29:33 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux

root@instance-20220320-0117:/home/ubuntu/dev/pyaf# apt-get --allow-unauthenticated install graphviz python3-sklearn python3-pandas python3-sqlalchemy python3-matplotlib jupyter-notebook python3-ipython python3-scipy python3-numpy python3-dill python3-pydot python3-statsmodels
antoinecarme commented 2 years ago

First functional tests OK at this time !!!!

ubuntu@instance-20220320-0117:~/dev/pyaf$ python3 tests/func/test_ozone.py 
     Month  Ozone       Time
0  1955-01    2.7 1955-01-01
1  1955-02    2.0 1955-02-01
2  1955-03    3.6 1955-03-01
3  1955-04    5.0 1955-04-01
4  1955-05    6.5 1955-05-01
INFO:pyaf.std:START_TRAINING 'Ozone'
INFO:pyaf.std:END_TRAINING_TIME_IN_SECONDS '['Ozone']' 1.1931898593902588
INFO:pyaf.std:TIME_DETAIL TimeVariable='Time' TimeMin=1955-01-01T00:00:00.000000 TimeMax=1967-09-01T00:00:00.000000 TimeDelta=<DateOffset: months=1> Horizon=12
INFO:pyaf.std:SIGNAL_DETAIL_ORIG SignalVariable='Ozone' Length=204  Min=1.2 Max=8.7  Mean=3.8357843137254894 StdDev=1.491559215940118
INFO:pyaf.std:SIGNAL_DETAIL_TRANSFORMED TransformedSignalVariable='_Ozone' Min=0.0 Max=1.0  Mean=0.35143790849673195 StdDev=0.19887456212534912
INFO:pyaf.std:DECOMPOSITION_TYPE 'T+S+R'
INFO:pyaf.std:BEST_TRANSOFORMATION_TYPE '_'
INFO:pyaf.std:BEST_DECOMPOSITION  '_Ozone_LinearTrend_residue_bestCycle_byMAPE_residue_AR(51)' [LinearTrend + Cycle_None + AR]
INFO:pyaf.std:TREND_DETAIL '_Ozone_LinearTrend' [LinearTrend]
INFO:pyaf.std:CYCLE_DETAIL '_Ozone_LinearTrend_residue_bestCycle_byMAPE' [Cycle_None]
INFO:pyaf.std:AUTOREG_DETAIL '_Ozone_LinearTrend_residue_bestCycle_byMAPE_residue_AR(51)' [AR]
INFO:pyaf.std:MODEL_MAPE MAPE_Fit=0.1668 MAPE_Forecast=0.1479 MAPE_Test=0.1426
INFO:pyaf.std:MODEL_SMAPE SMAPE_Fit=0.1565 SMAPE_Forecast=0.1637 SMAPE_Test=0.1527
INFO:pyaf.std:MODEL_MASE MASE_Fit=0.7155 MASE_Forecast=0.6304 MASE_Test=0.7351
INFO:pyaf.std:MODEL_CRPS CRPS_Fit=0.3393520318874523 CRPS_Forecast=0.26586525836868263 CRPS_Test=0.2827461815662519
INFO:pyaf.std:MODEL_L1 L1_Fit=0.6289210242456796 L1_Forecast=0.48935865210490537 L1_Test=0.3475195870267347
INFO:pyaf.std:MODEL_L2 L2_Fit=0.83001742194125 L2_Forecast=0.6681008123392113 L2_Test=0.43011933867802526
INFO:pyaf.std:MODEL_LnQ LnQ_Fit=6.489178708354776 LnQ_Forecast=1.7465131682383204 LnQ_Test=0.5857060588437416
INFO:pyaf.std:MODEL_MEDIAN_AE MedAE_Fit=0.5336042100469855 MedAE_Forecast=0.407257006454385 MedAE_Test=0.2542038976786749
INFO:pyaf.std:MODEL_COMPLEXITY 39.0
INFO:pyaf.std:SIGNAL_TRANSFORMATION_DETAIL_START
INFO:pyaf.std:SIGNAL_TRANSFORMATION_MODEL_VALUES NoTransf None
INFO:pyaf.std:SIGNAL_TRANSFORMATION_DETAIL_END
INFO:pyaf.std:TREND_DETAIL_START
INFO:pyaf.std:LINEAR_RIDGE_TREND LinearTrend (0.5096770848334515, array([-0.24361723]))
INFO:pyaf.std:TREND_DETAIL_END
INFO:pyaf.std:CYCLE_MODEL_DETAIL_START
INFO:pyaf.std:BEST_CYCLE_LENGTH_VALUES _Ozone_LinearTrend_residue_bestCycle_byMAPE None 0.0017292167436161465 {}
INFO:pyaf.std:CYCLE_MODEL_DETAIL_END
INFO:pyaf.std:AR_MODEL_DETAIL_START

ubuntu@instance-20220320-0117:~/dev/pyaf$ python3 tests/func/test_air_passengers.py 
INFO:pyaf.std:START_TRAINING 'AirPassengers'
INFO:pyaf.std:END_TRAINING_TIME_IN_SECONDS '['AirPassengers']' 0.8796365261077881
INFO:pyaf.std:TIME_DETAIL TimeVariable='time' TimeMin=1949.0 TimeMax=1956.91666666667 TimeDelta=0.08333333333336763 Horizon=12
INFO:pyaf.std:SIGNAL_DETAIL_ORIG SignalVariable='AirPassengers' Length=132  Min=104 Max=559  Mean=262.49242424242425 StdDev=106.22114554451818
INFO:pyaf.std:SIGNAL_DETAIL_TRANSFORMED TransformedSignalVariable='_AirPassengers' Min=0.0 Max=1.0  Mean=0.3483349983349982 StdDev=0.2334530671308092
INFO:pyaf.std:DECOMPOSITION_TYPE 'T+S+R'
INFO:pyaf.std:BEST_TRANSOFORMATION_TYPE '_'
INFO:pyaf.std:BEST_DECOMPOSITION  '_AirPassengers_LinearTrend_residue_bestCycle_byMAPE_residue_AR(33)' [LinearTrend + Cycle_None + AR]
INFO:pyaf.std:TREND_DETAIL '_AirPassengers_LinearTrend' [LinearTrend]
INFO:pyaf.std:CYCLE_DETAIL '_AirPassengers_LinearTrend_residue_bestCycle_byMAPE' [Cycle_None]
INFO:pyaf.std:AUTOREG_DETAIL '_AirPassengers_LinearTrend_residue_bestCycle_byMAPE_residue_AR(33)' [AR]
INFO:pyaf.std:MODEL_MAPE MAPE_Fit=0.0692 MAPE_Forecast=0.0724 MAPE_Test=0.1029
INFO:pyaf.std:MODEL_SMAPE SMAPE_Fit=0.0677 SMAPE_Forecast=0.0766 SMAPE_Test=0.11
INFO:pyaf.std:MODEL_MASE MASE_Fit=0.7564 MASE_Forecast=0.7904 MASE_Test=1.0237
INFO:pyaf.std:MODEL_CRPS CRPS_Fit=18.085457565409982 CRPS_Forecast=16.274110291616523 CRPS_Test=18.81743762563634
INFO:pyaf.std:MODEL_L1 L1_Fit=14.140766698293751 L1_Forecast=29.277902395005004 L1_Test=46.06841244485628
INFO:pyaf.std:MODEL_L2 L2_Fit=18.236783225048725 L2_Forecast=38.53679236353066 L2_Test=53.84516828297264
INFO:pyaf.std:MODEL_LnQ LnQ_Fit=0.7106484957272503 LnQ_Forecast=0.22173127836921702 LnQ_Test=0.18614862418400102
INFO:pyaf.std:MODEL_MEDIAN_AE MedAE_Fit=11.89639530520634 MedAE_Forecast=24.93454236107101 MedAE_Test=39.126842220988664
INFO:pyaf.std:MODEL_COMPLEXITY 25.0
INFO:pyaf.std:SIGNAL_TRANSFORMATION_DETAIL_START
INFO:pyaf.std:SIGNAL_TRANSFORMATION_MODEL_VALUES NoTransf None
INFO:pyaf.std:SIGNAL_TRANSFORMATION_DETAIL_END
INFO:pyaf.std:TREND_DETAIL_START
INFO:pyaf.std:LINEAR_RIDGE_TREND LinearTrend (0.02396754603991888, array([0.43429934]))
INFO:pyaf.std:TREND_DETAIL_END
INFO:pyaf.std:CYCLE_MODEL_DETAIL_START
INFO:pyaf.std:BEST_CYCLE_LENGTH_VALUES _AirPassengers_LinearTrend_residue_bestCycle_byMAPE None -0.005776413346313302 {}
INFO:pyaf.std:CYCLE_MODEL_DETAIL_END
INFO:pyaf.std:AR_MODEL_DETAIL_START
antoinecarme commented 2 years ago

CPU information :

https://en.wikichip.org/wiki/arm_holdings/microarchitectures/neoverse_n1

ubuntu@instance-20220320-0117:~/dev/pyaf$ lscpu 
Architecture:                    aarch64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
CPU(s):                          4
On-line CPU(s) list:             0-3
Thread(s) per core:              1
Core(s) per socket:              4
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       ARM
Model:                           1
Model name:                      Neoverse-N1
Stepping:                        r3p1
BogoMIPS:                        50.00
NUMA node0 CPU(s):               0-3
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:        Mitigation; __user pointer sanitization
Vulnerability Spectre v2:        Not affected
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp ssbs

ubuntu@instance-20220320-0117:~/dev/pyaf$ cat /proc/cpuinfo 
processor   : 0
BogoMIPS    : 50.00
Features    : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp ssbs
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x3
CPU part    : 0xd0c
CPU revision    : 1

processor   : 1
BogoMIPS    : 50.00
Features    : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp ssbs
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x3
CPU part    : 0xd0c
CPU revision    : 1

processor   : 2
BogoMIPS    : 50.00
Features    : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp ssbs
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x3
CPU part    : 0xd0c
CPU revision    : 1

processor   : 3
BogoMIPS    : 50.00
Features    : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp ssbs
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x3
CPU part    : 0xd0c
CPU revision    : 1
antoinecarme commented 2 years ago

Detailed test machine data (attached files)

cpuinfo.txt debian+ubuntu-packages.txt dmesg.txt dmidecode.txt hwinfo.txt pci.txt

antoinecarme commented 2 years ago

Run build tests (make -f tests/Makefile build-test), circle-ci equivalent

Perfect !!!