[FEATURE REQUEST] numpy.tolist()

[labodezao]

Hi, For serialize a numpy Array to json format I would need the function numpy.tolist(). This is to communicate beetween PC and micropython with Pyserial

[v923z]

Do you mean this method https://numpy.org/doc/stable/reference/generated/numpy.ndarray.tolist.html ? It could definitely be done. Give me a couple of days.

[labodezao]

Yes ! This method :)

Le ven. 31 déc. 2021 à 13:36, Zoltán Vörös @.***> a écrit :

Do you mean this method https://numpy.org/doc/stable/reference/generated/numpy.ndarray.tolist.html ? It could definitely be done. Give me a couple of days.

— Reply to this email directly, view it on GitHub https://github.com/v923z/micropython-ulab/issues/466#issuecomment-1003366435, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALPYWOJTWLBZLOHAMRA3HP3UTWPTXANCNFSM5LBAYAYA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

[v923z]

@labodezao Wouldn't https://numpy.org/doc/stable/reference/generated/numpy.ndarray.tobytes.html do the job?

[labodezao]

Maybe if you could do both, that would be great !

Message ID: @.***>

[v923z]

Maybe if you could do both, that would be great ! … Message ID: @.***>

tobytes is simpler, because it turns a tensor into a linear array of bytes, irrespective of the dimensionality. But tolist is not hard, either.

[v923z]

@labodezao tobytes is already supported: https://micropython-ulab.readthedocs.io/en/latest/ulab-ndarray.html?highlight=tobytes#tobytes

[v923z]

@labodezao Could you, please, check out the https://github.com/v923z/micropython-ulab/tree/complex2 branch? https://github.com/v923z/micropython-ulab/pull/456/commits/2b578ae333a03b675e9b0f9b6557f7c12bd91ef8 should implement your request. Let me know if anything else is missing.

[labodezao]

Hi I will try tomorrow.

Or do you have any solution / idea to save a matrix with the routine nump.save() on the sd card and then send the file through usb ?

Message ID: @.***>

[labodezao]

Ok, this is the try of this day :

I test this code :

import time
from micropython import const
from machine import Pin, I2C
from pyb import DAC
from step import Stepper
import utime
from bmp280 import *
from sfm3000 import SFM3000
import machine
import sys
import json
import math , time , array , gc
from ulab import numpy as np
import gc

gc.collect()
acquisition_time=1
sampling=100
n_exp_meas = math.floor(acquisition_time * math.ceil(sampling))
print(n_exp_meas)
sens1 = np.zeros(( 6 , n_exp_meas+1))
np.ndinfo(sens1)
print(sens1[3,50])      

n_act_meas = 0

        # read bytes as fast as possible
start = time.ticks_us()
while time.ticks_diff(time.ticks_us(), start) < acquisition_time * 1000000:
    curr_time = time.ticks_us()
    if time.ticks_diff(curr_time, start) < (n_act_meas * 999999. / sampling):
        continue
    # read sensor 1
    # grab the time of the measure 1
    sens1[0,n_act_meas] = n_act_meas
    # read sensor 2
    sens1[1,n_act_meas]= n_act_meas
    # grab the time of the measure 2
    sens1[2,n_act_meas] = n_act_meas
    # read sensor 3
    sens1[3,n_act_meas] = n_act_meas
    # grab the time of the measure 3
    sens1[4,n_act_meas] = n_act_meas
    sens1[5,n_act_meas] = n_act_meas      

    n_act_meas += 1

    # bytes --> lists of integers

        # remove exceeding zeros
sens1 = sens1[:,:n_act_meas]  

print("measured samples: ", n_act_meas)
print("expected samples: ", n_exp_meas)
print("actual sampling rate: ", n_act_meas / acquisition_time)

result = sens1.tobytes()
file = open('mes_all.csv','wb')
file.write(result)

file.close()

print(np.frombuffer(result))

And I get once over two this error : ValueError: tobytes can be invoked for dense arrays only

I can't understand the meanings, and why it's works not at all the time ...

[v923z]

Hi I will try tomorrow. Or do you have any solution / idea to save a matrix with the routine nump.save() on the sd card and then send the file through usb ?

numpy.save is not implemented.

[v923z]

Ok, this is the try of this day :

remove exceeding zeros

sens1 = sens1[:,:n_act_meas]

And I get once over two this error : ValueError: tobytes can be invoked for dense arrays only

I can't understand the meanings, and why it's works not at all the time ...

I think the problem is that you generate a slice in

sens1 = sens1[:,:n_act_meas]  

and then sens1 becomes "diluted", i.e., the strides cannot be calculated from the shape.

[labodezao]

Right, problem solved :)

[labodezao]

I suppose numpy.save is quite difficult to implement right ?

[v923z]

I suppose numpy.save is quite difficult to implement right ?

Not necessarily. If we leave out the allow_pickle option, then it is basically a combination of .tobytes() and .tolist() https://numpy.org/doc/stable/reference/generated/numpy.save.html

We could also think about allowing tobytes to accept diluted arrays. That would require a copy, because we cannot pass the array pointer into the bytes buffer: https://github.com/v923z/micropython-ulab/blob/a99e0b98787266369eaa33bdae271de761eca05b/code/ndarray.c#L1514-L1523

[labodezao]

Nice !

[labodezao]

np.save could be really nice because it's also byte array but the shapes are conserved ! :) Best

[v923z]

Closed via https://github.com/v923z/micropython-ulab/pull/456.