Help creating a custom program

[KF243]

try1.zip

On the vanilla Python environment, my example.py results in showing this error message. Please suggest how this problem is fixed.

[bicarlsen]

self.channels is documented as part of BiologicProgram. voltages should be a list of voltages keyed by channel. e.g.

voltages = { 0: [ -1, 0, 1 ], 1: [ 0, -1, 1 ] }

[KF243]

I seems to have no BiologicProgram.

C:\Users\kfush\AppData\Local\Programs\Python\Python39\Lib\site-packages\easy_biologic>dir ドライブ C のボリューム ラベルは OS です ボリューム シリアル番号は 7063-5E1C です C:\Users\kfush\AppData\Local\Programs\Python\Python39\Lib\site-packages\easy_biologic のディレクトリ 2021/11/20 17:50

. 2021/11/20 17:50 .. 2021/11/20 17:49 56,654 base_programs.py 2021/11/20 17:49 2,182 common.py 2021/11/20 17:49 22,507 device.py 2021/11/20 17:49 829 find_devices.py 2021/11/20 17:50 lib 2021/11/20 17:49 26,058 program.py 2021/11/20 17:50 techniques-5.35 2021/11/20 17:50 techniques-6.01 2021/11/22 10:20 techniques-6.04 2021/11/20 17:49 352 init.py 2021/11/20 17:50 pycache 6 個のファイル 108,582 バイト 7 個のディレクトリ 859,412,389,888 バイトの空き領域

Anyway, the error message was changed as follows.

Exception has occurred: TypeError
an integer is required (got type NoneType)
  File "C:\Users\kfush\data\secm131.py", line 158, in _run
    self.update_voltages( voltages )
  File "C:\Users\kfush\data\secm131.py", line 188, in <module>
    secm.run()

try16c.zip

[bicarlsen]

BiologicProgram is defined in the program.py file.

Unfortunately I'm not sure what is causing the new error. You may have to debug it yourself to find what is causing the issue.

[KF243]

In program.py, do I need to use self._channels and self.channels separately? How do I distinguish them?

[bicarlsen]

BiologicPrograms._channels is automatically set when the program is initialized so it is recommeneded to not modify it. BiologicProgram.channels returns the automatically configured channels, and is probably what you want to use.

[KF243]

param voltages should be "Dictionary of voltages list keyed by channel". "voltages = { 0: [ 0.1], 1: [-0.2] }" is not OK? Which kind of integer is required to fix "voltages"?

try17.zip

[bicarlsen]

I don't think the attached program is the correct one.

[KF243]

"set_channel_configuration" for 2 chs are added. But "set_channel_configuration" in line156 shows TypeError. You told a list of voltages keyed by channel. e.g. voltages = { 0: [ -1, 0, 1 ], 1: [ 0, -1, 1 ] }. How do I correct it?

try18.zip

[bicarlsen]

This is not an error with the parameters you are passing.

The error occurring is

File "C:\Users\kfush\AppData\Local\Programs\Python\Python39\lib\site-packages\easy_biologic\lib\ec_lib.py", line 988, in update_parameters
    idn = c.c_int32( idn )
TypeError: an integer is required (got type NoneType)

Meaning that the device is not connected when you are trying to update the parameters. My guess is this is occurring because on line 148 of secm14.py you call self._disconnect(), then try to update the voltages on line 157.

[KF243]

Version 0.3.2 was installed.

Since "asyncio.sleep()" at line 161 resulted in RuntimeWarning, the "asyncio.sleep()" is temporary not effective in secm15.py. secm15.py seems to run but not show the CA data at each coordinate. How is it modified?

PS C:\Users\kfush\data> & C:/Users/kfush/AppData/Local/Programs/Python/Python39/python.exe c:/Users/kfush/data/secm15.py <property object at 0x000001FDDFD68590> [1 1 0] [2 1 0] [3 1 0] [1 2 0] [2 2 0] [3 2 0] [1 3 0] [2 3 0] [3 3 0]

try19.zip

[bicarlsen]

The reason you got an error when trying to run asyncio.sleep() in SECM#_run is because #_run is not an async method.

I cleaned up some of the code and logic from the program above. All the logical changes I made are in SECM#_run. I have not tested the code, so you may have to address some minor errors to get it running. Please see if it is a bit closer to what you want.

secm15-edit.zip

[KF243]

Only the first data ([110]_data.csv) contains CA data without position data as attached.

Results_secm15-edit.zip

[bicarlsen]

I updated the program again. The functionality of update_parameters is a bit vague, as I'm not sure if the device "restarts" the program running the program from the beginning of the newly updated parameters, or if it continues from it's timed position, ignoring any parameters that have already been passed, even if they are updated. So you may have to play with the values of the voltages and durations parameters.

Also note that now data is appended to the data file, so if you run the program multiple times it will append data to the already created file. Thus, to separate the data from trial to trial you must move the data file (in this case hardcoded to be data.csv).

Again, I have not tested this code, so you may have to fix some errors.

secm15-edit.zip

[KF243]

Modified program (secm15-edit21.py) obtained the attached file (data.csv), in which CA data of [1 1 0] coordinate without x,y,z data.

try20.zip

[bicarlsen]

It seems that for some reason the x, y, and z variables are None. I suggest looking at SECM#go_to_position to see if self.position is being set correctly and at the _field_values function defined in SECM#__init__ to see why they aren't being set.

Also self.position_measurement_time should probably just be the sum of the duration from one of the channels, not both. As the channels run in parallel, summing their durations together will give double the actual run time.

[KF243]

The latest program (secm15-edit22.py) can move the position actually at each coordinate, because the movements are confirmed with naked eye. It seems work as a scanner. However, the saved file (data.csv) contains no header and x, y, z data for the first coordinate, and no data for the 2nd and more coordinates as attached. How are x, y, z data appended to CA data?

try21.zip

[bicarlsen]

The reason no header is being output is because append = True in the save_data call. You can add the header by creating a boolean called something like already_run and setting it to False outside the for coordinate in self.coordinates loop. Then after the save_data call, set it to True. e.g.

already_run = False
for coodinate in self.coordinates:
  # ...
  self.save_data( 'data.csv', append = already_run )
  already_run = True

I'm not sure why the coordinates are not bein saved in the data. I suggest adding some print statements in the _field_values function to try debugging. I would start by printing out self and self.position to see if those are being set correctly. If they are, then I would try making sure the tuple being returned is correct.

[KF243]

Now "data.csv" includes the header. "_field_values" function seems is carried out once before the for "coordinate in self.coordinates: "loop as the printout. How "_field_values" function is included in the loop? In the loop, "datum" and "segment" seems to be unknown.

try22.zip

[bicarlsen]

_field_values calculates the values to be written out as data from the DataSegment.

Try retrieving the position form a function. e.g.

def _field_values( datum, segment ):
  # ...
  position = self.get_position()
  if position is None:
    x = y = z = None
  else:
    x, y, z = self.position
  # ...

# ...
def get_position( self ):
  return self.position

[KF243]

Although it was difficult for me to retrieve the position as your suggestion, now "data.csv" obtained by "secm16.py" includes x, y, and z data for the 1st position. I want to add the following position from the 2nd. How do I retrieve "_field_values( datum, segmant)" as well as x, y, z data in for coordinate loop from line 198.

try23.zip

[bicarlsen]

self._field_values is function called whenever data segments are read from the device, and is passed each data segment to process it and create the data to be saved which is returned as a tuple.

The error you made in your script is that you reference controlX instead of something like self.controlX in SECM#get_position and _field_values, and PositionController#get_xyz. e.g.

class PositionController():
    def __init__( self, controlX, controlY, controlZ ):
        self.x = controlX
        self.y = controlY
        self.z = controlZ

    # ...

    def get_xyz( self ):
        xx = self.x.position
        yy = self.y.position
        zz = self.z.position
        return ( xx, yy, zz )

class SECM( blp.CALimit, PositionController ):
    def __init__( ... ):
        self.position_controller = position_controller
        # ...
        def _field_values( datum, segment ):
            # ...
            xx, yy, zz = self.get_position()

    # ...
    def get_position( self ):
        return self.position_controller.get_xyz()

You may also be able to call self.position_controller.get_xyz directly in _field_values and eliminate SECM#get_position, or you may have to modify SECM#get_position to be something like

def get_position( self ):
    xx = self.position_controller.x.position
    yy = self.position_controller.y.position
    zz = self.position_controller.z.position

    return ( xx, yy, zz )

or you could take the coordinates from those passed in rather than from the position controller. e.g.

def get_position( self ):
    return self.position

You may have to play around a bit with combinations of these.

[KF243]

"get_position" function is now in "_field_values" as SECM16-2.py. However, return values of " _field_values" are not shown with position scanning and in data-SECM file (in which only data of the 1st position is recorded but no data of the 2nd and more). How do I carry out "_field_values" or retrieve data for the following positions?

try24.zip

[bicarlsen]

My guess as to why the rest of the data isn't being collected is because of what I brought up earlier in that when updating the voltages the program does not restart running from the beginning, so the program actually finishes after the first run. You may try playing with the voltages and durations params, changing them to something like

num_positions = soordinates.shape[ 0 ]
base_interval = 0.1
time_interval = base_interval * 0.5
durations =  [ base_interval ]* num_position
voltages = {
    0: [  0.1 ]* num_positions,
    1: [ -0.2  ]* num_positions
}
params = {
        0: {'time_interval': time_interval, 'voltages': [ 0.1 ], 'durations': durations },
        1: {'time_interval': time_interval, 'voltages': [ -0.2 ], 'durations': durations }
}

or in #_run

def _run( self, technique, params, fields = None, interval = base_interval, retrieve_data = True ):
    # ... 

    for coordinate in self.coordinates:   
        self.go_to_position( coordinate )
        xx, yy, zz = self.get_position()

        # set voltages here
        self.update_voltages(
            self.params[ 0 ][ 'voltages' ],
            durations = self.params[0][ 'durations' ]
        )

        # perhaps change the sleep time to 0 to see if new positions are recorded
        time.sleep( self.position_measurement_time )
      # ...

At this point you may have to play around with some things to try to get the basic functionality working, then fine tune from there.

You can see how to manually retrieve data in the BiologicProgram#_retrieve_data_segment and BiologicProgram#_retrieve_data_segments methods.

[KF243]

Unnecessary commands were removed and the coding was arranged. Some "voltages" and "durations" params were tried. But I am still missing the rest of the data as attached. It seems not to be carried out line 144: asyncio.run( self._retrieve_data( interval ) ).

try25.zip

[bicarlsen]

asyncio.run( self._retrieve_data( interval * 5 ) ) is being run, otherwise no data would be retrieved. I think the issue has to do with the timing. I suggest going back to a version of the program where data was seemingly being retrieved for multiple positions, but the coordinates were not being saved, and step by step modifying it to save the coordinate positions as happens now.

[KF243]

Previously I think I failed to retrieve the rest data. The modified version excludes position scanning, but at line 124: def _run(), "fields" and "retrieve_data" are not accessed (which were also shown in the last coding: SECM16-3.py. May I restart the modification from this version or more primitive one? Anyway, no rest data data are saved.

try26.zip

[bicarlsen]

I'm sorry, I don't quite understand the issue you're having, or question you're asking. Could you try to clarfiy or provide a more indepth example?

[KF243]

I think that secm16-3retry.py in try26.zip, which has no relation with position scanning. However it cannot retrieve the 2nd and following data including CA output as data_SECM.csv. Why the 2nd data are ignored to save in the file.

[bicarlsen]

It appears that the program is running correctly. You commented out all the code that would iterate over the coordinates (lines 144 -167), and you only set one voltage in the parameters (line 44 - 47), so the additional durations are ignored.

[KF243]

Line 144-167 was modified. How are not ignored additional durations ?

try27.zip

[bicarlsen]

The voltages and durations parameters should have the same length. If they are not, the shorter one will cause the program to terminate once it is complete.

I think, however, you have a fundamental misunderstanding of how to use the program. e.g. Your calls to update_voltages are incorrect. Please look at the documentation or the base_progrmas.py file for examples of how to make these sorts of calls.

[KF243]

Smaller length (<= 8) of voltages and durations parameters are accepted but larger length (>=9) are not operated as attached. Why the update number is limited?

try28.zip

[bicarlsen]

I'm not 100% sure why this is occuring, but you are using the CALimit#update_voltage method incorrectly. To update the different channels, you should pass in a dictionary keyed by channel to each of the parameters. If a list is passed, then all channels are updated.

The error is

[-102] ERR_INSTR_TOOMANYDATA: too many data to transfer from the instrument (device error).

which isn't incredibly clear. Perhaps it would be worth contacting Biologic and asking them. I've worked with them before and they are quite responsive.

[KF243]

A dictionary keyed params for update_voltages was reconsidered, resulting in success to voltage update and save CALimit data. I am re-cording to combine stage scan and CALimit measurement. However, position data seem to be lost as csv-file, probably due to failure of synchronizing. How do I retrieve them?

try30.zip

[bicarlsen]

Glad to hear it's working better.

My guess as to why the position isn't working is because SECM both inherets PositionController and has a PositionController instance as an attribute (SECM.position_controller). Then, in the code you use the two interchangeably, which they shouldn't be. E.g.

# [line 130 in SECM#get_position]
return self.position_controller.get_xyz()  # gets the position of the position controller passed to the SECM instance.

# [line 155 in SEMC#_run]
self.go_to( coordinate )  # tells the PositionController represented by the SECM instance itself to go to the coordinate. 

So you can see you have two seperate instances of a PositionController. When you call self.go_to that is setting the position of the PositionController represented by the SECM instance, but has not changed the position of SECM.position_controller. So when you call SECM#get_position in _field_values it is returning the position of SECM.position_ccontroller which has not been changed.

My suggestion is to remove the inheritance of PositionController from SECM and only use the PositionController that is passed in as an argument (i.e. self.position_controller).

[KF243]

Position scanning might start after CALimit data reading as printout.txt.

In secm171.py, the modification for remove of the inheritance of PositionController from SECM and use of the PositionController is correct?

try31.zip

[bicarlsen]

To remove inheritence change class SECM( blp.CALimit, PositionController ) to class SECM( blp.CALimit ). You will then have to resolve any errors that come from removing the inheritence.

You seem to have done the opposite, keeping the inheritence, and removing the passed in PositionController which I would recommend against.

You also seem to retrienve the coordinates in an odd way using a class method rather than an instance metho on line 108.

xx, yy, zz = PositionController( controlX, controlY, controlZ ).get_xyz()

[KF243]

The inheritence of PositionController was removed from class SECM( blp.CALimit ). However, it still prints out datum (105) and then scan the position (from 128). How synchronize the scan to CALimit operation? Isn't it related to usage of _field_values( datum, segment )?

try32.zip

[bicarlsen]

It appears you have the experiment running for quite a short time. I suggest trying to increase the run time to see if that helps resolve the issue, especially in the beginning when everything is getting set up. I can also suggest removing teh update_voltages call from SECM#_run temporarily to see if you can take measurements and move the position controller at the same time. Once you have this simplified version of the program running, add the update_voltages call back in to coordinate the voltages and position.

[KF243]

I extended durations to 1 s, non-synchronization is not resolved. (Firstly, CALimit data are appeared and then stages are driven. The CALimit operation and stage drives are separately carried out.) Furthermore, a part of data for ch1 from 7.1 s are not saved.

try33.zip

[bicarlsen]

The reason all the measurements are happening before the positions change is because asyncio#run (line 126) is a blocking call. To allow both to run at the same time you can move the for coordinate in self.coordinates loop (lines 129 - 148) into an async coroutine, let's call it async def move_positions(). Then create another function to gather this and the self#_retrieve_data calls, lets call it async def main().

async def main():
    await asyncio.gather( [ self._retrieve_data( interval ), move_positions() ] )

Finally, instead of calling asyncio.run( self._retrieve_data( interval ) ) call asyncio.run( main() ) (line 126).

I don't know why the data did not get saved for channel 1 after a certain time.

[KF243]

await asyncio.gather( [ blp.CALimit._retrieve_data( base_interval ), move_positions() ] ) seems not to be operated as the attached output. Locations of async def move_position & async def main() are correct?

try34.zip

[bicarlsen]

async def main and async def move_positions should be methods of SECM.

Also, you've used asyncio#gather incorrectly. You should pass each coroutine as an argument to gather, not as a list. I believe this is why your are receiving the error.

[bicarlsen]

You'll also likely want to make Tasks out of your coroutines. You can see this StackOverflow thread for an explanation.

[KF243]

In the secm175.py, async def main and async def move_positions are methods of SECM. List was removed from gather. However, secm.run() seems to be a trouble maker as the printout.

try35.zip

[bicarlsen]

main should be a normal instance method not a class method. You should pass self to it, and run both the coroutines as instance methods. E.g.

async def main( self ):
        await asyncio.gather(
              self._retrieve_data( base_interval ), 
              self.move_positions() 
        )
...
asyncio.run( self.main() )

I also suggest not using base_interval in _retrive_data as it breaks encapsulation of the class. You should either pass that value in to SECM, calculate it from another property of the instance, or use a static value.

[KF243]

The stage is scanned and then CAlimit data for 10s (= duration) seem to be saved in csv.file. In csv.file, parts of voltages (0V at 0s, 0.1V at 0.5-3s, 0.2V at 3.5-4.5s, 0.3V at 5-7s, 0.4V at 7.5-8.5s, 0.5V at 9-10s for ch0) are irregularly recorded as well as the final position (x=200, y=200, z=100), and CAlimit data of 0.6-0.9V for ch0 are missed. First I want to start the position scanning and CALimit retrieving simultaneously.

try36.zip

[bicarlsen]

When calling SECM#save_data you set the append argument to append_run which is always False, so the data file is being overwritten.

We added the already_run variable, so the headers would be printed out during the first save when it is False, but needs to be set to True after the first save. i.e. Add the line already_run = True after the self.save_data() call in SECM#move_positions.

I'm not sure why not all the data is being recorded, but it may be worth calculating the voltages and durations for all position initially, rather than updating them each time.

For synchronization, you can read about the asyncio synchronization primatives. It's not immediately clear what th best way to synchronize is, though. Becase you can't reset the Biologic program at each position, it seems the only way to accomplish it would be to set a hold voltage, perhaps with an extended duration, at the end of each position measurement, then inspect the data when it is saved to see if it is in the holding period and move positions. e.g. You want to measure at 0.1, 0.2, and 0.3 V, each for 1 second, and have a hold voltage at 0 V. this measurement should take place at 3 positions. Then you could do

coordinates = [ 0, 1, 2 ]
voltages = [ 0.1, 0.2, 0.3, 0 ]* len( coordinates )
durations = [ 1, 1, 1, 10 ]* len( coordinates )

Then in SECM#move_positions you would check to see if the voltage was 0, and if so move to the next position. Note that this would not use the asyncio syncronization mechanisms.

[KF243]

'already_run = True' was added, but 'already_run' seems not to be referred successfully. try37.zip

While now, the desired operations for SECM are as follows, 1) Driving the position stages 2) Amperometry at constant potentials using two channel electrodes. Remark for 1): x stage drives frequently (with a constant interval) but sometimes xyz stages also drive (taking different interval), meaning that stages do not drive at certain interval totally. Remark for 2): at each x stage drive with a constant interval, current data should be recorded as well as stage position for current imaging. Only in tentative cordings, potentials (voltages) are changed to check the operation. On the other hand, CALimit or CA tends to collect data with the interval defined as params. So, I am wondering whether asyncio is available to retrieve current data with the intervals which is determined by stage drive.

In chronoamperometry of EC-Lab, current data are displayed with some intervals during continuous polarization. Is it also possible to display (or print out) data using easy-biologic?

[bicarlsen]

In terms of already_run the error speaks for itself. Because you are using already_run in several locations throughout the code, I would probaby create an instance variable for it, e.g. self.already_run.

That being said, this is a basic coding concept and you should expend some effort on fixing these errors yourself. I am happy to help you with coding the program, but you should not rely on me to fix every small bug that comes up in the program.

I don't understand your remarks enough to make any comment on them.

easy_biologic does not have any built in visualization tools. However you can easily add them by creating an external program that reads the data file being written to, or by creating a program that updates using the BiologicProgram#on_data callback.