Help creating a custom program

[KF243]

try1.zip

On the vanilla Python environment, my example.py results in showing this error message. Please suggest how this problem is fixed.

[KF243]

I think that "self.already_run" works now. But line 127 of "for self.coordinate in coordinates: "(line 115-131) in move_positions() seems not to save the data. Data are collected and saved after the "for self.coordinate in coordinates: " or the last coordinate in "for self.coordinate in coordinates: ".

try38.zip

[bicarlsen]

It appears that in SECM#move_positions you don't wait for any time at any of the positions. You need to use asyncio.sleep, not time.sleep in async functions, otherwise the entrie program will block.

[KF243]

Thanks every time. Addition of asyncio.sleep provides significant progress as attached files which include both position and CA data. However, some data are missed in saved file. Is it possible to be retrieved? Furthermore I am wondering the difference between total saved time and controlling "params". Although I will try to minimize or diminish them, are they essential using asyncio.sleep or program.py?

try39.zip

[bicarlsen]

Glad to hear it's working better.

If you are referring to the blank lines in the data file, those are due to the asynchoronous nature of the program, but do not indicate any data is missing. Looking at the data, it appears everything is there.

I not quite sure what you mean by total saved time or controlling "params".

[bicarlsen]

I should also mention, that I have written some utility functions for importing the data created from easy-biologic programs into pandas. It is part of the bric-analysis-libraries package, in the bric_analysis_libraries.jv.biologic_data_prep module.

You may also be interested in the bric_analysis_libraries.jv.jv_data_prep and bric_analysis_libraries.jv.jv_analysis modules.

[KF243]

I am changing some parameters at lines 35-42. 1) It is difficult to complete synchronize position and CA data. See the data. For example, x=100um at Time=1.5s but still Voltage=0.1V (should be 0.2V) for ch0. Do I need additional trick for "asyncio.gather()"? 2) I take ca. 1.5s to change the position or update voltage. To decrease this period, I am wondering to change any parameters other than those of lines 35-42?

try40.zip

[bicarlsen]

For the position synchronization, I would try exchanging the self.update_voltages and position_controller.go_to calls, so the voltage is updated only after the new position is obtained. I believe this will also allow you to decrease the time between positions.

[KF243]

I exchanged the self.update_voltages and position_controller.go_to calls. In saved data, a parts of CA data seems not to be synchronized as at 3.156s, 4.428s, 8.416s, 9.726s, 12.260s, and 13.561s.

try41.zip

[bicarlsen]

I see two options to try:

1. Try flushing the data by calling self._retrieve_data_segments before calling self.save_data in SECM#move_positions.
2. Increase the sleep time in SECM#move_positions so the data from the current position is flushed before moving to the next position. easy_biologic.program.BiologicProgram#_retrieve_data take an interval argument that dictates how often data is read from the device, in seconds. It defaults to 1 second. If you wait at least this amount of time extra, the data should automatically flush.

[KF243]

Since any sleep times during measurement directly will affect a total SECM imaging time, finally I want to decrease it as possible as small.
Anyways, now irregular synchronization is accomplished as followed. Most of position and CA data are changed by "for" loop (at 111). But the first voltage update seems to be ignored and then CA data at previous condition are saved at each position.

Can I set "interval" of "_retrieve_data" as line 128?

try42.zip

[bicarlsen]

It appears on line 117, in SECM#move_positions, you're not actually calling self._retrieve_data_segments which could be causing some of the issues. Also, it should go after the asyncio.sleep call, not before (but still before the self.save_data call, and should be awaited. i.e.

# your code
self._retrieve_data_segments
await asyncio.sleep( ... )
self.save_data( ... )

# should be
await asyncio.sleep( ... )
await self._retrieve_data_segments()
self.save_data( ... )

Yes, it appears you are calling self._retrieve_data with the interval parameter correctly.

[KF243]

The self.update_voltages and position_controller.go_to calls are exchanged. But I do not find the suitable intervals (line 36-45) for synchronization of position to update voltage. There are several hundred milliseconds-differences as the attached.

try43.zip

[bicarlsen]

I believe this is a consequence of running the position_controller.go_to and self.update_voltages synchronously. i.e. After position.go_to finishes it will take some time for self.update_voltages to actually update the voltages, and this is the delay you are seeing.

If you need a higher level of precision then you can try making both of these call asynchronous so the voltages will update while the position controller is moving. I see two ways to accomplish this.

1. I would start by making the position controller asynchronous using asyncio. You can then set the position controller to change position, then call self.update_voltages wrapped in a task, and use asyncio.gather to wait for both to complete before continuing with the loop, or possibly using an asyncio synchronization primative.

2. If that doesn't solve the issue you can also try replacing self.update_voltages with a manual call to easy_biologic.lib.eclib#update_parameters_async, again waiting for both.

Note that the async version of easy_biologic.lib.eclib functions are not well tested, so you may encouter erros. I also can not ensure how much improvement this will give you.

[KF243]

After each position control, an interval for stabilizing electrode reaction is necessary (line 72). I am wondering why return value (xx, yy, zz) of position_controller.go_to prior to update_voltages can not be retrieved. Asynchronous of position controlling can arrange in order correctly?

[bicarlsen]

You could retrieve the previous position because you have all of them stored in the coordinates list, or you could simply save the current position before going to the new position. However, I don't think this solves the synchronicity issue. The Biologic takes measurements as it sees fits, all we can do is poll it to see if it has taken any data. So the Biologic can take measurements during the movement process and there is no way to tell exaclty when the measurement was taken.

Think about what the case would be if you performed the steps manually. You would go to a position, set your voltage, and take your measurements. You would then either hold your voltage as you move, set the new voltage, and then perform your measurements at the new position. This is exactly what we are seeing in the data. So the question becomes, what do you want the voltage to do while you are in the process of changing positions.

There are only three options to the order the voltage update and position change can take place:

1. Position before voltage. Do you want the position to change before updating the voltages. If so, then this is what we have.
2. Voltage before position. Do you want to update the voltage before changing positions, instead of after? If so you can just swap the order of the position_controller.go_to and self.update_voltages calls, and adjust other code as needed.
3. Voltage and position change together. Do you want the voltage to change as the controller is moving to the new position? Then you need to use asynchronous methods as described previously.

I don't understand what you mean by

Asynchronous of position controlling can arrange in order correctly?

Also the for loop running over coordinates seems very poorly written. I believe what you want is

# currently
for self.coordinate in coordinates:

# should be
for coordinate in self.coordinates:

with the variables in the loop changing accordingly.

[KF243]

The for loop running over coordinates were corrected.

"self.update_voltages" with various voltages for each channel are used to check the data synchronization. After the confirmation of the synchronization it will change to that with fixed voltages.

Although synchronicity of position and current data are needed in the saved file, polarization and current measurement are conducted after the position set (position_controller.go_to). Find the attached pdf which shows diagrams for motion and time schedule in SECM operation.

try44.zip

[bicarlsen]

I'm not sure what question you have in your previous comment.

[KF243]

Please forget about "Asynchronous of position controlling can arrange in order correctly?" It is difficult to convey it well. However, I believe that the diagrams in the last pdf.file shows my desired timing of stage motion and current measurement very well.

[bicarlsen]

It does, but that still isn't a question. I can't help you unless you have a specific question.

[KF243]

During the for loop of self.pos = 0 to 1, difference of 0.762s was recorded. Similarly 0.816s fro 1 to 2, 2.977s from 2 to 3, 0.891s from 3 to 4, 0.827s from 4 to 5, 3.027s from 5 to 6, 1.042s from 6 to 7, and 0.819s from 7 to 8. Larger differences (ca. 3 s) of 2to3 and 5to6 are probably due to "position_controller.detour( coordinate )". I guess that "position_controller.detour( coordinate )" needs to be excluded from asyncio.gather()? How do I decrease or diminish 0.8s-interval after each position?

[bicarlsen]

As I described before, I think the best way to get better synchronization would be to make the PositionController asynchronous so the voltage and position update in parallel, rather than in series.

[KF243]

I think that secm1851.py controls the voltage and position updates being asynchronous in parallel. However, saved data show differences of time. Now, does "async def move_positions( self, already_run = False ):" or "async def main( self ):" make the problem?

try45.zip

[bicarlsen]

You haven't made either of the actual method you need to call -- SECM#update_voltages or PositionController#go_to asynchronous. All you've done is add an asyncio#sleep call after them.

To make SECM#update_voltages async you can do as I described above by manually calling easy_biologic.lib.eclib#update_parameters_async. To makePositionController#go_toasync, though, you'll need to reprogram it in an asyncronous way. It seems this can be done fairly easily by wrapping theoptosigma.GSC01#sleep_until_stop` method in an asynchronous way.

[KF243]

How do I call easy_biologic.lib.eclib#update_parameters_async manually? ecl.update_parameters_async() requests me idn, ch, and technique which seem to be int in ec_lib, as well as params? The following makes no sense.

await ecl.update_parameters_async(
      idn = 16,
      ch = 2,
      technique = 102,
      params = {
            0: {'time_interval': time_interval, 'voltages': voltages0, 'durations': durations },
            1: {'time_interval': time_interval, 'voltages': voltages1, 'durations': durations }
       }
)

[bicarlsen]

I'll refer you to the code that currently calls these methods as an example: easy_biologic.base_programs.CALimit#update_voltages, and easy_biologic.device.BiologicDevice#update_parameters

You can also inspect easy_biologic.device.BiologicDeviceAsync.

I would encourage you to browse through the source code for examples if you have trouble understanding how to implement something in the future.

[bicarlsen]

I will close this issue due to inactivity. If something else comes up, please feel free to open a new Issue.