HPInc / HP-Digital-Microfluidics

HP Digital Microfluidics Software Platform and Libraries
MIT License
3 stars 1 forks source link

Using the opentrons without the app #201

Open EvanKirshenbaum opened 8 months ago

EvanKirshenbaum commented 8 months ago

To run the opentrons with our own scripts:

python.py cs.py joey --pipettor ot2
--ot-ip 192.168.86.29 
--ot-config ../inputs/ot2.json 
--ot-reagents ../inputs/reagents.json 
--clock-speed=100ms 
--fragments 20
--shuttles=2 
--cycles=2

The --ot-ip argument is the IP address of the robot. You can get that from the app. --ot-config points to a JSON file describing what components are where on the deck. --ot-reagents specifies where the different reagents are in the input wells. The versions of the files that I use are in the inputs directory.

I recommend that you simply run the robot "dry", either with an empty board or with well plates in the positions specified in ot2.json. In any case, you probably want to not actually put tip racks there or you're going to be continually reloading them from the trash.

If you run without the --pipettor ot2, you'll get the DummyPipettor, which simply simulates the pipettor. (You don't need to worry about removing the --ot-... arguments.) If you do that, you'll probably want to add a --pipettor-speed 3 to speed up the simulation (by 3x, in this case), because I actually modelled a somewhat realistic pipettor.

@EvanKirshenbaum:

Migrated from internal repository. Originally created by Mark Huber on Sep 14, 2022 at 4:25 PM PDT.
EvanKirshenbaum commented 8 months ago

It should run a relatively complicated pipelined protocol that uses the pipettor a lot. If you leave off the --pipettor ot2 argument, it will just run on the display, printing out messages about what the pipettor would be doing. (This is equivalent to running with --pipettor dummy.) With --pipettor ot2, the robot should actually perform the requested aspirations and dispensing. The display will also show up, so you can follow along with what's going on.

Basically,

  1. Pull in a bunch of "fragment reagents" to the top right extraction point, and mix them together. Keep one drop and send the rest to waste. Thermocycle the drop.
  2. Dilute the drop 8x with Dilution Buffer, then mix in Master Mix and PM Primers. Keep one drop and send the rest to waste. Thermocycle the drop.
  3. Dilute 5x with Prep Mixture. Keep one drop and send the rest to waste. Thermocycle the drop.
  4. Dilute 5x with Prep Mixture and mix with Master Mix. Keep one drop and send the rest to waste. Thermocycle the drop.
  5. Mix with PF Primers, brought in via middle extraction point. Keep one drop and send the other to waste. Thermocycle the drop.
  6. Remove the drop as a product via the lower extraction point.

This is run as a pipeline, with up to eleven drops in progress at any time, each starting with a different random combination of fragments.

What happens when you run it?

To run a simpler script, the easiest thing is to create your own Task and create a script along the lines of interactive.

The Task is straightforward, something like (warning: I'm typing on the fly, so there are probably errors):

class TestTransfer(Task):
  def __init__(self) -> None:
    super().__init__(name="test-transfer", description="Simple pipettor transfer test")

  def run(self, board: Board, system: System, args: Namespace) -> None:
    r = Reagent.find("R")
    in_ep = board.extraction_points[0]
    out_ep = board.extraction_points[1]

    path = Path.teleport_into(in_ep, reagent=reagent).to_pad(out_ep.pad).teleport_out()
    path.schedule()

Then in your script (see interactive.py), you can do something like

exerciser = PlatormChoiceExerciser.for_task(TestTransfer,
                                            platforms=(joey.PlatformTask,),
                                            pipettors=(opentrons.PipettorConfig,))
exerciser.parse_args_and_run()

If you just run this, you'll get it simulated on the display. If you run it with --pipettor ot2 and the other ot2 params, it will drive the robot.

Finally, you need to create a reagents JSON file to pass as the --ot-reagents parameter to tell the robot where reagent R is stored, something like

{
    "reagents": [
        {
            "name": "R",
            "wells": [ 
                { "plate": 0, "well": "A1", "quantity": 200, "use": "input" }
            ]
        }
    ]
}
Migrated from internal repository. Originally created by @EvanKirshenbaum on Sep 15, 2022 at 11:21 AM PDT.
EvanKirshenbaum commented 8 months ago

The pipettor is just moving back and forth a little over the trash can (doesn't actually leave it's location). The console says this:

python cs.py joey --pipettor ot2 --ot-ip 192.168.1.113 --ot-config ../inputs/ot2.json --ot-reagents ../inputs/reagents.json --clock-speed=100ms --fragments 20 --shuttles=2 --cycles=2
Platform option 'devices.bilby.PlatformTask' requires 'pyglider' module, ignoring
   INFO|opentrons|Launching listener
======== Running on http://0.0.0.0:8087 ========
(Press CTRL+C to quit)
   INFO|opentrons|Temp protocol file is C:\Users\huberma\AppData\Local\Temp\protocol_jkslfyso.py
   INFO|opentrons|Create Protocol result: {'data': {'id': '1e386cd6-c586-45a3-9410-d59085383f81', 'createdAt': '2022-09-15T21:59:59.409447+00:00', 'files': [{'name': 'protocol_jkslfyso.py', 'role': 'main'}], 'protocolType': 'python', 'metadata': {'description': 'A generic looping client to use the OT-2 as a peripheral for Joey', 'author': 'Evan Kirshenbaum < evan.kirshenbaum@hp.com>', 'protocolName': 'Joey Peripheral Protocol', 'apiLevel': '2.11'}, 'analyses': [], 'analysisSummaries': [{'id': '84b39a73-5e04-484f-90f9-d658d3ef5a08', 'status': 'pending'}]}}
   INFO|opentrons|Created protocol "1e386cd6-c586-45a3-9410-d59085383f81".
   INFO|opentrons|Created session "090edf74-157a-4b29-9c2d-af900b45243d".
   INFO|opentrons|Started run.
   INFO|pipettor|Pipettor("OT-2") is not idle
   INFO|monitor|Setting tick to 100.0 ms

I did adjust the ot2.json to our setup (we only have the small pipettor on the right side.

I'm assuming it's hanging bc of this:

INFO|pipettor|Pipettor("OT-2") is not idle

Trying to figure out why this pipettor is not responding

Migrated from internal repository. Originally created by Mark Huber on Sep 15, 2022 at 3:13 PM PDT.
EvanKirshenbaum commented 8 months ago

I did adjust the ot2.json to our setup (we only have the small pipettor on the right side.

I'd actually be surprised if the OT2 protocol didn't die if you didn't specify both pipettors, as it was written with the assumption that they would both be there.

I can probably change things pretty easily to work with only the small pipettor. (If you only have the large one, you won't be able to dispense a small enough volume, but with only the small one, it'll just make more trips.) Let me see what I can do.

Migrated from internal repository. Originally created by @EvanKirshenbaum on Sep 15, 2022 at 3:38 PM PDT.
EvanKirshenbaum commented 8 months ago

Just out of curiosity, since when I started my Opentrons app it asked whether I wanted to update from 5.0.2 to 6.0.1, what version are you running? If you're on 6, it's possible (likely) that the HTTP protocol changed again, and I'll have to figure out how to talk to it again.

Migrated from internal repository. Originally created by @EvanKirshenbaum on Sep 15, 2022 at 4:23 PM PDT.
EvanKirshenbaum commented 8 months ago

We are on 6.0.1, I will test tomorrow. I think it should work since it's having a connection

Migrated from internal repository. Originally created by Mark Huber on Sep 15, 2022 at 7:35 PM PDT.
EvanKirshenbaum commented 8 months ago

I updated my branch to the latest:

image

I did modify ot2.json to:

"pipettes": {
    "small": {
        "name": "p20_single_gen2",
        "side": "right"
    }
},

and i'm still having the same issue and same log message, no errors thrown. The pipettor is moving though. This should mean that the http connection works right?

Migrated from internal repository. Originally created by Mark Huber on Sep 16, 2022 at 10:50 AM PDT.
EvanKirshenbaum commented 8 months ago

I think i was wrong, seems like they did change the HTTP endpoints between the versions, I'm checking out the docs here:

http://{ROBOT_IP_ADDRESS}:31950/redoc
http://{ROBOT_IP_ADDRESS}:31950/openapi.json
Migrated from internal repository. Originally created by Mark Huber on Sep 16, 2022 at 11:07 AM PDT.
EvanKirshenbaum commented 8 months ago

I downgraded the Opentrons version to 5.0.2 but I'm still having the same issue.

image

wondering if I'm doing something wrong on my end. The pipettes look like this:

image

and should match the setup in the json above.

Migrated from internal repository. Originally created by Mark Huber on Sep 16, 2022 at 11:35 AM PDT.
EvanKirshenbaum commented 8 months ago

Weird. Same trace? In particular, it gets a result from creating the session and gets to the "not idle" message (which indicates that there's work to do) and nothing further?

We may have to do this the hard way, with a debugging session over Zoom, since I can't connect to your robot from here.

Migrated from internal repository. Originally created by @EvanKirshenbaum on Sep 17, 2022 at 4:16 PM PDT.
EvanKirshenbaum commented 8 months ago

Just out of curiosity, can you check your firewall logs to see if you've been blocking connection requests on port 8087? That would do it.

If you need to pick a different port, we can easily make that a command-line parameter. (I'm not sure why it isn't already.) But we do need the robot to be able to open up an HTTP connection to your computer.

Migrated from internal repository. Originally created by @EvanKirshenbaum on Sep 17, 2022 at 4:22 PM PDT.
EvanKirshenbaum commented 8 months ago

same trace:

C:\microfluidics\thylacine\mpam\tools>python cs.py joey --pipettor ot2 --ot-ip 192.168.1.113 --ot-config ../inputs/ot2.json --ot-reagents ../inputs/reagents.json --clock-speed=100ms --fragments 20 --shuttles=2 --cycles=2
Platform option 'devices.bilby.PlatformTask' requires 'pyglider' module, ignoring
   INFO|opentrons|Launching listener
======== Running on http://0.0.0.0:8087 ========
(Press CTRL+C to quit)
   INFO|opentrons|Temp protocol file is C:\Users\huberma\AppData\Local\Temp\protocol_btbggrl4.py
   INFO|opentrons|Create Protocol result: {'data': {'id': '242b84e9-2958-4322-902a-b7929ca3770d', 'createdAt': '2022-09-19T18:26:11.900574+00:00', 'files': [{'name': 'protocol_btbggrl4.py', 'role': 'main'}], 'protocolType': 'python', 'metadata': {'protocolName': 'Joey Peripheral Protocol', 'apiLevel': '2.11', 'author': 'Evan Kirshenbaum < evan.kirshenbaum@hp.com>', 'description': 'A generic looping client to use the OT-2 as a peripheral for Joey'}, 'analyses': [{'id': '6af4e894-c58e-42e5-9627-aafaf8b8f75b', 'status': 'pending'}]}}
   INFO|opentrons|Created protocol "242b84e9-2958-4322-902a-b7929ca3770d".
   INFO|opentrons|Created session "1f41541a-1620-4611-ac2c-08876c1e3e72".
   INFO|opentrons|Started run.
   INFO|pipettor|Pipettor("OT-2") is not idle
   INFO|monitor|Setting tick to 100.0 ms

I don't seem to have this folder: %\System32\LogFiles\Firewall even though that's where it's specified:

image

I tried to turn my firewall off but it's managed by McAffee and needs a special password to turn it off. I tried to turn off the windows firewall here:

image

but it just turns itself back on again.

I checked the event logs in McAffee but there's only adobe events that got blocked.

Migrated from internal repository. Originally created by Mark Huber on Sep 19, 2022 at 11:47 AM PDT.
EvanKirshenbaum commented 8 months ago

I opened port 8087 but still the same issue

Migrated from internal repository. Originally created by Mark Huber on Sep 19, 2022 at 12:43 PM PDT.
EvanKirshenbaum commented 8 months ago

I tried connecting to the robot with curl and my own python script, following the tutorial here

but with curl and python and on both versions (5.0.2 and 6.0.1) I get the following error:

{'errors': [{'id': 'InvalidRequest', 'title': 'Invalid Request', 'detail': 'field required', 'source': {'pointer': '/files'}}]}

I downloaded the logs from the robot but it only said it's a 422 http error:

Sep 19 21:04:16 OT2CEP20210309B13 uvicorn[234]: INFO: - "POST /protocols HTTP/1.1" 422 Unprocessable Entity

Migrated from internal repository. Originally created by Mark Huber on Sep 19, 2022 at 2:27 PM PDT.
EvanKirshenbaum commented 8 months ago

Ah ok, this method only works with versions 4.7 and lower. @EvanKirshenbaum what version are you currently running?

https://support.opentrons.com/s/article/Downgrade-to-an-older-software-version

Migrated from internal repository. Originally created by Mark Huber on Sep 20, 2022 at 9:44 AM PDT.
EvanKirshenbaum commented 8 months ago

Ah ok, this method only works with versions 4.7 and lower. @EvanKirshenbaum what version are you currently running?

https://support.opentrons.com/s/article/Downgrade-to-an-older-software-version

I'm on 5.0.2. Getting it to work with version 5 was an effort in and of itself (#82). Looking at the log history, though, I'm not actually sure what made it start working. The last commit message was

issue #82. CKP. Can start protocol on OT-2, and messages work, but "ready" call doesn't come through

but it definitely works for me now.

Migrated from internal repository. Originally created by @EvanKirshenbaum on Sep 20, 2022 at 10:28 AM PDT.
EvanKirshenbaum commented 8 months ago

Look at opentrons.ProtocolManager.run() to see how I currently upload the protocol (and associated files).

If you're getting the "Created protocol" and "Created session" log messages, the OT-2 HTTP protocol is working fine. The next thing that there should be is a call on the message endpoint of our system that prints out a message saying "Entering main loop". If you're not getting that, then the HTTP call from the robot to your computer is failing.

Migrated from internal repository. Originally created by @EvanKirshenbaum on Sep 20, 2022 at 10:47 AM PDT.
EvanKirshenbaum commented 8 months ago

made some progress, followed this tutorial to install requests on the opentrons. It now turns on the light and throws an error.

Migrated from internal repository. Originally created by Mark Huber on Sep 21, 2022 at 12:00 PM PDT.
EvanKirshenbaum commented 8 months ago

The error message seem to have gone away with restarting my computer (was an error message in regards to port 8087. json seems to be installed on the opentrons:

 ssh -i ot2_ssh_key root@192.168.1.113
The authenticity of host '192.168.1.113 (192.168.1.113)' can't be established.
ECDSA key fingerprint is SHA256:ENkc9U3Xza2UCKR4O87yGhsHjXDUX/w2WIN7obLVTJw.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '192.168.1.113' (ECDSA) to the list of known hosts.

      @@@@@    @@@@@
    @@@@          @@@@
   @@@      @@      @@@    @@@@@@   @@@@@
  @@@      @@@@      @@@   @@@@@@  &@' '@@
  @@     @@@@@@@@    &@@     @@         @@
  @@    .@@@    @    #@@     @@        @@
  @@@    @      @    @@@     @@       @@
   @@@    @@..@@    @@@      @@      @@
    @@@@          @@@@       @@     @@@@@&
      @@@@@@@@@@@@@@         ##    &@@@@@#
         (@@@@@@.
# python
Python 3.7.1 (default, Aug  9 2022, 23:25:18)
[GCC 7.3.1 20180425 [linaro-7.3-2018.05 revision d29120a424ecfbc167ef90065c0eeb on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import json
>>>
Migrated from internal repository. Originally created by Mark Huber on Sep 21, 2022 at 12:22 PM PDT.
EvanKirshenbaum commented 8 months ago

getting the same behavior though, except light is turning on now, so i'm assuming we got further:

C:\microfluidics\thylacine\mpam\tools>python cs.py joey --pipettor ot2 --ot-ip 192.168.1.113 --ot-config ../inputs/ot2.json --ot-reagents ../inputs/reagents.json --clock-speed=100ms --fragments 20 --shuttles=2 --cycles=2
Platform option 'devices.bilby.PlatformTask' requires 'pyglider' module, ignoring
   INFO|opentrons|Launching listener
======== Running on http://0.0.0.0:8087 ========
(Press CTRL+C to quit)
   INFO|opentrons|Temp protocol file is C:\Users\huberma\AppData\Local\Temp\protocol_56y8c0_k.py
   INFO|opentrons|Create Protocol result: {'data': {'id': '2f2b6989-8d7b-4cf2-91fd-9c4f64f06cce', 'createdAt': '2022-09-21T19:24:22.603800+00:00', 'files': [{'name': 'protocol_56y8c0_k.py', 'role': 'main'}], 'protocolType': 'python', 'metadata': {'author': 'Evan Kirshenbaum < evan.kirshenbaum@hp.com>', 'apiLevel': '2.11', 'protocolName': 'Joey Peripheral Protocol', 'description': 'A generic looping client to use the OT-2 as a peripheral for Joey'}, 'analyses': [], 'analysisSummaries': [{'id': '7df29e47-e4ea-446c-941c-4e250790d802', 'status': 'pending'}]}}
   INFO|pipettor|Pipettor("OT-2") is not idle
   INFO|monitor|Setting tick to 100.0 ms
   INFO|opentrons|Created protocol "2f2b6989-8d7b-4cf2-91fd-9c4f64f06cce".
   INFO|opentrons|Created session "97f42cd7-2cea-4080-9141-8d7960a0fa63".
   INFO|opentrons|Started run.

Earlier I saw an error message in the terminal:

Traceback (most recent call last): File "C:\Users\huberma\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\models.py", line 910, in json return complexjson.loads(self.text, **kwargs) File "C:\Users\huberma\AppData\Local\Programs\Python\Python310\lib\json__init__.py", line 346, in loads return _default_decoder.decode(s) File "C:\Users\huberma\AppData\Local\Programs\Python\Python310\lib\json\decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\Users\huberma\AppData\Local\Programs\Python\Python310\lib\json\decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\huberma\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1009, in _bootstrap_inner
    self.run()
  File "C:\microfluidics\thylacine\mpam\src\devices\opentrons.py", line 420, in run
    self.trace_response("Deleted protocol", response)
  File "C:\microfluidics\thylacine\mpam\src\devices\opentrons.py", line 343, in trace_response
    json = response.json() if hasattr(response, "json") else response
  File "C:\Users\huberma\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\models.py", line 917, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: [Errno Expecting value] <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.15.7</center>
</body>
</html>
: 0

downloaded the log run:

{"id":"7b72b298-136c-4e0c-8c0a-301b4e72704d","createdAt":"2022-09-21T19:11:43.827856+00:00","status":"running","current":true,"actions":[{"id":"f6a90927-01c6-44e8-bf6f-c5f44c31fdcc","createdAt":"2022-09-21T19:11:48.416590+00:00","actionType":"play"}],"errors":[],"pipettes":[],"modules":[],"labware":[{"id":"fixedTrash","loadName":"opentrons_1_trash_1100ml_fixed","definitionUri":"opentrons/opentrons_1_trash_1100ml_fixed/1","location":{"slotName":"12"}}],"labwareOffsets":[],"protocolId":"e0042cb9-6b3e-4d47-9cbd-18e2c1671156","startedAt":"2022-09-21T19:12:00.544841+00:00","commands":[]}

seems like it cannot read ot2.json

Will downgrade the opentrons version to version 5 and see if it works with that

Migrated from internal repository. Originally created by Mark Huber on Sep 21, 2022 at 12:31 PM PDT.
EvanKirshenbaum commented 8 months ago

Downgrading to 5.0.2 got us somewhere but there is still an error thrown in the protocols tab:

ExceptionInProtocolError: ConnectionError [line 684]: HTTPConnectionPool(host='192.168.223.1', port=8087): Max retries exceeded with url: /message (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x6e0fc210>: Failed to establish a new connection: [Errno 110] Connection timed out'))

seems like this is an issue with my pc mentioned here [comment by @EvanKirshenbaum on Sep 17, 2022 at 4:22 PM PDT]

Migrated from internal repository. Originally created by Mark Huber on Sep 21, 2022 at 12:59 PM PDT.
EvanKirshenbaum commented 8 months ago

yah, seems to be my computer that is the problem, I changed the port to 9090 and still had the same error:

ExceptionInProtocolError: ConnectionError [line 684]: HTTPConnectionPool(host='192.168.223.1', port=9090): Max retries exceeded with url: /message (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x63e238d0>: Failed to establish a new connection: [Errno 110] Connection timed out'))

will check with IT on how to whitelist a port

Migrated from internal repository. Originally created by Mark Huber on Sep 21, 2022 at 1:04 PM PDT.
EvanKirshenbaum commented 8 months ago

getting the same behavior though, except light is turning on now, so i'm assuming we got further:

Yeah, the first thing that should happen is that the light turns on. Then it creates a Robot object, inside whose __init__() message is

        self.message(f"Creating the robot. Callback endpoint is {self.endpoint}")

which makes an HTTP call back to print the message. This is presumably what's failing.

What appears to be happening is that the response is (or contains, it's hard to tell) HTML rather than the JSON it expects, and the JSON decoder is unable to cope with it. But what printed out is a 502 Bad Gateway error, which implies that there's a proxy or gateway that's confused.

Downgrading to 5.0.2 got us somewhere but there is still an error thrown in the protocols tab:

ExceptionInProtocolError: ConnectionError [line 684]: HTTPConnectionPool(host='192.168.223.1', port=8087): Max retries exceeded with url: /message (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x6e0fc210>: Failed to establish a new connection: [Errno 110] Connection timed out'))

This lends credence to the notion that (1) version 6 changed the payloads of the messages and (2) there's something on the network actively preventing the connection.

Migrated from internal repository. Originally created by @EvanKirshenbaum on Sep 21, 2022 at 1:30 PM PDT.
EvanKirshenbaum commented 8 months ago

out of curiosity, how does your setup look like? Do you also have McAffee and all this big brother software on your machine? Do you also have windows 10? Wondering why this is blocking. The opentrons is not on the hp inc network but our own wifi (still at hp). So it should not have proxy issues. Hmm, wondering if it would work if I connect over usb and put that ip address in.

Migrated from internal repository. Originally created by Mark Huber on Sep 21, 2022 at 7:01 PM PDT.
EvanKirshenbaum commented 8 months ago

I've got Norton 360 on my machine, and I don't recall having to have added any special rules. You might try using a port in the ephemeral range: 49,152-65,535 (C000 - FFFF). You might also want to try to connect to your machine on port 8087 (from another computer) while the protocol's running. The Listener doesn't limit its allowed connections to the robot (it probably should), so you can try to go to http://[your ip]:8087/message. If your firewall allows it, you should get through, and the server will print an error due to the lack of message payload, but at least you'll know you're connecting.

Migrated from internal repository. Originally created by @EvanKirshenbaum on Sep 21, 2022 at 10:07 PM PDT.
EvanKirshenbaum commented 8 months ago

got a second machine (ubuntu only) and typed in the ip address of my laptop: http://[your ip]:8087/message

C:\microfluidics\thylacine\mpam\tools>python cs.py joey --pipettor ot2 --ot-ip 192.168.1.113 --ot-config ../inputs/ot2.json --ot-reagents ../inputs/reagents.json --clock-speed=100ms --fragments 20 --shuttles=2 --cycles=2
Platform option 'devices.bilby.PlatformTask' requires 'pyglider' module, ignoring
   INFO|opentrons|Launching listener
======== Running on http://0.0.0.0:8087 ========
(Press CTRL+C to quit)
   INFO|opentrons|Temp protocol file is C:\Users\huberma\AppData\Local\Temp\protocol_ljyn1xq2.py
   INFO|opentrons|Create Protocol result: {'data': {'id': '92dfbaef-ffb2-47e8-80ef-11f0e5b03895', 'createdAt': '2022-09-23T19:01:02.183395+00:00', 'files': [{'name': 'protocol_ljyn1xq2.py', 'role': 'main'}], 'protocolType': 'python', 'metadata': {'apiLevel': '2.11', 'protocolName': 'Joey Peripheral Protocol', 'description': 'A generic looping client to use the OT-2 as a peripheral for Joey', 'author': 'Evan Kirshenbaum < evan.kirshenbaum@hp.com>'}, 'analyses': [{'id': '9e8b1ebe-9d42-4a34-a4cc-82acb8c1cb44', 'status': 'pending'}]}}
   INFO|opentrons|Created protocol "92dfbaef-ffb2-47e8-80ef-11f0e5b03895".
   INFO|opentrons|Created session "55326dec-9e6e-4430-8285-06411cb78a9c".
   INFO|opentrons|Started run.
   INFO|pipettor|Pipettor("OT-2") is not idle
   INFO|monitor|Setting tick to 100.0 ms
  ERROR|web_protocol|Error handling request
Traceback (most recent call last):
  File "C:\Users\huberma\AppData\Local\Programs\Python\Python310\lib\site-packages\aiohttp\web_protocol.py", line 334, in data_received
    messages, upgraded, tail = self._request_parser.feed_data(data)
  File "aiohttp\_http_parser.pyx", line 551, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadStatusLine: 400, message="Bad status line 'Invalid method encountered'"
  ERROR|web_protocol|Error handling request
Traceback (most recent call last):
  File "C:\Users\huberma\AppData\Local\Programs\Python\Python310\lib\site-packages\aiohttp\web_protocol.py", line 334, in data_received
    messages, upgraded, tail = self._request_parser.feed_data(data)
  File "aiohttp\_http_parser.pyx", line 551, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadStatusLine: 400, message="Bad status line 'Invalid method encountered'"
  ERROR|web_protocol|Error handling request
Traceback (most recent call last):
  File "C:\Users\huberma\AppData\Local\Programs\Python\Python310\lib\site-packages\aiohttp\web_protocol.py", line 334, in data_received
    messages, upgraded, tail = self._request_parser.feed_data(data)
  File "aiohttp\_http_parser.pyx", line 551, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadStatusLine: 400, message="Bad status line 'Invalid method encountered'"
  ERROR|web_protocol|Error handling request
Traceback (most recent call last):
  File "C:\Users\huberma\AppData\Local\Programs\Python\Python310\lib\site-packages\aiohttp\web_protocol.py", line 334, in data_received
    messages, upgraded, tail = self._request_parser.feed_data(data)
  File "aiohttp\_http_parser.pyx", line 551, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadStatusLine: 400, message="Bad status line 'Invalid method encountered'"
  ERROR|web_protocol|Error handling request
Traceback (most recent call last):
  File "C:\Users\huberma\AppData\Local\Programs\Python\Python310\lib\site-packages\aiohttp\web_protocol.py", line 334, in data_received
    messages, upgraded, tail = self._request_parser.feed_data(data)
  File "aiohttp\_http_parser.pyx", line 551, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadStatusLine: 400, message="Bad status line 'Invalid method encountered'"
  ERROR|web_protocol|Error handling request
Traceback (most recent call last):
  File "C:\Users\huberma\AppData\Local\Programs\Python\Python310\lib\site-packages\aiohttp\web_protocol.py", line 334, in data_received
    messages, upgraded, tail = self._request_parser.feed_data(data)
  File "aiohttp\_http_parser.pyx", line 551, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadStatusLine: 400, message="Bad status line 'Invalid method encountered'"
  ERROR|web_protocol|Error handling request
Traceback (most recent call last):
  File "C:\Users\huberma\AppData\Local\Programs\Python\Python310\lib\site-packages\aiohttp\web_protocol.py", line 334, in data_received
    messages, upgraded, tail = self._request_parser.feed_data(data)
  File "aiohttp\_http_parser.pyx", line 551, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadStatusLine: 400, message="Bad status line 'Invalid method encountered'"
  ERROR|web_protocol|Error handling request
Traceback (most recent call last):
  File "C:\Users\huberma\AppData\Local\Programs\Python\Python310\lib\site-packages\aiohttp\web_protocol.py", line 334, in data_received
    messages, upgraded, tail = self._request_parser.feed_data(data)
  File "aiohttp\_http_parser.pyx", line 551, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadStatusLine: 400, message="Bad status line 'Invalid method encountered'"

so that means it's not a firewall issue.

Migrated from internal repository. Originally created by Mark Huber on Sep 23, 2022 at 12:07 PM PDT.
EvanKirshenbaum commented 8 months ago

I asked IT, he said there is no proxy on the wifi (the opentrons is not on the company wifi).

I also tried running the script from my ubuntu machine but had the same result. There was an error message related to the ui but i got the same logging infos in the console:

INFO|opentrons|Temp protocol file is C:\Users\huberma\AppData\Local\Temp\protocol_ljyn1xq2.py
   INFO|opentrons|Create Protocol result: {'data': {'id': '92dfbaef-ffb2-47e8-80ef-11f0e5b03895', 'createdAt': '2022-09-23T19:01:02.183395+00:00', 'files': [{'name': 'protocol_ljyn1xq2.py', 'role': 'main'}], 'protocolType': 'python', 'metadata': {'apiLevel': '2.11', 'protocolName': 'Joey Peripheral Protocol', 'description': 'A generic looping client to use the OT-2 as a peripheral for Joey', 'author': 'Evan Kirshenbaum < evan.kirshenbaum@hp.com>'}, 'analyses': [{'id': '9e8b1ebe-9d42-4a34-a4cc-82acb8c1cb44', 'status': 'pending'}]}}
   INFO|opentrons|Created protocol "92dfbaef-ffb2-47e8-80ef-11f0e5b03895".
   INFO|opentrons|Created session "55326dec-9e6e-4430-8285-06411cb78a9c".
   INFO|opentrons|Started run.

it did not print out these messages:

   INFO|pipettor|Pipettor("OT-2") is not idle
   INFO|monitor|Setting tick to 100.0 ms

but did turn on the light, will try to copy over the exact error message later.

When I checked in the opentrons app, it also had the same timeout error as on the windows machine.

Migrated from internal repository. Originally created by Mark Huber on Sep 23, 2022 at 12:22 PM PDT.
EvanKirshenbaum commented 8 months ago

here's the error from my ubuntu machine, doesn't look like it has anything to do with our issue of the opentrons making a request back to the laptop though:

baxter@baxter-HP-EliteBook-840-G5:~/thylacine/mpam/tools$ python3 cs.py joey --pipettor ot2 --ot-ip 192.168.1.113 --ot-config ../inputs/ot2.json --ot-reagents ../inputs/reagents.json --clock-speed=100ms --fragments 20 --shuttles=2 --cycles=2
Platform option 'devices.bilby.PlatformTask' requires 'pyglider' module, ignoring
   INFO|opentrons|Launching listener
  ERROR|base_events|unhandled exception during asyncio.run() shutdown
task: <Task finished name='Task-1' coro=<_run_app() done, defined at /home/baxter/.local/lib/python3.10/site-packages/aiohttp/web.py:289> exception=RuntimeError('set_wakeup_fd only works in main thread of the main interpreter')>
Traceback (most recent call last):
  File "/usr/lib/python3.10/asyncio/unix_events.py", line 105, in add_signal_handler
    signal.set_wakeup_fd(self._csock.fileno())
ValueError: set_wakeup_fd only works in main thread of the main interpreter

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/baxter/.local/lib/python3.10/site-packages/aiohttp/web.py", line 516, in run_app
    loop.run_until_complete(main_task)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/home/baxter/.local/lib/python3.10/site-packages/aiohttp/web.py", line 323, in _run_app
    await runner.setup()
  File "/home/baxter/.local/lib/python3.10/site-packages/aiohttp/web_runner.py", line 273, in setup
    loop.add_signal_handler(signal.SIGINT, _raise_graceful_exit)
  File "/usr/lib/python3.10/asyncio/unix_events.py", line 107, in add_signal_handler
    raise RuntimeError(str(exc))
RuntimeError: set_wakeup_fd only works in main thread of the main interpreter
   INFO|opentrons|Temp protocol file is /tmp/protocol_ioh9vdso.py
Exception in thread OT-2 listener:
Traceback (most recent call last):
  File "/usr/lib/python3.10/asyncio/unix_events.py", line 105, in add_signal_handler
    signal.set_wakeup_fd(self._csock.fileno())
ValueError: set_wakeup_fd only works in main thread of the main interpreter

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/home/baxter/thylacine/mpam/src/devices/opentrons.py", line 266, in run
    web.run_app(app,
  File "/home/baxter/.local/lib/python3.10/site-packages/aiohttp/web.py", line 516, in run_app
    loop.run_until_complete(main_task)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/home/baxter/.local/lib/python3.10/site-packages/aiohttp/web.py", line 323, in _run_app
    await runner.setup()
  File "/home/baxter/.local/lib/python3.10/site-packages/aiohttp/web_runner.py", line 273, in setup
    loop.add_signal_handler(signal.SIGINT, _raise_graceful_exit)
  File "/usr/lib/python3.10/asyncio/unix_events.py", line 107, in add_signal_handler
    raise RuntimeError(str(exc))
RuntimeError: set_wakeup_fd only works in main thread of the main interpreter
Traceback (most recent call last):
  File "/home/baxter/thylacine/mpam/tools/cs.py", line 16, in <module>
    exerciser.parse_args_and_run()
  File "/home/baxter/thylacine/mpam/src/mpam/exerciser.py", line 420, in parse_args_and_run
    self.run_task(task, ns, board=board)
  File "/home/baxter/thylacine/mpam/src/mpam/exerciser.py", line 798, in run_task
    super().run_task(self.task, args, board=board)
  File "/home/baxter/thylacine/mpam/src/mpam/exerciser.py", line 392, in run_task
    system.run_monitored(lambda _: do_run(),
  File "/home/baxter/thylacine/mpam/src/mpam/device.py", line 4990, in run_monitored
    monitor = BoardMonitor(self.board,
  File "/home/baxter/thylacine/mpam/src/mpam/monitor.py", line 958, in __init__
    self.legend = ReagentLegend(self)
  File "/home/baxter/thylacine/mpam/src/mpam/monitor.py", line 328, in __init__
    self.redraw()
  File "/home/baxter/thylacine/mpam/src/mpam/monitor.py", line 339, in redraw
    self.legend = self.board_monitor.figure.legend(handles=handles,
  File "/home/baxter/.local/lib/python3.10/site-packages/matplotlib/figure.py", line 1119, in legend
    l = mlegend.Legend(self, handles, labels, *extra_args,
  File "/home/baxter/.local/lib/python3.10/site-packages/matplotlib/_api/deprecation.py", line 454, in wrapper
    return func(*args, **kwargs)
  File "/home/baxter/.local/lib/python3.10/site-packages/matplotlib/legend.py", line 517, in __init__
    self._init_legend_box(handles, labels, markerfirst)
  File "/home/baxter/.local/lib/python3.10/site-packages/matplotlib/legend.py", line 792, in _init_legend_box
    len, np.array_split(handles_and_labels, self._ncols)):
  File "<__array_function__ internals>", line 180, in array_split
  File "/home/baxter/.local/lib/python3.10/site-packages/numpy/lib/shape_base.py", line 778, in array_split
    raise ValueError('number sections must be larger than 0.') from None
ValueError: number sections must be larger than 0.
   INFO|opentrons|Create Protocol result: {'data': {'id': '0c8f80ac-b692-445e-978c-b8e4c5a627cf', 'createdAt': '2022-09-23T18:16:27.049478+00:00', 'files': [{'name': 'protocol_ioh9vdso.py', 'role': 'main'}], 'protocolType': 'python', 'metadata': {'author': 'Evan Kirshenbaum < evan.kirshenbaum@hp.com>', 'apiLevel': '2.11', 'protocolName': 'Joey Peripheral Protocol', 'description': 'A generic looping client to use the OT-2 as a peripheral for Joey'}, 'analyses': [{'id': '60db5d76-ba5b-4b43-8cfe-c9276707f18c', 'status': 'pending'}]}}
   INFO|opentrons|Created protocol "0c8f80ac-b692-445e-978c-b8e4c5a627cf".
   INFO|opentrons|Created session "a155b65f-3976-459a-a674-a5dbb8d9bde7".
   INFO|opentrons|Started run.
Migrated from internal repository. Originally created by Mark Huber on Sep 23, 2022 at 5:16 PM PDT.
EvanKirshenbaum commented 8 months ago

I could ssh into the opentrons and try to ping my laptop and see if i get a response

Migrated from internal repository. Originally created by Mark Huber on Sep 23, 2022 at 5:16 PM PDT.
EvanKirshenbaum commented 8 months ago

I will connect the opentrons to its own wifi network router, just to rule out any funny things from the wifi we have in the lab (it's not hp corp wifi)

Migrated from internal repository. Originally created by Mark Huber on Sep 27, 2022 at 3:57 PM PDT.
EvanKirshenbaum commented 8 months ago

I tried to do a GET request with postman to port 80 but did not get a response. i checked the logs on the opentrons:

image

doesn't seem to find the pipettors. But somewhere in the logs earlier I was able to see that it found the correct pipettor. I don't think this is the problem though

Migrated from internal repository. Originally created by Mark Huber on Sep 28, 2022 at 3:58 PM PDT.
EvanKirshenbaum commented 8 months ago

tried this:

import random
import requests
IP = "192.168.1.113"
PORT = 31950

headers = {"Opentrons-Version": "2"}

def make_url(cmd: str) -> str:
    return f"http://{IP}:{PORT}/{cmd}"

def post_request(cmd: str, **kwd_args): 
    url = make_url(cmd)
    return requests.post(url=url, headers=headers,
                        **kwd_args).json()

pname = f"protocol-{random.randint(0,1000000)}"
response = post_request("protocols",
        files=[("protocolFile", (pname, open("my_protocol.py", "rb"))),
            ]
        )
print(response)

but got: {'errors': [{'id': 'InvalidRequest', 'title': 'Invalid Request', 'detail': 'field required', 'source': {'pointer': '/files'}}]}

Migrated from internal repository. Originally created by Mark Huber on Oct 03, 2022 at 1:13 PM PDT.
EvanKirshenbaum commented 8 months ago
Oct 03 20:30:38 OT2CEP20210309B13 opentrons-api[248]: Updating instrument model cache
Oct 03 20:30:38 OT2CEP20210309B13 opentrons-api[248]: Skipping configuration on LEFT
Oct 03 20:30:38 OT2CEP20210309B13 opentrons-api[248]: Skipping configuration on RIGHT
Oct 03 20:30:38 OT2CEP20210309B13 opentrons-api[248]: Instruments found: {<Mount.LEFT: 1>: None, <Mount.RIGHT: 2>: None}
Oct 03 20:30:38 OT2CEP20210309B13 opentrons-api[248]: Error response: 422 - LegacyError - 
Oct 03 20:30:42 OT2CEP20210309B13 opentrons-api[248]: Caught exception: {'message': 'Task was destroyed but it is pending!', 'task': <Task pending coro=<QueueWorker._run_commands() done, defined at usr/lib/python3.7/site-packages/opentrons/protocol_engine/execution/queue_worker.py:69> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x6fec5590>()]>>}

Hmm, looks like it really can't find the pipettors

Migrated from internal repository. Originally created by Mark Huber on Oct 03, 2022 at 1:32 PM PDT.
EvanKirshenbaum commented 8 months ago

same messages when I run cs.py:

Oct 03 20:39:43 OT2CEP20210309B13 opentrons-api[248]: Updating instrument model cache
Oct 03 20:39:43 OT2CEP20210309B13 opentrons-api[248]: Skipping configuration on LEFT
Oct 03 20:39:43 OT2CEP20210309B13 opentrons-api[248]: Skipping configuration on RIGHT
Oct 03 20:39:43 OT2CEP20210309B13 opentrons-api[248]: Instruments found: {<Mount.LEFT: 1>: None, <Mount.RIGHT: 2>: None}
Oct 03 20:39:44 OT2CEP20210309B13 opentrons-api[248]: Created protocol "47cd8146-0cb6-4c02-af3b-04b75502e5b9" and started analysis "89e1f9a8-cdb5-4203-a4e0-2298eb1d8953".
Oct 03 20:39:44 OT2CEP20210309B13 opentrons-api[248]: Created protocol run "979de54a-b516-4150-b848-193d228a9ce2" from protocol "47cd8146-0cb6-4c02-af3b-04b75502e5b9".
Oct 03 20:39:44 OT2CEP20210309B13 opentrons-api[248]: Starting run "979de54a-b516-4150-b848-193d228a9ce2".
Oct 03 20:39:45 OT2CEP20210309B13 opentrons-api[248]: Homing axes XYZAXZA in sequence ['ZA', 'X', 'Y']
Oct 03 20:39:46 OT2CEP20210309B13 opentrons-api[248]: Recovering from halt
Oct 03 20:39:46 OT2CEP20210309B13 opentrons-api[248]: Updating instrument model cache
Oct 03 20:39:46 OT2CEP20210309B13 opentrons-api[248]: Skipping configuration on LEFT
Oct 03 20:39:46 OT2CEP20210309B13 opentrons-api[248]: Skipping configuration on RIGHT
Oct 03 20:39:46 OT2CEP20210309B13 opentrons-api[248]: Instruments found: {<Mount.LEFT: 1>: None, <Mount.RIGHT: 2>: None}
Oct 03 20:39:46 OT2CEP20210309B13 opentrons-api[248]: Completed analysis "89e1f9a8-cdb5-4203-a4e0-2298eb1d8953".
Oct 03 20:39:51 OT2CEP20210309B13 opentrons-api[248]: Homing axes C in sequence ['C']

Pipettor can be found at the start of the opentrons though:

-- Logs begin at Fri 2018-06-22 11:11:49 UTC, end at Mon 2022-10-03 20:46:25 UTC. --
Oct 03 20:45:30 OT2CEP20210309B13 opentrons-api[245]: Application startup complete.
Oct 03 20:45:30 OT2CEP20210309B13 opentrons-api[245]: Uvicorn running on unix socket /run/aiohttp.sock (Press CTRL+C to quit)
Oct 03 20:45:30 OT2CEP20210309B13 opentrons-api[245]: Stopped opentrons-gpio-setup.
Oct 03 20:45:30 OT2CEP20210309B13 opentrons-api[245]: API server version: 5.0.2
Oct 03 20:45:30 OT2CEP20210309B13 opentrons-api[245]: Robot Name: OT2CEP20210309B13
Oct 03 20:45:31 OT2CEP20210309B13 opentrons-api[245]: Connecting to motor controller at port /dev/ttyAMA0
Oct 03 20:45:31 OT2CEP20210309B13 opentrons-api[245]: Registering Central Routing Board Revision GPIOs
Oct 03 20:45:31 OT2CEP20210309B13 opentrons-api[245]: Configuring GPIOs by central routing roard revision
Oct 03 20:45:31 OT2CEP20210309B13 opentrons-api[245]: Failed to detect central routing board revision gpio pins, defaulting to 2.1 (OG)
Oct 03 20:45:31 OT2CEP20210309B13 opentrons-api[245]: Setting up GPIOs
Oct 03 20:45:31 OT2CEP20210309B13 opentrons-api[245]: Connecting to smoothie at port /dev/ttyAMA0
Oct 03 20:45:33 OT2CEP20210309B13 opentrons-api[245]: Updating instrument model cache
Oct 03 20:45:33 OT2CEP20210309B13 opentrons-api[245]: smoothie: error:no L instrument found
Oct 03 20:45:34 OT2CEP20210309B13 opentrons-api[245]: smoothie: error:no L instrument found
Oct 03 20:45:34 OT2CEP20210309B13 opentrons-api[245]: Error response: 503 - LegacyError - 
Oct 03 20:45:35 OT2CEP20210309B13 opentrons-api[245]: Skipping configuration on LEFT
Oct 03 20:45:35 OT2CEP20210309B13 opentrons-api[245]: loaded: p20_single_v2.2, pipette offset: [0.9000000000000004, -4.600000000000001, -0.09999999999998721]
Oct 03 20:45:35 OT2CEP20210309B13 opentrons-api[245]: Doing full configuration on RIGHT
Oct 03 20:45:36 OT2CEP20210309B13 opentrons-api[245]: Instruments found: {<Mount.LEFT: 1>: None, <Mount.RIGHT: 2>: <Pipette: P20 Single-Channel GEN2 1928726000>}
Oct 03 20:45:36 OT2CEP20210309B13 opentrons-api[245]: Updating the window switch status: open
Oct 03 20:45:36 OT2CEP20210309B13 opentrons-api[245]: Homing Z axes
Oct 03 20:45:36 OT2CEP20210309B13 opentrons-api[245]: Homing axes ZA in sequence ['ZA']
Oct 03 20:45:38 OT2CEP20210309B13 opentrons-api[245]: Opentrons hardware API initialized
Migrated from internal repository. Originally created by Mark Huber on Oct 03, 2022 at 1:47 PM PDT.
EvanKirshenbaum commented 8 months ago

image

Migrated from internal repository. Originally created by Mark Huber on Oct 03, 2022 at 2:01 PM PDT.
EvanKirshenbaum commented 8 months ago
Oct 03 21:19:30 OT2CEP20210309B13 opentrons-api[245]: nmcli --mode multiline --escape no --terse --fields GENERAL.HWADDR,IP4.ADDRESS,IP4.GATEWAY,GENERAL.TYPE,GENERAL.STATE dev show eth0: stderr=(process:5026): libnm-WARNING **: 21:19:30.766: no object known for /org/freedesktop/NetworkManager/AccessPoint/383
Oct 03 21:19:49 OT2CEP20210309B13 opentrons-api[245]: Updating instrument model cache
Oct 03 21:19:49 OT2CEP20210309B13 opentrons-api[245]: Skipping configuration on LEFT
Oct 03 21:19:49 OT2CEP20210309B13 opentrons-api[245]: Skipping configuration on RIGHT
Oct 03 21:19:49 OT2CEP20210309B13 opentrons-api[245]: Instruments found: {<Mount.LEFT: 1>: None, <Mount.RIGHT: 2>: None}
Oct 03 21:19:49 OT2CEP20210309B13 opentrons-api[245]: Created protocol "7d9e6938-0d96-4ac2-ba30-dd94441c7aa4" and started analysis "33626d26-769f-4ab2-b856-8d8238231b01".
Oct 03 21:19:51 OT2CEP20210309B13 opentrons-api[245]: nmcli --mode multiline --escape no --terse --fields GENERAL.HWADDR,IP4.ADDRESS,IP4.GATEWAY,GENERAL.TYPE,GENERAL.STATE dev show eth0: stderr=(process:5096): GLib-GIO-WARNING **: 21:19:50.866: gdbusobjectmanagerclient.c:1589: Processing InterfaceRemoved signal for path /org/freedesktop/NetworkManager/AccessPoint/393 but no object proxy exists

                                                      (process:5096): GLib-GIO-WARNING **: 21:19:50.868: gdbusobjectmanagerclient.c:1589: Processing InterfaceRemoved signal for path /org/freedesktop/NetworkManager/AccessPoint/394 but no object proxy exists

                                                      (process:5096): GLib-GIO-WARNING **: 21:19:50.872: gdbusobjectmanagerclient.c:1589: Processing InterfaceRemoved signal for path /org/freedesktop/NetworkManager/AccessPoint/395 but no object proxy exists

                                                      (process:5096): GLib-GIO-WARNING **: 21:19:50.875: gdbusobjectmanagerclient.c:1589: Processing InterfaceRemoved signal for path /org/freedesktop/NetworkManager/AccessPoint/396 but no object proxy exists

                                                      (process:5096): GLib-GIO-WARNING **: 21:19:50.876: gdbusobjectmanagerclient.c:1589: Processing InterfaceRemoved signal for path /org/freedesktop/NetworkManager/AccessPoint/397 but no object proxy exists

                                                      (process:5096): GLib-GIO-WARNING **: 21:19:50.883: gdbusobjectmanagerclient.c:1589: Processing InterfaceRemoved signal for path /org/freedesktop/NetworkManager/AccessPoint/398 but no object proxy exists

                                                      (process:5096): GLib-GIO-WARNING **: 21:19:50.884: gdbusobjectmanagerclient.c:1589: Processing InterfaceRemoved signal for path /org/freedesktop/NetworkManager/AccessPoint/399 but no object proxy exists

                                                      (process:5096): GLib-GIO-WARNING **: 21:19:50.889: gdbusobjectmanagerclient.c:1589: Processing InterfaceRemoved signal for path /org/freedesktop/NetworkManager/AccessPoint/386 but no object proxy exists

                                                      (process:5096): GLib-GIO-WARNING **: 21:19:50.890: gdbusobjectmanagerclient.c:1589: Processing InterfaceRemoved signal for path /org/freedesktop/NetworkManager/AccessPoint/388 but no object proxy exists
Oct 03 21:19:52 OT2CEP20210309B13 opentrons-api[245]: Recovering from halt
Oct 03 21:19:52 OT2CEP20210309B13 opentrons-api[245]: Updating instrument model cache
Oct 03 21:19:52 OT2CEP20210309B13 opentrons-api[245]: Skipping configuration on LEFT
Oct 03 21:19:52 OT2CEP20210309B13 opentrons-api[245]: Skipping configuration on RIGHT
Oct 03 21:19:52 OT2CEP20210309B13 opentrons-api[245]: Instruments found: {<Mount.LEFT: 1>: None, <Mount.RIGHT: 2>: None}
Oct 03 21:19:52 OT2CEP20210309B13 opentrons-api[245]: Completed analysis "33626d26-769f-4ab2-b856-8d8238231b01".
Oct 03 21:19:52 OT2CEP20210309B13 opentrons-api[245]: Recovering from halt
Oct 03 21:19:52 OT2CEP20210309B13 opentrons-api[245]: Updating instrument model cache
Oct 03 21:19:52 OT2CEP20210309B13 opentrons-api[245]: smoothie: error:no L instrument found
Oct 03 21:19:52 OT2CEP20210309B13 opentrons-api[245]: smoothie: error:no L instrument found
Oct 03 21:19:54 OT2CEP20210309B13 opentrons-api[245]: Skipping configuration on LEFT
Oct 03 21:19:54 OT2CEP20210309B13 opentrons-api[245]: loaded: p20_single_v2.2, pipette offset: [0.9000000000000004, -4.600000000000001, -0.09999999999998721]
Oct 03 21:19:54 OT2CEP20210309B13 opentrons-api[245]: Doing full configuration on RIGHT
Oct 03 21:19:54 OT2CEP20210309B13 opentrons-api[245]: Instruments found: {<Mount.LEFT: 1>: None, <Mount.RIGHT: 2>: <Pipette: P20 Single-Channel GEN2 1937177008>}
Oct 03 21:19:54 OT2CEP20210309B13 opentrons-api[245]: Created protocol run "a762da90-478b-44fd-8e50-5368fc708759" from protocol "7d9e6938-0d96-4ac2-ba30-dd94441c7aa4".
Oct 03 21:19:54 OT2CEP20210309B13 opentrons-api[245]: Starting run "a762da90-478b-44fd-8e50-5368fc708759".
Oct 03 21:19:54 OT2CEP20210309B13 opentrons-api[245]: Homing axes XYZAXZA in sequence ['ZA', 'X', 'Y']
Oct 03 21:20:01 OT2CEP20210309B13 opentrons-api[245]: Homing axes C in sequence ['C']
Migrated from internal repository. Originally created by Mark Huber on Oct 03, 2022 at 2:21 PM PDT.
EvanKirshenbaum commented 8 months ago

-- Logs begin at Fri 2018-06-22 11:11:49 UTC, end at Mon 2022-10-03 21:25:16 UTC. -- Oct 03 21:24:20 OT2CEP20210309B13 opentrons-api[247]: Application startup complete. Oct 03 21:24:20 OT2CEP20210309B13 opentrons-api[247]: Uvicorn running on unix socket /run/aiohttp.sock (Press CTRL+C to quit) Oct 03 21:24:20 OT2CEP20210309B13 opentrons-api[247]: Stopped opentrons-gpio-setup. Oct 03 21:24:20 OT2CEP20210309B13 opentrons-api[247]: API server version: 5.0.2 Oct 03 21:24:20 OT2CEP20210309B13 opentrons-api[247]: Robot Name: OT2CEP20210309B13 Oct 03 21:24:21 OT2CEP20210309B13 opentrons-api[247]: Connecting to motor controller at port /dev/ttyAMA0 Oct 03 21:24:21 OT2CEP20210309B13 opentrons-api[247]: Registering Central Routing Board Revision GPIOs Oct 03 21:24:21 OT2CEP20210309B13 opentrons-api[247]: Configuring GPIOs by central routing roard revision Oct 03 21:24:21 OT2CEP20210309B13 opentrons-api[247]: Failed to detect central routing board revision gpio pins, defaulting to 2.1 (OG) Oct 03 21:24:21 OT2CEP20210309B13 opentrons-api[247]: Setting up GPIOs Oct 03 21:24:21 OT2CEP20210309B13 opentrons-api[247]: Error response: 503 - LegacyError - Oct 03 21:24:21 OT2CEP20210309B13 opentrons-api[247]: Connecting to smoothie at port /dev/ttyAMA0 Oct 03 21:24:23 OT2CEP20210309B13 opentrons-api[247]: Updating instrument model cache Oct 03 21:24:23 OT2CEP20210309B13 opentrons-api[247]: smoothie: error:no L instrument found Oct 03 21:24:24 OT2CEP20210309B13 opentrons-api[247]: smoothie: error:no L instrument found Oct 03 21:24:25 OT2CEP20210309B13 opentrons-api[247]: Skipping configuration on LEFT Oct 03 21:24:25 OT2CEP20210309B13 opentrons-api[247]: loaded: p20_single_v2.2, pipette offset: [0.9000000000000004, -4.600000000000001, -0.09999999999998721] Oct 03 21:24:25 OT2CEP20210309B13 opentrons-api[247]: Doing full configuration on RIGHT Oct 03 21:24:25 OT2CEP20210309B13 opentrons-api[247]: Instruments found: {<Mount.LEFT: 1>: None, <Mount.RIGHT: 2>: <Pipette: P20 Single-Channel GEN2 1928558288>} Oct 03 21:24:26 OT2CEP20210309B13 opentrons-api[247]: Updating the window switch status: closed Oct 03 21:24:26 OT2CEP20210309B13 opentrons-api[247]: Updating the window switch status: closed Oct 03 21:24:26 OT2CEP20210309B13 opentrons-api[247]: Homing Z axes Oct 03 21:24:26 OT2CEP20210309B13 opentrons-api[247]: Homing axes ZA in sequence ['ZA'] Oct 03 21:24:26 OT2CEP20210309B13 opentrons-api[247]: Error response: 503 - LegacyError - Oct 03 21:24:26 OT2CEP20210309B13 opentrons-api[247]: Error response: 503 - LegacyError - Oct 03 21:24:28 OT2CEP20210309B13 opentrons-api[247]: Opentrons hardware API initialized Oct 03 21:24:34 OT2CEP20210309B13 opentrons-api[247]: Error response: 404 - LegacyError - Oct 03 21:25:01 OT2CEP20210309B13 opentrons-api[247]: Storing protocols in /tmp/opentrons-protocols Oct 03 21:25:01 OT2CEP20210309B13 opentrons-api[247]: Updating instrument model cache Oct 03 21:25:01 OT2CEP20210309B13 opentrons-api[247]: Skipping configuration on LEFT Oct 03 21:25:01 OT2CEP20210309B13 opentrons-api[247]: Skipping configuration on RIGHT Oct 03 21:25:01 OT2CEP20210309B13 opentrons-api[247]: Instruments found: {<Mount.LEFT: 1>: None, <Mount.RIGHT: 2>: None} Oct 03 21:25:01 OT2CEP20210309B13 opentrons-api[247]: Created protocol "04a72a3a-e88e-47bf-b658-fe3a3421cdfe" and started analysis "fe252d72-754e-47f7-bca1-1db9e6bc60d6". Oct 03 21:25:02 OT2CEP20210309B13 opentrons-api[247]: Created protocol run "8ef82bcf-fdfc-4c33-ac6b-3913881373ea" from protocol "04a72a3a-e88e-47bf-b658-fe3a3421cdfe". Oct 03 21:25:02 OT2CEP20210309B13 opentrons-api[247]: Starting run "8ef82bcf-fdfc-4c33-ac6b-3913881373ea". Oct 03 21:25:02 OT2CEP20210309B13 opentrons-api[247]: Homing axes XYZAXZA in sequence ['ZA', 'X', 'Y'] Oct 03 21:25:03 OT2CEP20210309B13 opentrons-api[247]: Recovering from halt Oct 03 21:25:03 OT2CEP20210309B13 opentrons-api[247]: Updating instrument model cache Oct 03 21:25:03 OT2CEP20210309B13 opentrons-api[247]: Skipping configuration on LEFT Oct 03 21:25:03 OT2CEP20210309B13 opentrons-api[247]: Skipping configuration on RIGHT Oct 03 21:25:03 OT2CEP20210309B13 opentrons-api[247]: Instruments found: {<Mount.LEFT: 1>: None, <Mount.RIGHT: 2>: None} Oct 03 21:25:03 OT2CEP20210309B13 opentrons-api[247]: Completed analysis "fe252d72-754e-47f7-bca1-1db9e6bc60d6". Oct 03 21:25:09 OT2CEP20210309B13 opentrons-api[247]: Homing axes C in sequence ['C']

Migrated from internal repository. Originally created by Mark Huber on Oct 03, 2022 at 2:25 PM PDT.
EvanKirshenbaum commented 8 months ago

unfortunately, factory resetting doesn't actually factory reset the robot, it just deletes scripts (mb does other things). I selected all options and then clicked restart and it just loaded for about 3min and then was up and alive again. Reinstalling the current installation (also an option from the UI) is faster, so i don't think it did that.

image

Will try to attach the other pipettor and then see

Migrated from internal repository. Originally created by Mark Huber on Oct 03, 2022 at 3:34 PM PDT.
EvanKirshenbaum commented 8 months ago

switched the current pipettor from right to left and added another one. Recalibrated them:

image

logs don't say anything in addition but the comments from self.message actually show up in the ui:

image

So we do know we get to here

and then initializing the board fails. Will add some more logging info to see where it hangs, might be spec['labware'] here

Migrated from internal repository. Originally created by Mark Huber on Oct 03, 2022 at 5:18 PM PDT.
EvanKirshenbaum commented 8 months ago

This is really weird, I did a checkout of opentrons_support.py, so I brought it back to its original form (before we were only adding more comments). Then I ran it and it now does not get to the robot call anymore. Logs don't really show anything else than before.

Migrated from internal repository. Originally created by Mark Huber on Oct 03, 2022 at 5:27 PM PDT.
EvanKirshenbaum commented 8 months ago

after a couple of reboots I was able to get it to work again. I tried printing log messages after the "Creating the robot..." but it wouldn't print. So I commented that line out and instead added other log messages:

image

my code looks like this: image

so it seems like the first log message: "robot init" gets printed out twice because I'm assuming try_remote is None, so I guess the post request is what fails.

Now I'm getting a bunch of log messages

image

this seems to fail

Migrated from internal repository. Originally created by Mark Huber on Oct 03, 2022 at 6:29 PM PDT.
EvanKirshenbaum commented 8 months ago

Tried adding a bunch of log messages and exceptions according to the docs but now it seems to be stuck in this post request exception, it just sits in the exception and doesn't time out. Will try more things on a different day.

Migrated from internal repository. Originally created by Mark Huber on Oct 03, 2022 at 7:19 PM PDT.
EvanKirshenbaum commented 8 months ago

I think the ip address is wrong? It seems to post the call to 192.168.223.1, the opentrons' ip is 192.168.1.113, mine is 192.168.1.117.

@EvanKirshenbaum I'm not sure I understand where the ip is coming from. It seems to come from config, here which has an endpoint key. config gets passed in the looping_protocol when initializing the robot and created here. And the file seems to be config.json but I can't find it anywhere. Where does this file come from?

Migrated from internal repository. Originally created by Mark Huber on Oct 07, 2022 at 10:55 AM PDT.
EvanKirshenbaum commented 8 months ago

Okay, I can answer the last question easily, if not satisfactorily. There's a reason the code in that file keeps checking for the presence of a variable named "COMBINED_FILES_KLUDGE". In Opentrons version 4, I could send files along with the protocol file, and a cobbled-together config.json was one of those files (along with opentrons_support.py and schedule_xfers.py).

With Opentrons version 5, that was no longer possible, so the protocol that's actually uploaded is (more-or-less) a concatenation of the three Python files, an assignment of what would have been loaded from config.json to config, and setting COMBINED_FILES_KLUDGE to True. So when the code in run() checks to see whether config is None, it isn't because it's actually already been assigned. You just can't see it in the file yet.

Now you understand why I called the flag variable a kludge.

Migrated from internal repository. Originally created by @EvanKirshenbaum on Oct 07, 2022 at 11:07 AM PDT.
EvanKirshenbaum commented 8 months ago

lmao, that is the reason, I hardcoded it as my ip address and now it's doing something 192.168.1.117

Migrated from internal repository. Originally created by Mark Huber on Oct 07, 2022 at 11:08 AM PDT.
EvanKirshenbaum commented 8 months ago

so whatever happens, it uses the wrong ip address, it's working now

Migrated from internal repository. Originally created by Mark Huber on Oct 07, 2022 at 11:10 AM PDT.
EvanKirshenbaum commented 8 months ago

How long does the looping protocol run ?

Migrated from internal repository. Originally created by Mark Huber on Oct 07, 2022 at 11:10 AM PDT.
EvanKirshenbaum commented 8 months ago

The callback endpoint can be specified explicitly in ot2.json. If it isn't (and it almost certainly shouldn't be), it's looked up dynamically:

        if not "endpoint" in config:
            host_name = socket.gethostname()
            ip = socket.gethostbyname(host_name)
            config["endpoint"] = {
                    "ip": ip,
                    "port": listener_port
                }
Migrated from internal repository. Originally created by @EvanKirshenbaum on Oct 07, 2022 at 11:12 AM PDT.
EvanKirshenbaum commented 8 months ago

It runs until the local protocol tells it it's done. In the case of cs, this will depend on the number of products you're trying to make and the number of cycles each thermocycle goes through. If you just want to test things, I'd set the first to 1 or 2 (that's the number immediately after the joey argument) and use --cycles 1 or --cycles 2. You can also set --clock-speed to something reasonably fast, like 100ms.

What arguments are you using?

Migrated from internal repository. Originally created by @EvanKirshenbaum on Oct 07, 2022 at 11:15 AM PDT.
EvanKirshenbaum commented 8 months ago

And do I understand by your comment that the robot is actually moving now? Congratulations!

Migrated from internal repository. Originally created by @EvanKirshenbaum on Oct 07, 2022 at 11:15 AM PDT.
EvanKirshenbaum commented 8 months ago

this part seems to be correct: this is from the opentrons: image

I am hardcoding it to my laptops ip address though:

image

Migrated from internal repository. Originally created by Mark Huber on Oct 07, 2022 at 11:17 AM PDT.