search

2i2c-org / utoronto-image

User image for the UToronto Hub
BSD 3-Clause "New" or "Revised" License
0 stars 4 forks source link

r2d-action broken? #20

Open sgibson91 opened 2 years ago

sgibson91 commented 2 years ago

Referencing PR #19

I haven't tracked down the actual error message yet since the logs from the r2d-action are very long and verbose.

Is there a known issue regarding not being able to push this image to quay.io from CI/CD?

cc: @yuvipanda @damianavila

sgibson91 commented 2 years ago

Ok, I found some logs by using the raw logs instead of the endless scrolling in the GitHub Actions UI

2022-03-08T12:07:08.7444889Z Traceback (most recent call last):
2022-03-08T12:07:08.7446166Z   File "/usr/lib/python3.9/site-packages/urllib3/response.py", line 438, in _error_catcher
2022-03-08T12:07:08.7469113Z     yield
2022-03-08T12:07:08.7469935Z   File "/usr/lib/python3.9/site-packages/urllib3/response.py", line 519, in read
2022-03-08T12:07:08.7474051Z     data = self._fp.read(amt) if not fp_closed else b""
2022-03-08T12:07:08.7474426Z   File "/usr/lib/python3.9/http/client.py", line 462, in read
2022-03-08T12:07:08.7499803Z     n = self.readinto(b)
2022-03-08T12:07:08.7500550Z   File "/usr/lib/python3.9/http/client.py", line 496, in readinto
2022-03-08T12:07:08.7500870Z     return self._readinto_chunked(b)
2022-03-08T12:07:08.7501183Z   File "/usr/lib/python3.9/http/client.py", line 591, in _readinto_chunked
2022-03-08T12:07:08.7501488Z     chunk_left = self._get_chunk_left()
2022-03-08T12:07:08.7501791Z   File "/usr/lib/python3.9/http/client.py", line 559, in _get_chunk_left
2022-03-08T12:07:08.7505064Z     chunk_left = self._read_next_chunk_size()
2022-03-08T12:07:08.7505433Z   File "/usr/lib/python3.9/http/client.py", line 519, in _read_next_chunk_size
2022-03-08T12:07:08.7505735Z     line = self.fp.readline(_MAXLINE + 1)
2022-03-08T12:07:08.7506032Z   File "/usr/lib/python3.9/socket.py", line 704, in readinto
2022-03-08T12:07:08.7510496Z     return self._sock.recv_into(b)
2022-03-08T12:07:08.7511385Z socket.timeout: timed out
2022-03-08T12:07:08.7511555Z 
2022-03-08T12:07:08.7511710Z During handling of the above exception, another exception occurred:
2022-03-08T12:07:08.7511913Z 
2022-03-08T12:07:08.7512026Z Traceback (most recent call last):
2022-03-08T12:07:08.7512523Z   File "/usr/bin/jupyter-repo2docker", line 8, in <module>
2022-03-08T12:07:08.7518867Z     sys.exit(main())
2022-03-08T12:07:08.7519368Z   File "/usr/lib/python3.9/site-packages/repo2docker/__main__.py", line 418, in main
2022-03-08T12:07:08.7525546Z     r2d.start()
2022-03-08T12:07:08.7526020Z   File "/usr/lib/python3.9/site-packages/repo2docker/app.py", line 834, in start
2022-03-08T12:07:08.7533417Z     self.push_image()
2022-03-08T12:07:08.7533890Z   File "/usr/lib/python3.9/site-packages/repo2docker/app.py", line 533, in push_image
2022-03-08T12:07:08.7534252Z     for chunk in client.push(self.output_image_spec):
2022-03-08T12:07:08.7534721Z   File "/usr/lib/python3.9/site-packages/docker/api/client.py", line 349, in _stream_helper
2022-03-08T12:07:08.7548670Z     data = reader.read(1)
2022-03-08T12:07:08.7549617Z   File "/usr/lib/python3.9/site-packages/urllib3/response.py", line 541, in read
2022-03-08T12:07:08.7550221Z     raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
2022-03-08T12:07:08.7561654Z   File "/usr/lib/python3.9/contextlib.py", line 137, in __exit__
2022-03-08T12:07:08.7562069Z     self.gen.throw(typ, value, traceback)
2022-03-08T12:07:08.7562847Z   File "/usr/lib/python3.9/site-packages/urllib3/response.py", line 443, in _error_catcher
2022-03-08T12:07:08.7563433Z     raise ReadTimeoutError(self._pool, None, "Read timed out.")
2022-03-08T12:07:08.7564649Z urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out.
damianavila commented 2 years ago

I faced some timeouts using r2d in the past and this is why I ended up pushing with docker on this one: https://github.com/2i2c-org/utoronto-image/pull/9.

I think we should be able to pass a timeout value now: https://github.com/jupyterhub/repo2docker/issues/711, but I am not sure if this was released and propagated to the action.

@yuvipanda, since you implemented the fix for the above r2d issue, do you have any more context here?

yuvipanda commented 2 years ago

@sgibson91 So #21, #22, #23 have fixed this, and I've submitted https://github.com/jupyterhub/repo2docker-action/pull/84 upstream as well. This is still temporary though, will need to finish up the 'real' fix which would be to pass a larger timeout to the repo2docker action itself.

sgibson91 commented 2 years ago

Thank you for your hard work on this @yuvipanda!