Closed humphd closed 2 years ago
We're discussing doing this on the Thursday before study week, and then fix things over the break.
According to https://docs.github.com/en/repositories/creating-and-managing-repositories/transferring-a-repository, this should mostly work:
People will need to update their upstream
remote with git remote set-url origin new_url
.
I think people with access will still retain it too, though, we might need to tweak permissions and teams. I'm not sure what's going to happen with Projects, especially those that are attached to the Seneca-CDOT org.
I looked to see what we could create. The following team names (e.g., XXXX/telescope) are (currently) available:
Other ideas?
I looked to see what we could create. The following team names (e.g., XXXX/telescope) are (currently) available:
- SenecaOpen
- OpenSourceCourse
- LearningOpen
- PlanetSeneca
- SenecaCodes
- code4seneca
- osd-dps
- SenecaBlogs
Other ideas?
I love PlanetSeneca
name
- SenecaOpen
- OpenSourceCourse
- LearningOpen
- PlanetSeneca
- SenecaCodes
- code4seneca
- osd-dps
- SenecaBlogs
I don't really like having Seneca in name, OpenSourceCourse
sounds so generic to be available.. Voting for that
I really want to stick with Seneca-CDOT, but I'll go with SenecaOpen if I have to choose.
@humphd Why doesn't Docker Hub work for us? I think they have a free plan for education/open-source.
We could try. What I don't love about is the fact that it would be tied to my account vs. a team (blocked on me to do any fixes), and that we get rate limited "200 image pulls per 6 hours." Without good caching, this is going to mean that CI builds, deployments, local dev, etc. will just randomly fail.
I don't know, maybe I'm wrong and we should try it first.
We could also try https://www.docker.com/blog/expanded-support-for-open-source-software-projects/
This seems like the easier solution, but is easier better?
Do we meet the other two requirements, or should issues be made to address them?
We use an OSI approved license, https://opensource.org/licenses/BSD-2-Clause.
Our images would be under the same license. I think we are good for all of this.
Docker Open Source Community Application Thank you for completing the Docker Open Source Survey. We will review your application, and get back to you as quickly as we can. We have received a large of applications, so please bear with us, as we go through the review process
OK, I filled out the form. I'll see what they say.
Another follow-up. I had a good meeting today with Chris Tyler. He and I agreed that if Docker says 'no', or it doesn't make sense to use Docker Hub, we'll look at hosting our own Docker Registry in CDOT. He has a machine with 1/2 TB SSD that we could use pretty quickly, and a lot more NAS storage that we could expand into later if needed.
Running our own registry is well documented at https://docs.docker.com/registry/deploying/. Essentially, we need to run the a registry Docker container, and configure it for proper storage volumes, access, and security.
We might be able to do this using our existing infrastructure, which would get us the TLS and security for free. If not, we'll have to do Let's Encrypt, DNS, etc ourselves.
Oh thank god, we don't have to move out of Seneca-CDOT, I was having an identity crisis over what name Telescope's org was gonna get.
This is interesting, re: docker build
:
By default, the build cache is based on results from previous builds on the machine on which you are building. The --cache-from option also allows you to use a build-cache that’s distributed through an image registry refer to the specifying external cache sources section in the docker build command reference.
With our own registry, we could re-use the build cache in CI or on development machines too.
I've reached out to Chris to start the process of creating our own Docker Registry.
Those that are interested in being involved should begin by reading https://docs.docker.com/registry/deploying/. Once I talk to him and get more details, I'll file specific issues that we can start fixing.
I've just finished a meeting regarding this, and here's the update:
TODO items for this work:
docker.cdot.systems
domain with Let's Encrypt, and have them auto-update. Ideally we can steal/learn from what @manekenpix has done on Telescope (I don't know all the steps)push
to the registry.I'm sure there's more to do, but that's the list I have right now. Let me know if and how you want to be involved.
I've just finished a meeting regarding this, and here's the update:
* Chris has to go to campus to move some SSDs around so we can take over one of his servers. * He's going to create a VM that we can use to host the Docker Registry, and we'll initially have access to a few hundred Gigs of storage. * Later, he's hoping to move this into a rack with faster network and access to a larger set of network storage, but that won't happen this term. * He hopes to have this set up done by Friday afternoon * We'll run the registry at https://docker.cdot.systems
TODO items for this work:
* we need to get SSL certs setup for the `docker.cdot.systems` domain with Let's Encrypt, and have them auto-update. Ideally we can steal/learn from what @manekenpix has done on Telescope (I don't know all the steps) * we should probably run this behind nginx, which is documented at https://docs.docker.com/registry/recipes/nginx/#setting-things-up * setup Basic Auth username/password pairs for any service (e.g., GitHub Actions) or person who needs to `push` to the registry. * we should see if we can set-up pull through mirroring https://docs.docker.com/registry/recipes/mirror/
I'm sure there's more to do, but that's the list I have right now. Let me know if and how you want to be involved.
I would like to be involved. I don't have many issues assigned right now for anticipation of this, so load me up with a decent workload for over the next couple of weeks if possible.
What will be the network bandwidth of the current system?
Just wondering, I read a tutorial, about self-hosting docker registry. Is it something similar to this https://www.digitalocean.com/community/tutorials/how-to-set-up-a-private-docker-registry-on-ubuntu-18-04 we gonna do?
What will be the network bandwidth of the current system?
Eventually 10 Gigabit between the docker registry and staging/prod, but not until we move it into the server rack later in the term. For now, I don't know the answer, but should be pretty fast.
Just wondering, I read a tutorial, about self-hosting docker registry. Is it something similar to this https://www.digitalocean.com/community/tutorials/how-to-set-up-a-private-docker-registry-on-ubuntu-18-04 we gonna do?
Yes, very similar, but on Fedora vs. Ubuntu.
@TDDR, @rclee91, @Kevan-Y for the docker registry, here is what I think we need to do:
docker-compose.yml
created in https://github.com/Seneca-CDOT/docker.cdot.systems, to define our services (registry, nginx, outh2-proxy)@TDDR I've put your name on all of this, but we can add more people and should split this up into separate issues. I know @rclee91 and @Kevan-Y want to join in as well, so maybe they can speak up. I'm happy to take any of it you don't want and do reviews.
I would suggest trying to create all this for your local fork without SSL to learn how the proxy works (i.e., try to get it setup so you have to authenticate with GitHub on your local machine)
OK, server is setup, docker.cdot.systems created. Thank you @ctyler!
Now the fun begins.
This is exciting, I added my name to a task.
Assigned myself to some tasks too.
I think oauth2-proxy might be hard to use for automated login scenarios (e.g., pushing to the registry from CI). I think we should look at https://github.com/cesanta/docker_auth too, which should let us do it with tokens, see https://github.com/cesanta/docker_auth/blob/main/examples/reference.yml#L122-L163
Auth discussion happening in https://github.com/Seneca-CDOT/docker.cdot.systems/issues/3
@TDDR if there by chance you uploaded the PR by tonight, please also file following up issues for 2.9 and close this.
Following-up from https://github.com/Seneca-CDOT/telescope/issues/1743#issuecomment-1026157483, we need a solution for pushing docker images from CI to a registry. Our options seem to be:
I don't think there is another (free) option. Docker Hub is not going to work for a Team, and Amazon ECR will cost us. Are there other options I'm missing?
If we go with 1., which is probably the simpler of the two, we need to figure out what would break, and what has to be re-created. That includes:
I have no idea what else would break. If we do this, we'd have to plan it carefully and execute it in such a way that it wouldn't cause major downtime for the project/community.
Thoughts?