Closed PeterJCLaw closed 4 years ago
Status here:
Vagrantfile
(https://github.com/srobo/server-puppet/commit/b68e191846d110bd5e9cd94ce602428d43ff20df), however this obviously isn't ideal as it hits end-of-life in November 2019For the Python 2 packages issue, I'm currently exploring moving the things we need into virtualenvs (for example https://github.com/srobo/server-puppet/commit/883d382f144c39d488249dc166f264484a13fad3), which will allow us to install from PyPI (thankfully Python 2 is still present in the distro even though many packages aren't). I've no idea what happens in January (will PyPI stop serving request for Python 2 packages?), but that's likely a bridge we'll need to cross anyway.
For PHP, the only think I think this affects is the forums. https://hub.docker.com/r/bitnami/phpbb looks like it might help with that, though many questions remain about how we connect it to the rest of the stuff we need (extensions, MySQL, etc.)
It might be worth looking at Fedora Modularity. It's for exactly this situation (wanting a specific version of a language or framework that is older than the one shipped by default in Fedora). I imagine you can install the necessary version of PHP.
On Sun, 6 Oct 2019, 16:24 Peter Law, notifications@github.com wrote:
Status here:
- Fedora 29 works with a simple version update to the Vagrantfile ( srobo/server-puppet@b68e191 https://github.com/srobo/server-puppet/commit/b68e191846d110bd5e9cd94ce602428d43ff20df), however this obviously isn't ideal as it hits end-of-life in November 2019
- Fedora 30 has two main issues:
- lots of the Python 2 packages we want are missing (which is reasonable due to the impending end-of-life of Python 2; see #287 https://github.com/srobo/tasks/issues/287 for our general task around that)
- PHPBB 3.2.x requries PHP <= 7.2, but only PHP 7.3 is available in the repos
For the Python 2 packages issue, I'm currently exploring moving the things we need into virtualenvs (for example srobo/server-puppet@883d382 https://github.com/srobo/server-puppet/commit/883d382f144c39d488249dc166f264484a13fad3), which will allow us to install from PyPI (thankfully Python 2 is still present in the distro even though many packages aren't). I've no idea what happens in January (will PyPI stop serving request for Python 2 packages?), but that's likely a bridge we'll need to cross anyway.
For PHP, the only think I think this affects is the forums. https://hub.docker.com/r/bitnami/phpbb looks like it might help with that, though many questions remain about how we connect it to the rest of the stuff we need (extensions, MySQL, etc.)
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/srobo/tasks/issues/363?email_source=notifications&email_token=AAMUUFIZDUMFHVVVC3N57CDQNH7I7A5CNFSM4IUUZFZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAOMSJA#issuecomment-538757412, or mute the thread https://github.com/notifications/unsubscribe-auth/AAMUUFIHBGNMSH5RPGMZYITQNH7I7ANCNFSM4IUUZFZQ .
It might be worth looking at Fedora Modularity. It's for exactly this situation (wanting a specific version of a language or framework that is older than the one shipped by default in Fedora). I imagine you can install the necessary version of PHP.
Nice! I was unaware of this.
Unfortunately it doesn't seem to be the case that PHP has any modules available. When running dnf module list | grep -i php
in a Fedora 30 VM, I'm not getting any results. If there's something I'm missing about how to use modules, please do say as they definitely do sound like the ideal solution here.
You possibly have to enable the modular repos first. I don't think they're enable by default yet (Fedora generally introduces new features in an off-by-default state for a few releases before defaulting them to on). I think there are some guides/a manual that will probably say how to enable it.
On Sun, 6 Oct 2019, 19:31 Peter Law, notifications@github.com wrote:
It might be worth looking at Fedora Modularity. It's for exactly this situation (wanting a specific version of a language or framework that is older than the one shipped by default in Fedora). I imagine you can install the necessary version of PHP.
Nice! I was unaware of this.
Unfortunately it doesn't seem to be the case that PHP has any modules available. When running dnf module list | grep -i php in a Fedora 30 VM, I'm not getting any results. If there's something I'm missing about how to use modules, please do say as they definitely do sound like the ideal solution here.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/srobo/tasks/issues/363?email_source=notifications&email_token=AAMUUFPDFYLMJ6QZ3D4UOALQNIVJTA5CNFSM4IUUZFZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAOQYBA#issuecomment-538774532, or mute the thread https://github.com/notifications/unsubscribe-auth/AAMUUFKOFSH6C2FTEELK6ETQNIVJTANCNFSM4IUUZFZQ .
Hrm. I was following the guide at https://docs.fedoraproject.org/en-US/modularity/installing-modules/ (from which I got dnf module list
) and I do get other things listed when running dnf module list
(in two sections: "Fedora Modular 30 - x86_64" and "Fedora Modular 30 - x86_64 - Updates").
I know that with apt
, there are extra repos which can be enabled (universe
and multiverse
etc.) which offer more packages. Might it be something like that and/or that I need to add a repo specifically for PHP?
Update here: as of https://github.com/srobo/server-puppet/commit/da9524a8c4e85c053a75c7c2f5694f9755d057c3 all the Python stuff seems to work under Fedora 30
Update on the forums:
I'm really struggling to get PHPBB working in a Docker. Fundamentally it comes down to a Docker container having no simple mechanism to access ports on the host machine. We need to do that because PHPBB needs access to at least LDAP and MySQL which are running on the host.
So far the options I've found are:
<rant subject="Docker">
It seems crazy to me that Docker has the ability to expose listening ports from within a container to the host, but not the other way around.
One hacky solution I've seen some internet posts suggest is to setup an ssh connection which does the port forwarding I want. While I know exactly how to make ssh do this ordinarily (and it's a really handy piece of functionality), in this context it feels like a huge hack.
I also have security concerns around running things in Docker anyway -- anything in the container (all the PHPBB containers I've found include a whole webserver stack as well as just PHP) doesn't get updates when you update things in the normal way.
(This feels like a huge downside to using containers in general) (edited)
</rant>
I'm very much open to other ideas on how to get us some forums which work.
At this point, I'm concerned that we're getting really rather close to Kickstart and the team leaders don't have accounts yet. That said, I also don't want to commit to an platform which means we can't make the forums work.
Have you tried running the mysql in a container? That would be the usual way to connect it to the phpbb container. You can then expose that server on the host to give services on the host access.
Raw docker isnt usually recommended without an orchestrator to handle things like updates. Usually you'd run everything in containers and use something like k8s to handle that. I don't think it's worth the effort to do that here.
Are the issues with a specific version of phpbb, or any version?
Alternatively, we could investigate alternative software options, such as Discourse, but I'm concerned about time constraints.
Have you tried running the mysql in a container? That would be the usual way to connect it to the phpbb container. You can then expose that server on the host to Enel
Thanks for the suggestion. I did consider that approach, however if we were to follow down that route we'd need to end up essentially rebuilding a substantial portion of our stack in containers -- the forums need at least LDAP too and we have the same issue there. I've previously explored replicating our LDAP setup in containers in order to get CI around nemesis
and spent several days failing to do so.
I'd therefore expect to spend at least the same again on the LDAP side, plus a similar amount of time as I have done so far on the forums (which are by no means themselves complete; there's at least another half day or so on things other than this connectivity issue), meaning this doesn't feel like a likely avenue of success.
That aside, containerising everything also introduces significant new complexity to our stack in terms of ongoing maintenance.
(Being forced to change our overall architecture to enable one piece of software feels like a very back to front way to do things.)
Ok, I think I have a solution to the Docker networking shenanigans:
docker0
to reach the hostI've tested this using https://github.com/vinodpandey/python-port-forward and it works for the trivial case of poking a connection manually. That may not be how the port forward works in the final solution, but I'm happy it works.
This avoids needing to change the listen behaviour of the services. While it's still a bit of a hack, I'm now happy enough that this can be made to work reliably to call it a solution.
Update: patience
is now running (Fedora 30), though I've not sent out the team leader or mentor accounts. That will probably need to wait until tomorrow now.
Progress on the forums: after some firewall munging, I have a somewhat working local config I believe I can reproduce. It's a lot more manual than I'd like and it's not complete, but will probably work.
Ok, we're very nearly done here.
There's a bug in the container we're using (https://github.com/bitnami/bitnami-docker-phpbb/issues/54), which breaks the styling. I've got workarounds for that in PR (https://github.com/srobo/website/pull/178, https://github.com/srobo/reverse-proxy/pull/9).
I'm calling this done, but I'll keep the forums-specific task open until the styling PRs merge.
We provide a hosted IDE and online forums to the teams to use during the year. Typically the microgames at Kickstart get the competitors using these, so they need to be ready before the event. Our current approach is to spin up a new hosted "pet" machine running the
competitor-services
variant of our puppet configuration.Original
kickstart/services/main
Dependencies
357 Prepare forums for SR2020
359 Prepare the IDE for SR2020
360 Setup mentor accounts for SR2020
362 Setup team leader accounts for SR2020