URL Shortening service

[robertquitt]

CSUA used to have a URL shortener at csua.org/u/... , it would be a good idea to rebuild it! Not sure who owns csua.org, but it forwards to csua.berkeley.edu.

[wylliec]

should we roll our own implementation or try to find an existing package to do it for us?

[robertquitt]

I don't think it should take long to make our own, plus, it's something that I think would be interesting and very doable as a first PR

[wylliec]

yep! agreed. and there doesn't seem to be any one app out there that does exactly what we want. i think the most important feature would be for pb to create named shortlinks (i.e. csua.org/u/application would link to the officer application); the current amalgamation of tinyurls and bitlys is hard to follow, hard to remember, and hard to keep track of. this could initially just be managed by the admin interface, but we could potentially open this up for all members to have access to our shortlink service (like i used hkn's shortlink service to take attendance in my ee16a sections). i don't think we should have it open to the public tho, i remember seeing porn sites linked from the old csua.org/u service. features i'd like to track would be: what the custom shortlink is, what the destination url is, when the link was created (and/or when it was last updated), and who created the link.

some resources: this seems to be a useful guide to test driven development overall, but the specific example is a shortlink service in django - https://arunrocks.com/understanding-tdd-with-django/ this hasn't been updated since django 1.4 and python2 and their base62 thing seems pretty jank, but it seems pretty close to what we'd want from a shortlink app - https://github.com/tehranian/django-url-shortener

[toulouse]

Some alumni probably owns it. We started using that domain for mail in March 2011, and the earliest mail in my mailbox with us using it as a URL shortener is in 2007. Fun historical excerpt from the announcement thread:

You misunderstand. Nothing is happening to @csua.berkeley.edu. Politburo is merely migrating some lists and official emails to csua.org to ensure that, even if mail goes down (as it has been doing often recently) that mail operation continues normally. The impetus for this was that politburo recently failed to respond to certain emails in a timely manner due to the mail outage, and this resulted in some issues during the Startup Fair.

It is, of course, impossible for us to move the CSUA wholesale to Google Apps anyway, given the per-head cost for >100 users.

There is no mystery as for the problem with our servers (in relation to this problem). Mail is overloaded because our NFS I/O is shit. Cooler has a load average of 12.0. FYI, when the load average hits 5.0, generally you're in a critical situation. Cooler is pretty much beyond critical and barely functioning, and all access to soda and mail depend on Cooler. Root has considered some options, but improvements to increase stability may not be put into effect until the end of the semester, lest messing with it now and running out of time result in an extended outage that nobody has time to fix.

steggy (@mjsteger) might remember more context about the domain.

A URL shortener is a very long-term commitment, however so be wary of rolling your own vs. just using a custom domain with bitly or whatever the cool kids are doing nowadays. Even if you have to pay a couple of dollars, it's easier than trying to recover lost data or deal with downtime.

[wylliec]

Yeah the domain was owned by David Bushong, who transferred it to root staff last May. I haven't done anything with it besides point it at CSUA's servers (it was originally pointing to some droplets in digital ocean).

Whether we want to migrate CSUA services into the cloud is something worth discussing, but hitherto we've opted to maintain our own physical servers because we felt like that was something that made us CSUA (HKN has moved their emails to a google apps non-profit organization and their web hosting to the OCF). @toulouse thanks for the historical exerpt; I feel like a lot of logs of conversations/threads gets lost in each turnover of politburo.

I definitely feel like creating a url shortener is worth the commitment and hopefully can resolve at least some of the needs of setting up something like bunny1 mentioned in #22 . As for lost data, we really need regular, verifiable backups to the cloud (unlimited free storage on google drive through berkeley yeet).

[toulouse]

I think that most of the servers could live in the cloud with little consequence, given a good configuration. You want disk and shell servers to live on premises, but LDAP, web, and the like living on EC2 would be an improvement, especially as it's so easy to use terraform and ansible to provision them (I've got scripts I can donate). And given provisioning scripts, that'll allow you to keep rather minimal (and probably verifiable) backups, as well as spin up new services on demand with less hassle.

I actually recommend t2.nanos over docker containers when it's a choice, btw, as docker networking and state is a lot harder to reason about than just configuring a VM and a restrictive security group (firewall rule set). Docker is great for collecting a bunch of tiny services, VMs are good for clear division of responsibility/security.

The wrinkle would be users being able to publish csua.berkeley.edu/~username pages,, but I don't think they hold much utility anymore, and if you really want to, you could configure nginx (or whatever) to reverse proxy or just rewrite/redirect the url to soda.csua.berkeley.edu/~username or its equivalent.

Depending on the amount of data, i would put backblaze b2 under consideration for a backup service.

Duplicati seems like a relatively user-friendly backup tool; I used it at one point but am currently on CrashPlan (and migrating off to a local FreeNAS server auto-backing-up to backblaze).

[robertquitt]

Exists with the django.contrib.redirects app set up.