New website - hosting + forum + other dynamic services

[ohlidalp]

Let's analyze and discuss what we need and expect from RoR's future website. The content and services of the website will largely determine the hosting parameters we'll be looking for. Further, our content gathered from the old website needs a lot of sorting and pruning work.

Required services:

• Forum. Needs PHP + Database. Generates moderate traffic. Not very demanding on server space, except for attachments (which may be limited).
• Content repository. Needs PHP + Database. Generates low to moderate traffic. Quite demanding on server space: Our current repo has 19GB of data + 3GB of images. We need to find a new software to run the repo. Unlike the old repo which was upload-based, I'd like to make the new one primarily a link-based of all public content, whereever it's hosted.
• Multiplayer serverlist. Needs PHP + Database. Generates low traffic, no space required.

Required static content

• Copy of old forum - A very important source of information - tutorials, tests, hints. Also possibly a HUGE bulk of data. The full backup run by TDev at http://html.rigsofrods.com is currently 120GB and counting; it also duplicated gigabytes wiki/repo content and possibly other dead weight. Also, some data-heavy threads/sections are not really worth archival, for example "random game media thread". Overall, the forum snapshot will need a lot of additional pruning and cleanup
• Doxygen docs - The no.1 resource for developers. Contains generated API-reference + additional text docs. On the old web, most of dev info was on wiki, but I'd like to put all developer info to the doxy. Server: no traffic, light on data.

Static or dynamic? This content can be handled in 2 ways: Using a CMS/Wiki software (database, logins, permissions) or a github.io-style content management: create a github repository and manage the webpages as a project.

• Homepage - General info, overview, links, downloads: ror, rortoolkit, repo link. Low traffic, no data.
• Knowledge base/Wiki - Manuals/tutorials for players and content creators.

Work to be done

• Old forum copy: We need an enthusiastic programmer to write a couple scripts to prune and polish the pages: Duplicate/useless pages from links like "first unread; user profile; last post" should be removed. The helper links like "first unread; last post" should be removed. Further, we need to selectively remove threads which contain outdated info or are useless in any other way.
• Repository: Content needs to be sorted and synced with external conversions/updates. A lot of updates hapenned since the repo got locked. A lot of mods are hosted externally. I don't want obsolete content in the repo.

Stats TDev told me in skype-chat that rigsofrods.com's web traffic in January 2016 was 1.5TB/month. That's quite a bulk, considering the forum activity isn't so big.

[Strykr1]

Since I have no idea where else to post this I'll post it here. I have a great offer that I encourage you all to take advantage of.

• PHP: Latest version supported (5.4.45)
• Database: MYSQL and PostgreSQL Supported. No maximum amount of databases and no database size limit.
• Traffic: My web host is able to easily handle the traffic imposed on the RoR site.
• Webspace: Again, my web host says not to worry about space a bit. It will all be handled.

At the moment, I have business level hosting with my provider. I am able and willing to extend my web server to you guys for no cost whatsoever. I do not want to see RoR die and I also know that there are other people that do not want to see it die either. Yes, I know some people may not like me simply because I own Allied Gaming but I can assure you that if you decide to allow us to host Rigs of Rods, that there will be no involvement with Allied Gaming whatsoever.

[mikadou]

@only-a-ptr First of all thanks for negotiating with Tdev and leading the transition effort!

I assume you plan on gradually reestablishing the infrastructure starting with the forum? I believe the content repository and old forum backup should have a lower priority (and will certainly require alot more time and effort).

Forum Regarding forum software here is the outcome of my little research:

• Recent developments in the forum software world are Discourse and Flarum which try to somewhat redefine the concept of forums (whether you think that's a good idea or not is up to you ...). I'd also like to mention FlaskBB here which is written in python as an inspiration (it is definitely not mature and tested enough for our use case).
  • Personally I don't like discourse, cannot tell exactly why. It feels cluttered. Besides that deployment is not easy (not written in PHP)
  • To me Flarum looks really nice. It seems to be quite usable already, although it is not ready for use in production yet (=> not not an option for us now).
• There are many scripts that implement the more traditional forum approach.
  • I quickly skipped over SMF and Phorum, they simply appeared much more inactive and less organized than the available alternatives.
  • FluxBB is advertised as being very lightweight. I was somewhat put off after browsing around their codebase and seeing that PHP code and HTML output is intermingeled in the files.
  • MyBB at first seemed to me to be the best contenter (and I still think it is a viable option). However, I learned that template styles are not simply stored in files but in the database instead. I'd rather have a custom/adapted theme for the RoR forum being hosted on Github. So anyone who wants to propose a visual enhancement can simply do so via pull request. It should also be mentioned that MyBB's development is currently stalled as their devs are rewriting a new 2.0 version from scratch.
  • phpBB This piece of software has been around for a long time. It has also suffered quite some bad press regarding security vulnerabilites in the past. Furthermore it has a reputation of being a hassle to maintain. From what I've learned this was mainly due to the fact that extending the base functionality was accomplished by partly instructed manual modifications of the code and database. Obviously this becomes troublesome when trying to update with multiple such mods installed. It seems, however, that there has been alot of progress in recent years and the current version 3.1 has many improvements compared to older versions.

Summary: Despite my initial rejection of it, I'd currently prefer phpBB among the available opensource forum solutions.

[AnotherFoxGuy]

I've made a website here it runs 100% on PHP fusion

Features Forum. there is a simple forum. it doesn't feature anything special, it just does the job. Content repository. there is a Downloads section, once again it is nothing special.

Also, I don't see the advantages for us to have a static website. We don't have 7,790 people viewing the website at once and the differents in loading times are a few milliseconds

Static website php fusion (I've compared the download pages, because there is a lot of junk on the homepage that I've made)

[Hiradur]

Also, I don't see the advantages for us to have a static website. We don't have 7,790 people viewing the website at once and the differents in loading times are a few milliseconds

I suspect that the difference will get higher with more simultaneous accesses. Regardless, there are other reasons to choose static over dynamic:

• the static part will still be visible when the software for the dynamic stuff crashed
• once set up you never have to worry about it again
• it's secure
• no maintenance
  • anecdote: I set up a CMS website to play around with years ago and I got weekly security update notifications, sometimes daily. All the updating got annoying very quickly.
• the static part of the website could be put in a repo like rigsofrods.github.io where everyone can help develop and maintain it. This is not possible with CMS afaik.

having forum, repo and wiki separated also means the scope of a single attack is smaller.

@only-a-ptr I worry that a link-based repository has a negative impact on user experience:

• with a central place for uploads the content is available as long as the repo itself is unlike other hosting places
  • but it's also a SPOF
• it keeps the downloads clean from ads, malware, captchas, 30s waiting times, bandwidth limits and all that other inconvenient stuff

[mikadou]

@AnotherFoxGuy I do greatly appreciate your efforts.

As @Hiradur already said, I don't think that site performance is the main argument for static HTML. I have to admit that I'm quite sceptical towards php-fusion. I did not find much (useful) information about it on the web and their support forum does not really look like a bustling place in my eyes. I fear there is a realistic possibility of unfortunately locking ourselves in with a single product if we choose to go with php-fusion.

Regarding the repository I have this vague idea. I'm a bit reluctant to share it because it is totally not realistic at this point in time, but anyway. I think it would be really cool to have something similar to package management in many Linux distributions. Imagine a core repository with just few high quality vehicles under a permissive license. Vehicles from the core repo are expected to work flawlessly with the latest RoR version (i.e. they are tested before a new release). This would take some burden from devs because there is a defined set of vehicles that must be made to work, either by hacking RoRs codebase or simply modifying the vehicle, whatever fits better. Less ugly workarounds for broken vehicle files. In addition there might be a community repository which contains a much larger number of vehicles (also using permissive license) and is moderated by a few community members. These vehicles are not guaranteed to work flawlessly with the latest version, but can be modified by the community to make them compatible. Finally there would still be the need for a repository which can hold all remaining vehicles with unclear license specification.

[DirtGamer301]

@mikadou I just got this tiny idea while reading your post for the repository: I have no idea at all and it might just be a stupid thing coming in my mind now which I'll be shamed of tomorrow, but is the Ubuntu SoftwareCenters code open source? If so, it could probably be used as a base for the repository, it got all the Repo would need (I think, but as I said, I have hardly any idea of what I'm talking about).

[monwarez]

Maybe it's off topic, but it could be really great to have at least three topic for additional content(such as map, vehicle, ...) : the one with know proprietary license, an other with open source license , and finally one for unknown license

[disloyalpick]

https://domains.google.com/about/home/?utm_expid=87793071-1.pATaM0MLQqGW0IRNnbAJ1Q.1&utm_campaign=2016-q1-na-newdomainendings&utm_medium=social&utm_source=facebook&utm_content=manifesto&tld=business,company,guru,email,today&utm_referrer=https%3A%2F%2Fwww.facebook.com%2F

Check that out. Lokks cheap/affordable. I thought maybe someone would like to take a look

@only-a-ptr if it interests you, I thought I would mention you.

[Hiradur]

Maybe it's off topic, but it could be really great to have at least three topic for additional content(such as map, vehicle, ...) : the one with know proprietary license, an other with open source license , and finally one for unknown license

Yes, the repository should make clear what license the content is under and also offer an option to sort by license (category).

I like @mikadou's idea, I also thought about a package manager for mods once. It could also help save user disk space and traffic if it would check if textures etc. are already on the machine, though it may be too late to get that going with all the already existing mods and their possibly redundant names for different textures.

but is the Ubuntu SoftwareCenters code open source? If so, it could probably be used as a base for the repository, it got all the Repo would need (I think, but as I said, I have hardly any idea of what I'm talking about).

Not sure about the Ubuntu Software Center in particular but there are many open source package managers so it shouldn't be to hard to find one.

IMO research into package managers should only start when the rest of the website is established and stable.

[RoRAustin]

I'd like to give my two cents.

1. The idea of a link-based system for a file repository is terrible for us. That's one major way to turn people away. I don't want to click a download button and then go through 5 different ad.fly links to download a simple .zip file. I know that storage is a major concern but it's worth it. A better repository than what we had is definitely a plus though especially if it is better moderated.
2. Honestly I like the idea of a static home page that allows a person to see what the game is all about and download it without the clutter of a bunch of dynamic widgets that gather useless data from a forum. Something clean and professional that doesn't look like another vbAdvanced CMS.
3. If any financial support is needed feel free to contact me. I've been a supporter of this game since 2007, I have no problem lending a hand to keep it going.

[kskarthik]

Hello, I am following this game since a long time. I have a few suggestions now regarding website.

1. Github is best to host static website for the game. Existing website is cool !
2. For the game binaries sourceforge is good since we have unlimited space.
3. For discussions & support Google groups is best since I have seen many big Foss projects are using it.

These ideas are suggested keeping the cost effective way in mind.

[GimmickNG]

For the game binaries sourceforge is good since we have unlimited space.

IIRC sourceforge had an incident where they'd bundle adware along with the installer, I don't think they're doing that anymore, though.

[kskarthik]

In that case, github's large file storage can help I think.

[Hiradur]

In that case, github's large file storage can help I think.

Keep in mind that we are talking about >20gb of content and 1.5TB monthly traffic here. GitHub LFS offers 1GB space and 1 GB bandwidth/month for free. Since there is no way to get such high volume for free we can also host our own forum. This will give us much more control.

[ohlidalp]

Hi everybody.

Webhosting After a lot of thinking, I picked a webhosting for us:

• Type: Shared webhosting - i.e. pre-configured with PHP+MySQL+sFTP access. Reason: I don't want a VPS/dedicated server at my responsibility. I'm not a regular linux user and I never managed my own server. Further, I'm not a web/server enthusiast - I want to do C++ coding for fun, not do annoying server maintenance. Further, a shared webhosting will definitely be enough for RoR in close future. For more distant future ... we'll see.
• Plan: http://hosting.wedos.com/en/webhosting/extra.html - WEDOS is the #1 hosting provider in my home country (Czech republic). I wanted to use a domestic provider because of legal obligations associated with running a forum.
• Disk space: unlimited. I talked to the provider about our ~200GB initial requirement. It appears OK.
• Traffic: unlimited. I can't really evaluate our close/distant future demand for traffic. So let's rely on an "unlimited" offer and if the host complains, we'll re-evaluate.
• PHP max processes: 10. Totally enough for forum/wiki/blogs (will we use static content?). Forum downloads thru PHP scripts may be an issue, but we'll see.
• PHP upload limit: 128M. Quite enough. We should focus on having slim and space-effective content rather than a lot of space for junk.
• PHP per-process memory limit: 256M. Generous, an usual page-view process takes around 2M.
• Auto-backup: Yes. This is an advantage.

Forum A next thing for us to decide a forum software. And, boy, what an ungrateful task this is. There are so many options and all of them flawed in a way. This is a general picture:

• Old, proven forums - all criticised for ugly codebases and many of them for miserable security.
• WIP forums - all I've seen are a waste of development time, bringing little useful features and a lot of built-in distractions and socialnetwork-like junk.
• Old, other forums - plenty of these, but usually abandoned due to the WIP ones.

My preferences for the forum are simple:

• Forums/threads like in traditional PhpBB
• Slim, bloat-free posts: No sigs, no repetitive data under user's avatar, no thick "post tools" panels - just thin text box and that's it.
• Attachments: images and files. Simple formatting utilities.
• Bloat-free HTML/CSS output. Minimum of JS. No image popup galleries, just plain links.
• Simple maintenance

Resources I'm looking into:

• https://en.wikipedia.org/wiki/Comparison_of_Internet_forum_software
• http://www.forum-software.org/forum-comparator/fluxbb-vs-mybb-vs-phpbb3-vs-simple-machines-2
• http://adamyamada.com/best-free-forum-software/
• http://fluxbb.org/docs/unfeatures
• http://community.mybb.com/forum-137.html - showcase

Feedback

To adress your posts:

• @thatrorguy: Thanks for the offer, but I want the webhosting to be officially contracted to me and located in CZ. We may use the services you provide for some content, tho.
• @mikadou Thx for the overview. Both Flarum+Discourse have nice visual design, but otherwise I don't see much novelty in them. FluxBB/MyBB look interesting.
• @AnotherFoxGuy Thanks for the analysis.
• @Hiradur I'm not suggesting to using a link-only repo, but a hybrid link/upload solution would be good IMO. The poin is, many users upload their stuff to various external hosts anyway. Some things are in the old forum only. Some things are on people's dropboxes. I simply want a single hub where players can find everything.
• repo/better sorting&license sorting&official content: This is also on my mind.
• repo/package-system: This is a nice idea, but it has some prerequisites. First we need to have sort of a content workflow. Currently it's total chaos: Documentation lacks, making content is awfully and stupidly manual process, we only use the bare essentials of modern 3D graphics + crude and outdated utils like editorizer/RoRToolkit/blender-4x plugins. If we don't tell people how to make content, what sense does it make to tell them how to package the content? Sounds daft to me.
• @animosus-wolf: I agree on the link-based repo, but as I said above, I just wanted to account for existing content uploaded elsewhere.
• @kskarthik / Sourceforge: Any association with SF is unacceptable for me. SF is a zombie of FOSS, packing adware to installers and triggering web-security filters in browsers.
• @kskarthik / GoogleGroups: Unacceptable IMO. Not everyone is fan of google. I'm not. The idea of forcing all our players to register to their services is pretty lame.

[ghost]

PHP upload limit: 128M. Quite enough. We should focus on having slim and space-effective content rather than a lot of space for junk.

@only-a-ptr Only 128MB? Large map projects take up much more space than that - with Rockton 2016 taking up 160MB+ and the latest Community Map version on the forum takes 79MB, not including the changes @DarthCainRor did after that, which would probably skyrocket the size. This is 2016, we shouldn't be limited on size as much anymore.

[ohlidalp]

@hotrod55: Actually, Rockton is pretty much what I reffered to by "We should focus on having slim and space-effective content". While I appreciate Brickman's work, the content used there is just awful. The heavisest thing there are PNG textures (PNG...textures!!! In 2016!?!?! GIMP has a DDS plugin you know!) having up to 3MB and containing essentially un-edited photographs of building walls/uselessly detailed brickwall textures/other space-inconsiderable stuff. I didn't check the meshes, but I'd bet my neck they're converted without the "-e" flag, thus carrying a dead-weight of "edge-lists", useful only for stencil-shadows (obsoleted by both OGRE and RoR years ago). And did I mention auto-generated LODs (OgreXMLconverter's default is IIRC 5) which are essentially duplicates of the geometry, and for mostly-boxy models, they're useless? More deadweight.

TLDR: If the content file has more than 100G, then something's wrong with it.

Of course, I admit the 128M isn't much. But at the same time, none of our downloads should really exceed that. If a map uses so much objects, then those should be separated out to some mesh-pack to share among multiple maps

[DarthCainRor]

The Community map as it is on my hard Drive is currently 128Mb, All of the images are DDS and the map has one of the biggest areas to cover that I'm aware of in RoR at 18Km x 18Km, and it still has a long way to go to even cover parts of the map. https://i.gyazo.com/173adfb741a051e5c0499bebdb05e11b.png

[ohlidalp]

@DarthCainRor Ok, I admit, map of this size won't fit under 128MB. In fact, if I read the screenshot correctly, it will even hardly fit under 1GB. Questions: Is it a mesh-only map, or does it use heightmap? If yes, what's the heightmap size alone?

[DarthCainRor]

As it is right now, there is no height map, and is mesh only. At some point, it might get a height map, but it will probably remain mesh only for quite some time.

[fidoj]

@only-a-ptr 128mb is to low my remade map at its smallest is 72mb and i would still like to add content. we should encourage people to make dense and detailed mods for ror, and that would include full city's, especially if we want to progress ror and make it valid in today's "market". as for shared content you first need the content to share people cant make maps full of nothing. and everything made prior is content locked into packs. so that's all of us making content so it might be used by someone maybe one day once maybe, not to mention mesh theft from ror, heavy limiting factor in users making content of that quality unless its claimed and owned by content creator X. the idea of ror outright claiming ownership and content locking is against its gnu. so why would i want to spend 20 days working on something that hasnt got my name linked to it and any nfs/lfs/gta console boy can rip and claim ownership off. ror cant claim ownership over anything.

[ohlidalp]

@fidoj @DarthCainRor @Hotrod55 The 128M limit only restricts uploads thru PHP scripts (forum).

What about FTP accounts made individually for projects/teams who require larger space? My preffered hosting provider allows unlimited extra FTP accounts into separated directories. FTP might even be more comfortable for uploading big packages (PHP uploads often lack any progress-indicator)

[fidoj]

@only-a-ptr could that be an optional flag as in-line "upload through ftp" what you do is call a script to upload to a public directory on the server a "holding zone" then wait for admin/mod to move it to a linkable directory and activate the forum thread/repo download

[ohlidalp]

@fidoj Repository uploads will be approved by repo admins of course.

FTP uploads should be on written request only. I see no benefit in opening this to wider public, even if there's post-upload approval.

[nick149]

Have you checked out drupal forums? I have used Drupal before (not the forum but will be checking it out soon) Setup On Youtube

[mikadou]

My preferences for the forum are simple:

• Forums/threads like in traditional PhpBB
• Slim, bloat-free posts: No sigs, no repetitive data under user's avatar, no thick "post tools" panels - just thin text box and that's it.
• Attachments: images and files. Simple formatting utilities.
• Bloat-free HTML/CSS output. Minimum of JS. No image popup galleries, just plain links. Simple maintenance

@only-a-ptr What's wrong with phpBB? To me FluxBB is not an option after looking at some of its code. The remaining alternative MyBB would be ok, although the following points should be considered.

• MyBB development is on halt, their devs are doing a complete rewrite for version 2.0. phpBB is in continuous development with fairly regular releases.
• phpBB uses Twig for templating. We could simply fork the default phpBB theme, customize it and put it on github. This allows any community to issue a pull request to change the forum theme. I don't know if this is possible with MyBB since it stores its themes inside its database (haven't really looked into this though ...)

@nick149 A few years ago I had a website (for testing purposes) setup with Drupal. About one year after I stopped maintaining it, it was hacked. However, this is probably a problem that every popular web application will suffer from.

[ohlidalp]

@mikadou Thx for the feedback. I decided agains phpBB3 because according to reviews I read, it has complicated maintenance and confusing admin panel. Admittably, those resources might have been out of date. I have zero experience installing/administering any forum, so I can't really make a qualified decision.

At the moment, there's already a test installation of MyBB (link is private!). The theme is good enough and it appears the stable MyBB is still actively supported: http://docs.mybb.com/versions/1.8.7/. Honestly, at the moment I have no energy to invest into evaluating another forum (my daily job happened), so I'd like to just close this task. And, I personally like MyBB.

@nick149 Drupal (or any other CMS of this scale) is a no-go. Pointlessly complex for our needs. And according to most feedback I got, PITA to maintain.

[wilkgr76]

I'm willing to try to do this, but there's the question if you trust me with this :wink:

We need an enthusiastic programmer to write a couple scripts to prune and polish the pages: Duplicate/useless pages from links like "first unread; user profile; last post" should be removed. The helper links like "first unread; last post" should be removed. Further, we need to selectively remove threads which contain outdated info or are useless in any other way.

[ohlidalp]

This issue is resolved. Webite is online and hosted by Avrintech.

For more info, see http://forum.rigsofrods.org/thread-496.html