At the moment this website is hosted on a dedicated machine leased from Bytemark. The Bytemark team are superb, and their service is great; if you’re looking to run machines (virtual or physical) that aren’t in your house or company, go and see them. However, if there’s one thing that I’ve learned in the last fifteen years of running a website, it’s that I hate being a sysadmin (and Jono, with whom I share the dedicated machine, is if possible even worse than me). If the site goes down I just bury my head in my hands, and I hate, hate, hate having to fix it. Whether it’s because Apache’s crashed or the backups have stopped working or the machine’s out of disc space or there was a kernel panic or we’ve been slashdotted again or it’s not responding or… man, I just don’t care. I used to find this sort of thing fascinating — having my own server, being able to fiddle with it and tweak it and deploy what I liked — and now I just want it to go away. So I’ve been looking for alternatives.
This site, what with having been running for fifteen years and all, has built up a pretty large collection of both stuff and infrastructure. So, here are the things that I want, that I need, to preserve.
A blog
This blog. Lives at kryogenix.org/days. Permalinks at /days/YYYY/MM/DD. Can be themed.
Comments on the blog
People can comment here. The commenting requires JavaScript already, so I’m not averse to continuing to require that for a new thing.
Arbitrary files at arbitrary URLs
Over these many years, I have written and uploaded a whole bunch of stuff here. There are images and random little uploaded things and HTML and CSS and JavaScript files which are not part of the blog. Take, for example, sorttable, probably the most popular thing I’ve ever written. So, these things are not part of the blog at all. I have them already: I do not want to have to manually type them all back in again to make them “static pages” controlled by the blogging engine.
Redirect farms
Back in the early days, before we (that is: mpt and Mark Pilgrim, both of whom have since killed their sites so I’m not sure how to link to them, and others) thought up stable permalink URLs in a standard format (/blog/YYYY/MM/DD), and by virtue of this place having been run by about six different blogging engines (ones I wrote myself, Moveable Type, pyblosxom, WordPress, the list goes on) posts here have had many, many, many different permalinks. Also, the RSS feed has moved around a number of times. I have a forest of redirect rules to make sure that all those old links keep working (and 302 redirect to the newest link), and I do not want to give that up. (Nor do I want to publish a given thing in all possible places.)
So, how can I host this site with no sysadmin overhead and still do all the above? I’ll list the options I’ve looked at, and whether there’s some sort of a problem with each (hint: there is) and then you clever people can tell me about ways I can fix that or options I’ve missed. Nota bene: I am prepared to pay for whatever I come up with, as long as it’s not too expensive; I get some money back by no longer having the server, anyway.
Completely static hosting on S3/Rackspace Cloud Files/similar
It is possible to host a static website entirely on Amazon S3 and its competition, and this seems like a good idea. It would involve using Disqus for comments (which I am not keen on: anyone who remembers e-net-ation or netcomments should know why (they were disqus, and went bust taking everyone’s comments with them)), but I can live with that if I have to. The big problem here is redirects: you cannot do them, as far as I can tell. So any old links to my site or my resources or my RSS feeds will just break, or I have to multiply-publish everything that needs redirecting, which is lame and wrong.
Completely managed site, on WordPress.com/SquareSpace/Tumblr/similar
I quite like this idea: it overcomes one of the other big weaknesses I have, which is that I’m horseshit at design, by allowing me to one-click-choose a new design, yay! However, this is a big problem because of arbitrary files. I quite often write a new little JavaScript experiment or something — one HTML file, with a CSS file and a JavaScript file, in its own folder and scp’ed to my server. This is fourteen million times harder and more annoying if I have to manage those pages as “Pages” controlled by the CMS. I don’t want that. I want to be able to store static content by just ftp’ing it up or something similar. As far as I can tell, none of the full-site-management stuff lets me do this — understandable, since they are full-site-management and what I’m asking for is an unmanaged bit, but a problem nonetheless.
(Extra bonus deficit for SquareSpace: their blog URLs are wrong! They’re /YYYY/M/D, so 2012/8/5, rather than with the 0s in place, which will break every URL on this blog. WTF, SquareSpace? Why would you do that? Huge disappointment.)
However, I’m keen on this method if someone does it in a way I can use.
Old-sk00l web hosting
You know, a company which gives you a cPanel control panel and FTP access and so on. My annoyance here is that suddenly I’m a sysadmin again, but one level up; when a new version of WordPress comes out, I have to upgrade it myself. It would be nice to avoid that.
PaaS-style hosting, like Google App Engine/heroku/dotcloud/appfog
Also under consideration. I’d have to be a bit of a sysadmin again — that is, I’d need to write (or maybe use an existing) blog engine designed for these things, and publishing’s a bit harder (I likely have to use their command line app, rather than just FTP or similar, although dotcloud do SSH, which makes it much harder to do from someone else’s machine), and I’d need a bunch of complicated redirection to handle all the static content, but it’s doable. One thing here is that they tend to be oriented around “let’s make it trivially easy for you to horizontally or vertically scale your site by spinning up new instances”, which is nice but my site ain’t gmail. I don’t need that, really, and so it suggests that if they’re devoting their attention to making that great, they’re not doing other stuff that I do care about.
Just have a server, man! it’s easy, really!
Go away.
So, LazyWeb, here’s your chance. Fill me with your knowledge and ideas!
I went through same thought process few years ago. Ended up w a normal host (Dreamhost, tho if I were choosing now would probably go Hostgator – would switch if I could be arsed) and investing a little time to set it up so WordPress automatically updates itself. Works alright, definitely better than sysadminning my own server and I can SFTP in if I want to mess with anything, whack something in it’s own sub domain or folder or whatever. HTH.
Little bit of a faff to set up and involves git.. But WordPress hosted on heroku. If I remember rightly you’d be able to control the apache conf to preserve the link farms. And your little static scripts are just files in a DVCS. 1 web server process is free. 50,000 row postgres is free. Decent backups built in. You’d have to update WordPress versions yourself or write a script to do it.
You will hate this recommendation, but I _love_ Octopress. But that depends on how much you like/dislike working with ruby tools.
There are some ways to do redirects, I haven’t investigated though: http://rawsyntax.com/blog/blogging-on-jekyll-url-redirects/
For me personally since I am not a famous personality I just broke all of my URLs and let google handle it.
Also I think I prefer hosted comments due to the management tools on disqus that let me deal with content across the web; like a nice shared banlist for myself.
After doing this for about 6 months I’m liking the static hosting, the most expensive my blog ever cost me was 27 cents for a month.
jorge: that redirect thing isn’t a redirect; it puts the same content in both places. That’s lame.
(I have no problem with static site generators, and I’d use one if I were to go the S3 route — probably nikola rather than octopress, but that’s fine.)
Whoa nikola looks great! (Man real redirects are hard on static storage, I’ve been looking and can’t really find anything)
One more vote for Dreamhost! Maybe there are meanwhile better ones out there (as Meri suggests) but I never had the need to check for.
Dreamhost runs Debian on their machines and I have written XODA always directly SSH-ing into the hosting machine and using nano. WordPress (as a “One-Click-Install”) is being offered with automatic updates but I have never seen a fast enough WordPress setup yet including my professional (WordPress powered) blog. So, I ended up writing an own thing for xoda.
I think Dreamhost offers a money-back period of time, so you could give it a try.
I am with jorge on the Octopress, I’ve been using it a little while now and could not be more happy.
When I moved from WP to Octopress I made sure that the permalinks ( config option ) matched so no real need for redirects here, but if I were to need them I’d likely try a blank page with a meta-refresh, not ideal but a small price to pay for all the other conditions to be met and be able to use static hosting ( github pages for brandonholtsclaw.com in my case not s3 like jorge )
Old-school web hosting on a virtual system, with static content (rather than WordPress or similar).
I use bzr to manage my website on my local machine, and bzr upload to push it out (uses sftp in the background). I also use PHP includes, which means that there is a little bit of building when the page is served. This means I don’t have to ‘rebuild’ my site if I want to change the look. I just have to change the header file (or CSS in the majority of cases).
Gives you:
Arbitrary files at arbitrary URLs
Redirect farms
Very little sys admin stuff required once it is setup
Problem: comments
Suggested solution: dynamic comments (i.e. stored in a database)
Problem with solution: more sys admin stuff (but falls to near zero after setup, except comment managing (which you would have to do anyway) and maybe db backup)
After much searching I found some potential tools:
https://github.com/phusion/juvia
http://www.tildehash.com/comments.php?source (though doesn’t appear to save to a db, but rather xml files, which could be a problem)
How I think I would go about it:
Store all the comments in a db (so that they aren’t affected by bzr upload), and push them or pull them depending on the URL. Include on every page where comments should be, a script that include the form and also pulls comments. The comment form posts to a specific script that checks to make sure that it is valid (perhaps checking for spam stuff by sending to an external service), the URL is valid and so on. If so, it says OK and redirects to the original page (which will now pull the new comment in addition to the other ones). Else it gives an error.
Comment administration is certainly possible, and would require extra coding.
(Also, I like the plain design you have here. No distractions, and nothing getting in the way.)
https://github.com/mhoofman/wordpress-heroku
I recently went through (as in, the last two weeks) the whole “I’m sick of being a sysadmin” experience. What I ended up doing is getting a tiny slice from Linode to serve up some basic static content with nginx. The blog is now hosted on WordPress.com with the $13/year use it as a subdomain option. Email, calendar, and god only knows what else is going through Google Apps for Business. I did break the interwebs by not redirecting all of my old posts/content from www -> blog, but I figure our Google overlords will re-index it soon enough.
This was already mentioned, but I want to put a second vote in for Github Pages. I admin 5 domains and I’m using github pages for every single one. It’s essentially painless to set up, jekyll is a sexy static templating engine, and you get arbitrary files for free.
Meta refreshes are ugly, but much less ugly than duplicating the content at all possible URLs. It shouldn’t be difficult to write a small script that scans your list of redirects and copies a basic blank HTML page with a meta refresh into the right place to achieve the same redirect.
Robert: I’m not too worried about meta refreshes for HTML. However, for images and RSS feeds, that won’t work…
Some updates, after further investigation:
github pages don’t do redirects.
WordPress.com flat-out bans all JavaScript.
“Managed WordPress Hosting” is apparently a thing: wpengine seem to be the leaders. It is not clear to me that you can host random static content there which isn’t managed by WP, though. wpengine themselves are a big no because they won’t let me host the blog at /days.
Staticloud (http://staticloud.com/) would be great, if (a) they had done anything in the last year, (b) they had some sort of API better than “upload your whole site as a zip file every time”, and (c) I could pay them to be sure they’d stick around. They do static hosting but also handle 301 redirects, which NO-ONE ELSE does as far as I can tell.
I do about 400MB of transfer per day, ~2GB/week, ~10GB/month. 250,000 requests/week, ~1m/month. (These figures are useful because some people quote their pricing based on them.)
I have a worrisome list of referrers which I need to look into. Also, a shitload of people hotlink sorttable.js and smoothscroll.js.
Windows Azure: no concept of redirects. More importantly, there’s no “default blob” in the Azure Blob Storage (http://stackoverflow.com/questions/8639484/howto-upload-setup-default-html-page-for-azure-blob-storage) — if you CNAME http://www.kryogenix.org to your blob storage thing, you can create a magic “container” called $root and then foobar.html in $root becomes http://www.kryogenix.org/foobar.html, but you can’t say “serve index.html if I don’t specify a specific thing”. S3 can do this: Azure can’t. Fail.
Managed WordPress hosting: wpsites seem to do what I want, but it’s early days; they have no control panel, everything’s handled by ringing them up. page.ly only host WP, not other static content. wpengine let you host at /days if you get their multisite package, but that’s 60 quid a month.
My suggestion would be: get a regular hosting solution with a ASP option for WordPress, which still is editable/customize-able by the client. I know there are a few of those around, at least in Germany, cause I did some theme and plugin adaption jobs of such installations (approx 2 years ago).
Next option (might be seen a bit off-topic, but still …): Try Habari, and be happy ever after. My personal blog and sites are all running (more or less) with it, cause the major rush to update just about EVERYTHING every few months because some major bugshit, including having to change the admin UI every second major release, like it is with WP, just DOES NOT happen. It’s developed both by former and still well deep wordpress (plugin/core/whatever) developers, so migrating stuff from WP is rather easy (mostly themes thou). It simply works (rock-stable, for 4 years by now), doesnt pester you all the time with update nag links, and if you get in trouble after a major two-version release upgrade jump, the developers themselves might appear for help if you whine or curse loud enough (like I do)
Third option: Webhosting with SSH access, cron jobs et more, so you get all the benefits of both worlds .. like automatizing the WP update crap with cron jobs (eg. first: backup all data, second: update), and the option to dwelve back into the matter if you feel the urge for (ie. more control). There aint many of these companies around, but IMHO it’s one of the better solutions so far. .. and yes, I am also with one.
Got all the happiness – regular webhosting company for my main site(s), secondary webhosting with shell access (as mentioned before), DynDNS options for your domains (which comes in rather handy when showing clients the current state of their projects on your local Apache), proper control panel etc. pp. plus my own virtual server at the Hetzner AG data center, althou mostly for playing around, “githubbing” and recording some webradio shows (including mine).
cu, w0lf.
FYI, Amazon S3 now supports redirects for individual web pages:
http://aws.typepad.com/aws/2012/10/amazon-s3-support-for-website-redirects.html