There is something very odd about seeing a grown-up Will Wheaton playing TableTop games; in my head, he’s still about 15 and on the Enterprise bridge (TNG). The Flog is also worth a watch to see all the gaming and meme references.

1. I mostly download games through the likes of Steam, no need to wait for postie;
2. I have a 50Mb internet connection (soon to be 100Mbit, thank you Virgin Media)

So, my dilemma isn’t: Should I pre-order this so that I can get it on release day? No, my dilemma is: Should I pre-order this so that I can get the pre-order DownLoadable Content (DLC)?

Don’t get me wrong, back in the day when this sort of thing used to be called an expansion, I used to love it. Many modern games continue this tradition of adding new and fresh content to games, see my ‘good’ point below. However, DLC nowadays seems to be a byword for revenue generation, with actual gameplay a distant second. I shall explain…

Expansion DLC is the only DLC that I’ll pay good money for. This is the stuff that adds genuine new content to the game, doesn’t require any prerequisites (in terms of game play), and is available to anyone who already owns the game. Examples include the Total War series, most recently Total War: Shogun 2: Rise of the Samurai, but there are examples in everything from Wing Commander to Railworks. It doesn’t give [m]any advantages over those who don’t own it, it’s available to all, it can be played in isolation to the main plot. In all, most of the DLC for the Total War series has fallen into this category, adding additional units and starts/campaigns to the game.

Then we get into the DLCs that I dislike. There are a few, but the mainly apply to Bethesda and EA games:

Pre-order DLC,  the “buy the game in advance and receive an in-game X” content. Whilst I can appreciate that game publishers want you to pony up the cash in advance, my last few pre-orders (which I shall blog about another day) have been lame horses when finally released. Moreover, this DLC is often a one-shot; if you don’t pre-order then there’s only the slim chance that the DLC will be in a Gold or Complete edition of the game  released a few months/years down the line. This isn’t an old concept, so there are plenty of examples, but Mass Effect 2 is the one that demonstrates it well.

Retailer specific DLC, the “pre-order from retailer X and receive exclusive in-game item Y” content. This is a relatively new concept, but it annoys the hell out of me. If you buy from retailer X you’ll receive item X, order from retailer Y and you’ll receive item Y. In many cases these are also region locked, there won’t necessarily be a retailer in your region that sells the game with item Y. If I want item X or Y, I’m locked into a specific vendor, and often have to source and then wait for a physical copy of the game (Amazon and Game don’t do digital distribution). A recent example of this is Deus Ex:Human Revolution, where there were both retailer specific and region specific exclusive items.

Overpowered DLC, is the sort of DLC that changes the balance of the game. Usually these items are massively overpowered, and can be used throughout the entire game, negating the need to search for and upgrade starting items. Conversely, the item is overpowered, but ammunition is so rare that the item is useless once it’s been ‘used’. Again, Mass Effect 2 is a good example here, where the pre-order weapons were more powerful than many late-game items.

Vanity DLC, does absolutely nothing to the game, other than make it look visually different; there’s no advantages here other than showing off to your mates. Fortunately, this concept is so stupid that it’s relatively rare, but a good example is the weapon skins in Gears of War 3.

Promotional DLC, is the sort of thing where you get DLC for doing/purchasing something completely unrelated to the game. Again, Mass Effect 2 is the example here, where there was a promotion run with Dr Pepper. I’ve recently heard that although these codes are easy to find on the internet, Bioware no longer allows users to redeem them.

Other purchases DLC (play demo, get DLC) generally requires you to own another game by the same developer or publisher. Sometimes just playing a demo of the other game will suffice, but often you have to own the game outright, and have reached a certain point in the game or unlock a certain achievement. Bioware’s games are a shining example of this, where the Blood Dragon Armour persists across several games; unlocking it in an earlier game will unlock it in a later one. Mass Effect 3 is also guilty of doing this, requiring you to play demos of other games in order to unlock Mass Effect 3 items.
In fact, Mass Effect 3 goes comically further on this one, its been recently announced that purchasing physical figurines of in-game characters will unlock DLC.

If you’ve read this far, you’ll realise that a lot of games mix these types of ‘bad’ DLC *cough*MassEffect*cough*. It wouldn’t be a problem if these items were only a few pounds, and available to all; but often they are locked into significant additional expense (>£5) and only available for a limited time.
So, that’s the end of my rant. I have until the beginning of March to decide whether I should pre-order some games…

For the past three months, since October 2011, our Moodle serve hasn’t been compiling statistics.

I can’t fully demonstrate the symptoms here, as I only decided to write this blog post after I had fixed things. But if you look at the graph below, you’ll see an earlier month where statistics also failed. 1st April 2011 has no actions logged.

Example Moodle statistics

This fix works on Moodle 1.9.3+, it should also work on Moodle 2.0 but some of the lines might be slightly different. YMMV

So… how to fix it… well…

First off, log in to your Moodle server and look at the Server > Statistics settings. Ensure that the following are set:

• Enable statistics – checked (duh!)
• Maximim processing interval – all
  (This is the earliest date for which Moodle will try to compile statistics.)
• Maximum runtime – until complete
• Days to process – 2
  (This is how many days worth of statistics will be processed each day, best to keep this >1 and <31
• Run at – whatever you like, we’ll ignore this later
• User threshold – 0
• Maximum parent categories – 1

Then, open up your favourite database editor and look at the Moodle database. There are a few rows that you’ll need to adjust, which can best be found by running:

SELECT [id]
      ,[name]
      ,[value]
  FROM [moodle].[moodle].[mdl_config]
  WHERE name LIKE '%stats%'

The fields you want to edit, are:

• statslastdaily = 0
• statslastweekly = 0
• statslastmonthly = 0
• statslastexecution = 0

Finally, you’ll want to change the /admin/cron.php so that it only compiles statistics, and it ignores when it was last run. The best way of doing this is to take a copy of the existing cron.php, and rename it to something sensible, eg. cronstats.php. Then, editing our copy of cronstats.php:

• Keep everything above:

  mtrace("Server Time: ".date('r',$timenow)."\n\n");

• Keep everything below the line:

  //Unset session variables and destroy it

• For the rest of the document, remove everything except the following:

      if (!empty($CFG->enablestats) and empty($CFG->disablestatsprocessing)) {
          require_once($CFG->dirroot.'/lib/statslib.php');
          // check we're not before our runtime
          $timetocheck = stats_get_base_daily() + $CFG->statsruntimestarthour*6$
  
          if (time() > $timetocheck) {
              // process configured number of days as max (defaulting to 31)
              $maxdays = empty($CFG->statsruntimedays) ? 31 : abs($CFG->statsrunt$
              if (stats_cron_daily($maxdays)) {
                  if (stats_cron_weekly()) {
                      if (stats_cron_monthly()) {
                          stats_clean_old();
                      }
                  }
              }
              @set_time_limit(0);
          } else {
              mtrace('Next stats run after:'. userdate($timetocheck));
          }
      }

• Then, comment out some lines so that it reads:

      //if (!empty($CFG->enablestats) and empty($CFG->disablestatsprocessing)) {
          require_once($CFG->dirroot.'/lib/statslib.php');
          // check we're not before our runtime
          //$timetocheck = stats_get_base_daily() + $CFG->statsruntimestarthour*6$
  
          //if (time() > $timetocheck) {
              // process configured number of days as max (defaulting to 31)
              $maxdays = empty($CFG->statsruntimedays) ? 31 : abs($CFG->statsrunt$
              if (stats_cron_daily($maxdays)) {
                  if (stats_cron_weekly()) {
                      if (stats_cron_monthly()) {
                          stats_clean_old();
                      }
                  }
              }
              @set_time_limit(0);
          //} else {
          //    mtrace('Next stats run after:'. userdate($timetocheck));
          //}
      //}

This last step will stop the cronstats.php script from checking that it’s already run that day, meaning that you can run the script several times in succession. This is handy because something must have prevented the statistics from being compiled in the first place, and we want to determine what that is without having to reset the mdl_config table each time.

Warning! Running the cronstats.php will use a significant amount of CPU/RAM on your database server. The queries that Moodle uses to compile statistics are quite complex. It’s fine to run on a live server, just bear in mind that things will slow down.

Now, if you run the crontstats.php from the command line (or directly via the web, if you prefer). It should go off and compile the statistics for a few days (whatever you set ‘Days to process’ to, above). If you’re lucky, the script will start to list hours of the day, and then eventually complete with the date that it’s just processed. eg. 0:1,1:20,2:30 … January 1 2012.

If you’re unlucky, like me, your script will get to the 0:1,1:20,2:30 part, then fail miserably with a SQL error. Which is probably what caused the script to fail in the first place, 3 months ago. But, at least we can now run this script repeatedly, and see what’s going on behind the scenes.

Warning! Until now, we’ve just been altering some configuration settings and timestamps, we’ve not changed anything serious. These next steps will vary wildly, depending on your database platform (we use MSSQL), and your experiences. You have been warned.

Part of the reason the cronstats.php script is not completing is because its taking too long. I can’t prove this, but the 100% CPU usage on our SQL server, and the 3 minutes it took the script to run before it gave up, are good indicators. For MSSQL we can run the profiler and database tuning wizards to decide whether additional indexes would improve things. Again, YMMV, but in our case the SQL server suggested a 30% improvement if we added some indexes, mainly to the mdl_context table.

After these index improvements, cronstats.php runs successfully, and manages to build statistics. In our case there was one last error where the SQL server timed out whilst trying to run some DELETE statements on the mdl_stats_… tables, but fortunately the cronstats.php script will list the statements that have failed, and you can run these by hand.

Final big warning! This generates a very large number of transactions in your database. Processing 3 months worth of statistics created a MSSQL transaction log in the region of 20GB. Don’t be a numpty like me and ignore this. I’ve spent most of this morning cleaning up transaction logs and disk space on our database server. It might be better to spread this entire process over a few days, or change the transaction logging method on your database whilst you run this.

Finally, a few observations from this:

• MSSQL Profiler and Database Tuning Wizard are excellent tools. The Moodle database isn’t the most efficient, especially on big installations, and inefficient indexes can cripple performance.
• Make sure you have enough disk space on your database servers, no, really!
• Always set your “Days to process” to more than 1 day. Since the statistics are only compiled once per day, you’ll want to have this set somewhere between 2 and 7; any less and we could start to see a backlog.
• Keep a regular eye on the statistics. It’s very quick and easy to recover a week or two of statistics, but any mocan be problematic (see my server snafu, above).

That’s it. I am sure there will be more Moodle adventures soon.

I’ve decided on a bit of a fresh start, partly driven by the new blogs of my colleagues (see links), but also driven by the new year.

Hopefully, this attempt will be a bit more regular.