Xbox Live Tracker Keeping track of the games.
One of the things I enjoy doing in my free time is playing games on my Xbox. Now, you might not find that fact all that surprising, given that pretty much the only remotely significant thing that is on this site at the moment is the Xbox Tracker, and that most of my posts in the last week have made mention of the tracker.
Anyway, I thought I’d bring it all together to talk about what I did and how I did it, for two reasons really. So I had something to write a post about, and because I found writing it really really interesting.
I have, in the past, built a node.js based API so that myself, and some of my select friends, can directly query Xbox Live achievement data without having to go through the rigomorole of handling the login and html scraping of the ruefully awkward Xbox Live site. It’s a simple wrapper that uses Phantom JS (and Casper JS) to log in to the site, load the appropriate page, retrieve the HTML, and then pull the appropriate data from it, and return it all as a JSON string.
So, my plan was to utilise the data from this API, to automatically update files on GitHub using the API to commit the data. This did, however, drive my requirement to build a Jekyll website that would be compiled by the version of Jekyll available on the GitHub pages platform, and therefore leave me somewhat limited as to the functionality I could include on the site.
The basic logic I wanted to follow was:
Most of it was pretty straight forward, really, but the biggest hurdle was building the Jekyll pages such that the game pages would be generated appropriately. I didn’t want to get to the point of having a hardcoded page layout within the code, or some kind of template that was external from my Jekyll site. Mainly because this meant that it would be hard to backdate any changes to the templates, plus it means that any data would be difficult to migrate to a new system. I also wanted the data to be easy for the Tracker to read. Afterall, it would have to read it in some manner to get the scores and achievements ready for comparison.
Reading From GitHub
Handily, the GitHub API provides a nice method to read a file from a repository - assuming you know the path to the file you want to read. And this works fine for the overview page, as we can set where that’s generated. In my case it would be within
The API methods I use for this is the get contents call - and that’s pretty straight forward. The reason that I expplicitly state
master branch is because I’ve built this code to be repo agnostic - and I want to be able to use the collections of libraries I built to potentially power a collection of Jekyll sites, using
gh-pages setup, and so forth.
Getting the data from the individual game pages is a bit more involved, because we need to load a bunch of pages. Fortunately this is pretty simple with the GitHub API using a combination of the above mentioned get contents call and the get recursive tree call. However this requires a tree hash, so this is a 3 step process:
I needed a way to uniquely identify any game, and any achievement if I was to reliably to any kind of matching between what was stored within GitHub and what was being returned by the Xbox Live scraper. Towards this end I wrote a very simple
String.slugify() function, that would generate slugs for games and achievements in such a way that I could count on the consistency of the output. What we wanted to do was convert all the potential multi-byte characters either into non-entities (for example ™ and © aren’t really required for a game name) or into meaningful representations of themselves (ü into
ue, & into
and, and so forth) so that the slugs would still make sense in the context of the games and achievements they were representing.
This also ensured that the identification strings would be in
UTF-8 format, which is much simpler to deal with, and allows me to easily create filenames that make sense as URLs.
So games are a slug of the gamename - and the achievements have a slug of the game name concatonated with a slug of the achievement name. So, for example, Lego Lord of The Rings (game page) is identified on the achievement page as Lego LOTR - meaning its slug is
lego-lotr. The achievement “This is no mine… it’s a tomb.” has the slug
YAML Front Matter
I realised that the YAML Front Matter of the pages could be used to this end pretty effectively, and using the layouts stored within the
_layouts directory to act as renderer for that data. This meant that, handily, all the design and layout for the data from Xbox Live would be kept as an internal part of Jekyll.
I always try to write any code, even stuff that only I am ever to use, in such a way that in theory it’s completely independant to what I require - and the final stuff that’s specific to me is handled by configuration, or at the final presentation layer. I find staying in that mindset helps me stay pretty on the ball in any project.
I used js-yaml to both parse and serialise the YML from the Front Matter for use within the node.js code. This meant that once I had loaded the page content from GitHub by the API, I could extract and load the YML into a JSON object with something as simple as (it’s worth noting that in the code I implemtented there was more error checking - just in case):
Thus leaving me with the page content, and the YAML handily seperated, along with the path that the file had (allowing me to ensure that when I saved modifications back to GitHub they would overwrite the old file correctly). This meant that any modifications to the file outside of the Tracker would be preserved. For example, if I wanted to add some MarkDown to talk about the game page, it would be there, and the page would just be updated with new achievements.
I wanted the xbox specific to be contained in a single variable within the YAML, and for it to store as much data as is possible - just so that when I came to present the data, I would have as much to play with as I wanted. As you can see, however, from the page itself - I didn’t end up using all the data, but it did give me flexibility when it came to the design itself. I also kept a list of the 20 latest achievements, allowing me to show such a list on my homepage, or on the overview page - just to make things a bit nicer.
The game pages were going to require a bit more thought, as I can’t really hardcode for them within the templates, so I needed to use an idea already extant within Jekyll Bootstrap which is to add an attribute of “group” so that I can iterate through all the game pages, and only display the ones that are games (along with some other conditionals that I don’t want to go through at this point).
Bringing it all together
To view the outputted pages you can view the games volder within this sites github repository here. The best way I can think of to bring all elements of the build together in an easy to read format is with another web sequence diagram that explains all the logic that the system goes through (warning; this is pretty exhaustive):
This was actually a really fun and interesting project to work on, and i can only hope I explained something in an understandable way. I’ve gone into the main challenges that I worked through, along with a sequence diagram to explain the full logic on how the system works, and hopefully that is understandable. It turns out that using the GitHub API and making sure my Jekyll site can be compiled by GitHub Pages itself means that I can add some really nice bits of “dynamic” functionality to my site without having to worry about backend code securities, or even hosting. Plus it means that everything on my site is versioned, including all the achievement updates I get. This means GitHub can tell my how much time i’m spending on my Xbox, so I can feel guilty when I see it on day to day work hours…
Related Posts They might not be good...
- Tech - 7 posts