Author Topic: 2013 September 4 - The Strategy of Playtesting  (Read 5087 times)

Jon Shafer

  • Lord of the Forest
  • *
2013 September 4 - The Strategy of Playtesting
« on: September 04, 2013, 04:32:08 PM »


Hey all, just a short update on recent goings-on, plus a special a peek behind the curtain regarding how developers decide when to pull back said curtain. I know many of you are very eager to get your hands on the game, and we'll be kicking the alpha off in a few weeks. So what have we been up to, and why the wait?



AI Status Report

I'm in the process of finishing up the first batch of key AI functionality. The basic framework for all AI systems (except for diplomacy) is now in. The AI knows how to explore, account for supply, train new units, defend itself, and survey/improve resource deposits. The last two items on my list are teaching the AI how to take out nearby hostile camps, and migrating once an area's resources start running out. At that point they'll be at least semi-competent at the early game, and - officially - able to play the game.

On a related front, Jonathan is almost done untangling the complex web of pain that is pathfinding, a feature needed by the AI and for the issuing of multi-turn move orders. Among many other benefits, I'm particularly excited about the ability to see which tiles you're able to move to during the turn. Blind trial and error is not most people's preferred method of learning!

Once these tasks are wrapped up we'll be doing a week of internal playtesting and tweaking to ensure the game can be played for at least a few minutes before devolving into a charred mass of fail.

Okay, sure, the game isn't done yet - this is neither news nor a concern if you're a member of our dedicated initial test group. After all, if the game were ready then it wouldn't be an alpha test! So what's the rationale behind holding off for another few weeks?



Deciding When to Start Playtesting

First impressions are incredibly valuable. In fact, probably the most important feedback a developer can get. They offer a nearly-unbiased metric for what's actually working and what needs the most attention. It's tempting to shrug off early complaints and say "that's no big deal, we'll fix it later." But sometimes "later" never comes. And even if it does, that seemingly minor problem might have been masking one far more insidious. And now that you only have a month left it's far too late to do anything about it. Woops!

Veteran playtesters are crucial in a variety of other ways, but after playing for a while you become blind to certain high-level flaws that newbies spot easily. Your stock of these bullets is limited, and it's unwise to start firing them at an early experience you know to be completely unrepresentative of your final vision. This is true with any game, but let's take a look at a couple tangible examples from AtG.

The supply feature has yet to be tested rigorously and it's pretty much guaranteed the amount provided by each terrain, consumed by each Unit, the effect radius of Supply Camps and so on are all completely out of whack. If we started testing right now most players would find the system completely brutal and (gulp) unfun. But is this because our design vision for supply is completely broken? Or is it simply the result of a few numbers slightly askew? This determination is hard under ideal circumstances, and at this early stage you honestly have no clue until you're sure reality matches up at least roughly with the idealized design that lives in your head.

Exploration might be an even more poignant illustration of this. Uncovering a shrouded map is fun because you're constantly discovering something new - but after you've seen it all that excitement diminishes somewhat. We've invested heavily in making this aspect of AtG very strong, but does it really work? Until the AI is playing the same game as the human there's no way you can accurately gauge this. If you're able to grab all of the goodies you'll develop a highly skewed opinion of the reward pacing. And unlike supply, once you've seen the surprises there's no going back. Sure, exploration is still fun the hundredth time you play, but in a very different way. That's not the kind of blind spot you want to just shrug off.

Playtesting too early isn't even the biggest risk, as it's much easier - and far more dangerous - to pursue the opposite extreme. Everyone is sensitive to criticism and there's always a voice in the back of your head encouraging you to spend too much time polishing your work before opening it up to the world. The problem is that the more you invest the more you waste when you change gears. And trust me, change gears you will. A lot. In the best-case scenario you've wasted a lot of time, and in the worst your game is far worse than it could - and should - have been.

Developers have a tough balancing act in deciding when to reveal their game to the outside world. There's no formula that works in all situations, but there's no doubt in my mind that it's better to err on the side of early.

And lean that direction we shall. Just... not quite yet! Thanks again for your patience and support!

- Jon
If you have any questions, please send me a private message here on the forums or an email at [Contact@ConiferGames.com]. Thanks for your support!

Re: 2013 September 4 - The Strategy of Playtesting
« Reply #1 on: September 05, 2013, 11:00:51 PM »
There have been many times in playtesting where the game changed from its original incarnation. Most of the time this doesn't bother me as I'm expecting change (hopefully improvement), but sometimes even a small change can turn the game into something completely different. The issue isn't so much the changes themselves, but my expectations of what the game is to become.

I like the idea of immutable design goals (mentioned on one of the podcasts?) that set the intent of the game. Other areas may chop & change to get the rest of the game to work, but keeping everyone clear of what areas are the backbone of the design can keep player's expectations in line with the eventual vision of the completed game.
« Last Edit: September 06, 2013, 04:23:12 PM by VRBones »
  • OS: English Win 7 with Steam
  • Hardware: AMD Phenom X4, 8Gb RAM, nVidia GeForce 560 @ 1680x1050

Jon Shafer

  • Lord of the Forest
  • *
Re: 2013 September 4 - The Strategy of Playtesting
« Reply #2 on: September 06, 2013, 04:15:52 PM »
There have been many times in playtesting where the game changes from its original incarnation. Most of the times this doesn't bother me as I'm expecting change, but sometimes it changes the game into something completely different. The issue isn't so much the changes themselves, but my expectations of what the game is to become.

I like the idea of immutable design goals (mentioned on one of the podcasts?) that set the intent of the game. Other areas may chop & change to get the rest of the game to work, but keeping everyone clear of what areas are the backbone of the game (in terms of design) can keep player's expectations in line with the eventual vision of the completed game.

It's indeed a tough balance to strike. Your ultimate goal is an awesome game, but how much are you willing to change along the way?

Most features (e.g. shrouded resource deposits) should always remain up in the air, but you'll likely also have some that are fundamental to "the game" you're making, and know will be included in some form or another (e.g. seasons, supply).

I have indeed spoken on the podcast about my general philosophy, which is to decide the game's theme and feel up front, then basically just go where the currents take you. It's easy to fall in love with very specific ideas you've come up with, but that is very dangerous.

A designer needs to be as objective as possible. I often like to use the analogy of the designer as a sieve, where your job isn't so much to make all of the decisions so much as to be the arbiter as to what ideas that make sense within the framework of the overall vision.

I'm very fortunate that I have a temperament very open to feedback. Hearing criticism aimed at something I came up with still makes me wince, but I recognize the value it has and that there's a reason why someone has something negative to say. Even if the specific feedback isn't on the mark, the feeling behind it always real.

The key to success with any creative endeavor is a strong vision coupled with the flexibility to change details as necessary. I expect AtG to be a success story born out of that approach. But I suppose we'll see!

- Jon
If you have any questions, please send me a private message here on the forums or an email at [Contact@ConiferGames.com]. Thanks for your support!