Author Topic: 2013 April 2 - AI - Mini Update  (Read 18554 times)

Jon Shafer

  • Lord of the Forest
  • *
2013 April 2 - AI - Mini Update
« on: April 02, 2013, 12:16:57 PM »
Woohoooo, the first update! This is going to be a fairly short one, but I wanted to let you guys know what I've been up to, lest you worry that I've run off to a Caribbean island something. My intention is to post one major update every month, with smaller unplanned ones in-between going up here on the forums.

So for the past couple weeks I've been heavily focused on designing the basic structure of the AI. As I've noted in previous articles, the basic goals are effective behavior and minimal mistakes, achieved with simple, targeted systems.

I've been creating several "scenario sandboxes" in the AI brainstorming docs to establish what an AI response should be to various situations, along with the process for how decisions are made. [Shameless Plug] If you'd like access to these and other documents, you can use PayPal on our website to get up to the $125 tier! [/Shameless Plug]

Right now my brainstorming has led me to an AI design with four main systems:

  • Objectives AI (OAI) - The meat of the AI where strategic plans are laid. Its varied Objectives range from "I'm out of Metal... spawn an Objective to grab that Iron deposit over there" to "The resources in this area are diminishing, I need find and head for a more fertile one" to "I don't like the Picts and they'd be easy to attack, so let's prepare an invasion."
  • High Command AI (HCAI)- This divvies up the player's Units between various "Military Goals" and orders them around. If we're trying to muster an invasion force on the eastern flank and need to draw strength from various parts of the kingdom, the HCAI is what makes that happen.
  • Tactical AI (TAI) - This system takes over from the HCAI when there are very specific, intricate tasks that must be accomplished (typically when there are enemies nearby). It runs through and scores a large number of possible action combos, choosing the best option. An approach like this is necessary if the goal is to reduce the number of mistakes to the absolute minimum - trying to teach it to "do everything right" in one go is basically impossible when there's as much to consider as there is in AtG. If the TAI slips up just once and forgets to protect a single Supply Camp, it could very easily be the end of everything.
  • Economic AI (EAI) - This handles production, spending of resources, etc. For the most part it hums along happily enough, just keeping things running. However, the OAI and HCAI can make "requests" of the EAI, such as "build more Units, ASAP!" (because we're being invaded from two sides - but the EAI doesn't know or care about that). The EAI does its best to balance these competing priorities.

There's a lot more detail I could go into here, but I'll save that for future updates, as I want to keep this one fairly high-level. In your guys' opinion, what should the AI always/never do? What's the most important thing for it (and me!) to focus on?

- Jon
« Last Edit: April 02, 2013, 01:09:04 PM by Jon Shafer »
If you have any questions, please send me a private message here on the forums or an email at [Contact@ConiferGames.com]. Thanks for your support!

Re: 2013 April 2 - AI - Mini Update
« Reply #1 on: April 02, 2013, 01:26:40 PM »
I'll look forward to seeing how those map out.  I'd be very curious to see how many thresholds you have to put in ("NEVER do X if Y is below Z") to keep it smart.

How will you fit the exploration priorities for units?  Will that be in the "Objectives AI?"

Also, when do we get access to the design docs if we pledged at the appropriate level?

Re: 2013 April 2 - AI - Mini Update
« Reply #2 on: April 02, 2013, 01:47:38 PM »
I quite like the breakdown into Objective, High Command, and Tactical levels, as that appears to pretty closely follow the highly effective hierarchical model of Strategy, Operations, and Tactics.

Some things that occur to me based on that reading:

1. There's also the highest level of Grand Strategy (basically a "vision" level of action). Although AtG seems intended to be more a fun conflict-focused game than a general nation-simulator, one element of grand strategy could be distinctive cultural characteristics. These would be the top-level qualities that influence Objectives.

To what extent will a simulated player's actions be guided by cultural elements?

2. Related to that is diplomacy ("war by other means" ;).

Is diplomacy in AtG intended to rise to the level of benefiting from its own AI management subsystem? If not, but some diplomacy is desired (as I think is the case), where would that AI go?

JB

  • *
Re: 2013 April 2 - AI - Mini Update
« Reply #3 on: April 02, 2013, 01:59:02 PM »
Well, i'm no expert on AI but i guess my first req. would be that the AI doesn't have knowledge that it should not be able to have.
But it does need to remember past intel it does have (type, amount units spotted). Pretty difficult maybe. Like panzer general where it didnt know anything about turns past so it would be easy to ambush the AI. Hope that makes sense. AI scouting should be a priority (or spying but i dont like that mechanic myself)

The rest is a bit too difficult for me, i guess the AI should try to play to win :)
But then are we going like hardcore (see player winning, allying to destroy player) or more civ-like. I dont know.


Re: 2013 April 2 - AI - Mini Update
« Reply #4 on: April 02, 2013, 03:15:10 PM »
Also, to offer one answer to the question that was actually asked  :P, the one thing I'd like to see for sure is clarity.

I'd like to be able to to see an AI opponent's actions and think, "ouch, but that makes sense." Roman actions should seem like things the Romans would do (however those are defined); they should be consistent throughout the game; and they should be properly signaled ahead of time.

These are all features that support the appearance of fairness in a game with AI players. If they aren't consistent, that appears random. And if the human player isn't given an opportunity to form plausible strategies based on observed opponent behaviors, that takes away much of the fun in a strategy game.

So "strategic clarity" would be at the top of my must-have list for opponent AI.

Re: 2013 April 2 - AI - Mini Update
« Reply #5 on: April 02, 2013, 03:20:28 PM »
I like the idea of using diplomacy from Flatfingers. Maybe it should be a little like poker where you are not sure if they are bluffing or not. Say you got them on the ropes and they ask for mercy, which could be true or a trap. Or conversely, the envoy could threatens us by disclosing intel that an army (theirs or allied) is going to attack our flanks/supply lines/cities if we don't back off, this could be empty bluster or not. All this of course can't be used all the time the AI is under pressure, that would be annoying, but it would be good for a change of pace I imagine.   

Re: 2013 April 2 - AI - Mini Update
« Reply #6 on: April 02, 2013, 03:24:06 PM »
Hello there everybody.. :)

well thinking about how my opponent should act, this comes into my mind:
we will fight against Romans, right? So... The AI should be as close as possible to a Romans habit.
Now Im no Romans specialist, but I still remember some (basic) stuff from school.

Romans were VERY organized and had magnificent leaders. When I am not wrong - a weakpoint was their unability to act very fast and spontaneaously. Right? So a "long", but effective build up of army could be considerable.
As programming such a massive weakpoint would probably end up in a very boring game - Romans did something against it:

The ability and trust of the dictators appointing regional leaders is another thing, which could be awesome. Because these leaders could differ from each other a lot. Starting from "killing their own units, because they are weak"  to "anxious leaders with luck" and ending with "Rebels in their own system (heading for the throne)". Their weakpoint "slow" could be balanced by a young regional leader, who tries to impress everybody and goes out for a quick hunt (sometimes too quick probably...).



JB wrote the opponent should be able to remember the past. Means "Learn" basically. I totally agree with this and would find it astonishing to read a message pointing out what i did earlier. Example for a narcisstic, cocky/jaunty regional leader: "My merchant-spys on your market/streets just reported me you save ressources/build archer army/big army/whatever... I will have an eye on you." Then later: "I assume you  saved enough now and you built your second barrack already. Come over NOW!!" (He prepared a better defense already.. etc.)
Or when I attacked him with several small troops in winter weakening him, he will (dependig on his character) change his defense for next winter or strike back ridicously, because he got angry of my teasing.
Learning can as well simply be to remember whether I used more riders or archers at my last attack and adjusting his army to his observations at his next strike.
All this would keep my head busy during playing and it is not just gathering ressources, defending a little, building up a huge army - strike once. win.

Jon.. Sorry, if I drifted off - Im no programmer and never set up an AI, so Im just writing what comes into my mind and what I would consider as "WOW! Awesome reaction!". As well I dont know yet how exactly the gameplay will be - so maybe excuse suggestions, which are simply impossible to implement.


Cheers
Peter

Jon Shafer

  • Lord of the Forest
  • *
Re: 2013 April 2 - AI - Mini Update
« Reply #7 on: April 02, 2013, 05:07:12 PM »
I'll look forward to seeing how those map out.  I'd be very curious to see how many thresholds you have to put in ("NEVER do X if Y is below Z") to keep it smart.

The plan is for every Objective to have custom code - so I imagine there will be a ton of them!


Quote
How will you fit the exploration priorities for units?  Will that be in the "Objectives AI?"

Priorities such as "Explore the area around us so that we can see what Resources are nearby" will indeed be decided at the OAI level. However, exploration tends to be pretty straightforward - you generally want to maximize the number of tiles uncovered with each move and avoid getting too close to other explorers and enemies.

Outside of those things, what sorts of priorities were you thinking of?


Quote
Also, when do we get access to the design docs if we pledged at the appropriate level?

Couple more weeks. Thanks for your patience! :)

- Jon
If you have any questions, please send me a private message here on the forums or an email at [Contact@ConiferGames.com]. Thanks for your support!

Re: 2013 April 2 - AI - Mini Update
« Reply #8 on: April 02, 2013, 05:32:58 PM »
Not sure about the feasibility, but I'd love the ability to integrate these mechanics into a player-controlled empire. Turning on the Economics AI, so I can focus on the combat would be one example present in other games (ie Civilization.)
Is this a possibility, if not in the base game, then with mods?
I've always wanted a game where you could delegate uninteresting bits, such as a predictable tactical battle, to an AI, and then 'jump in' if you see a particular opportunity.

Jon Shafer

  • Lord of the Forest
  • *
Re: 2013 April 2 - AI - Mini Update
« Reply #9 on: April 02, 2013, 05:34:45 PM »
I quite like the breakdown into Objective, High Command, and Tactical levels, as that appears to pretty closely follow the highly effective hierarchical model of Strategy, Operations, and Tactics.

Some things that occur to me based on that reading:

1. There's also the highest level of Grand Strategy (basically a "vision" level of action). Although AtG seems intended to be more a fun conflict-focused game than a general nation-simulator, one element of grand strategy could be distinctive cultural characteristics. These would be the top-level qualities that influence Objectives.

To what extent will a simulated player's actions be guided by cultural elements?

Cultural preference will definitely be represented, and you're right that an overarching game strategy will have to be represented in some manner. I haven't decided whether I want it to be a distinct "grand strategy" level of AI, or simply a blend of important Objectives and built-in leader personality preferences. You nearly always want the simplest system that can get the job done, so it's a matter of whether or not this can be done with what I've outlined here, or if another layer is needed.


Quote
2. Related to that is diplomacy ("war by other means" ;).

Is diplomacy in AtG intended to rise to the level of benefiting from its own AI management subsystem? If not, but some diplomacy is desired (as I think is the case), where would that AI go?

The plan is for diplomacy to be handled by the OAI. But don't think that's because it's not important!

The role of the OAI is to serve as the "prioritization" part of the AI brain. It decides how the player should be spending its time and resources, and ensures there's no confusion as to whether it's more important to grab that Iron deposit near Rome, to make friends with the Goths and avoid moving Units near them, or amassing all hands on deck to fight off the Huns.

In my mind, a unified system is the easiest way to ensure the AI is always consistent and working towards a clear set of goals.

This could also be done with a variety of subsystems, but I want something stupid simple to understand and debug. If the AI's #1 Objective is making friends and yet it happens to be declaring war on everyone, we know there's a bug that needs fixing. On the other hand, if we see the AI is prioritizing something that we feel isn't worthwhile we can dig into the scoring and correct that.

Not only will having a single OAI system make it super easy to work with what we already have, but it also lends itself well to the addition of new features. Is it time to teach the AI how to migrate? No problem - we'll just add a new Objective with the appropriate triggers and desired priority. Once it's smartly deciding when to migrate, we can then work out the details for how that should be done. It would be much harder to get every piece in the right place if we were starting from scratch with a completely independent "Migration AI subsystem."

That's the theory, anyways. :) I still have plenty of design work left to do, and there's always a chance I completely change my mind in the next few weeks!

- Jon
If you have any questions, please send me a private message here on the forums or an email at [Contact@ConiferGames.com]. Thanks for your support!

Jon Shafer

  • Lord of the Forest
  • *
Re: 2013 April 2 - AI - Mini Update
« Reply #10 on: April 02, 2013, 05:42:32 PM »
Well, i'm no expert on AI but i guess my first req. would be that the AI doesn't have knowledge that it should not be able to have.
But it does need to remember past intel it does have (type, amount units spotted). Pretty difficult maybe. Like panzer general where it didnt know anything about turns past so it would be easy to ambush the AI. Hope that makes sense. AI scouting should be a priority (or spying but i dont like that mechanic myself)

Good request. I definitely want the AI to have some sort of memory of the map, but I haven't yet figured out the best way to accomplish this.


Quote
The rest is a bit too difficult for me, i guess the AI should try to play to win :)
But then are we going like hardcore (see player winning, allying to destroy player) or more civ-like. I dont know.

A strategy game AI should always be able to present a challenge. It's much easier to make a smart AI dumb than "teaching" a dumb AI to be smart! :) It's particularly important that the AI in AtG knows what it's doing since combat is so heavily dependent on supply, rather than sheer numbers.

- Jon
If you have any questions, please send me a private message here on the forums or an email at [Contact@ConiferGames.com]. Thanks for your support!

Jon Shafer

  • Lord of the Forest
  • *
Re: 2013 April 2 - AI - Mini Update
« Reply #11 on: April 02, 2013, 05:59:52 PM »
Also, to offer one answer to the question that was actually asked  :P, the one thing I'd like to see for sure is clarity.

I'd like to be able to to see an AI opponent's actions and think, "ouch, but that makes sense." Roman actions should seem like things the Romans would do (however those are defined); they should be consistent throughout the game; and they should be properly signaled ahead of time.

These are all features that support the appearance of fairness in a game with AI players. If they aren't consistent, that appears random. And if the human player isn't given an opportunity to form plausible strategies based on observed opponent behaviors, that takes away much of the fun in a strategy game.

So "strategic clarity" would be at the top of my must-have list for opponent AI.

I agree 100%. The main goal behind the OAI is to provide computer players with strategic vision and game-wide consistency. If our number one priority is to be friends with the Romans, then that should permeate every decision we make. And if the AI is too focused you can always make it more random or unpredictable.

- Jon
If you have any questions, please send me a private message here on the forums or an email at [Contact@ConiferGames.com]. Thanks for your support!

Jon Shafer

  • Lord of the Forest
  • *
Re: 2013 April 2 - AI - Mini Update
« Reply #12 on: April 02, 2013, 06:04:32 PM »
I like the idea of using diplomacy from Flatfingers. Maybe it should be a little like poker where you are not sure if they are bluffing or not. Say you got them on the ropes and they ask for mercy, which could be true or a trap. Or conversely, the envoy could threatens us by disclosing intel that an army (theirs or allied) is going to attack our flanks/supply lines/cities if we don't back off, this could be empty bluster or not. All this of course can't be used all the time the AI is under pressure, that would be annoying, but it would be good for a change of pace I imagine.

This does sound awesome, but it's very tough to achieve this with computer opponents. By default, humans tend to think of AI as dumb and random (rightfully so), and your task as a developer is climbing the mountain and convincing them that the AI does have a plan and are in fact competent. Once you get to that point, you can then play a few tricks, but it's all for naught if never get to the point where the AI appears to know what it's doing. Very, very few games have even gotten to that point.

- Jon
If you have any questions, please send me a private message here on the forums or an email at [Contact@ConiferGames.com]. Thanks for your support!

Jon Shafer

  • Lord of the Forest
  • *
Re: 2013 April 2 - AI - Mini Update
« Reply #13 on: April 02, 2013, 06:12:48 PM »
Hello there everybody.. :)

well thinking about how my opponent should act, this comes into my mind:
we will fight against Romans, right? So... The AI should be as close as possible to a Romans habit.
Now Im no Romans specialist, but I still remember some (basic) stuff from school.

Romans were VERY organized and had magnificent leaders. When I am not wrong - a weakpoint was their unability to act very fast and spontaneaously. Right? So a "long", but effective build up of army could be considerable.
As programming such a massive weakpoint would probably end up in a very boring game - Romans did something against it:

The ability and trust of the dictators appointing regional leaders is another thing, which could be awesome. Because these leaders could differ from each other a lot. Starting from "killing their own units, because they are weak"  to "anxious leaders with luck" and ending with "Rebels in their own system (heading for the throne)". Their weakpoint "slow" could be balanced by a young regional leader, who tries to impress everybody and goes out for a quick hunt (sometimes too quick probably...).

JB wrote the opponent should be able to remember the past. Means "Learn" basically. I totally agree with this and would find it astonishing to read a message pointing out what i did earlier. Example for a narcisstic, cocky/jaunty regional leader: "My merchant-spys on your market/streets just reported me you save ressources/build archer army/big army/whatever... I will have an eye on you." Then later: "I assume you  saved enough now and you built your second barrack already. Come over NOW!!" (He prepared a better defense already.. etc.)
Or when I attacked him with several small troops in winter weakening him, he will (dependig on his character) change his defense for next winter or strike back ridicously, because he got angry of my teasing.
Learning can as well simply be to remember whether I used more riders or archers at my last attack and adjusting his army to his observations at his next strike.
All this would keep my head busy during playing and it is not just gathering ressources, defending a little, building up a huge army - strike once. win.

Jon.. Sorry, if I drifted off - Im no programmer and never set up an AI, so Im just writing what comes into my mind and what I would consider as "WOW! Awesome reaction!". As well I dont know yet how exactly the gameplay will be - so maybe excuse suggestions, which are simply impossible to implement.

Haha, no problem!

It's easy to get caught up in the nitty-gritty details of architecture and efficient algorithms, but at the end of the day the goal is to have an opponent that improves the feel of the game. Some players want ruthless, cunning opponents while others want ones that have distinct, interesting personalities. These are not mutually exclusive goals, but you can only achieve them if you keep your eye on the end target.

This is why I'm spending so much time on "sandbox scenarios," as this is by far the best way to ensure your design is accomplishing more than some basic math under the hood. Writing an elaborate system for AI naval invasions is pointless unless you have a clear idea of how it's actually supposed to play out, and whether the computer is even equipped to handle the variety of in-game situations it might encounter.

In many ways, designing the AI is more important than designing the gameplay. It's easy to iterate on the latter, as humans interact directly with the mechanics and can decide for themselves if they're fun or not. With AI you're at least one and sometimes many more steps removed, and many times you won't even know what it's doing, let alone whether it's behaving intelligently. This is also why working on AI is one of my favorite parts of game development. :)

- Jon
If you have any questions, please send me a private message here on the forums or an email at [Contact@ConiferGames.com]. Thanks for your support!

Jon Shafer

  • Lord of the Forest
  • *
Re: 2013 April 2 - AI - Mini Update
« Reply #14 on: April 02, 2013, 06:16:00 PM »
Not sure about the feasibility, but I'd love the ability to integrate these mechanics into a player-controlled empire. Turning on the Economics AI, so I can focus on the combat would be one example present in other games (ie Civilization.)
Is this a possibility, if not in the base game, then with mods?
I've always wanted a game where you could delegate uninteresting bits, such as a predictable tactical battle, to an AI, and then 'jump in' if you see a particular opportunity.

In theory, this should be very easy to do - it's just a matter of flipping a switch and handing the reigns to the AI instead of the human. The trickier part is just keeping track of everything. You're not going to see every move the AI makes step-by-step, and if you want to keep track of when it, say, chooses new Romanization Perks you're going to have to be on top of that. Well, either that or you need to be looking at the AI logs every turn, but that's not quite the same!

- Jon
If you have any questions, please send me a private message here on the forums or an email at [Contact@ConiferGames.com]. Thanks for your support!