Modding the AI

Discussion in 'Modding' started by Ftoomsh, Oct 29, 2017.

  1. Ftoomsh

    Ftoomsh Well-Known Member

    The question has been asked, can we do something about making the AI better in OCMOD3? The answer is yes but it is a hard project. I am mainly a parameter modder which means I am not good at new programming. A certain amount can be done with parameter modding. Examples would be changing market sales and changing units produced by nations and doing this to suit the overall new balance in OCMOD3. For example, with a 5,000s resources start, Algeria can rush best in the current OCMOD3 by selling all remaining iron for stone after making blacksmith, all coal for wood and selling 4,000 gold (2,000 for stone and 2,000 for wood). This requires an early iron mine too. Then an AI can be programmed to spam light infantry which are good in OCMOD3.

    However, those kind of changes only begin to scratch the surface of making a true AI. What I described above is a scripted AI. It follows a set script for resource starts, nations and units produced. The trouble with a set script is that it is unresponsive. It also does the same thing every time except that a few random parameters may be used to introduce a little variety. I mean like making pikes or muskets, heavy horse or dragoons.

    What is really needed is to get away from running a scripted, non-responsive AI. How would we do that? A responsive AI would need to respond to three key things. One, it would respond to the resource start. That is to say it would make different decisions based on how many resources it started with. Two, it would respond to the mod it was in. That is to say it would figure out best builds based on the parameters a particular mod set or the vanilla for that matter. Three, it would respond to opponent actions intelligently by using scouting and not cheating. How could an AI be made to do these things?

    To respond as above, the AI needs to be goal seeking in the way that a chess program, for example, is goal seeking. However, an RTS AI cannot rely on brute force searches of a move tree as chess programs mainly do. The RTS "tree" of branching possibilities is vastly bigger than the chess "tree". As a start, the C3 AI should use some basic heuristics (rules of thumb) and possibly some limited tree searches.

    To get the AI project under way we would have to use the divide and conquer approach. Divide the problem into sub-components. Try to solve these one at a time and then try to put the components together (which would make a system of sub-systems needing feed-backs between the sub-systems).

    The first problem to solve is the economy problem. Let us consider a general rule. The bigger economy you make, the bigger military you can make. Thus, the first problem to solve is the economy problem. The idea would be to solve this problem in isolation first. In other words, make an experimental mod which only made economic units and structures. Then try to solve the problem of how to make the AI grow the economy as fast as possible.

    Here we can leverage a heuristic, a rule of thumb, of all RTS. It is well known that you cannot grow your economy as fast as possible if you leave too many resources in store. The heuristic rule is "Convert all your resources into buildings and units as fast as possible. Do not have idle resources." However, this rule in turn can run afoul of the need (a bit later) to save resources for upgrades. Nevertheless, we would put this concern aside at first and just build the routines to run the "Spend!" heuristic as we might call it.

    How can we make goal-seeking routines for the "Spend!" heuristic? The answer is to give a bonus to resources embodied in structures and units as compared to resources in store. First, you convert all resources to "gold equivalent" not with actual sales but just with calculations (easy with the flat market of OCMOD3). Then you can get a stored resource score which is the value of resources in store in gold equivalent. Then you give a bonus, probably simply of 2x (times two) to resources embodied in structures (and later in units). A tree search routine would search possible sales and builds and pick the highest scoring answer. Then it would follow that build path.

    However, even this is probably too sophisticated for a first cut attempt. It would be easier to "one-step" the build by having the AI do this;

    (1) Sell all resources to a floor for each resource. (I will come back to the floor issue).
    (2) Build the most valuable building available.
    (3) build the next most valuable building available and so on.
    (4) Return to Step 1.

    Here, the most important thing would be the floors set on market resources. These floors would have to float and indeed rise as the game goes on. Currently, the vanilla AI has "don't starve" routines. These rely on more than market floors. They will reassign peasants to the field if I recall correctly. In our case, the AI would set market floors in special ways and these floors would rise as the game goes on. These floors would prevent selling a resource that is adequate for a "next needed building" and there are ways of determining that. These floors would keep reserves of food to reduce starvation risks. These floors would keep reserves of iron and coal, once these are needed for possible musket fights. These floors would allow for saving towards "next required upgrade".

    To cut a long story short, the above are just ideas to get a smart economic system. Then there is the military build and actual military tactics and strategies. However, as a first start of learning the AI system, I could improve as follows;

    (a) Improve the scripting. (The above would be a longer term project).
    (b) Improve unit choices.
    (c) Get the AI to use proper formations as OCMOD3 mandates formation use, if you want to win.

    I have several projects on the go. I will look at this when I can.
    Last edited: Oct 29, 2017
    Loner likes this.
  2. General WVPM

    General WVPM Member

    I like your idea as I mentioned before. But one AI for all mods will be extremely hard to do.
    I believe the easier way is to get the AI to have a number of plans.
    A plan would be made for a certain amount of time in the game as well as the starting conditions.
    For example in a 20+ peace time game the AI would buy wood and stone to get 3 town centres, whereas in a match where the PT is less and the AI isn't isolated on an island, they should quickly prepare for war.
    Standardised build orders might make it more boring, but a lot of players use this as well and you could always create multiple variants which are picked depending on nation or other conditions.

    A good example of a 20 PT 5k starting resources match would be like this:
    1. Build 2 town centres.
    2. Build a mill and storehouse
    3. Build a market
    4. Sell metals for wood, stone and food. (Save a bit for early upgrades and unit production).
    5. Build a 3rd town centre. (By now they could task peasants to gather stone and wood.)
    6. Build 2 barracks and an academy. (This would obviously be different for nations like Ukraine).
    7. Put the barracks on endless pike production. (Place some guards at mines, blacksmith and other places.)
    8. Once have enough pikes, form them into formations and move them to the enemy.
    9. Research the pike upgrades just before the peacetime runs out.
    10. This would be the end of this plan and depending on conditions, a different one will be activated.
    Variations of this script could be merc spam, 18th century rush and mixed pike and musket.

    Such a plan could be different per mod.
    I believe the AI already somewhat behaves like this, so this would be a smaller step.
    As for the plans, ask different players how they behave in certain scenario's or watch replays.
    You could even estimate the player's skill and based on that assign his plan to a certain AI difficulty.
  3. Hansol333

    Hansol333 Active Member

    I stopped modding mainly due the terrible bad AI.
    I gave them insane high boni like +300% gathering speed or that every private upgrade also increased building time of that unit by 5%.

    But even then I was able to win most of the time.
    compare to BTW the AI is terrible.

    Especially if the AI trains pike it is so easy to win with muskets. they never reach you, they die and you lose no units. With the buffs of 17c muskets 17c pike became really bad if not supported by shooters. And it takes forever until the AI builds a 18c barracks.

    I tried that the AI starts with a 18c barracks (or a stable) to make them more aggressive at the start but I failed so far.

    if you replace 17c pike with 17c musket and cavalry on range then they can become dangerous but with melees only they always die in your gunfire.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice