Predicting the Effects of Issues

[Starrie]

Large communities have been built around the gameplay and roleplay aspects of the NationStates, but at the core of the game, less mentioned, are the issues that manifest in the differences between each and every nation. There is almost an art to interpreting issues -- a game one participates in to nudge the numbers closer and closer to perfection. For years, many have tried to divine the mechanisms that govern the effects of issues. GM Frisbeeteria has even claimed that it is completely impossible to do so. However, this article attempts to detail how one might, with a great deal of time and effort, lay the mystery to rest.


It helps, of coures, to take a look at what we already know. In issues 34 and 35 of The Rejected Times, Dr. George gives a few pointers of maximizing or minimizing some stats. Most of the article is pretty common sense, but there are a few hints. Dr. George mentions some bias toward high economy and low civil and political rights, although it is unclear whether this bias is statistically significant. He also mentions general guidelines a nation may take to get low stats in all three trends -- be paternalistic, extreme, and brutal. Unfortunately, it is also unclear whether the relationship between low stats and paternity, extremism, or brutality is causal or merely concommitant.

Perhaps the most extensive theory about nations' statistics and issue effects is New South Hell's "Spectral Theory", in which each nation and issue has '(macro)-themes' like Liberalism, and subthemes (named micro-themes), like culture, which are parts of macro-themes. Each theme a nation possesses is given a value from -9 to 9, and together, the themes and values of each nation make up a "spectrum". A nation's spectrum succintly and accurately characterizes its nature. Accordingly, each issue can also have theme values depending on how it affects the host nation, and here, a database of issues and their thematic values can be found.

While New South Hell's spectral classification is certainly interesting, there is little evidence to suggest it accurately mimics the behind-the-scenes workings of NationStates. Moreover, the classification cannot, and was not designed to, allow someone to predict the effects of issues on their nation.

Two years ago, I collected a list of about 10,000 decisions made over a period of three months by some puppet nations. Not wanting to manually figure out the effects of each issue, I shoved them through a shiny new neural network library. If you've never heard of a neural network, you can think of it as an statistical model, or a predictor, based off an obsolete model of the brain, which predicts outputs from inputs. Sadly, the experiment was an utter failure and the predictor I got was unusable. This was mainly due to the "curse of dimensionality". If each nation has fifty variables of input (for example, public/private sector, economy, civil/political rights, etc), and there needs to be at least three points of resolution along each axis (that is, low, middle, and high economy), then 3^50 = 7.2E23 datapoints are needed -- for each and every issue and decision. Clearly a larger sample is needed.

Indeed, many people have already attempted to create listings of the effects of each issue.
However, they all fail for one of several reasons.
1. The sample size is too small. Most collections don't even have stats from all of the issues and decisions, much less multiple samples.
2. Changing game mechanics. Given the refactoring, we know there has been at least one major change in the way the game works. Much of the old data is therefore probably now obsolete.
3. Current nation state. Most records don't note the stats of the nation enacting the decision, which has a major role in determining the effects of the issue. A communist and capitalist nation may react to the same issue very differently.
4. Bias. Manual sampling will always be biased, even when exact numbers are visible, in the kinds of issues that tend to be recorded, the nations which tend to record the issues, and many other factors.

Automation is therefore necessary. There are two main methods to go about collecting data, both with their own merits and demerits.
1. Monitor the world's decisions through happenings pages.
2. Create a bunch of puppets and control their every decision.

When monitoring decisions through the happenings api, there are a few things to keep in mind.
1. One must first compile a dictionary of "following new legislation ..." phrases so that you can match the text of each happening to the corresponding decision and issue. This could be quite a pain.
2. There are just too many nations to keep track of without breaking the api rate-limit. Therefore, it is necessary to use some clever thinking and determine which nations are and aren't actively making decisions in order to effectively sample.
3. The sampler has no control of the decisions made by the nations here. It is entirely possible that out of the hundreds of issues and thousands of decisions, some decisions will be made so rarely that very little data is acquired on them. In order words, the data isn't collected uniformly over the sample space.
4. Many nations tend to make multiple decisions per update, so the change in the nation's stats must be explained as a combination of multiple issues. This vastly complicates the statistical analysis.

There are also a few things to keep in mind when collecting data through puppets.
1. Given the scripting rate-limits, an individual can only operate 7200 nations simultaneously. Which sounds like plenty, but actually yields a lower data collection rate than monitoring the entire world's decisions does.
2. One "benefit" of this method is that it's scalable -- multiple people working can collect vast amounts of data. Of course, creating tens of thousands of puppets will probably get someone in some serious trouble.

But there are some major advantages to collecting data through puppets.
1. The sampler has full control over how the nations act, so the problem with the non-uniform sampling distribution just becomes a matter of cleverly controlling your nations to explore a wide variety of states and categories (which is admittedly rather non-trivial).
2. Tabula rasa. Newly created puppet nations are not tainted with the bureacratic inertia of already active nations.

Wait what? Bureacratic inertia?

One might notice that issues appear to have smaller effects over time. Perhaps it is the case that issues lose influence as a nation's age increases. However, a small and informal sample conducted revealed this is not actually the case. Instead there is something Frisbeetaria calls "bureacratic inertia", which he refused to elaborate on. Nonetheless it is not too difficult to imagine how this inertia works -- as a nation continuously legislates in one direction, they will find it harder to reverse course, or even halt.

More recently, a comment by Ballotonia lent more insight, or rather, confusion, to this concept. A nation with no police force but which wanted one needed to overcome a "steep negative" before they would reach their goal. But this raises some questions, because there is a difference between a negative and "bureacratic inertia" -- anyone who has taken a physics class will know the difference between inertia and displacement. While this discrepancy might seem to be quite the problem, the distinction is actually not essential. The important take-away from these comments is that there are, very clearly, internal variables which influence the statistics of nations, and this is another reason why shoveling data into a neural network won't just work.

So how can we use our newly minted data?

On the lower end of the ambition scale is the option of just compiling a database of decisions and matching a given input nation, with decision, to the closest datapoint in the database in order to "predict" what some possible effects will be. However, this isn't really a predicative system, and ignores the internal states of each nation.

Fortunately, there are many machine learning techniques (long short-term memory neural networks (LSTM) and hidden markov machines (HMM) come to mind), which keep an internal memory or representation of previous inputs -- the series of decisions which each nation makes, in order to account for a nation's internal state. Since I'm no machine learning expert, I won't go into the specifics of how this is accomplished, but it is likely that these techniques will produce the useful predictor that a naive neural network is incapable of.

The downside is that these "deep learning" techniques are often black boxes -- they work, but it is impossible to explain how they work. Even if they can successfully predict issues, they provide little information into how the game actually models nations. In fact, the reverse-engineering of these black-boxes is one of the most active fields of research in machine learning.

I haven't painted a very rosy picture, but if you have a vast amount of data, or are willing to create thousands of nations to collect it, and if you can work around refactorings and the constant addition of new issues, if you can compensate for the effects from WA resolutions, if you know a bit, or a lot, about machine learning, and if you can spend some thankless years working on a doubtful endeavor, then the rest should be smooth sailing, and I wish you luck!

[Starrie]

I see what you mean, but there's no neat way of figuring out the "backroom". Even if we were given the names of every hidden variable, we wouldn't know how the census measures are computed from them, and since we can't see their values, we can't gather data on them either. In a way, that's what this is all about -- collecting data about the visible variables to figure out the hidden ones.

violet did mention Economic Freedom so I'm betting that it really does exist.

[Gruenberg]

This was really interesting. I wish everyone were as methodologically exacting as you in terms of understanding the limits of predicting issues.