Today I’d like to talk about metatheorycrafting, or what essentially amounts to theorycrafting about the overall picture of theorycrafting.
I’m going to examine the different types of places that theorycrafting generally starts and how theories evolve and progress throughout the general WoW community. To my knowledge, this isn’t something that anyone really ever even talks about, let alone trying to formally describe, so we might be going into uncharted territory. As such, most of what I will say here comes from my own perspective and my own experiences.
The purpose of this venture is to help people understand where they might go to get the most raw information they can, or where they might go if they want refined information a little later down the line. Hopefully I can also help others that produce and analyze the information that comes out of theorycrafting identify where the best places to start might be or how they want to be involved in the way that suits them most.
The Question
All theorycrafting starts because someone wants to answer a question. If you wanted to get very deep into epistemology (the “how do we know what we know?” side of philosophy) you might say that the search for any kind of knowledge always starts with a question. The question can be anything, but there must be an objective of theorycrafting to aim the search and the analysis.
The question can come from anywhere, but it has to be heard by someone that will actually do some theorycrafting for it to mean anything. In the case that it is your own question and you are doing your own theorycrafting, that’s great. If you just wonder out to thin air whether Crit or Mastery provides greater defensive benefits for a Brewmaster and leave it at that, your question is pretty useless.
This is where the theorycrafting community comes in. Most people don’t understand everything about the actual mechanics and mathematical forces behind the game, and that’s fine. You don’t really need to know what the damage reduction from armor formula is unless you plan on doing some theorycrafting with it, so most people just don’t. There are a few people though that could be considered experts in the mechanical and mathematical way that World of Warcraft works, and they’re usually the ones that answer the questions. The best way to get a question answered is to make sure the question is heard by someone that can and will answer it. If you want to know something, all you really have to do is find someone that can answer it for you, but that can be tricky because theorycrafters practice their craft in a variety of places, some more accessible to others.
The Answer
If you want to answer a question, you need a place to work. Most theorycrafting involves long equations and many variables, which means that the sheer math involved in the work practically requires the ability to write things down as you go along to keep everything straight. It is possible to do a lot of mental math, but a large part of the strength of the theorycrafting community is the ability to have other people look at your work and agree that it makes sense (also called peer review). For this reason, theorycrafting in-game or on Twitter is almost impossible. Character limits and an inability to format things into a logical progression that can make sense to other people destroy the ability to theorycraft as well as cause them to be lost in a flood of other things, which makes reproducing the work require doing it all over again.
Because ample space is needed, most theorycrafting actually occurs on forums and blogs (sometimes with links to publicly viewable spreadsheets via Google Docs as well). It should be no surprise then that the best place to get a question answer is on whatever forum the theorycrafters for your area usually frequent (might take some looking around, maybe specifically asking a few of them yourself) or to comment on a blog post to try to get in contact with someone. Some theorycrafters also put themselves out on Twitter or are available in-game, but that’s only really good for getting the question across. The answer (and its supporting work) usually comes in the form of a forum or blog post where it can be seen by the most people possible in its full version.
Peer Review of the Answer
No one is infallible. That is why it is extremely important that theorycrafting work be done in public and be viewable by other theorycrafters so that mistakes can be identified. I would advise everyone to be very suspicious of anyone that claims to be a theorycrafter but keeps all of their work hidden, or anyone that just gives an answer and expects to automatically be believed without having the work available to show on request.
When it comes down to it, you want to make sure the answer that you are getting is the correct one. Posting work in a place where peer review is not only possible but also encouraged has numerous benefits.
1) It drastically increases the chances that mistakes are found and corrected.
2) It lets everyone else know that the work is done and can be borrowed for future use. There’s no sense in 20 people all doing the same work if the first person that did it could just put it out in the open.
3) Sometimes someone else’s theorycrafting can spawn new questions that other theorycrafters might think of themselves, which spurs the advancement of new work without it having to be directly asked.
4) Public, preserved, and peer reviewed work can simply be shown to anyone that has the same question in the future. It only has to be done once, checked, and archived to be referred to any time the question comes up again. This way, we can have the most accurate knowledge possible for the most amount of people while doing the least amount of work.
The Right Places to Be
With all of the above in mind, there are a few places to accomplish each of these things much more easily than anywhere else.
Asking/Receiving Questions – Twitter, direct contact (in-game or otherwise), or on a forum are the best places to get a question to a theorycrafter. Blog comments can go unread and be superfluous and Reddit posts can be forgotten or buried.
Publishing Theorycrafting Answers – As mentioned, forum and blog posts are the best ways to go about this. If in a forum, the post should be in a stickied thread that won’t disappear and the specific post should be bookmarked for later use. Personally I prefer keeping math theorycrafting to forum posts where they can be more easily reviewed and keeping more philosophical things (such as this) to blog posts because they have little to no need to be reviewed.
Reviewing Work – Realistically the best and only place to easily review work is in a public forum. This ensures multiple participants, a conversation, and usually the ability to quickly edit mistakes so false information is eliminated as quickly as possible. The problem with most things WoW-related though is to find the appropriate forum, because each class and/or spec usually only has one very good peer review site due to the problems with trying to coordinate discussion with multiple people in different places. IRC can work in a pinch though.