A Guide to Diagnostic Data Costs

What is the cost of implementing diagnostic data in a development of a video game?

As the answer vary on different factors, I will try to do my best here to explain some general cost guidelines. Creating a department of diagnostic data can be compared to any other department as code, art, animation, QA etc.

The process of implementing and using diagnostic data tools in your game is divided into different stages and it’s cost can vary depending on several factors that I’ll describe in this article.

The process can be divided into 8 steps:

1. Identify the most important metrics for your game. What do you want to learn about players behavior and in which aspects do you want to improve your game? This process takes several days and it requieres involvement of several team members. It is a strategical and analytical stage in which a lot of planning is done.
2. Implement a diagnostic tool in your project and configure the initial data reading for the most generic metrics. It is the first operational work done in diagnostics. At this stage is necessary an involvement of a programmer who is going to add to the game’s code a serie of “Events” with “Parameters” to send them as data.
3. Test the functionality of the diagnostic data tool and prepare a pilot version of a report. At this stage you gain the perspective of the potentiality of using diagnostics in your game and you check if the data is sent correctly. As gathering data and preparing reports is a repetitive process, it’s recomended to prepare a standardization for that process and template for a report. It will save you time and effort in the future. At this stage each functionality should be tested by QA department.
4. Measure the KPIs and set goals for improvements. (Balance the difficulty between levels, decrease amount of players leaving the game before finishing a level, etc). Which metrics are the most relevant in your game?
5. Learn from the data about players behaviour. Compare different metrics and dependencies between them. Draw first conclusions about different aspects of your game. What changes will translate to the improvement in gameplay and balance of your game?
6. Analyse the use of diagnostic data tools and add more metrics. What does work for you? What have you learned in the process? Which processes can be automatized? Create more processes and standardization for use of data analytics.
7. Iterate on game design. Conduct A/B tests, conduct experiments, pivot on what you consider necessary to change, adjust parameters, compare the data from different versions. It requieres some micromanagement as the focus is not on global
8. Finalize the implementation of the diagnostic data tools into your organization. Set a strategy for it’s use in a longterm. How do you want to improve your game and what results do you want to achieve in each milestone and post-release? How often are you going to read the data and iterate?
   

The highest cost of diagnostic data is in the first steps, in the implementation of the tools and it’s configuration. The cost of maintanence and cyclical analytics of the data is relatively low. Depending on a project, it’s development stage reports can be generated biweekly, once a month or by quarterly.