A More Robust Data System

And a virtual version of that pond from last time


Posted 08/01/2016 20:59:18 in Homestead

Updated 09/06/2017 02:33:52

Most of my time spent on Homestead over the past month has been on a big redesign of its data handling system. I was deep into a set of database courses, and decided to apply what I was learning to Homestead instead of to the silly chapter exercises! Here's a breakdown of what I did, and how I designed the system (and a little eye-candy at the end, too!)

Realizing the Inventory class was getting too big

I came to the realization that I was trying to do too much with the Inventory class. When you're comparing it to things like WeatherController, PlayerController, or LivestockManager, it sounds like one component. On closer inspection, there are actually a couple of layers-- the class was handling everything from keeping track of the items that exist in the game, to storing the meshes and sprites, to handling requests for item changes, to building the UI (blech!). Deciding to break it up led to a time investment, but the game is more modular, powerful, and clean!

Database Design

As I mentioned in an older post, one of my courses inspired me to rethink how Homestead was storing and handling its data. One of the most interesting realizations that I came to was that instead of storing instances of items in a collection-based, hierarchical format (a set of inventory "containers" that hold instances of items), I could store the data in an associative way. That's a little dense, so let me put it a different way. Instead of having two "tables", an ItemType table and an Inventory table that stores Inventories that store Items, I now have three tables: an ItemType table, an Inventory table that just stores InventoryIDs and Size information, and an associative table that stores the actual item instances. The interesting part is that associative table-- each entry has a reference to an InventoryID, an ItemType, and a couple other values that are particular to each item instance (like a crop's quality, for example). It seems a little roundabout, but doing it this way makes swapping items between Inventories much easier and less error-prone! Instead of searching through several Inventory collections, it's all in one table, and updating an item means just changing its InventoryID reference!

SQL Backend

I decided to go with SQLite as a backend, since there's a nice lightweight plug-in already available for Unity, it saves the database into a single file, and it's fast! Since the system is modular, it won't be too hard to swap it out for something else if the need arises. It's also much more development-friendly-- instead of working in a spreadsheet and exporting to a .CSV file every time I change something, I can just work in the database directly! Even though it's about as fast as you get get as far as SQL goes, the disk latency still causes a couple frames worth of delay on reads. To get around this, I load all* of the data into memory when the game starts, and write the data back when the game saves.

  • I'm probably going to have to get choosy about what data to load later on, when there's a lot more of it, but for development right now the memory impact is minimal :D

In-Memory Model

Above the physical layer, I have an MVC-ish pattern set up. The "model" is a class that maintains the loaded data and provides simple database-like methods for other classes to use. This keeps the in-memory data the same regardless of how it's actually stored on disk (or across a network!). It exposes methods like Select(Table t, int id) and Update (Table t, int id, string value). It isn't as granular as a typical MVC model, but it works as more of an interface between the Controller and the physical data.


The next layer up is what most of the game actually interfaces with. It exposes higher level functions like TryAddItem(int InventoryID, Item i) and GetLivestockName(int livestockID), and does the actual processing to accomplish those things. It's nice having all the database-ish code in one place-- it avoids having to do mostly unrelated database heavy lifting in classes like FarmPlot.


And of course, we have the View, which no longer has anything to do with anything besides the Controller. It sits next to my Canvas, it just handles UI stuff, and I can turn it on and off with no effect on the rest of the game, which is awesome for taking screenshots, and videos like this one :)

Thanks for reading this far! Hopefully this has been interesting for some of you-- a lot of times stuff like this is pretty game-dependent, so it seems like there isn't really a lot written online about how to actually go about designing an inventory system at a higher level. Was this interesting, or would you like me to elaborate on any of this? Am I setting myself up for disaster? Have you implemented anything similar? Let me know in the comments or on Twitter!



The bit about loading all data into memory at the beginning and then writing it all out at the end reminds me of an accidental discovery I made with Animal Crossing (for GameCube): Once when I was finished playing and had saved my game, I took out the disc and tried to load my save to see when the game would crash or show an error about putting the disc back in. My save loaded and the game worked just fine and never crashed. It seems Nintendo loaded all the data needed for the entire game* into memory off the disc by the time the game reached the title screen, after which point you could remove the disc entirely from the GameCube and Animal Crossing could continue to be played without any issues. *except save data, which was stored on (and read from/written to) the memory card.

08/02/2016 23:58:54


Cool, yeah that definitely makes sense! Reading from a CD drive is even slower than a hard drive, so I'm sure they wanted to load up as much as they could at first in order to avoid load times once you get into the game. I remember doing something similar with Metroid Prime-- except it would throw an error when you would try to open a door, so I think in that case they would load a room at a time. That helped explain why doors sometimes took a second or two to open though!

08/06/2016 17:25:45