Encapsulation in Warcraft Addons - Inheritance

05 Dec 2014

Using Inheritance (sort of)

When we actually need inheritance, things get a little more complicated. We need to use two of lua's slightly harder features to get it to work: metatables and colon notation. A little background on these will help:

MetaTables

All "objects" in lua are tables, and tables can something called a metatable added to them. Metatables can have special methods on them which run under certain circumstances (called metamethods), such as keys being added. A full list of metamethods is available here.

The metamethod we are interested in is called called __index, which gets triggered when a key is not found in the table.

There are two ways of using __index. The first is to assign it a function, which gets passed two arguments: table, and key. This is useful if you want to provide a default value if a key in a table isn't found, which I use in the spellData example in the previous post.

The other way of using __index is to pass it another table of methods to call, like in this example:

local meta = {
    print = function()
        print("Hi from the metatable")
    end
}

local actual = {
    test = function()
        print("testing")
    end
}

--wont work:
-- actual.print()

setmetatable(actual, { __index = meta })

-- now it will!
-- actual.print()

By calling setmetatable on actual, we provide actual with all the methods on meta. A table can only have one meta table though, and you might break things by overwriting it (example, don't call setmetatable on a Frame or ActionButton...)

Colon Notation

All methods on a table can be called in two ways; with a colon, or with a period. The colon can be thought of as "fill in the first parameter with the table this is being called on". For example, these two statements are equivalent:

local x = string.gsub("hello world", "hello", "bye")
local x = "hello world":gsub("hello", "bye")

In the example above, the signature of gsub is something like this:

local string = {
    gsub = function(self, searchTerm, replacement)
        --self is the input string
    end,
}

The convention used is to call the first parameter self. We can now use this colon notation with metatables to make our version of inheritance.

Combining

local base = {
    increase = function(self)
        self.count = self.count + 1
    end,
    print = function(self)
        print("The count is " .. self.count .. ".")
    end
}

local first = {
    count = 0
}
setmetatable(first, { __index = base })

local second = {
    count = 100
}
setmetatable(second, { __index = base })

--usage
first:increase()
second:increase()

first:print()      -- prints 1
first:print()      -- prints 101

Due to the way the colon operator works, the self parameter is filled in with the table calling the method, not the table the method is defined on. So calling first:increase() is the same as base.increase(first)

Usage

We can now take these elements, and craft a set of classes designed for reuse. We start off with our root object (think System.Object if you are from a .net world.)

local class = {

    extend = function(self, this)
        return setmetatable(this, { __index = self })
    end,

    new = function(self, ...)

        local this = setmetatable({}, { __index = self })
        this:ctor(...)

        return this

    end,

    ctor = function(self, ...)
    end,
}

We have two methods here, extend and new. The new method is nice and straight forward - it creates a new table, assigns the meta to be class and calls the ctor method (which is the one you would want to replace in sub classes).

The extend method takes in a new table, and applies and sets the meta to class. This is what is used to inherit and add new functionality.

For example, in my control library, I have a base class with some common methods:

local control = class:extend({

    size = function(self, config)
        self.frame:SetSize(unpack(config))
    end,

    point = function(self, config)
        self.frame:SetPoint(unpack(config))
    end,

    parent = function(self, value)
        self.frame:SetParent(value)
    end,
})

And then many other classes which extend the base, cilling in the ctor method with how to actually create the frame:

local label = control:extend({

    ctor = function(self, name, parent)
        self.frame = CreateFrame("Frame", name, parent)
        self.label = self.frame:CreateFontString()
        self.label:SetAllPoints(self.frame)
        self.label:SetFont(fonts.normal, 12)
    end,
})

local textbox  = control:extend({

    ctor = function(self, name, parent)
        self.frame = CreateFrame("editbox", name, parent, "InputBoxTemplate")
        self.frame:SetAutoFocus(false)
        self.frame:SetFont(fonts.normal, 12)
    end,

    text = function(self, value)
        self.frame:SetText(value)
    end,
})

Some classes, such as the textbox provide other methods where they make sense.

Calling Base Class Methods

If we wish to start overriding a method and then call the original method within, things start to get a lot more complicated.

local class = {
    extend = function(self, this)
        this.base = self
        return setmetatable(this, { __index = self })
    end,
}

local child = class:extend({
    method = function(self)
        self.name = "child"
    end,
})

local grandchild = child:extend({
    method = function(self)
        self.base:method()
    end
})

While this looks like it will work, it will cause some strange and hard to debug problems (I know it will, it took me ages to figure out.)

The problem is that when you do self.base:method() you are effectively doing self.base.method(self.base), which means the base method is referencing the wrong table!

We can solve this, but it requires a certain level of voodoo. First we need to change our extend method:

extend = function(self, this)

    this.super = function(child)

        local parent = {
            __index = function(_, methodName)
                return function(_, ...)
                    self[methodName](child, ...)
                end
            end
        }

        return setmetatable({}, parent)
    end

    return setmetatable(this, { __index = self })
end

This took me far too long to come up with and get working. Essentially what it does is take all calls, and replace the self parameter with the correct table.

This method has some restrictions, in that you can only go 'up' one level in the class hierarchy, e.g. you cannot do item:super():super():super(). In practice though, I have never needed to do this.

The entirety of my class file can be found on my github.

Problems

There are two disadvantages to this method of creating objects. The first is using a table like this, you can no longer totally hide variables as you could do in the closure version. The other is the complexity added - especially if you wish to allow base method calling, however in balance, you only need to write the super() functionality once (or use mine!)

When writing addons, I use both methods of encapsulation where they fit best - as like everything else in development the answer to what to use is "it depends".

design, code, lua, warcraft

---

Encapsulation in Warcraft Addons - Closures

28 Nov 2014

In the last post I alluded to the fact that if you put in a little leg work, you could write well encapsulated objects in lua. There are two main ways to do this; with closures, and with metatables. In this post we will deal with using closures, and in the next post we will cover using metatables.

Using Closures

The simplest way to write an object in lua is with a closure to hide all the variables from the outside world. For example, we can write a counter class like so:

local counter = {

    new = function()

        local count = 0

        local this = {

            increase = function()
                count = count + 1 end
            end,

            print = function()
                print("The count is " .. count .. ".")
            end,
        }

        return this

    end,
}

We are using a table to give us a class name, and the closure is the only method on it (called new). My standard convention is to call the actual object we return this. The this object contains the public surface of our object, in this case two methods called increase() and print(). You can use the counter like this:

local first = counter.new()

first.increase()
first.print() -- prints "The count is 1"

By using a closure, we limit the use of the count variable to only methods defined in the body of the function new. This prevents anyone who uses the class from knowing how it is implemented, which is important as we are now at liberty to change the implementation without affecting our users.

A good example of this technique is in my Dark.Combat addon. While writing cooldown tracking, I needed to know how many stacks of Maelstrom Weapon was the maximum, so that I could trigger a glow effect on the icon. The problem is that the Warcraft API doesn't have a way of querying this (you can call GetSpellCharges for spells such as Conflagurate, but sadly this doesn't work on an aura.)

To solve this, rather than hard coding values into the view, or forcing the user to specify some kind of "glow at xxx stacks" parameter in the config, I wrote an object which you can be queried. This could also be expanded later to hold additional spell data which is not available in the API.

local addon, ns = ...

local spellData = {

    new = function()

        local charges = {
            [53817] = 5,
            ["Maelstrom Weapon"] = 5,

            [91342] = 5,
            ["Shadow Infusion"] = 5,
        }

        setmetatable(charges, { __index = function(key) return 1 end })

        return {
            getMaxCharges = function(spellID)
                return charges[spellID]
            end,
        }

    end
}

ns.spellData = spellData.new()

As the implementation of getMaxCharges is hidden, I can change it at will - perhaps splitting my charges table into two separate tables, or if Blizzard kindly implemented a GetMaxStacks(spellName) I could call this instead and remove my charges table altogether.

Composition

We can utilise composition to create objects based off other objects, by decorating an instance with new functionality. A slightly cut down version of the grouping code from my Dark.Bags addon makes good use of this:

local group = {

    new = function(name, parent, options)

        local frame = CreateFrame("Frame", name, parent),
        layoutEngine.init(frame, { type = "HORIZONTAL", wrap = true, autosize = true })

        return {
            add = function(child)
                frame.add(child)
            end,
        }
    end,
}

local bag = {

    new = function(name, parent)

        local this = group.new(name, parent)

        this.populate = function(contents)

            for key, details in pairs(contents) do
                this.add(itemView.new(details))
            end

        end

        return this

    end,
}

Here we have two classes group and bag. The group acts as our base class; it just creates a frame, and initialises a layout engine which does the heavy lifiting of laying out child frames.

In the bag.new() function, we create an instance of a group and add a populate method to it, and return it. We can continue creating new classes which use bag and group as base types as we need.

Problems with Closures

The down side to using closures is that inheritance is not really possible. To take the counter example again, if you wanted to create a stepping counter, you couldn't do this:

local evenCounter = {
    new = function()

        local this = counter.new()

        this.increase = function()
            -- how do we access count?!
        end

        return this
    end
}

Not only can you not access the original count variable, but you would also have to reimplement the print function as it would not have access to your new counting variable.

These problems can be solved using the metatables methods in the next post, however depending on what you are doing, you could just use composition instead as outlined below.

design, code, lua, warcraft

---

Good Design in Warcraft Addons/Lua

23 Nov 2014

Lack of Encapsulation in Addons

I first noticed a lack of good design in addon code when I started trying to tweak existing addons to be slightly different.

One of the stand out examples was a Threat Meter (you know which one I mean). It works well, but I felt like writing my own, to make it really fit into my UI, with as little overhead as possible. Not knowing how to even begin writing a Threat Meter, I downloaded a copy, and opened its source directory... to discover that the entire addon is one 3500+ line file, and 16 Ace.* dependencies.

When I had finished my Threat Meter, I had two files (170 lines and 130 lines), and one dependency (Dark.Core, which all my addons use). I learnt a lot while reading the source for the original threat meter - it is very customisable, is externally skinable, and has some very good optimisations in it. But it also has a lot of unused variables (which are named very similarly to used ones), and so much of it's code could be separated out, making it easier to modify by newer project members.

This set of observations goes on forever when concerning addons. The three main problems I see are:

  • Pollution of the global namespace
  • All code in one file
  • No separation of concerns

All of this makes it harder for new developers to pick up and learn how to maintain and write addons. They are all fairly straight forward to solve problems, so lets address them!

Pollution of the Global Namespace

A lot of addons you find declare many variables as global so they can access them anywhere within their addon. For example, this is pretty standard:

MyAddonEvents = CreateFrame("Frame", "MyAddonEventFrame")

MyAddonEvents:RegisterEvent("PLAYER_ENTERING_WORLD")
MyAddonEvents:SetScript("OnEvent", MyAddonEventHandler)

MyAddonEventHandler = function(self, event, ...)
    
    if event == "PLAYER_ENTERING_WORLD" then
        --do something useful
    end
end

This is an example of poluting the global namespace, as now the entire UI has access to: MyAddonEvents, MyAddonEventFrame, MyAddonEventHandler. This is very trivial to rewrite to not expose anything to the global namespace:

local events = CreateFrame("Frame")
local handler = function(self, event, ...)
    
    if event == "PLAYER_ENTERING_WORLD" then
        --do something useful
    end

end

events:RegisterEvent("PLAYER_ENTERING_WORLD")
events:SetScript("OnEvent", handler)

This version exposes nothing to the global namespace, and performs exactly the same function (you can even get rid of the handler variable and just pass the function directly into SetScript).

However, by writing your code like this, you can't access any of this from another file (either a lua file, or shudder a frameXml file), but using namespaces we can get around this limitation without polluting the global namespace.

Splitting into Separate Files

So, how to access local variables in other files? Well Warcraft addons come with a feature where all lua files are provided with two arguments: addon and ns. The first of these is a string of the addon name, and the second is an empty table. I almost never use the addon parameter, but the ns (or "namespace") parameter is key to everything.

You can access these two variables by writing this as the first line of your lua file:

local addon, ns = ...

print("Hello from, " .. addon)

By using the ns, we can put our own variables into it to access from other files. For example, we have an event system in one file:

eventSystem.lua

local addon, ns = ...

local events = CreateFrame("Frame")
local handlers = {}

events:SetScript("OnEvent", function(self, event, ...)
    
    local eventHandlers = handlers[event] or {}

    for i, handler in ipairs(eventHandlers) do 
        handler(event, ...)
    end

end)

ns.register = function(event, handler)
    
    handlers[event] = handlers[event] or {}
    table.insert(handlers[event], handler)

    events:RegisterEvent(event)

end

Note how the register function is defined on the ns. This means that any other file in our addon can do this to handle an event:

goldPrinter.lua

local addon, ns = ...

ns.register("PLAYER_MONEY", function() 
    
    local gold = floor(money / (COPPER_PER_SILVER * SILVER_PER_GOLD))
    local silver = floor((money - (gold * COPPER_PER_SILVER * SILVER_PER_GOLD)) / COPPER_PER_SILVER)
    local copper = mod(money, COPPER_PER_SILVER)

    local moneyString = ""
    local separator = ""

    if ( gold > 0 ) then
        moneyString = format(GOLD_AMOUNT_TEXTURE, gold, 0, 0)
        separator = " "
    end
    if ( silver > 0 ) then
        moneyString = moneyString .. separator .. format(SILVER_AMOUNT_TEXTURE, silver, 0, 0)
        separator = " "
    end
    if ( copper > 0 or moneyString == "" ) then
        moneyString = moneyString .. separator .. format(COPPER_AMOUNT_TEXTURE, copper, 0, 0)
    end

    print("You now have " .. moneyString)

end)

A pretty trivial example, but we have managed to write a two file addon, without putting anything in the global namespace.

We have also managed to separate our concerns - the goldPrinter does not care what raises the events, and the eventSystem knows nothing about gold printing, just how to delegate events. There is also an efficiency here too - anything else in our addon that needs events uses the same eventSystem, meaning we only need to create one frame for the entire addon to receive events.

Structure

Now that we can separate things into individual files, we gain a slightly different problem - how to organise those files. I found over time that I end up with roughly the same structure each time, and others might benefit from it too.

All my addons start with four files:

  • AddonName.toc
  • initialise.lua
  • config.lua
  • run.lua

The toc file, other than the usual header information is laid out in the order the files will run, for example this is the file segment of my bags addon's toc file:

initialise.lua
config.lua

models\classifier.lua
models\classifiers\equipmentSet.lua
models\itemModel.lua
models\model.lua

groups\group.lua
groups\bagGroup.lua
groups\bagContainer.lua

views\item.lua
views\goldDisplay.lua
views\currencyDisplay.lua
views\bankBagBar.lua

sets\container.lua
sets\bag.lua
sets\bank.lua

run.lua

The initialise lua file is the first thing to run. All this tends to do is setup any sub-namespaces on ns, and copy in external dependencies to ns.lib:

local addon, ns = ...

ns.models = {}
ns.groups = {}
ns.views = {}
ns.sets = {}

local core = Dark.core

ns.lib = {
    fonts = core.fonts,
    events = core.events,
    slash = core.slash,
}

By copying in the dependencies, we not only save a global lookup each time we need say the event system, but we also have an abstraction point. If we want to replace the event system, as long as the replacement has the right function names, we can just assign the new one to the lib: ns.lib.events = replacementEvents:new()

The sub namespaces correspond to folders on in the addon (much the same practice used by c# developers), so for example the classifier.lua file might have this in it:

local addon, ns = ...

local classifier = {
    new = function() end,
    update = function() end,
    classify = function(item) end,
}

ns.models.classifier = classifier

The config file should be fairly simple, with not much more than a couple of tables in it:

local addon, ns = ...

ns.config = {
    buttonSize = 24,
    spacing = 4,
    screenPadding = 10,
    currencies = {
        823, -- apexis
        824,  -- garrison resources
    }
}

And finally, the run.lua file is what makes your addon come to life:

local addon, ns = ...

local sets = ns.sets

local pack = sets.bag:new()
local bank = sets.bank:new()

local ui = ns.controllers.uiIntegration.new(pack.frame, bank.frame)
ui.hook()

--expose 
DarkBags = {
    addClassifier = ns.classifiers.add
}

If you need to expose something to the entire UI or other addons, that's fine. But make sure you only expose what you want to. In the example above the DarkBags global only has one method - addClassifier, because that is all I want other addons to be able to do.

Wrapping Up

I hope this helps other people with their addons - I know I wish that I had gotten to this structure and style a lot sooner than I did.

There will be a few more posts incoming covering encapsulation, objects and inheritance in more detail, so stay tuned.

design, code, lua, warcraft

---

Edge.js for Embedded Webuis

04 Aug 2014

We work we have a number of windows services which each have a lot of stats they could expose. Currently they are only interrogatable by the logfiles and from any notifications we receive.

I have been toying with the idea of hosting a website in-process which would give a simple dashboard ui and access to a live view of the log file. The idea first struck me when I was experimenting with FubuMvc, as they have an EmbeddedFubuMvcServer, which is very easy to use:

FubuMvcPackageFacility.PhysicalRootPath = @"Backend\";

using (var server = EmbeddedFubuMvcServer.For<EmbeddedBackend>(FubuMvcPackageFacility.PhysicalRootPath))
{

    Console.WriteLine("Some long running process, with a web-backend on :5500");

    var p = server.Services.GetInstance<IProcessor>();

    var t = new Task(p.Start);
    t.Start();

    Console.ReadKey();
}

But while I like this, FubuMvc embedded seems like overkill.

Wouldn't it be nice if we could host an expressjs app inside our process? They are very lightweight, and to get one setup is almost no coding (especially if you use the express commandline tool).

Enter Edgejs

The Edge.js project provides an in-process bridge between the .net and nodejs worlds, and allows for communication between the two...

Steps:

  • Create a new application (eg: ServiceWithEdge)

  • Create a subdirectory for the webui in your applications root (eg, next to the csproj file)

    • ServiceWithEdge\ServiceWithEdge\webui
  • If you don't have express-generator installed, get it:

    • npm install -g express-generator
  • Cd to your webui directory, and create an express application:

    • express - there are some options if you want, see the guide
  • In visual studio, include the webui directory

    • Mark all files as content and copy if newer
  • Add a new js file in your webui root:

var options;

exports.set = function (m) {
    options = m;
};

exports.getModel = function (modelName, action) {

    options.getModel(modelName, function (error, result) {

        if (error) throw error;

        action(result);
    });

};
  • add the edgejs package:

    • PM> install-package edge.js
  • The following function will run the webui, and inject a callback for getting models from .net

private static void RunWebui(ModelStore store)
{
    var func = Edge.Func(@"
        var app = require('../webui/app');
        var com = require('../webui/communicator');

        app.set('port', process.env.PORT || 3000);

        var server = app.listen(app.get('port'));

        return function(options, callback) {
            com.set(options);
        };
    ");

    var getModel = (Func<object, Task<object>>)(async (message) =>
    {
        return store.GetModel((string)message);
    });


    Task.Run(() => func(new
    {
        getModel
    }));
}
  • The last step to getting this to work is running npm install in the webui directory of the build output folder. I use a rake file to build everything, so its just an extra task (see the entire Rakefile here):
task :npm do |t|

    Dir.chdir "#{project_name}/bin/debug/webui" do
        system 'npm', 'install'
    end

end
ny route needing data from .net just needs to require the communicator file and call `getModel`:
var com = require('../communicator');

router.get('/', function (req, res) {

    com.getModel("index", function(value) {

        res.render('index', {
            title: 'Express',
            result: value.Iterations
        });

    });

});

All the code is available on github.

How I am aiming to use it

I am planning on constructing a nuget package to do all of this, so that all a developer needs to do is add the package, and configure which statistics they wish to show up on the web ui.

design, code, net, typing, sql, database, orm

---

Configuring Dapper to work with custom types

22 Jul 2014

In the last post we looked at using custom ID types to help abstract the column type from the domain.

This works well until you start trying to load and save entities using an ORM, as the ORM has not way to know how to map a column to a custom type. ORMs provide extension points to allow you to create these mappings. As I tend to favour using Dapper, we will go through setting it up to work with our custom ID types.

We need to be able to get the raw value out of the id type, but without exposing this to the outside world. To do this we internal interface:

internal interface IValueID
{
    object Value();
}

Then update our id struct with a private implementation of the interface, and also mark the only constructor as internal:

public struct PersonID : IValueID
{
    private readonly Guid _id;

    internal PersonID(Guid id)
    {
        _id = id;
    }

    object IValueID.Value()
    {
        return _id;
    }
}

We now can define a class which Dapper can use to do the mapping from uuid to id:

public class PersonIDHandler : SqlMapper.TypeHandler<PersonID>
{
    public override void SetValue(IDbDataParameter parameter, PersonID value)
    {
        parameter.Value = ((IValueID)value).Value();
    }

    public override PersonID Parse(object value)
    {
        return new PersonID((Guid)value);
    }
}

We then need to regiter the command with Dapper once on start up of our application:

SqlMapper.AddTypeHandler(new PersonIDHandler());

Now when Dapper loads an object with a property type of PersonID it will invoke the Parse method on PersonIDHandler, and populate the resulting object correctly. It will also work when getting a value from the PersonID property, invoking the SetValue method on PersonIDHandler.

Extension

While the PersonIDHandler works, I really don't want to be creating essentially the same class over and over again for each ID type. We can fix this by using a generic id handler class, and some reflection magic.

We start off by creating a generic class for id handling:

public class CustomHandler<T> : SqlMapper.TypeHandler<T>
{
    private readonly Func<Object, T> _createInstance;

    public CustomHandler()
    {
        var ctor = typeof(T)
            .GetConstructors()
            .Single(c => c.GetParameters().Count() == 1);

        var paramType = ctor
            .GetParameters()
            .First()
            .ParameterType;

        _createInstance = (value) => (T)ctor.Invoke(new[] { Convert.ChangeType(value, paramType) });
    }

    public override void SetValue(IDbDataParameter parameter, T value)
    {
        parameter.Value = ((IValueID)value).Value();
    }

    public override T Parse(object value)
    {
        return _createInstance(value);
    }
}

The constructor of this class just finds a single constructor on our ID type with one argument, and creates a Func which will create an instance of the id passing in the value. We put all this constructor discovery logic into the CustomHandler's constructor as this information only needs to be calculated once, and can then be used for every Parse call.

We then need to write something to build an instance of this for each ID type in our system. As all of our IDs need to implement IValueID to work, we can scan for all types in the assembly implementing this interface, and then operate on those.

public class InitialiseDapper : IApplicationStart
{
    public void Initialise()
    {
        var interfaceType = typeof(IValueID);

        var idTypes = interfaceType
            .Assembly
            .GetTypes()
            .Where(t => t.IsInterface == false)
            .Where(t => t.IsAbstract == false)
            .Where(t => t.GetInterfaces().Contains(interfaceType));

        var handler = typeof(CustomHandler<>);

        foreach (var idType in idTypes)
        {
            var ctor = handler
                .MakeGenericType(new[] { idType })
                .GetConstructor(Type.EmptyTypes);

            var instance = (SqlMapper.ITypeHandler)ctor.Invoke(new object[] { });

            SqlMapper.AddTypeHandler(idType, instance);
        }
    }
}

This class first scans the assembly containing IValueID for all types implementing IValueID which are not abstract, and not interfaces themselves. It then goes through each of these types, and builds a new instance of CustomHandler for each type, and registers it with Dapper.

You might notice this is in a class which implements IApplicationStart - In most of my larger projects, I tend to have an interface like this, which defines a single void Initialise(); method. Implementations of the interface get looked for on startup of the application, and their Initialise method called once each.

design, code, net, typing, sql, database, orm

---