Imagine the application that receives some data from the external source. For the sake of an example let’s assume the data is currency rates stream.
The application has a set of rules to filter the incoming data stream. Let’s say we have a list of currencies we are interested in, and we want only the currencies from this list to pass through. Also, sometimes we receive invalid rates (nobody is perfect, our rates provider is not an exception.) So we maintain a long-lived validator that ensures that the rate in the stream looks fine for us and only then we allow the machinery to process it. Otherwise we just ignore it.
Naïve Approach
The naïve approach to handle this use case would be to maintain a map, containing currency pairs as keys and rules as values, and apply rules to the incomind rates to check whether the rate is of interest or not. Rules would be simple maps specifying the acceptable interval for the rate as min and max values. Something like this (for the sake of an example let’s assume rates are coming as maps already, for instance from RabbitMQ or like):
defmodule Validator do
use Agent
def start_link,
do: Agent.start_link(fn -> %{} end, name: __MODULE__)
def update_rules(currency_pair, rules),
do: Agent.update(__MODULE__, &(Map.put(&1, currency_pair, rules))
def valid?(%{currency_pair: pair, rate: rate}) do
with %{} = rules <- Agent.get(__MODULE__, & &1),
rule when not is_nil(rule) <- rules[pair],
r when r > rule.min and r < rule.max <- rate do
{:ok, r},
else
_ -> :error
end
end
end
This is good, and this works. But can we improve the performance?—Sure thing!
Pattern matching
Instead of looking up the map with rules, we might simply generate the module, that will have one function valid? with as many clauses as we have rules. These clauses will directly pattern match the input and return {:ok, rate} tuple (the existence of this clause guarantees that the rate is good.) The last clause will accept all the non-matched garbage to return :error.
Sounds smart?—Indeed. Let’s implement it. Imagine we still have this map with rules.
defmodule Validator do
def instance!(rules) do
mod = Module.concat(["Validator", "Instance"])
if Code.ensure_compiled?(mod) do
:code.purge(mod)
:code.delete(mod)
end
Module.create(mod, ast(rules), Macro.Env.location(__ENV__))
end
defp ast(map) do
[
quote(do: (def valid?(_), do: :error)) |
Enum.map(map, fn {pair, %{min: min, max: max}} ->
quote do
def valid?(%{currency_pair: unquote(pair), rate: rate})
when rate > unquote(min) and rate < unquote(max),
do: {:ok, rate}
end
end)
] |> :lists.reverse()
end
end
When we need to update our rules, we just call Validator.instance!/1 and receive back the module named Validator.Instance. It has several clauses for valid/1 function. Let’s see this in action
iex|1 ▶ Validator.instance!(%{"USDEUR" => %{min: 1.0, max: 2.0}})
{:module, Validator.Instance,
<<70, 79, 82, 49, 0, 0, 4, 244, 66, 69, 65, 77, 65, 116, 85, 56, 0, 0, 0, 167,
0, 0, 0, 17, 25, 69, 108, 105, 120, 105, 114, 46, 86, 97, 108, 105, 100, 97,
116, 111, 114, 46, 73, 110, 115, 116, 97, ...>>, [valid?: 1, valid?: 1]}
iex|2 ▶ Validator.Instance.valid?(%{currency_pair: "USDEUR", rate: 1.5})
{:ok, 1.5}
iex|3 ▶ Validator.Instance.valid?(%{currency_pair: "USDEUR", rate: 0.5})
:error
iex|4 ▶ Validator.Instance.valid?(%{currency_pair: "USDGBP", rate: 1.5})
:error
Exactly what we needed, and blazingly fast.
Further Improvement
If the amount of incoming rates is big enough, like thousands per a second, we might use Flow on GenStage to validate them in bulks. That would be probably the topic of the next writing on the subject.
Happy generating!