❖ |> 🌢2024-03-03T07:31:24+00:00https://rocket-science.ruAleksei Matiushkinam@mudasobwa.ruFinitomata FTW2024-03-03T00:00:00+00:00https://rocket-science.ru/hacking/2024/03/03/finitomata-for-the-win<p>It’s been a year since I started to implement my own <em>FSM</em> library in <em>Elixir</em>. Now it’s funny to read my writings from the time when I had only a gut feeling I’m doing it right.</p>
<ul>
<li><a href="/hacking/2022/04/02/finitomata">The Proper FSM for Elixir</a></li>
<li><a href="/hacking/2022/04/30/finitomata-ex-docs">First Class Documentation</a></li>
<li><a href="/hacking/2023/03/06/finitomata-mox-testing">Finitomata ❤ Mox</a></li>
</ul>
<p>Now I have a proof that my premonitions were correct.</p>
<p>When I shared my approach to keeping the local cache of some external incoming value as a bunch of <a href="https://hexdocs.pm/finitomata"><code class="language-plaintext highlighter-rouge">Finitomata</code></a> instances to my boss, he replied with the following.</p>
<blockquote>
<p>The simple value as an FSM? Have you gone mad man?</p>
</blockquote>
<p>No, I haven’t. And here is why.</p>
<hr />
<p>We do receive the external values via some kind of stream, where we might receive a full snapshot for the key, a delta against the previously received snapshot for that key, and a delta against the snapshot for another key. The flow might contain tens thousands values per a second and the total number of keys might be up to one hundred thousand. This all might need more sophisticated infrastructure than a bare <a href="https://hexdocs.pm/elixir/DynamicSupervisor.html"><code class="language-plaintext highlighter-rouge">DynamicSupervisor</code></a> or even a <a href="https://hexdocs.pm/elixir/PartitionSupervisor.html"><code class="language-plaintext highlighter-rouge">PartitionSupervisor</code></a>. We need a handy lookup, and we might later decide to scale it horizontally.</p>
<p><a href="https://hexdocs.pm/finitomata/Infinitomata.html"><code class="language-plaintext highlighter-rouge">Infinitomata</code></a> runs transparently on the cluster, which means all one needs to use more memory/cores to process state transitions, would be adding new nodes to the cluster. <code class="language-plaintext highlighter-rouge">Infinitomata</code> will use them transparently upon addition.</p>
<p>But what’s wrong with leveraging a standard <a href="https://www.erlang.org/doc/man/pg.html"><code class="language-plaintext highlighter-rouge">:pg</code></a>, would probably ask a careful reader. After all, <code class="language-plaintext highlighter-rouge">Infinitomata</code> is built on top of <code class="language-plaintext highlighter-rouge">:pg</code> and we unlikely would need any <em>FSM</em> goodness to carry simple values. Negative. We definitely might rely on conditionals within values updates, but it would quickly become clumsy making it hard to reason about. On the contrary, we might keep the current state of the value in the <em>FSM</em> state, abstracting it out of the business logic. We won’t need to validate anything, because transitions are either possible (allowed,) or ignored.</p>
<p>As I said, we might receive a delta, which implies to get to the actual value, the snapshot must be already there. Yes, we might do somewhat along</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">value</span> <span class="o">=</span>
<span class="k">if</span> <span class="n">state</span><span class="o">.</span><span class="n">snapshot?</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="n">apply_delta</span><span class="p">(),</span> <span class="k">else</span><span class="p">:</span> <span class="ss">:ignore</span>
</code></pre></div></div>
<p>or even pattern-match in the function clause, but it’d be cumbersome, because deltas might rely on the snapshots for other keys. Instead, we might handle the transition for “local” delta if and only we already have a snapshot, meaning we are in the state <code class="language-plaintext highlighter-rouge">ready</code>. The transition handler for deltas relying on “external” keys would be as easy as</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">on_transition</span><span class="p">(</span><span class="n">current_state</span><span class="p">,</span> <span class="ss">:foreign_delta</span><span class="p">,</span> <span class="p">{</span><span class="n">foreign_key</span><span class="p">,</span> <span class="n">delta_values</span><span class="p">},</span> <span class="n">state</span><span class="p">)</span> <span class="k">do</span>
<span class="n">target_state</span> <span class="o">=</span>
<span class="k">case</span> <span class="no">Infinitomata</span><span class="o">.</span><span class="n">state</span><span class="p">(</span><span class="n">key</span><span class="p">)</span> <span class="k">do</span>
<span class="p">{</span><span class="ss">:error</span><span class="p">,</span> <span class="ss">:not_started</span><span class="p">}</span> <span class="o">-></span> <span class="no">Logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s2">"…"</span><span class="p">)</span>
<span class="p">%{</span><span class="ss">state:</span> <span class="ss">:idle</span><span class="p">}</span> <span class="o">-></span> <span class="no">Logger</span><span class="o">.</span><span class="n">warn</span><span class="p">(</span><span class="s2">"…"</span><span class="p">)</span>
<span class="p">%{</span><span class="ss">state:</span> <span class="ss">:ready</span><span class="p">}</span> <span class="o">-></span> <span class="n">apply_delta</span><span class="p">(</span><span class="n">delta_values</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">if</span> <span class="n">target_state</span> <span class="o">==</span> <span class="ss">:ok</span><span class="p">,</span>
<span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">current_state</span><span class="p">,</span> <span class="n">state</span><span class="p">},</span>
<span class="k">else</span><span class="p">:</span> <span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">target_state</span><span class="p">,</span> <span class="n">state</span><span class="p">}</span>
<span class="k">end</span>
</code></pre></div></div>
<p>This is clean, applies to <code class="language-plaintext highlighter-rouge">:foreign_delta</code> event only, and leaves the <em>FSM</em> in the correct state no matter what. We do gain the sequential processing for free as well. Also, there is no chance one forgets about some combination of state and event and leaves it unhandled. The <code class="language-plaintext highlighter-rouge">:finitomata</code> compiler would tell the <em>FSM</em> is inconsistent, if there are orphans or unhandled transitions.</p>
<p>All the <a href="https://hexdocs.pm/finitomata/Finitomata.html#c:on_transition/4"><code class="language-plaintext highlighter-rouge">on_transition/4</code></a> callbacks might be as a matter of fact implemented as a <code class="language-plaintext highlighter-rouge">with/1</code> chain, logging the error, if any, and leaving the <em>FSM</em> in the state it was before the event came. There is no chance to end up with some inconsistency.</p>
<p>Even if the <em>FSM</em> definitions is as simple as</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nv">@fsm</span> <span class="sd">"""
idle --> |snapshot| ready
ready --> |snapshot| ready
ready --> |delta| ready
ready --> |foreign_delta| ready
ready --> |shutdown| down
"""</span>
<span class="kn">use</span> <span class="no">Finitomata</span><span class="p">,</span> <span class="ss">fsm:</span> <span class="nv">@fsm</span><span class="p">,</span> <span class="ss">auto_terminate:</span> <span class="no">true</span>
</code></pre></div></div>
<p>I prefer to use <em>FSM</em> because it abstracts a lot of boilerplate out and makes me to implement business logic only, with all the guarantees and validations, provided by <code class="language-plaintext highlighter-rouge">Finitomata</code> <em>and</em> all the necessary <code class="language-plaintext highlighter-rouge">on_transition/4</code> handlers I am to implement, ready to pattern-match on meaningful states and events.</p>
<p>With a sleek learning curve, first-class test support with <a href="https://hexdocs.pm/finitomata/Finitomata.ExUnit.html#test_path/3"><code class="language-plaintext highlighter-rouge">Finitomata.ExUnit</code></a>, and plain text FSM declaration as both code and documentation, it’S a pleasure to deal with.</p>
<p>Happy infinite automating!</p>
Продолжая писать в то самое время, когда технологии поломали все социальные договоренности в сети2023-05-20T00:00:00+00:00https://rocket-science.ru/translation/2023/05/20/writing-when-tech-has-broken-social-contract<blockquote>
<p>Это перевод, если есть такая возможность, лучше прочитать <a href="https://www.baldurbjarnason.com/2023/tech-broke-the-webs-social-contract/">оригинал на английском</a>.</p>
</blockquote>
<p>Я тяжело переживаю по поводу того, куда нас завела эта дорожка, и стараюсь по мере сил бороться с этими переживаниями. Я постоянно думаю о том, <a href="https://www.baldurbjarnason.com/2023/what-next-he-asks/">какое место я занимаю в отрасли, которая, похоже, больше не заботится о том, что она производит</a>. Стало очевидно, что индустрии программного обеспечения попросту <a href="https://wandering.shop/@vaurora/110399176261147147">наплевать на программное обеспечение</a>.</p>
<h2 id="становится-только-хуже">Становится только хуже.</h2>
<blockquote>
<p>Кто-нибудь еще замечает, что программное обеспечение просто …ну… больше не работает подобающим образом? В каждой отдельной системе бронирования, какую я бы ни пыталась использовать на прошлой неделе, была серьезная ошибка, влияющая на доход (авиакомпания, бронирование такси, краткосрочная аренда, прием к врачу). Две из них были связаны с неправильной интерпретацией часового пояса, одна — с ошибкой проверки обязательности заполнения некоторых полей формы, а последняя так просто звучала как «вы не сможете получить назначенное место, если не позвоните нам».
— <a href="https://wandering.shop/@vaurora/110399176261147147">Valerie Aurora</a></p>
</blockquote>
<p>Разумеется, меня можно подозревать в предвзятости, ведь пару лет назад я написал <a href="https://www.baldurbjarnason.com/2021/software-crisis-2/">длинное эссе на эту тему</a>, а в прошлом году — <a href="https://softwarecrisis.baldurbjarnason.com/">целую книгу</a>.</p>
<p>Моя теория, которую я изложил в книге, заключается в том, что качество программного обеспечения и успех в бизнесе, особенно для отдельных менеджеров, — вообще никак не связаны. Настаивать на методах разработки программного обеспечения, которые зарекомендовали себя как уменьшающие количество дефектов и улучшающие пользовательский опыт, — скорее навредит вашей карьере, чем поможет. Менеджеры высокого уровня получают вознаграждение за масштаб проекта, а не за долгосрочный успех. Индустрия программного обеспечения никогда не могла похвастаться особенным качеством производимого продукта, но с каждым днем все становится только хуже.</p>
<p>Вот, например, все эти увольнения.</p>
<p>Любой руководитель, загнавший свою компанию в такую ситуацию, что им приходится увольнять 10-20% сотрудников — некомпетентен <em>по определению</em>.</p>
<p>Они привели свои компании к ужасающей финансовой и организационной катастрофе с не поддающимися никакой оценке пагубными последствиями, которые будут отдаваться эхом в течение целого десятилетия. Они — своим бесталанным руководством — поставили всех вокруг в жуткое положение. На адекватном рынке они были бы вынуждены уйти в отставку, да и то, если бы их попросту не уволили одним днем.</p>
<p>Вместо этого они стали образцом для подражания. Так же, как и раньше, во всех <em>иных</em> поразительно неудачных решениях по разработке программного обеспечения, принятых этими компаниями, все остальные игроки в индустрии решили следовать за ними, копируя их подход.</p>
<p>А их руководители еще и получают премии.</p>
<p>В принципе, то, что наше программное обеспечение ухудшается, ни для кого не должно быть сюрпризом. Это иррациональный рынок, управляемый людьми, которые не знают, что они делают.</p>
<h2 id="ииодно-сплошное-разочарование">ИИ — одно сплошное разочарование.</h2>
<p>Нигде это не проявляется так явно, как в случае с ИИ. Несколько недель назад генеральный директор <em>Google</em> Сундар Пичаи выступил в программе «60 минут» с рядом вопиющих облыжных заявлений, которые либо свидетельствуют о <a href="https://www.thedailybeast.com/60-minutes-made-a-shockingly-wrong-claim-about-googles-ai-chatbot-bard">полном непонимании технологии</a>, которую он рекламирует, либо о <a href="https://medium.com/@emilymenonbender/google-ceo-peddles-aihype-on-cbs-60-minutes-4a0e080ef406"><em>непорядочности</em></a>.</p>
<p>Я предпочитаю думать, что он просто абсолютно невежественен, но это лишь укрепляет мою точку зрения: всех этих людей следовало давно уволить.</p>
<p>Я немного отошел от темы, но индустрия программного обеспечения идет по пути, который прямо очень разочаровывает.</p>
<p>Наблюдать, как руководители дают обещания, основанные на фундаментальном непонимании технологии, которую они рекламируют, в то время как каждое их действие, кажется, специально направлено на увеличение количества дефектов, багов и ухудшения пользовательского опыта, — как бы охаректиризовать ощущения от всего этого?</p>
<p>Разочарование это, вот что это такое. <em>Разочарование.</em></p>
<p>Постоянным спутником перманентного ухудшения ситуации в отрасли является их неспособность думать в рамках более широкого контекста, в том числе о различных социальных контрактах, которые определяют положение технологии в обществе.</p>
<h2 id="с-помощью-искусственного-интеллекта-технологии-нарушили-социальные-договоренности-в-сети">С помощью искусственного интеллекта технологии нарушили социальные договоренности в сети.</h2>
<p>Проблема в том, что довольно много людей в сфере технологий <em>не</em> верят ни в какой общественный договор. Их концептуализация общества упрощена до предела: оно представляется им равновесием доминирующих намерений, мотивированных миметическим желанием. В этой картине мира богатые находятся на вершине; остальные из нас стремятся быть похожими на них; любая критика в их адрес рождена из зависти. Улучшить мир могут только те, у кого есть власть над другими. Любая форма просоциальных рассуждений, достижения консенсуса или подлинных переговоров кажется им чуждой.</p>
<p>Эти люди — реакционные либертарианские сволочи, и они — истеблишмент мира технологий. Они видят себя добродетельными пастырями человечества, особенно в долгосрочной перспективе, но, по большому счету, они всего лишь жадные до власти либертарианские сволочи.</p>
<p>Вот почему они оставляют после себя выжженную землю.</p>
<p>Не секрет, что многое из произведенного на свет технологической индустрией, не лишено того, что экономисты выразили бы эвфемизмом «негативные побочные эффекты».</p>
<p>Неспособность <em>Microsoft</em> справиться с дефектами программного обеспечения привела к тому, что в течение почти двух десятилетий обществу приходилось нести внушительные расходы по борьбе с фундаментально негодной безопасностью большинства версий <em>Windows</em>. Их некомпетентность создала рынок антивирусного программного обеспечения объемом 4 миллиарда долларов на «пике» незащищенности <em>Windows</em> в 2004 году, и никто не знает, во сколько миллиардов долларов реально обходится обществу заражение программного обеспечения вирусами и прочие взломы.</p>
<p>(В прошлой жизни я работал на поставщика антивирусного программного обеспечения. Если что, мы сильно недооцениваем разрушительные издержки, которые <em>Microsoft</em> нанесла нашей экономике из-за своей некомпетентности в области безопасности в течение этих лет.)</p>
<p>И ничего не меняется.</p>
<p><em>Android</em> — это куча вредоносных программ и не исправленных телефонов.</p>
<p><em>AirBnB</em>, похоже, был специально разработан для повышения цен на недвижимость за пределами того, что могут себе позволить местные жители.</p>
<p>«Контрактная» экономика предназначена для того, чтобы полностью подорвать общую оплату труда и гарантии занятости.</p>
<p>А уж их ориентированная на рекламу всеобщая слежка — это просто заветная мечта авторитарного правительства. Примеров слишком много, чтобы перечислить из все.</p>
<p>Но искусственный интеллект — это последняя капля.</p>
<h2 id="старый-общественный-договор">Старый общественный договор.</h2>
<p>Вообще-то договор, решулирующий «сетевое пространство», всегда был, как правило, довольно прост:</p>
<ul>
<li>Мы и медиа-компании размещаем материалы в интернете бесплатно. Некоторые из нас делают это по деловым соображениям. Кое-что из этого носит личный характер. Во многом это просто результат культурного обмена. Люди есть люди.</li>
<li>Технологические компании используют этот материал, но чаще всего экономически дополняющим образом. Поисковые системы используют его, чтобы помочь вам найти то, что вы ищете. Сайты социальных сетей показывают вам рекламу. Что-то обрабатывается с целью улучшения предоставляемых сервисов, таких как проверка орфографии или автозаполнение.</li>
</ul>
<p>Все это сломалось не сразу, а рушилось постепенно. Переводчики сильно пострадали от программного обеспечения для перевода, а ведь само его создание оказалось возможным только благодаря обработке текстов, сделанных переводчиками-людьми.</p>
<p><em>Google</em> неумолимо манипулирует результатами своих поисковых систем, чтобы все чаще и чаще подменять страницы в результатах выдачи. Некоторые из подобных действий привели к судебным искам или даже прямому изменению законодательства. Многие юридические проблемы <em>Google</em> в <em>ЕС</em> связаны именно с этим.</p>
<p>Повсеместная технологическая слежка также раздвинула границы того, что многие считали приемлемым. Теперь даже мои родители используют блокировщики рекламы.</p>
<p>Но языковые и диффузионные модели идут дальше. Договор, который технологии предлагают нам на этом поле, тоже не особо запутан:</p>
<ul>
<li>Мы и медиа-компании размещаем материалы в интернете бесплатно. Некоторые из нас делают это по деловым соображениям. Кое-что из этого носит личный характер.</li>
<li>Технологические компании используют этот материал для создания систем, которые могут создавать некачественные, упрощенные, неработоспособные версии того, что сделали мы, а еще — создавать убедительные подделки как результатов нашей работы, так и нас самих: например, для астротурфинга, разрушая нашу работу, наш бизнес и наши социальные взаимодействия.</li>
</ul>
<h2 id="это-плохая-сделка">Это плохая сделка.</h2>
<p>Это и близко не справедливая сделка для тех из нас, кто придерживается принципа «бесплатного размещения материалов в сети». Не имеет значения, незаконно это или нет — хотя законодательство, вероятно, единственный способ заставить технологическую индустрию остановить это, — потому что общественный договор уже нарушен.</p>
<p>Стимул сделать что-то новое уходит след в след за вытесненными из интернета нашими прошлыми работами. Вытесненными — контентом, который был создан с использованием искусственного интеллекта. По мере того, как желание что-то делать отступает, все меньше и меньше людей и организаций будут вносить свой вклад в цифровое наследие. Все больше и больше материалов будет спрятано за неприветливой дверью с табличкой «вход платный».</p>
<p>Все это уже повлияло на мои отношения с сетью.</p>
<p>В процентном соотношении я опубликовал в интернете меньше выдержек из «<a href="https://illusion.baldurbjarnason.com/"><em>Иллюзии интеллекта</em></a>», чем из «<a href="https://softwarecrisis.baldurbjarnason.com/"><em>Кризиса программного обеспечения</em></a>». Я также почти перестал публиковать фотографии. Возможно, я больше пишу в блог, но меня все сильнее беспокоит, а сто́ит ли мне это делать.</p>
<p>Я нахожусь ровно в той точке, когда при обычных обстоятельствах стратегически верным решением было бы продолжать писать больше и больше. Обычно это был бы самый эффективный способ подразогнать мою карьеру в данных обстоятельствах.</p>
<p>Но теперь против всех публикаций стоит жирный вопросительный знак.</p>
<p>Побился ли бы я об заклад, что поток шаблонных текстов, созданных языковыми моделями, выведет вдумчивые авторские эссе в премиальный сегмент? А что, если распространение этих эссе — просто рабский труд на благо улучшения моделей?</p>
<p>Подумываю ли я о том, чтобы сделать значительную часть своей писанины платной, или, хотя бы, закрыть все это за логином, даже если это было бы контрпродуктивно для моей карьеры? Не приведет ли это вообще к разрыву отношений с моими друзьями и интернет-сообществом в целом?</p>
<p>Неужели я просто игнорирую тот факт, что помогаю воспитывать универсального производителя, который, как надеются многие в индустрии, заменит всех нас?</p>
<p>Я не знаю.</p>
<p>Я знаю, что буду продолжать писать об этом. Даже просто написание этой заметки помогло мне прояснить свои мысли и привести в порядок эмоции.</p>
<p>Буду ли я продолжать публиковать свои работы в интернете?</p>
<p><em>Я надеюсь на это.</em> Я надеюсь, что онлайн-писательство переживет любые передряги.</p>
<p>Но я больше в этом не уверен.</p>
<hr />
<p><em>Лучший способ поддержать автора — купить одну из книг, «<a href="https://illusion.baldurbjarnason.com/">Иллюзия интеллекта: практическое руководство по бизнес-рискам, связанным с генеративным ИИ</a>«, или «<a href="https://softwarecrisis.baldurbjarnason.com/">Выход из кризиса программного обеспечения</a>.</em></p>
Finitomata ❤ Mox2023-03-06T00:00:00+00:00https://rocket-science.ru/hacking/2023/03/06/finitomata-mox-testing<p>While pushing for FSMs in general and <a href="https://hexdocs.pm/finitomata"><code class="language-plaintext highlighter-rouge">Finitomata</code></a> in particular, I needed to provide a robust and handy way to test the beast. The ideology of <code class="language-plaintext highlighter-rouge">Finitomata</code> approach is to spawn a process per each entity alive, where <em>alive</em> means <em>in some intermediate state</em>. Generally speaking, that does not make the conventional testing a charm. Consider a process which might change its state (or, in terms of <code class="language-plaintext highlighter-rouge">Finitomata</code>, <em>can transition from one state to another</em>,) not only as a result of an explicit intervention, like a message sent to it, or, which is nearly the same, via <code class="language-plaintext highlighter-rouge">Finitomata.transition/4</code>, but also as a consequence or even a coincidence of some several external occasions.</p>
<p><img src="/img/camina-ronda.jpg" alt="Camina Ronda" /></p>
<p>Testing asynchronous interoperation is not the most trivial part of dealing with OTP, although we still might cook it right and make delicious. To simplify it and make testing pleasant and easy, I needed to inject some synchronization points, somewhat like <em>mutexes</em>, or, if you wish, erlang <em>schedulers</em>.</p>
<p>I always have been a big fan of callbacks. I tend to allow listener injection to each and every process I am implementing. If there are several phases of the process, I’d allow listening on phases change. If there is an input parsed, I’d welcome a listener on each line parsed. You got the point. FSM literally implies the existence of <code class="language-plaintext highlighter-rouge">on_transition</code> callback for some listener(s).</p>
<p>This listener would be an ideal <a href="https://dashbit.co/blog/mocks-and-explicit-contracts"><em>noun to mock</em></a>. During testing, this listener is to play the single role of <strong>process synchronization guard</strong>, but it must play it great. It can and should be used to synchronize control flow between testing and tested processes (and the whole outer world, if desired.) By introducing several synchronization points which are roughly speaking the callbacks themselves, it provides the catch-up scenario for the test code via <code class="language-plaintext highlighter-rouge">assert_receive/2</code>.</p>
<hr />
<p>The main issue with implementing this idea was <code class="language-plaintext highlighter-rouge">Mox</code> demanded the process to call mocks to be started at the moment of mock declaration, which results is what I call <em>alive-lock</em>, as a contrary to notorious <em>dead-lock</em> from Java world. To declare mock, I needed a process and to test a process I needed the already declared mocks. To make my initial plan happen, I provided a pull request to <code class="language-plaintext highlighter-rouge">Mox</code> to allow deferred <code class="language-plaintext highlighter-rouge">pid</code> resolution, based on the process name. It has been merged and it might be tested right away with a dependency specified as <code class="language-plaintext highlighter-rouge">git: :master</code>.</p>
<p>So far so good, now I have a clean interface of any possible interop between testing process and the test itself.</p>
<p>Here is an example taken from <code class="language-plaintext highlighter-rouge">Finitomata</code> tests, which shows the approach.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Mox</span><span class="o">.</span><span class="n">defmock</span><span class="p">(</span><span class="no">Finitomata</span><span class="o">.</span><span class="no">Test</span><span class="o">.</span><span class="no">Listener</span><span class="p">,</span> <span class="ss">for:</span> <span class="no">Finitomata</span><span class="o">.</span><span class="no">Listener</span><span class="p">)</span>
<span class="k">defmodule</span> <span class="no">Finitomata</span><span class="o">.</span><span class="no">Test</span><span class="o">.</span><span class="no">Mox</span> <span class="k">do</span>
<span class="nv">@fsm</span> <span class="sd">"""
idle --> |start!| started
started --> |do| done
"""</span>
<span class="kn">use</span> <span class="no">Finitomata</span><span class="p">,</span> <span class="ss">fsm:</span> <span class="nv">@fsm</span><span class="p">,</span> <span class="ss">auto_terminate:</span> <span class="no">true</span><span class="p">,</span> <span class="ss">listener:</span> <span class="no">Finitomata</span><span class="o">.</span><span class="no">Test</span><span class="o">.</span><span class="no">Listener</span>
<span class="err">…</span>
<span class="k">end</span>
</code></pre></div></div>
<p>This is the standard declaration of <code class="language-plaintext highlighter-rouge">Finitomata</code> worker: <em>FSM</em>, having three states, an unconditional transition to <code class="language-plaintext highlighter-rouge">started</code> and a normal transition to <code class="language-plaintext highlighter-rouge">done</code>. After <code class="language-plaintext highlighter-rouge">start!</code> event we might want to test initialization and after <code class="language-plaintext highlighter-rouge">do</code> we might want to test actual behaviour.</p>
<p>Here is the code in test doing exactly that, and I finally like how this code looks like.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="n">parent</span> <span class="o">=</span> <span class="n">self</span><span class="p">()</span>
<span class="no">Finitomata</span><span class="o">.</span><span class="no">Test</span><span class="o">.</span><span class="no">Listener</span>
<span class="o">|></span> <span class="n">allow</span><span class="p">(</span><span class="n">parent</span><span class="p">,</span> <span class="k">fn</span> <span class="o">-></span> <span class="no">GenServer</span><span class="o">.</span><span class="n">whereis</span><span class="p">(</span><span class="n">fsm_name</span><span class="p">)</span> <span class="k">end</span><span class="p">)</span>
<span class="o">|></span> <span class="n">expect</span><span class="p">(</span><span class="ss">:after_transition</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="k">fn</span> <span class="n">id</span><span class="p">,</span> <span class="n">state</span><span class="p">,</span> <span class="n">payload</span> <span class="o">-></span>
<span class="n">parent</span> <span class="o">|></span> <span class="n">send</span><span class="p">({</span><span class="ss">:on_transition</span><span class="p">,</span> <span class="n">id</span><span class="p">,</span> <span class="n">state</span><span class="p">,</span> <span class="n">payload</span><span class="p">})</span> <span class="o">|></span> <span class="n">then</span><span class="p">(</span><span class="k">fn</span> <span class="n">_</span> <span class="o">-></span> <span class="ss">:ok</span> <span class="k">end</span><span class="p">)</span>
<span class="k">end</span><span class="p">)</span>
<span class="err">…</span>
<span class="no">Finitomata</span><span class="o">.</span><span class="n">start_fsm</span><span class="p">(</span><span class="err">…</span><span class="p">)</span>
<span class="err">…</span>
<span class="c1"># test payload after initialization</span>
<span class="n">assert_receive</span> <span class="p">{</span><span class="ss">:on_transition</span><span class="p">,</span> <span class="o">^</span><span class="n">fsm_name</span><span class="p">,</span> <span class="ss">:idle</span><span class="p">,</span> <span class="p">%{</span><span class="ss">foo:</span> <span class="o">^</span><span class="n">bar</span><span class="p">}}</span>
<span class="c1"># test payload after unconditional state advance</span>
<span class="n">assert_receive</span> <span class="p">{</span><span class="ss">:on_transition</span><span class="p">,</span> <span class="o">^</span><span class="n">fsm_name</span><span class="p">,</span> <span class="ss">:started</span><span class="p">,</span> <span class="p">%{</span><span class="ss">data:</span> <span class="n">_</span><span class="p">}}</span>
<span class="no">Finitomata</span><span class="o">.</span><span class="n">transition</span><span class="p">(</span><span class="s2">"MyFSM"</span><span class="p">,</span> <span class="p">{</span><span class="ss">:do</span><span class="p">,</span> <span class="n">event_payload</span><span class="p">})</span>
<span class="c1"># test payload after conditional transition</span>
<span class="n">assert_receive</span> <span class="p">{</span><span class="ss">:on_transition</span><span class="p">,</span> <span class="o">^</span><span class="n">fsm_name</span><span class="p">,</span> <span class="ss">:done</span><span class="p">,</span> <span class="p">%{</span><span class="ss">outcome:</span> <span class="n">_</span><span class="p">}}</span>
</code></pre></div></div>
<hr />
<p>Next step would be to export helper macros to decrease an amount of code needed to be written in each test.</p>
<hr />
<p>Happy synchrotesting!</p>
Wash your dishes2023-01-17T00:00:00+00:00https://rocket-science.ru/hacking/2023/01/17/wash-your-disches<p>Many years ago I’ve read Joe Armstrong’s brilliant description of how they invented <a href="https://www.erlang.org"><em>erlang</em></a>. It was based on the principles of functioning of society in general and the human brain in particular. Instead of inventing behavioral rules and laws out of the blue, the authors of the new programming language simply transferred patterns that are well known from history and sociology as useful in communication.</p>
<p><img src="/img/baeza.jpg" alt="Baeza" /></p>
<p>This is how the <em>erlang</em> syntax appeared, copying, as far as possible, the English text not only with the spelling of keywords, but also with syntax and punctuation. Similarly, an actor model was chosen because it imitates the standard algorithm of human interaction (“I can’t read minds”).</p>
<blockquote>
<p>We don’t have shared memory. I have my memory. You have yours. We have two brains, one each. They are not joined. To change your memory, I send you a message: I talk, or I wave my arms.<br />
You listen, you see, and your memory changes; however, without asking you a question or observing your response, I do not know that you have received my messages.<br />
<em>~ Joe Armstrong, Programming Erlang</em></p>
</blockquote>
<p>Since then, I have been trying to solve all the problems I encounter in my work by projecting them onto my (or someone else’s) life experience. Such examples help a lot to understand where you are right and where you are very wrong.</p>
<p>I often cite culinary analogies in disputes about the industry. Not even because cooking is in many ways akin to software development (although this is also the case), but to heighten the contrast. A different subject area allows you to focus more clearly on important aspects and not go into terminological discussions about minor details.</p>
<p>Surely everyone is familiar with the type of “great chef by chance.” You know, a person who doesn’t like to cook much, considers the time spent in the kitchen lost, but has a couple of signature dishes in stock, which they invariably strive to impress new acquaintances. “Nah, Harley, you never change!”</p>
<p>These dishes, thanks to repeated permutations and a mechanical procedure honed over the years, always turn out really, really tasty. Without a twist, like ones you’ve got in a good restaurant having no Michelin stars. There is nothing surprising here, a hare can be taught to smoke, if there would be some time and diligence invested. That’s why I always sneak a peek into the kitchen after the accomplished ritual of cooking the crown foie gras under olive marmalade with truffles. And—if a mountain of unwashed dishes stealthily looks at us from the sink—I’d rather have a lime slice for a snack waiting to buy a pizza on my way home. A good chef does not expect an errand boy to wash the dishes for him.</p>
<p>Software baking is exactly the same. While industry gurus rank code by execution speed, readability, maintainability, scalability, and God knows what other criteria, I approach the assessment very superficially. The code just needs to be neat. Sloppy code always tends to be slow, unreadable and difficult to amend let alone maintain. The neatness of the code—think of the dishes washed after cooking—will tell the observant viewer a whole story: about the experience of supporting your own code, fixing someone else’s bugs, maintaining projects, and many other very important (contrary to so-called ‘design patterns’) things.</p>
<p>Aesthetic taste in general is an exceptional (and perniciously underrated) attribute of a really good programmer. That’s why I get a little tense when people praise <em>Go</em> or use <em>IDEA</em> in their work. That’s why I spend an hour adjusting the color scheme, fonts and environment, even of alien laptops I borrow to take to the conference for three days. Unfortunately, people who lack good taste are also very <em>influenceable</em> by fashion trends. Hence all these iphones, louisvuittons and ugly jeeps of recent years. It’s hard to imagine that twenty years ago, all these phones and cars (not to mention clothes), they managed to make them stylish. But then the trees were tall, and the grass was green.</p>
<p>A well-designed programming language literally prevents you from writing dirty, aesthetically unpleasant code. Every time I find myself in the wilds of a conditional operator on elixir, I mercilessly erase the entire piece I just wrote that got me there, and start with a clean slate. Confusing, clumsy, unsightly? “So you did something wrong, man. Rewrite it from scratch.”</p>
<p>Unfortunately, most languages simply do not provide the ability to write elegant code. It is enough to run the autoformatter in a rather non-trivial piece on <em>Haskell</em>, and it becomes obvious. Even <em>Java</em>, with its verbosity that defies any intelligible explanation, does not look as disgusting. Triple-cover your everything with algebraic types, but they would not allow you to write intelligible code when you have five hundred functions in <em>Prelude</em> with indistinguishable four—letter names meaning whoever knows what. Which line you need to put a closing bracket on and how many spaces should be in the tab is a matter of habit, nothing else. But the python code, in which the eye literally has nothing to cling to, is beyond good and evil. Therefore, any python project on the hundredth line of code turns into a mess, understandable only to the author. Even <em>Perl</em> looks more profitable in this regard, despite it is confusing, invented by aliens, understandable only to Larry Wall,—but it is aesthetically elegant.</p>
<p>Returning to the original thesis, I just wanted to say that if instead of tricky questions about design patterns in general and closures in the javascript in particular—you’d simply take the candidate to the kitchen and see how they cook curry—much more will come out. If they do not know what chicken carry stands for, this is a <em>YAGNI</em> type with a narrow outlook and a desire to do everything according to the only pattern that they have with them since the first year in the university. If they demand thyme and sage besides a mill for cilantro grains, they are a pedant who will torment you with claims to the tools, the environment and five hundred third—party libraries. Will they put chicken in a saucepan and sprinkle unpeeled garlic cloves on top?—This is a work-life balancer who has a job instead of a bedside table from which he gets money. But if the candidate silently cooks something, washes the dishes after themselves and only then calls you, hiding a smile—release the offer out of your pocket. This is your developer. Don’t miss the oportunity.</p>
Finitomata Marries Ecto2023-01-01T00:00:00+00:00https://rocket-science.ru/hacking/2023/01/01/finitomata-marries-ecto<p>When people around me talk about metrics, the most used words are service reliability, latency, throughput, load capacity and their siblings. All that is indeed extremely important until it gets to the data. In the data world, where we deal with customer’s private information, business nuances, and, after all, money, the <em>data consistency</em> is what we should take care of in the first place. I even <a href="https://rocket-science.ru/hacking/2022/08/14/fsm-driven-development">wrote a rant about it a while ago</a>.</p>
<p><img src="/img/roses-are-red.jpg" alt="Roses Are Red" /></p>
<p>Data consistency is harder than rumors say. Data consistency, contrary to popular misconception, cannot be preserved by <em>RDBMS</em> itself. It cannot be represented as a diagram, let alone graph of relations. And, after all, when people talk about consistency, they too often use a wrong vocabulary. There is even the brand new wording invented by people who had most likely never been to the university. <a href="https://learn.microsoft.com/en-us/azure/architecture/reference-architectures/saga/saga">Saga</a>, if one would squeeze out all the water from the paper, is a poor-man FSM. OK, rich-man FSM, it’s coming from Microsoft after all.</p>
<p>I blindly used mocks for decades until I read José Valim’s ‘<a href="https://dashbit.co/blog/mocks-and-explicit-contracts">I always consider “mock” to be a noun, never a verb.</a>’ and it clicked. Not that I had no clue what mocks are, but I surely was abusing the clean concept with somewhat nasty implementation.</p>
<p>Data consistency has not clicked for me for a while too. I felt there should be a clean way to ensure it in general, but I missed the details. I knew it should most likely rely on <a href="https://en.wikipedia.org/wiki/Finite-state_machine">finite automata</a> and I tended to vote for it to be <a href="https://en.wikipedia.org/wiki/Nondeterministic_finite_automaton">non-deterministic</a> because the absense of side effects would ruin any business (including but not limited to programming languages relying on pure functions only.)</p>
<p>I worked a lot with a handy <a href="https://github.com/geekq/workflow"><code class="language-plaintext highlighter-rouge">workflow</code></a> gem in ruby, and it did what I wanted to some extent, but it has two glitches that cannot be circumvented. The state is attached to an entity (to the data object directly,) and there is no restriction to modify entity state with transitions only. One can easily update the amount of order manually, bypassing a transition, which obviously ruins the whole math behind finite automata. It miserably becomes an <em>infinite state machine</em>.</p>
<p>That said, while attaching <em>FSM</em> to data entities might ensure their consistency alone, it cannot validate relationships between them. Which are usually called <strong>business processes</strong>. Also, business processes is something what runs the business, might be spoken about with non-tech deps, and, the last but not the least, they are easily described with goddamn diagrams. Which are barely differ from a ready-to-use <em>FSM</em> definition.</p>
<hr />
<p>Consider <a href="https://en.wikipedia.org/wiki/Actor_model">actor model</a> applied to data. Business processes are processes. Events provoking transitions are messages. Data is nothing but an internal state of a business process. Don’t you feel the smell of fault tolerance through business processes supervision trees yet?</p>
<hr />
<p>In simple systems, the business process might be attached to the data entity, but still it would be better be a separate beast. When I started to work on <em>FSM</em> application to data consistency, I wrote an <a href="https://github.com/am-kantox/finitomata/tree/main/examples/ecto_intergation">example application</a> managing a <code class="language-plaintext highlighter-rouge">Post</code> in a wild. The most impressive feature of <a href="https://elixir-lang.org/"><em>Elixir</em></a> as of language is it literally yells at you when you are trying to implement things in a wrong way. The code starts to look ugly, whispering “Hey, you are likely missing a better approach here.”</p>
<p>Despite that I managed the example to work and even look not so ugly, I discovered the whole idea was crucially misconcepted. The <code class="language-plaintext highlighter-rouge">Post</code> itself should not keep track of its changes. This is not scalable, not extandable, and not supportable. Adding a comment would lead the approach to disintegrating like a house of cards. There must be a separated business process, like <code class="language-plaintext highlighter-rouge">PostLifecycle</code>, which would take care of a <code class="language-plaintext highlighter-rouge">Post</code> …ahem… lifecycle. And—which is extremely important—the <code class="language-plaintext highlighter-rouge">Post</code> itself should not allow any modifications outside of the transitions in this process.</p>
<p>Here we get to the intriguing and cost-free side effect. Storing transitions alone would give us a perfectly valid event log. It would not be re-playable in general, due to non-pure transitions (consider the post which embeds an <a href="https://ogp.me/"><code class="language-plaintext highlighter-rouge">opg</code></a>, which is to be downloaded from some external source with not idempotent responses,) but despite that, the event log would be fully plausible.</p>
<hr />
<p>This approach works well with web applications. Instead of random requests, modifying the internal data of the application directly (as every single protocol known to me allows in a nutshell,) the endpoints in the web application should expose transition handlers only. The outmost world should not be allowed (and therefore even have an access) to <em>modify</em> data directly. It should be permitted to initiate transitions only. As by Joe Armstrong’s analogy, the computer interoperation should mimic the human communication; we cannot read minds, we can only send messages, like saying words or waving our hands, and the interlocutor might or might not decide to change their mind or even share some of it with us.</p>
<p>The described approach is extremely easy to reason about, which is also true for most of other approaches broadly accepted in computer science. Its uniqueness is it drastically simplifies the project vision exchange with the people of business. Transitions are exactly what they think of, and everything else magically becomes <em>actual</em> implementation details.</p>
<hr />
<p>I am to come up with another example, demostrating the power of <code class="language-plaintext highlighter-rouge">Finitomata</code> in a concord with business processes automation, mapped to data internally, but in the meanwhile I wanted to share this agenda as a pure idea.</p>
<p>Happy business process driven development!</p>
FSM Driven Development2022-08-14T00:00:00+00:00https://rocket-science.ru/hacking/2022/08/14/fsm-driven-development<blockquote>
<p>There are only two hard things in Computer Science: cache invalidation and naming things.<br />
— Phil Karlton</p>
</blockquote>
<p>Well, actually my whole experience says there are three, and the third one is the most challenging one. It’s <strong>data consistency</strong>.</p>
<p>All the paradigms, methodologies, best practices, and all these accumulated decades of knowledge teach us how to store data, derive data, operate data, transform data, but none tells us how to keep data consistent. Which is way more important compared to naming (hey, Fortran 77,) and cache invalidation which might always be fixed by switching the cache heartfully off.</p>
<p><img src="/img/rainbow.jpg" alt="Rainbow in Montgat" /></p>
<p>HTTP verbs, REST, GraphQL, all the ORMs, even databases… Each and every <em>framework</em> does literally nothing to help keeping the data consistent. Yeah, databases are guilty less than others, but still indices and foreign keys fail miserably on cases when the relationship involves <em>states of entities</em>. Consider the cloying, stuck in the teeth, example: posts with comments. Any mediocre developer would tell you that the comment might be added to the <em>published</em> posts only, not to drafts or deleted posts. Can we express this in DB constraints?—Hell, no.</p>
<p>In the evolving applications, number of ties and relationships grows exponentially. One cannot tag the suspended author, a legacy post cannot receive comments after two weeks, you name it. It all results in the cumbersome full-of-conditional-statements spaghetti business logic, applied here and there in the random places of the application. It quickly becomes unreadable let alone maintainance. What would be the silver bullet to fix this crap?</p>
<p>Well, there is no silver bullet, but there is definitely a pill. <em>FSM</em> with an attached to the entity <em>state</em> and validated transitions would allow us to make sure that the entity, or a set of entities, is always in the consistent state (or in the middle of a rollable back transition.) These transitions behave like database transactions, but on the application business logic level. Adding a comment would look like (in pseudocode) <code class="language-plaintext highlighter-rouge">Post.transition(:new_comment) && Comment.transition(:added)</code>. The example looks a bit contrived (and it surely is,) but it shows the technique <em>and</em> is expandable to more complicated relationships. Once the <code class="language-plaintext highlighter-rouge">Comment</code> is in added state, we are sure it has been attached to the published <code class="language-plaintext highlighter-rouge">Post</code> (otherwise <code class="language-plaintext highlighter-rouge">Post.transition(:new_comment)</code> would have failed,) and this <code class="language-plaintext highlighter-rouge">Post</code> is aware of this <code class="language-plaintext highlighter-rouge">Comment</code>.</p>
<hr />
<p>I tend to bring the <em>FSM</em> to everywhere, it drastically simplifies the development process and maintainability of the resulting code. My effort resulted in <a href="https://rocket-science.ru/hacking/2022/04/02/finitomata"><code class="language-plaintext highlighter-rouge">Finitomata</code></a> library several months ago, but I still felt discontented because too much of a boilerplate was still to be produced and/or copy-pasted from project to project. That’s how <a href="https://hexdocs.pm/siblings"><code class="language-plaintext highlighter-rouge">Siblings</code></a> library was born. Built on top of <code class="language-plaintext highlighter-rouge">Finitomata</code>, it enforces all the business logic to be written in the dedicated handlers and callbacks. One cannot interfere the internal state of anything managed by the library in any way but initiating a <em>transition</em> and doing stuff in the callback.</p>
<p>All the stateful processes are <em>FSM</em> implementations underneath, supervised by a partitioned <code class="language-plaintext highlighter-rouge">DynamicSupervisor</code> and accessible by a fast <code class="language-plaintext highlighter-rouge">O(1)</code> lookup. To update the state, one should start a transition. <em>And</em> even to create a new object, one should start a transition on the supervising entity. So yeah, <strong>everything is a transition</strong> within the context of this library.</p>
<hr />
<p>That makes me think about exploring the idea further to introducing <code class="language-plaintext highlighter-rouge">TRANSITION</code> <em>HTTP</em> verb. Well, let it be a <code class="language-plaintext highlighter-rouge">POST</code> with transition data in the body (<code class="language-plaintext highlighter-rouge">event</code>, <code class="language-plaintext highlighter-rouge">event_payload</code>.) The shape of the transition input would automatically validate the <code class="language-plaintext highlighter-rouge">event_payload</code>, making the validation <em>and</em> even client code generation obvious and straightforward.</p>
<p>I’m currently in the very beginning of an enthralling journey to a complete specification, but I see a bright future for this particular feature and I’d be glad to hear opinions on the matter.</p>
<p>Happy transitioning!</p>
Md. Another Word About Markdown2022-08-07T00:00:00+00:00https://rocket-science.ru/hacking/2022/08/07/md-under-the-hood<p><strong><em>NB</em> This post is in English, <a href="/2022-08-06-md-under-the-hood.md">Russian version is here</a></strong></p>
<h2 id="intro">Intro</h2>
<p>I never liked visual tools to cook text (well, except for <em>Delphi32</em>, which fed me a few years in the 1990s). I used <em>LaTeX</em> for my masters and other hand-written poems, but it always made me feel a little sick to know that I was using a microscope to nail up the screws. As you probably already guessed, there were quite a few formulas in my diploma, and even fewer in my poems.</p>
<p>Then I learned about <em>SGML</em>, and with the arrival of the web in our homes, about <em>HTML</em>; I learned a couple of tags by heart and began to design texts for my three and a half loyal readers using it. Even then, in 1996, I was incredibly angry that for some standard design samples (three asterisks for an empty verse title, for example) I have to literally copy-paste a few lines of unreadable spaghetti from my snippet store. This is how my first self-written blog engine was created. It was a single <em>XML</em> file with all the texts and an <em>XSLT</em> transformation that created a bunch of <em>HTML</em> pages from it. Unfortunately, the code for this work of art has been lost.</p>
<p><img src="/img/anis-del-mono.jpg" alt="Anis del Mono" /></p>
<p>This solution worked perfectly for me, because the tag <code class="language-plaintext highlighter-rouge"><verse caption="none"></code> literally without my participation was uniformly converted into the very exact spaghetti that annoyed me in pure <em>HTML</em>. Then <em>LiveJournal</em> appeared and I abandoned my blog for several years, then I don’t remember what, but when I wanted to publish something in my personal space again, markdown has bloomed everywhere. Which at first glance is fine, because lists of asterisks and a slope of underscores are exactly what you need to design a neat text that is not required to approved by <em>NASA</em> standards. Still, the same problem with the very title of the verse straightened my shoulders again. I absolutely do not have the opportunity to pretty-format a link to the <em>Twitter</em> account in the form <a href="https://twitter.com/mudasobwa"><code class="language-plaintext highlighter-rouge">@mudasobwa</code></a> or to a reddit thread using <code class="language-plaintext highlighter-rouge">[rd /f/foo/1234]</code>, or something like that. So I decided to write my own parser.</p>
<h2 id="self-baked-parser">Self-baked Parser</h2>
<p>Home-cooked parser is good because one can explicitly set the proper grammar for themselves (I often use superscript, and each time typing <code class="language-plaintext highlighter-rouge"><sup>foo</sup></code> instead of the obvious and absolutely natural markdown <code class="language-plaintext highlighter-rouge">^foo^</code> is kinda weird.) The problem is, I hate everything that’s nailed down hardcodedly. I am an apologist for screws. I want to be able to grab a screwdriver, unscrew the chandelier, and screw on the gallows hook instead. I write all the code in this way, which often leads to closing tasks from the backlog as if retroactively. That said, I decided that the grammar should be customizable.</p>
<p>I also knew that I wasn’t going to set a goal to fully pass the existing tests for common markdown. People who need tables inside a list inside a table are not my clients (as a result, oddly enough, almost all such exotics were supported naturally by the code which was not meant to support them.)</p>
<p>First of all, I tried to organize the subspecies of the standard markdown markup. Here’s what comes to mind right away:</p>
<ul>
<li>paragraphs (separated by two or more line breaks)</li>
<li>code (I’m a programmer, it’s important to me)</li>
<li>“parentheses” (bold, emphasized, anything that looks and feels as <em>parentheses</em>)</li>
<li>escaped characters</li>
<li>simple substitution (the <code class="language-plaintext highlighter-rouge"><</code> sign should become <code class="language-plaintext highlighter-rouge">&lt;</code> and so on)</li>
<li>headers, etc. (from here to the end of the line)</li>
<li>separators (<code class="language-plaintext highlighter-rouge"><hr/></code> siblings)</li>
<li>lists</li>
</ul>
<p>That’s enough for starters. Then I added comments, tags, footnotes, and whatnot, and you can see the full list today in the <a href="https://hexdocs.pm/md/Md.html#module-markup-handling">documentation</a>.</p>
<h2 id="how-do-we-parse">How Do We Parse?</h2>
<p>I’ve always been a fan of the <em>XMPP</em> protocol because it allows to work with an infinite stream of bytes rather than with a file. For the same reason, I prefer <em>SAX</em> parsers. We devour the available part of the stream, process it, call third-party registered handlers, and sit there in peace waiting for new data.</p>
<p>I must say that I always, in all projects, without exception, provide the possibility of connecting callback listeners. Has something, that the outside world might potentially want to know about, happen? Well, “register a listener and I’am to call it back.” Therefore, the consumer might connect a listener to my parser and quickly learn about the twists and turns of parsing.</p>
<p>Summing up, there are to exist: custom syntax (within compliance with the original markdown;) the ability to connect listeners; one pass. The last requirement was born simply because in an infinite stream it is impossible to provide support for lookbehind, sooner or later I will be forced to cut off the tail of this snake.</p>
<p>Well, cool. It’s now time to do the actual parsing. I’d like to make a statement that this task is not necessarily feasible in languages without strong support for pattern matching. In pure <code class="language-plaintext highlighter-rouge">C</code>, it can be solved by simulating it by looking ahead to the length of the maximum <em>tag</em> (in the current version of markdown, this number is 3, for <code class="language-plaintext highlighter-rouge">---</code> and similar markup, and for custom syntax it can be calculated.) <em>Idris</em> would allow to solve this problem beautifully without explicit pattern matching using dependent types (and <em>Haskell</em> sucks on the matter, of course.) But hey, I have <em>Elixir</em> in which pattern matching is the first class citizen. So I’d simply sort the opening tags from the grammar description by length and match the input to these samples, recursively calling the same function on the remaining thread.</p>
<p>Everything should be fine, but. There is still a hierarchy of tags (e. g. escaping is stronger than parentheses,) context dependence (inside a piece of code, an underscore is just an underscore, not the emphasized text marker,) and terminating sequences of characters (the beginning of a new paragraph should close all open tags). Thus, we are in a need to push some knowledge about the already processed text further.</p>
<h2 id="custom-grammars">Custom Grammars</h2>
<p>Strictly speaking, it would be possible to just read the grammar from the config file and pile up all sorts of conditional operators, but this would be damn slow, and the pattern matching would have to be emulated. I wanted the parser to work with the selected grammar as if it were written specifically for it, without redundant checks and superfluous processor ticks. Therefore, based on the configuration of the selected grammar, I do generate a processing code. All of it. If someone understands <em>Elixir</em> code while not being afraid of harsh metaprogramming, here’s the <a href="https://github.com/am-kantox/md/blob/v0.8.1/lib/md/parser/engine.ex#L50-L81">generation process</a> code. For the simplest version (like slack grammar,) the generated code will be about 8-10 times faster than code with full markdown support, due to a much smaller number of pattern matching function clauses.</p>
<h2 id="custom-handler">Custom Handler</h2>
<p>In addition to mandatory callbacks from wherever it makes sense, I always provide the opportunity to write a custom handler in addition to the existing ones. When I laid out this opportunity, I didn’t even know why, but it finally came in handy. It really did, as always. What does a standard markdown do when it encounters a separate link? <a href="https://github.com/am-kantox/md">https://github.com/am-kantox/md</a> — at best, it would highlight it. The existence of custom handlers and the ability to connect them to each and every <em>tag</em> allowed me to implement <a href="https://github.com/am-kantox/md/blob/v0.8.1/lib/md/transforms.ex#L7">pulling out a <em>TwitterCard/OG</em> by link</a> and showing a preview, as <em>Twitter</em> does. When the card isn’t there, I’m still able to show the correct title for the anchor and the descriptionit as an alternative text. And such a plan of expansion is possible almost for free, thanks to custom handlers.</p>
<p>That’s probably it. I received some feedback about this library, and all negative reviews came down to the fact that I was violating the standard. Well, yes, I probably shouldn’t have mentioned <em>markdown</em> at all; if you need a perfect mathematically proven parser which passes all the tests, just go for something else, probably. But if you want to have full control over parsing, callbacks and your own grammars… Hmmm, maybe <a href="https://hexdocs.pm/md"><code class="language-plaintext highlighter-rouge">md</code></a> would suit your expectations.</p>
<p>Happy marking it upside-down!</p>
Md — еще раз о маркдауне2022-08-06T00:00:00+00:00https://rocket-science.ru/hacking/2022/08/06/md-under-the-hood<p><strong><em>NB</em> This post is in Russian, <a href="/2022-08-07-md-under-the-hood.md">English version is here</a></strong></p>
<h2 id="вечер-воспоминаний">Вечер воспоминаний</h2>
<p>Никогда в жизни мне не нравились визуальные средства работы с текстом (ну, кроме дельфей, которые меня кормили несколько лет в девяностые). Для диплома и прочих своих стихов я использовал LaTeX, но у меня всегда немного свербило от осознания того, что я заколачиваю микроскопом шурупы. Формул в моем дипломе, как вы наверное уже догадались, было немного, а в стихах — и того меньше.</p>
<p>Потом я узнал про <em>SGML</em>, а с приходом в наши дома веба — и про <em>HTML</em>; я выучил наизусть пару тегов и стал оформлять тексты для моих трех с половиной верных читателей с его помощью. Уже тогда, в 1996 году, меня неимоверно злило, что для некоторых типовых образцов оформления (три звездочки для пустого заглавия стиха, например) мне приходится буквально копипастить из записной книжки несколько строчек нечитаемой лапши. Так появился мой первый самописный движок для блога. Это был один <em>XML</em> файл со всеми текстами и трансформация <em>XSLT</em>, которая создавала из него пучок <em>HTML</em> страничек. К сожалению, код этого произведения искусства утрачен.</p>
<p><img src="/img/anis-del-mono.jpg" alt="Anis del Mono" /></p>
<p>Такое решение меня устраивало буквально всем, потому что тег <code class="language-plaintext highlighter-rouge"><verse caption="none"></code> или типа того буквально без моего участия единообразно преобразовывался в ту самую лапшу, которая меня раздражала в чистом <em>HTML</em>. Потом появился ЖЖ и я на свой блог на несколько лет подзабил, потом уже не помню, в общем когда мне снова захотелось что-то публиковать в личном пространстве — фактически везде пышным цветом цвел маркдаун. Что, в принципе, на первый взгляд прекрасно, потому что списки из звездочек и наклон из подчеркиваний — это ровно то, что нужно для оформления опрятного текста не под ГОСТ. И все же, проблема с тем самым заголовком стиха снова распрямила плечи. Мне категорически не хватает возможности дать ссылку на учетку в твиттере в виде <code class="language-plaintext highlighter-rouge">@mudasobwa</code> и на запись на реддите в виде <code class="language-plaintext highlighter-rouge">[rd /f/foo/1234]</code>, или типа того. В общем, я решил писать свой парсер.</p>
<h2 id="свой-парсер">Свой парсер</h2>
<p>Свой парсер хорош тем, что можно сразу задать грамматику под себя (я часто использую верхний регистр, например, или как он там называется, и каждый раз печатать <code class="language-plaintext highlighter-rouge"><sup>foo</sup></code> вместо напрашивающегося и абсолютно естественного для маркдауна <code class="language-plaintext highlighter-rouge">^foo^</code> — как минимум странно). Проблема в том, что я ненавижу все, прибитое гвоздями. Я — апологет шурупов. Взял отверточку, отвинтил люстру, привинтил вместо нее крюк для виселицы. Я весь код так пишу, что часто приводит к закрытию задач из бэклога как бы задним числом. В общем, я решил, что грамматика должна быть настраиваемой.</p>
<p>А еще я сразу решил, что не собираюсь ставить перед собой цель полностью пройти существующие тесты. Люди, которым нужны таблицы внутри списка внутри таблицы — не моя клиентура (в результате, как ни странно, почти вся такая экзотика поддержалась сама собой).</p>
<p>Первым делом я попытался систематизировать подвиды стандартной разметки маркдауна. Вот что приходит на ум сразу же:</p>
<ul>
<li>параграфы (разделенные двумя и более переносами строк)</li>
<li>код (я программист, мне важно)</li>
<li>«скобки» (жирный, наклонный, все то, что оформляется как скобки)</li>
<li>экранированные символы</li>
<li>бесхитростная замена (знак «<code class="language-plaintext highlighter-rouge"><</code>» должен стать «<code class="language-plaintext highlighter-rouge">&lt;</code>» и типа того)</li>
<li>заголовки и т. п. (отсюда до конца строки)</li>
<li>разделители (<code class="language-plaintext highlighter-rouge"><hr/></code> сотоварищи)</li>
<li>списки</li>
</ul>
<p>На первое время достаточно. Потом я добавил комментарии, теги, сноски и что-то там еще, полный список на сегодняшний день можно увидеть в <a href="https://hexdocs.pm/md/Md.html#module-markup-handling">документации</a>.</p>
<h2 id="как-парсить">Как парсить?</h2>
<p>Мне всегда нравился протокол <em>XMPP</em>, потому что он позволяет работать с бесконечным потоком байтиков вместо файла. По той же причине я предпочитаю <em>SAX</em>-парсеры. Сожрали доступную часть потока, обработали, позвали сторонние зарегистрированные обработчики, сидим, ждем новых данных.</p>
<p>Надо сказать, я всегда, во всех без исключения проектах, предусматриваю возможность подключения обработчиков. Что-то случилось, о чем потенциально может захотеть узнать окружающий мир? — зарегистрируй колбек, я его вызову. Поэтому к моему парсеру можно подключить <em>listener</em> и оперативно узнавать о перипетиях парсинга.</p>
<p>Итак: свой настраиваемый синтаксис (в рамках соответствия оригинальному маркдауну), возможность подключения обработчиков, один проход. Последнее требование родилось просто потому, что в бесконечном потоке невозможно предусмотреть поддержку <em>lookbehind</em>, рано или поздно я буду вынужден отрезать этой змее хвост.</p>
<p>Ну, круто. Пора заняться собственно парсингом. Сразу оговорюсь, что эта задача не обязательно выполнима в языках без уверенной поддержки сопоставления с образцом. На чистом си ее можно решить имитируя его заглядыванием вперед на длину максимального «тега» (в текущей версии маркдауна это число — 3, для <code class="language-plaintext highlighter-rouge">---</code> и подобных, для настраиваемого синтаксиса его можно посчитать). Идрис позволит решить эту задачу красиво без явного сопоставления с образцом на зависимых типах (хаскель прососет, естественно). Но у меня есть эликсир, в котором сопоставление с образцом занимает самое почетное место. Поэтому я просто отсортирую открывающие теги из описания грамматики по длине и буду сопоставлять ввод с этими образцами, рекурсивно вызывая одну и ту же функцию на оставшемся потоке.</p>
<p>И все бы ничего, но еще есть иерархия тегов (экранирование сильнее скобок), зависимость от контекста (внутри куска кода — подчеркивание — это просто подчеркивание, а не начало наклонного текста) и терминирующие последовательности символов (начало нового параграфа должно закрыть все открытые теги). Таким образом, нам потребуется протаскивать некие знания об уже обработанном тексте дальше.</p>
<h2 id="настраиваемые-грамматики">Настраиваемые грамматики</h2>
<p>В принципе, можно было бы просто прочитать грамматику из конфига и нагородить всяких условных операторов, но это было бы чертовски медленно, да и сопоставление с образцом приглось бы имитировать. А хочется, чтобы парсер работал с выбранной грамматикой так, как будто бы он был написан именно для нее, без лишних проверок и тиков процессора. Поэтому на основе конфигурации выбранной грамматики я <em>генерирую</em> код обработки. Весь. Если кто-то понимает код на эликсире и не боится сурового метапрограммирования, вот <a href="https://github.com/am-kantox/md/blob/v0.8.1/lib/md/parser/engine.ex#L50-L81">процесс генерации</a>. Для простейшего варианта (грамматика слака) сгенерированный код будет быстрее кода с поддержкой полного маркдауна примерно в 8–10 раз за счет гораздо меньшего количества функции сопоставления с образцом.</p>
<h2 id="custom-handler">Custom handler</h2>
<p>Помимо обязательных колбеков отовсюду, где это имеет смысл, я всегда предоставляю возможность написать свой обработчик в дополнение к существующим. Когда я закладывал такую возможность, я даже не представлял, зачем, но — пригодится. И она пригодилась. Что сделает стандартный маркдаун, встретив обособленную ссылку <a href="https://github.com/am-kantox/md">https://github.com/am-kantox/md</a> — в лучшем случае подсветит ее. Существование кастомных обработчиков и возможность их подключить к любому «тегу» позволили мне за полчаса реализовать <a href="https://github.com/am-kantox/md/blob/v0.8.1/lib/md/transforms.ex#L7">вытаскивание <em>TwitterCard/OG</em> по ссылке</a> и показ предпросмотра, как это делает твиттер. Когда карточки нет, я все еще способен показать правильный заголовок и запихнуть <code class="language-plaintext highlighter-rouge">description</code> в <code class="language-plaintext highlighter-rouge">alt</code>. И такого плана расширения возможны практически бесплатно.</p>
<p>Вот, наверное, и все. Я получил некоторое количество отзывов на эту библиотеку, все негативные сводились к тому, что я нарушаю стандарт. Ну да, наверное, мне не нужно было вообще упоминать маркдаун; если вам нужен идеальный сферический парсер в вакууме, проходящий все тесты — возьмите что-нибудь другое, наверное. Но если вы хотите иметь полный контроль над парсингом, колбеки и собственные грамматики — возможно, <code class="language-plaintext highlighter-rouge">md</code> подойдет и вам.</p>
URL Shortener2022-05-27T00:00:00+00:00https://rocket-science.ru/hacking/2022/05/27/url-shortener<p>One of my friends pointed me to <a href="https://www.geeksforgeeks.org/system-design-url-shortening-service/">the solution</a> of kinda popular nowadays interview task. It is “How would you design an URL shortener service like <a href="https://tinyurl.com/">TinyURL</a>?”</p>
<p>To my pity, I am an inquisitive person, that’s why I immediately jumped into reading it. What I read made me promise to propose my own solution, because accepting what is suggested there was hard if not impossible.</p>
<p><img src="/img/tramvia.jpg" alt="Tramvia" /></p>
<h3 id="how-it-started">How It Started</h3>
<p>The aforementioned post starts with the wise bit of advice</p>
<blockquote>
<p>When you’re asked this question in your interviews don’t jump into the technical details immediately.</p>
</blockquote>
<p>This is a great statement, I would even broaden it to “When you have a task to solve, don’t jump into the technical details immediately.” The time you actually type a solution using your keyboard should not increase 10–20% of the whole time spent on the task. Thinking in advance pays back better than fixing issues afterward.</p>
<p>But then the author rushes into the abyss of unjustified assumptions, framing themselves in the illusion of a lovingly constructed ivory tower.</p>
<blockquote>
<p>Let’s assume our service has 30M new URL shortenings per month.</p>
</blockquote>
<p>No, please don’t. Do assume the service <em>might</em> grow to handle millions new shortenings per month and be ready to scale. If you are hired to handle the existing traffic of 30M/m, you’ll be asked very different questions during the interview, like “what the existing service are we to buy,” trust me.</p>
<blockquote>
<p>To convert a long URL into a unique short URL we can use some hashing techniques like <em>Base62</em> or <em>MD5</em>.</p>
</blockquote>
<p>Eh. Before discussing what hashing technique to use, we should clarify the most important requirement: should we generate the new short URLs for the same long URL upon subsequent requests, or shall we return the already generated one. The latter approach is way harder to implement, but usually we don’t care about that. Here I went to <a href="https://bit.ly">BitLy</a> and made sure they don’t. So won’t we.</p>
<p>The ability to not keep track of backward references (<em>Long</em> → <em>Short</em>) makes it easy to forget about hashing at all and just use random sequences as <em>ID</em>, btw.</p>
<blockquote>
<p>Database</p>
</blockquote>
<p>Wait, what? Who said we ever need a database? Then the author compares RDBMS against NoSQL, and comes to the controversial conclusion “handle this amount of huge traffic on our system relational databases are not fit and also it won’t be a good decision to scale the RDBMS.” That was the exact place in the original post where I decided, OK, I’ll write down how would <em>I</em> handle it.</p>
<h3 id="how-is-it-going">How Is It Going</h3>
<p>My approach would be slightly less biased from scratch.</p>
<ul>
<li>the service must be optimized for reads over writes</li>
<li>we don’t choose the backend to store the links right away</li>
<li>we start with an approach to handle <em>some</em> traffic, keeping in mind the scalability</li>
</ul>
<p>That’s it.</p>
<p>The cool story about the task is we don’t need the highly coupled storage, even more: we don’t need it to be coupled at all. The best approach to shuffle the requests between different “shards” (whatever it means in this context) would be <a href="https://en.wikipedia.org/wiki/Consistent_hashing"><code class="language-plaintext highlighter-rouge">Hashring</code></a>, but there is an issue: changing the size of the hashring (scalability!) on the fly would result in rehashing, and this is not something we can afford. That said, the short link <code class="language-plaintext highlighter-rouge">abcdefg</code> should always go to the same <em>shard</em>, no matter what, and the number of shards is surely to be increasing.</p>
<p>Okey-dokey. Then what?</p>
<p>They say, programmers accept only three kinds of numbers: <code class="language-plaintext highlighter-rouge">0</code>, <code class="language-plaintext highlighter-rouge">1</code>, <code class="language-plaintext highlighter-rouge">∞</code>. Everything should be infinitely extendable. If I could create the third shard, I must be able to create the 1_000_000_003<sup>rd</sup> one. But I am not a real programmer by this definition, I’m perfectly fine the initial scaling up to <code class="language-plaintext highlighter-rouge">62</code> shards (and when we run out of it, I’d sell the business, or, shame to admit, will switch to 8-symbols short URLs.) That said, the first letter in the short URL would point to the physical shard, hardcoded. I am to start with the single instance, serving early adopters, named <code class="language-plaintext highlighter-rouge">a</code>. All the links would have <code class="language-plaintext highlighter-rouge">a</code> as a first letter. Like, <code class="language-plaintext highlighter-rouge">abcdefg</code> or <code class="language-plaintext highlighter-rouge">a8GH0ff</code>. Then, after first billion or so users, I’ll go with <code class="language-plaintext highlighter-rouge">b</code>. This is a nasty, inelegant hack, I hear you. But still, bear with me.</p>
<p>So far, so good. Now what do we do <em>within the same instance</em>? Oh, here is where <code class="language-plaintext highlighter-rouge">Hashring</code> shines. I’d create <code class="language-plaintext highlighter-rouge">N</code> separated storages (read: tables in the DB, or even directories on the file system,) where <code class="language-plaintext highlighter-rouge">N</code> is fixed. I am too lazy to calculate the exact number here, but it could be got by dividing the whole storage size by acceptable for the quick access substorage size. Some benchmarking would probably be required here. Let’s for now assume <code class="language-plaintext highlighter-rouge">N</code> is also <code class="language-plaintext highlighter-rouge">62</code>, just because <code class="language-plaintext highlighter-rouge">62</code> is a great number after all.</p>
<p>Upon the request to shorten the URL, I would produce the short <code class="language-plaintext highlighter-rouge">6</code>-symbols gibberish, prepend <code class="language-plaintext highlighter-rouge">a</code> to it (the current “prefix” denoting the actual physical shard/node), and check if it already exists in the storage.</p>
<p>If it exists, I’m to retry with the another one. If it does not, I would run <code class="language-plaintext highlighter-rouge">Hashring.who?(six_symbols)</code>, get to its local shard and store the key-value pair there.</p>
<p>An access to the long URL would be drastically fast:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>abcdefg
^ physical node/shard name
^^^^^^ local shard key to hashring
</code></pre></div></div>
<p>And then select from the storage by the key itself.</p>
<h3 id="implementation">Implementation</h3>
<p>Remember, I said, there might be no need for <em>DB</em> at all? All we need from scratch would be an interface looking like a pair of <code class="language-plaintext highlighter-rouge">get/1</code> and <code class="language-plaintext highlighter-rouge">put/2</code> (and maybe <code class="language-plaintext highlighter-rouge">delete/1</code>, but we can think about it later) functions. The naïve but robust implementation of this interface would be a bunch of directories on the file system, organized in the same way as linux distros keep packages: for the 7-symbols key it would be a path like <code class="language-plaintext highlighter-rouge">/hashring-id/b/c/d/e/f/g</code> where <code class="language-plaintext highlighter-rouge">g</code> is the file having content representing the long original URL. <code class="language-plaintext highlighter-rouge">inode</code>s work slowly when there are too many files/subdirectories in the directory, but here we have <code class="language-plaintext highlighter-rouge">62</code> tops on each level, so we are all set and the access would be instant. If it won’t, we’ll move to <code class="language-plaintext highlighter-rouge">62</code> tables in the RDBMS, or <code class="language-plaintext highlighter-rouge">62</code> shards in <code class="language-plaintext highlighter-rouge">NoSQL</code>, or whatever else.</p>
<p>That’s basically it. We have a clean, backend-storage agnostic, scalable solution.</p>
<h3 id="further-improvements">Further Improvements</h3>
<p>We might easily keep the number of accesses via shortened link, as well as any other attached data. Even more, we might extend this approach to re-use short links, which would increase the write-time, but keep the access time the same (dealing with collisions would be the most interesting part of this solution, I’d leave its design to the astute reader, feel free to ping/DM me with your suggestions.)</p>
<hr />
<p>Happy shortening everything!</p>
Shining Access2022-05-21T00:00:00+00:00https://rocket-science.ru/hacking/2022/05/21/shining-access<p>Probably, the most irritating expierence in immutable languages for those coming from the OOP, would be updating the deeply nested structures. I have answered a ton of questions on <a href="https://stackoverflow.com/questions/tagged/elixir">StackOverflow</a> related to how to deal with GraphQL-like structures.</p>
<p>When my <em>Elixir</em> journey has been just starting, I even wrote the library <a href="https://github.com/am-kantox/elixir-iteraptor"><code class="language-plaintext highlighter-rouge">Iteraptor</code></a>, allowing the deep structure traversal with updates, doing somewhat similar to what <a href="https://hexdocs.pm/elixir/Macro.html#traverse/4"><code class="language-plaintext highlighter-rouge">Macro.traverse/4</code></a> does to <em>AST</em>. I would not recommend to use it though, because <em>Elixir</em> comes equipped with much better solution.</p>
<p><img src="/img/roses-en-tossa.jpg" alt="Roses en Tossa de Mar" /></p>
<p>This solution is extremely underrated and newbies tend to either have no idea about it, or find it too cumbersome, which is a complete nonsense. It’s the most beautiful and clean architectural pattern I have ever seen. Welcome <a href="https://hexdocs.pm/elixir/Access.html"><code class="language-plaintext highlighter-rouge">Access</code></a> behaviour.</p>
<p><code class="language-plaintext highlighter-rouge">Access</code> is already implemented for the internal erlang structures, like maps and lists; what is more impressive, everyone might implement it for their custom structs by defining <a href="https://hexdocs.pm/elixir/Access.html#callbacks">three callbacks</a> only.</p>
<p>When I ran <code class="language-plaintext highlighter-rouge">mix new estructura</code> in my shell, I was mostly after bringing the default <code class="language-plaintext highlighter-rouge">Access</code> implementation to user structs for free. That is how <a href="https://hexdocs.pm/estructura"><code class="language-plaintext highlighter-rouge">Estructura</code></a> library was born. Soon I was even asked whether I have plans to evolve it, and I answered, probably no. I was wrong.</p>
<p>When <a href="https://twitter.com/wojtekmach">@wojtekmach</a> <a href="https://twitter.com/wojtekmach/status/1525226000283467776?s=20&t=V8g0hIFa-WXT7rcKibHmrQ">announced</a> <a href="https://github.com/wojtekmach/req"><code class="language-plaintext highlighter-rouge">Req</code></a> library, I was like, eh, wait, we need lazy maps to store a potentially huge rarely accessed values.</p>
<p>Consider the file system viewer written in <em>Elixir</em>. It’d be kinda natural to keep the directory tree in the deeply nested map. In the <code class="language-plaintext highlighter-rouge">v2</code> we would be likely to implement a file viewer. Which should probably cache content at least for a while. Somewhat like</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">%{</span>
<span class="s2">"/"</span> <span class="o">=></span> <span class="p">%{</span>
<span class="s2">"home"</span> <span class="o">=></span> <span class="p">%{</span>
<span class="s2">"am"</span> <span class="o">=></span> <span class="p">%{</span>
<span class="s2">".zshrc"</span> <span class="o">=></span> <span class="sx">??????</span><span class="p">,</span>
<span class="o">...</span>
<span class="p">}</span>
<span class="p">},</span>
<span class="o">...</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Greedy loading contents of all the files does not sound as a good idea (unless you are utilizing NASA clusters to list files on your local.) What can we do to load it lazily?—Of course. <code class="language-plaintext highlighter-rouge">Access</code>.</p>
<p><code class="language-plaintext highlighter-rouge">listing["/"]["home"]["am"][".zshrc"]</code> should result is loading the file content and returning it. That was easy. But how would we approach caching?</p>
<hr />
<p>I spent some time writing and deleting code then. And, I hope, I find the least annoying way to accomplish this task. I have introduced <code class="language-plaintext highlighter-rouge">Lazy</code> struct in the first place, keeping the lazy getter, possibly loaded and cached value, and some internal stuff.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">@type</span> <span class="n">t</span> <span class="p">::</span> <span class="p">%{</span>
<span class="ss">__struct__:</span> <span class="no">Lazy</span><span class="p">,</span>
<span class="ss">expires_in:</span> <span class="n">non_neg_integer</span><span class="p">()</span> <span class="o">|</span> <span class="ss">:instantly</span> <span class="o">|</span> <span class="ss">:never</span><span class="p">,</span>
<span class="ss">timestamp:</span> <span class="no">nil</span> <span class="o">|</span> <span class="no">DateTime</span><span class="o">.</span><span class="n">t</span><span class="p">(),</span>
<span class="ss">payload:</span> <span class="n">any</span><span class="p">(),</span>
<span class="ss">value:</span> <span class="n">cached</span><span class="p">(),</span>
<span class="ss">getter:</span> <span class="p">(</span><span class="no">Map</span><span class="o">.</span><span class="n">key</span><span class="p">(),</span> <span class="no">Map</span><span class="o">.</span><span class="n">value</span><span class="p">()</span> <span class="o">-></span> <span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">value</span><span class="p">()}</span> <span class="o">|</span> <span class="p">{</span><span class="ss">:error</span><span class="p">,</span> <span class="n">any</span><span class="p">()})</span>
<span class="p">}</span>
</code></pre></div></div>
<p>As one might see, I prematurily optimized the struct to be able to deal with expiration, as all the caches should do. I also have a <code class="language-plaintext highlighter-rouge">payload</code>, which I tend to have everywhere (who knows what users would want to keep alongside this structure?—it’s like a process state, it might be empty all the day long, but it must be provided to the user.)</p>
<p>Upon external call to retrieve the data, the <code class="language-plaintext highlighter-rouge">value</code> will be retrieven, <code class="language-plaintext highlighter-rouge">timestamp</code> will be amended on successful update, etc. The only thing left now would be to make it friend with <code class="language-plaintext highlighter-rouge">Access</code>.</p>
<p>Here I introduced <code class="language-plaintext highlighter-rouge">LazyMap</code>, the whole idea of which is shamelessly stolen from <code class="language-plaintext highlighter-rouge">MapSet</code> implementation. I have a “lazy data,” used to retrieve the real data (it’d be a file name in the file-system-listing example above,) and key-value pairs where values are instances of <code class="language-plaintext highlighter-rouge">Lazy</code>. The contrived example below, taken from tests, shows how it all works.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">parse_int</span><span class="p">(</span><span class="n">bin</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">with</span> <span class="p">{</span><span class="n">int</span><span class="p">,</span> <span class="n">_</span><span class="p">}</span> <span class="o"><-</span> <span class="no">Integer</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="n">bin</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">int</span><span class="p">}</span>
<span class="o">...</span>
<span class="n">lazy</span> <span class="o">=</span>
<span class="no">LazyMap</span><span class="o">.</span><span class="n">new</span><span class="p">([</span>
<span class="ss">foo:</span> <span class="no">Lazy</span><span class="o">.</span><span class="n">new</span><span class="p">(</span><span class="o">&</span><span class="n">parse_int</span><span class="o">/</span><span class="mi">1</span><span class="p">),</span>
<span class="o">...</span>
<span class="p">],</span>
<span class="s2">"42"</span>
<span class="p">)</span>
<span class="o">...</span>
<span class="n">lazy</span><span class="p">[</span><span class="ss">:foo</span><span class="p">]</span> <span class="o">=</span> <span class="mi">42</span>
</code></pre></div></div>
<p>This works, but it does not update the map itself, the next call to <code class="language-plaintext highlighter-rouge">get_in/2</code> would pass through the getter, missing cache. Clear, that’s because everything is immutable, including the map’s values. Let’s play smarter.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="n">value</span><span class="p">,</span> <span class="n">lazy</span><span class="p">}</span> <span class="o">=</span> <span class="n">get_and_update_in</span><span class="p">(</span><span class="n">lazy</span><span class="p">,</span> <span class="p">[</span><span class="ss">:foo</span><span class="p">],</span> <span class="o">&</span><span class="p">{</span><span class="nv">&1</span><span class="p">,</span> <span class="nv">&1</span><span class="p">})</span>
</code></pre></div></div>
<p>Now <code class="language-plaintext highlighter-rouge">lazy</code> contains the updated version of the map, with <code class="language-plaintext highlighter-rouge">value</code> and <code class="language-plaintext highlighter-rouge">timestamp</code> set in the value associated with key <code class="language-plaintext highlighter-rouge">:foo</code>. Subsequent call to <code class="language-plaintext highlighter-rouge">get_in/2</code> would hit the cache (until the value gets expired.)</p>
<p>That approach might be helpful when one deals with a data which is unlikely to be accessed, or with remote data having expiration time, or even with data, that should not be long-lived in the memory.</p>
<hr />
<p>Happy accessing everything!</p>
Parser for Markdown Family2022-05-02T00:00:00+00:00https://rocket-science.ru/hacking/2022/05/02/md-customizable-parser<p>Last several months I’ve been lazily working on markdown parser. My goal was not to compete with <a href="https://github.com/pragdave/earmark"><code class="language-plaintext highlighter-rouge">earmark</code></a> or any other markdown parser already existing in the wild. Even more, from the day one I knew I am not going to make it fully compatible with <a href="https://commonmark.org/"><code class="language-plaintext highlighter-rouge">Commonmark</code></a>. What I actually was after would be to produce a blazingly fast, <em>customizable</em> markup parser.</p>
<p>I created a tool for myself in the first place and I wanted to allow custom syntax in markdown spirit, like <code class="language-plaintext highlighter-rouge">^2^</code> for superscript, <a href="https://twitter.com/mudasobwa">@mudasobwa</a> for twitter handles, <code class="language-plaintext highlighter-rouge">#hashtag</code> for hashtags etc. Markdown itself has a lot of contrived features, which are barely known and literally never used by regular adopters. Did you know you could have a bulleted list inside a blockquote which lives withing an item of another numbered list? Well, now you know; would you ever use it?</p>
<p><img src="/img/el-masnou-beach.jpg" alt="Seaview in El Masnou" /></p>
<p>I have no clue who had introduced the alignment in tables with colons in the head separator (<code class="language-plaintext highlighter-rouge">|:---:|</code>,) but this fellow developer surely never thought about how drastically they broke the whole markdown paradigm by introducing a markup that has an effect on <em>the content before declaration</em>. Yes, these colons change the alignment of the text in head’s columns, introducing the necessity for lookbehinds and ruin the performance (and the original idea in general.)</p>
<p><a href="https://hexdocs.pm/md"><code class="language-plaintext highlighter-rouge">Md</code></a> is SAX-like parser, which goes through the input, pattern-matching the control sequences as they come, and emits parsed stuff as <em>XHTML</em>. It does not do lookbehinds and it’ll never do. Whenever one needs to be full-compliant with <em>Commonmark</em> tests, I’d suggest to pick any other library, generously presented on the market.</p>
<p><code class="language-plaintext highlighter-rouge">Md</code> is five times faster than <code class="language-plaintext highlighter-rouge">earmark</code> because of its approach (and lack of some features which I voluntarily decided to be redundant, like the aforementioned table column alignment.) That does not mean <code class="language-plaintext highlighter-rouge">Md</code> is unusable for the average user, dealing with markdown; I have it tested against the whole <em>Elixir</em> documentation and it gets parsed without a glitch.</p>
<p>Still, the main advantage of using <code class="language-plaintext highlighter-rouge">Md</code> compared to other markdown implementations is its full customization. Let’s see how one would implement <em>Slack</em> parser.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">SlackParser</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">Md</span><span class="o">.</span><span class="no">Parser</span>
<span class="nv">@syntax</span> <span class="p">%{</span>
<span class="ss">flush:</span> <span class="p">[{</span><span class="s2">"---"</span><span class="p">,</span> <span class="p">%{</span><span class="ss">tag:</span> <span class="ss">:hr</span><span class="p">,</span> <span class="ss">rewind:</span> <span class="no">true</span><span class="p">}}],</span>
<span class="ss">paragraph:</span> <span class="p">[{</span><span class="s2">">"</span><span class="p">,</span> <span class="p">%{</span><span class="ss">tag:</span> <span class="ss">:blockquote</span><span class="p">}}],</span>
<span class="ss">list:</span> <span class="p">[</span>
<span class="p">{</span><span class="s2">"- "</span><span class="p">,</span> <span class="p">%{</span><span class="ss">tag:</span> <span class="ss">:li</span><span class="p">,</span> <span class="ss">outer:</span> <span class="ss">:ul</span><span class="p">}},</span>
<span class="p">{</span><span class="s2">"* "</span><span class="p">,</span> <span class="p">%{</span><span class="ss">tag:</span> <span class="ss">:li</span><span class="p">,</span> <span class="ss">outer:</span> <span class="ss">:ul</span><span class="p">}}</span>
<span class="p">],</span>
<span class="ss">block:</span> <span class="p">[</span>
<span class="p">{</span><span class="s2">"```"</span><span class="p">,</span> <span class="p">%{</span><span class="ss">tag:</span> <span class="p">[</span><span class="ss">:pre</span><span class="p">,</span> <span class="ss">:code</span><span class="p">],</span> <span class="ss">mode:</span> <span class="ss">:raw</span><span class="p">,</span> <span class="ss">pop:</span> <span class="p">%{</span><span class="ss">code:</span> <span class="ss">:class</span><span class="p">}}}</span>
<span class="p">],</span>
<span class="ss">brace:</span> <span class="p">[</span>
<span class="p">{</span><span class="s2">"*"</span><span class="p">,</span> <span class="p">%{</span><span class="ss">tag:</span> <span class="ss">:b</span><span class="p">}},</span>
<span class="p">{</span><span class="s2">"_"</span><span class="p">,</span> <span class="p">%{</span><span class="ss">tag:</span> <span class="ss">:i</span><span class="p">}},</span>
<span class="p">{</span><span class="s2">"~"</span><span class="p">,</span> <span class="p">%{</span><span class="ss">tag:</span> <span class="ss">:s</span><span class="p">}},</span>
<span class="p">{</span><span class="s2">"`"</span><span class="p">,</span> <span class="p">%{</span><span class="ss">tag:</span> <span class="ss">:code</span><span class="p">,</span> <span class="ss">mode:</span> <span class="ss">:raw</span><span class="p">,</span> <span class="ss">attributes:</span> <span class="p">%{</span><span class="ss">class:</span> <span class="s2">"code-inline"</span><span class="p">}}}</span>
<span class="p">],</span>
<span class="ss">magnet:</span> <span class="p">[</span>
<span class="p">{</span><span class="s2">"@"</span><span class="p">,</span> <span class="p">%{</span><span class="ss">transform:</span> <span class="o">&</span><span class="no">SlackHandle</span><span class="o">.</span><span class="n">apply</span><span class="o">/</span><span class="mi">2</span><span class="p">}}</span>
<span class="p">]</span>
<span class="p">}</span>
<span class="k">end</span>
</code></pre></div></div>
<p>What if we wanted to support headers as well?</p>
<div class="language-diff highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="gd">- paragraph: [{">", %{tag: :blockquote}}],
</span><span class="gi">+ paragraph: [
+ {"##", %{tag: :h2}},
+ {"###", %{tag: :h3}},
+ {">", %{tag: :blockquote}}
+ ],
</span></code></pre></div></div>
<p>Table?—Nothing could have ever been easier.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="ss">matrix:</span> <span class="p">[</span>
<span class="p">{</span><span class="s2">"|"</span><span class="p">,</span> <span class="p">%{</span>
<span class="ss">tag:</span> <span class="ss">:td</span><span class="p">,</span>
<span class="ss">outer:</span> <span class="ss">:table</span><span class="p">,</span>
<span class="ss">inner:</span> <span class="ss">:tr</span><span class="p">,</span>
<span class="ss">first_inner_tag:</span> <span class="ss">:th</span><span class="p">,</span>
<span class="ss">skip:</span> <span class="s2">"|-"</span><span class="p">}}],</span>
</code></pre></div></div>
<p>Allow some custom tag?—Sure, despite we suggest to use a dedicated syntax, why not?</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="ss">tag:</span> <span class="p">[{</span><span class="s2">"sup"</span><span class="p">,</span> <span class="p">%{}},</span> <span class="p">{</span><span class="s2">"sub"</span><span class="p">,</span> <span class="p">%{}}]</span>
</code></pre></div></div>
<hr />
<p>Besides the standard syntax definition through <code class="language-plaintext highlighter-rouge">@syntax</code> attribute (it’s accumulated, btw, so one might break the definition into several ones,) we support <em>DSL</em>. For instance, defining the allowed tags via DSL would look like</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">MyDSLParser</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">Md</span><span class="o">.</span><span class="no">Parser</span>
<span class="nv">@syntax</span> <span class="p">%{</span><span class="err">…</span><span class="p">}</span>
<span class="kn">import</span> <span class="no">Md</span><span class="o">.</span><span class="no">Parser</span><span class="o">.</span><span class="no">DSL</span>
<span class="n">tag</span> <span class="s2">"sup"</span><span class="p">,</span> <span class="p">%{}</span>
<span class="n">tag</span> <span class="s2">"sub"</span><span class="p">,</span> <span class="p">%{}</span>
<span class="o">...</span>
<span class="k">end</span>
</code></pre></div></div>
<p>With all that freedom, one might easily extend the syntax to support their business needs (custom tags, named entities, youtube embeds, etc.)</p>
<hr />
<p>Happy marking it right!</p>
Finitomata :: First Class Documentation2022-04-30T00:00:00+00:00https://rocket-science.ru/hacking/2022/04/30/finitomata-ex-docs<p>I have <a href="https://rocket-science.ru/hacking/2022/04/02/finitomata">already introduced</a> the <a href="https://github.com/am-kantox/finitomata"><code class="language-plaintext highlighter-rouge">finitomata</code></a> library, yet another implementation of <a href="https://en.wikipedia.org/wiki/Finite-state_machine">Finite Automata</a>. The main reason I’ve decided to climb the low hill that has not been yet conquered by lazy developers only, was I felt like DSL does not fit well the main purpose of <em>FSM</em> declaration. It’s not …ahem… declarative. The <em>FSM</em> is best described with a diagram, not with a plain English text.</p>
<p><img src="/img/mambie.jpg" alt="Mambie" /></p>
<p>This is Mambie, by the way. He is 31 and he happily lives in Valencia zoo. He has nothing to do with finite automata at all, but look, how declarative he is. Anyway.</p>
<p>I wanted <em>FSM</em> implementation to be as easy and succinct as possible. That’s why I have chosen <a href="https://plantuml.com/en/state-diagram"><code class="language-plaintext highlighter-rouge">PlantUML</code></a> syntax to declare it. Later I was given <a href="https://elixirforum.com/t/elixir-blog-post-finitomata-the-proper-fsm-for-elixir/46983/11">an advise</a> to support <a href="https://mermaid.live/"><code class="language-plaintext highlighter-rouge">Mermaid</code></a> as well.</p>
<p>The huge advantage of supporting <code class="language-plaintext highlighter-rouge">Mermaid</code> is that it can be natively embedded into library documentation. Actually, <code class="language-plaintext highlighter-rouge">ex_doc</code> can do way more of a cool stuff rather than the regular user (read: me) could ever imagine. It natively supports <a href="https://hexdocs.pm/ex_doc/readme.html#admonition-blocks">admonition blocks</a>, <a href="https://hexdocs.pm/ex_doc/readme.html#rendering-math">KaTeX math expressions</a>, <a href="https://hexdocs.pm/ex_doc/readme.html#rendering-vega-lite-plots">Vega-Lite plots</a>, and, yes, <a href="https://hexdocs.pm/ex_doc/readme.html#rendering-mermaid-graphs"><em>Mermaid graphs</em></a>.</p>
<p>It’s all possible with introducing <code class="language-plaintext highlighter-rouge">before_closing_body_tag</code> keyword parameter for <code class="language-plaintext highlighter-rouge">docs</code> section in <code class="language-plaintext highlighter-rouge">mix.exs</code> file and implementing the respective handler as shown below (the excerpt is taken from <code class="language-plaintext highlighter-rouge">ex_doc</code> docs.)</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="ss">docs:</span> <span class="p">[</span>
<span class="c1"># ...</span>
<span class="ss">before_closing_body_tag:</span> <span class="o">&</span><span class="n">before_closing_body_tag</span><span class="o">/</span><span class="mi">1</span>
<span class="p">]</span>
<span class="c1"># ...</span>
<span class="k">defp</span> <span class="n">before_closing_body_tag</span><span class="p">(</span><span class="ss">:html</span><span class="p">)</span> <span class="k">do</span>
<span class="sd">"""
<!-- HTML injected at the end of the <body> element -->
"""</span>
<span class="k">end</span>
<span class="k">defp</span> <span class="n">before_closing_body_tag</span><span class="p">(</span><span class="n">_</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="s2">""</span>
</code></pre></div></div>
<hr />
<p>That said, I was able to generate <em>FSM diagrams</em> directly in the documentation of the generated <em>FSM implementation</em>.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">MyFSM</span> <span class="k">do</span>
<span class="nv">@moduledoc</span> <span class="s2">"FSM implementation for 3-state conditional flow"</span>
<span class="nv">@fsm</span> <span class="sd">"""
s1 --> |to_s2| s2
s1 --> |to_s3| s3
"""</span>
<span class="kn">use</span> <span class="no">Finitomata</span><span class="p">,</span> <span class="p">{</span><span class="nv">@fsm</span><span class="p">,</span> <span class="no">Finitomata</span><span class="o">.</span><span class="no">Mermaid</span><span class="p">}</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The above code produces the following documentation</p>
<p><img src="/img/fsm-mermaid.png" alt="FSM drawn with Mermaid" /></p>
<hr />
<p>Happy rich documenting!</p>
Finitomata :: The Proper FSM for Elixir2022-04-02T00:00:00+00:00https://rocket-science.ru/hacking/2022/04/02/finitomata<p>I’m a big fan of <a href="https://en.wikipedia.org/wiki/Finite-state_machine">Finite Automata</a>. Always have been. Whenever we deal with a long-lived objects, they eventually have states and <em>FSM</em> does an enormously great job by attesting consistency, eliminating human errors and leaving the implementation with a business logic only. In a mediocre project, fifty <code class="language-plaintext highlighter-rouge">if</code>-<code class="language-plaintext highlighter-rouge">then</code>-<code class="language-plaintext highlighter-rouge">else</code> conditionals might perform as well as one <em>FSM</em>, but unless you are paid for the number of LoCs, <em>FSM</em> is drastically easier to carry on.</p>
<p><img src="/img/sakura.jpg" alt="Cherrytree" /></p>
<p>Of course, I am not alone, and there are many implementations of <em>FSM</em> in different languages. During my <em>Ruby</em> journey, I loved <a href="https://github.com/geekq/workflow"><code class="language-plaintext highlighter-rouge">workflow</code></a> gem for its simplicity, clarity, and robustness. Nowadays I mostly do <em>Elixir</em> and I surely searched through available libraries to minimize <em>FSM</em> boilerplate.</p>
<p>The package that comes up first when doing an internet search would be <a href="https://github.com/sasa1977/fsm"><code class="language-plaintext highlighter-rouge">fsm</code></a> by Saša Jurić. But the author explicitly says on the very top of <em>README</em></p>
<blockquote>
<p>[…] I don’t advise using [this library]. Pure functional FSMs are still my preferred approach (as opposed to <code class="language-plaintext highlighter-rouge">gen_statem</code>), but you don’t need this library for that. Regular data structures, such as maps or structs, with pattern matching in multiclauses will serve you just fine.</p>
</blockquote>
<p>That said, the proposed approach would be to have a <code class="language-plaintext highlighter-rouge">GenServer</code> with multiple clauses of, say, <code class="language-plaintext highlighter-rouge">transition/2</code> function and messages containing events. Somewhat like</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">@impl</span> <span class="no">GenServer</span>
<span class="k">def</span> <span class="n">handle_call</span><span class="p">(</span><span class="ss">:event1</span><span class="p">,</span> <span class="p">%{}</span> <span class="o">=</span> <span class="n">state</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="err">…</span>
<span class="k">def</span> <span class="n">handle_call</span><span class="p">(</span><span class="ss">:event2</span><span class="p">,</span> <span class="p">%{}</span> <span class="o">=</span> <span class="n">state</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="err">…</span>
<span class="k">def</span> <span class="n">handle_call</span><span class="p">(</span><span class="ss">:event3</span><span class="p">,</span> <span class="p">%{}</span> <span class="o">=</span> <span class="n">state</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="err">…</span>
</code></pre></div></div>
<hr />
<p>I admire Saša’s contribution in <em>Elixir</em> ecosystem, and he is definitely one of the smartest persons I ever met, so I used this approach for years. Until I found myself in a position of copy-pasting a boilerplate for that from one project to another. I surely felt it might be done better, but the actual implementation eluded me.</p>
<p>Last week I had been presenting the <em>FSM</em> concept to non-tech auditory during one of our internal tech talks, and it finally clicked. That’s how <a href="https://github.com/am-kantox/finitomata"><code class="language-plaintext highlighter-rouge">finitomata</code></a> library was born.</p>
<hr />
<p>The most important thing I wanted to <em>automate</em> would be the <em>FSM</em> pure description itself. I wanted something, that is fault-tolerant, not error-prone, and easy to grasp. I wanted the result to be drawable out of the box. That’s why I recalled <a href="https://plantuml.com/en/state-diagram">PlantUML format</a>. Instead of scrolling the editor window back and forth and memorizing all the transitions handled, the definition of the entire <em>FSM</em> would be in the very single place! That sounded as a great idea.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[*] --> s1 : to_s1
s1 --> s2 : to_s2
s1 --> s3 : to_s3
s2 --> [*] : ok
s3 --> [*] : ok
</code></pre></div></div>
<p>So I took <a href="https://hexdocs.pm/nimble_parsec"><code class="language-plaintext highlighter-rouge">NimbleParsec</code></a>, the great parser combinators library by <a href="https://dashbit.co/">Dashbit</a> and started with <em>PlantUML</em> parsing. I ensured the parsed <em>FSM</em> consistency as: one single state, no orphan states, at least one final state. Also I have plans to implement a <code class="language-plaintext highlighter-rouge">mix</code> task to generate a visual representation of <em>FSM</em> as a diagram (the respective <a href="https://github.com/am-kantox/finitomata/issues/1">issue</a> is marked as <em>Help Wanted</em> FWIW :))</p>
<p>I already had the approximate architecture in my mind. I wanted <em>FSM</em> to leverage <code class="language-plaintext highlighter-rouge">GenServer</code> functionality under the hood, and several callbacks for the <em>real</em> consumer actions when the transaction gets performed. These callbacks were intended to allow the consumer to concentrate on business logic only, without a necessity to deal with processes, messages and supervision trees. It should have been absolutely imperative for the consumer yet fully <em>OTP</em> under the hood.</p>
<p><em>Draft implementation:</em> the consumer initiates a transition by calling somewhat like <code class="language-plaintext highlighter-rouge">transition(object, event)</code>, then the <code class="language-plaintext highlighter-rouge">GenServer</code> does its magic and the callback <code class="language-plaintext highlighter-rouge">on_transition</code> gets called. From inside this callback, the consumer implements the business logic and returns the result (the next state to move the <em>FSM</em> to.) Soon I figured out we need also <code class="language-plaintext highlighter-rouge">on_failure</code> and <code class="language-plaintext highlighter-rouge">on_terminate</code> callbacks to allow easy handling of errors and to perform a final cleanup respectively.</p>
<hr />
<p>All the callbacks do have a default implementation, which would perfectly handle transitions having a single <code class="language-plaintext highlighter-rouge">to</code> state and not requiring any additional business logic attached.</p>
<p>Upon start, it moves to the next to initial state and sits there awaiting for the transition request. Then it would call an <code class="language-plaintext highlighter-rouge">on_transition/4</code> callback and move to the next state, or remain in the current one, according to the response.</p>
<p>Upon reaching a final state, it would terminate itself, that’s where <code class="language-plaintext highlighter-rouge">on_terminate/3</code> callback is called from. The process also keeps all the history of states it went through, and might have a payload in its state.</p>
<p>That’s how the whole implementation would look like</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">MyFSM</span> <span class="k">do</span>
<span class="nv">@fsm</span> <span class="sd">"""
[*] --> s1 : to_s1
s1 --> s2 : to_s2
s1 --> s3 : to_s3
s2 --> [*] : ok
s3 --> [*] : ok
"""</span>
<span class="kn">use</span> <span class="no">Finitomata</span><span class="p">,</span> <span class="nv">@fsm</span>
<span class="k">def</span> <span class="n">on_transition</span><span class="p">(</span><span class="ss">:s1</span><span class="p">,</span> <span class="ss">:to_s2</span><span class="p">,</span> <span class="n">event_payload</span><span class="p">,</span> <span class="n">state_payload</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="ss">:s2</span><span class="p">,</span> <span class="n">state_payload</span><span class="p">}</span>
<span class="k">def</span> <span class="n">on_transition</span><span class="p">(</span><span class="err">…</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="err">…</span>
<span class="k">def</span> <span class="n">on_failure</span><span class="p">(</span><span class="err">…</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="err">…</span>
<span class="k">def</span> <span class="n">on_terminate</span><span class="p">(</span><span class="err">…</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="err">…</span>
<span class="k">end</span>
</code></pre></div></div>
<hr />
<p>The library includes a supervision tree with a <a href="https://hexdocs.pm/elixir/DynamicSupervisor.html"><code class="language-plaintext highlighter-rouge">DynamicSupervisor</code></a> carrying all the <em>FSM</em> and the <code class="language-plaintext highlighter-rouge">Registry</code> instance to allow anything to be an <em>FSM</em> name with <a href="https://hexdocs.pm/elixir/GenServer.html#module-name-registration"><code class="language-plaintext highlighter-rouge">{:via, module, term}</code></a>.</p>
<p>The snippet below demonstrates how one would use the <em>FSM</em> implementation above</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Starts an FSM supervised</span>
<span class="c1"># ⇓ IMPL ⇓ NAME ⇓ PAYLOAD</span>
<span class="no">Finitomata</span><span class="o">.</span><span class="n">start_fsm</span> <span class="no">MyFSM</span><span class="p">,</span> <span class="s2">"My first FSM"</span><span class="p">,</span> <span class="p">%{</span><span class="ss">foo:</span> <span class="ss">:bar</span><span class="p">}</span>
<span class="no">Finitomata</span><span class="o">.</span><span class="n">transition</span> <span class="s2">"My first FSM"</span><span class="p">,</span> <span class="p">{</span><span class="ss">:to_s2</span><span class="p">,</span> <span class="no">nil</span><span class="p">}</span>
<span class="no">Finitomata</span><span class="o">.</span><span class="n">state</span> <span class="s2">"My first FSM"</span>
<span class="c1">#⇒ %Finitomata.State{current: :s2, history: [:s1], payload: %{foo: :bar}}</span>
<span class="no">Finitomata</span><span class="o">.</span><span class="n">allowed?</span> <span class="s2">"My first FSM"</span><span class="p">,</span> <span class="p">:</span><span class="o">*</span> <span class="c1"># state</span>
<span class="c1">#⇒ true</span>
<span class="no">Finitomata</span><span class="o">.</span><span class="n">responds?</span> <span class="s2">"My first FSM"</span><span class="p">,</span> <span class="ss">:to_s2</span> <span class="c1"># event</span>
<span class="c1">#⇒ false</span>
<span class="no">Finitomata</span><span class="o">.</span><span class="n">transition</span> <span class="s2">"My first FSM"</span><span class="p">,</span> <span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="no">nil</span><span class="p">}</span> <span class="c1"># to final state</span>
<span class="c1">#⇒ [info] [◉ ⇄] [state: %Finitomata.State{current: :s2, history: [:s1], payload: %{foo: :bar}}]</span>
<span class="no">Finitomata</span><span class="o">.</span><span class="n">alive?</span> <span class="s2">"My first FSM"</span>
<span class="c1">#⇒ false</span>
</code></pre></div></div>
<p>That’s it for now, but the library is still in the kindergarten, it’s <code class="language-plaintext highlighter-rouge">v0.1.0</code> so one might expect more sugar to come soon.</p>
<hr />
<p>Happy finite automating!</p>
Plugins in Elixir Applications2022-02-12T00:00:00+00:00https://rocket-science.ru/hacking/2022/02/12/plugins-in-elixir-apps<p>Each and every application has code duplication. Well, not yours, of course. All but yours.</p>
<p>Eventually, the similar functionality tends to suffer from discrepancies in the details. Parameters are slightly different, return values are discrepant, one might raise, another one should not. Commercial projects often have an optional features to be switched on only once the customer pays. Libraries want to support external modules to extend their basic abilities. All that comes to the question “how do we deal with the induced complexity?”</p>
<p><img src="/img/plugins.jpg" alt="Lizard" /></p>
<p><code class="language-plaintext highlighter-rouge">C</code> uses header files to claim a demand. Erudite developers with a rich vocabulary yell “polymorphism.” OOP coaches whisper “interfaces.” Pattern adepts get probably already bored reading this because the “Command” pattern is the one. Dynamic languages vote for duck-typing. Erlang provides behaviours for the nearly same purpose. Elixir inherits behaviours and introduces a notion of protocols. RESTful notation has its own word. Even Facebook is now Meta (although I doubt it has anything with the real meaning of the beautiful Greek word.)</p>
<p>OK, jokes aside. We are talking about generic plugins here. Once plugged in, such a beast provides some rich additional functionality. If there is no suitable adapter in the house, nothing crashes, despite the significant part of the functionality is not available anymore. Sounds quite familiar. You cannot listen to the music on your not-so-smart-phone if you are using your grandma’s jack’ed earpods, but answering calls and playing snake is still an option.</p>
<p>Writing plugins in a well-designed plugin architecture is drastically easy. Designing the wellformed plugin architecture is …ahem… a bit harder. The good news is, it’s still not a rocket-science.</p>
<p>We need a duplex communication here. The main application should be able to identify a plugin and ask it to perform its duties when needed, the plugin itself must understand the input and provide a meaningful outcome.</p>
<p>I’d distinguish three different categories of plugins.</p>
<ul>
<li>side effect</li>
<li>pure functions</li>
<li>mix of both</li>
</ul>
<h3 id="side-effect">Side-effect</h3>
<p>Plugins, providing side-effects only (like extendable <code class="language-plaintext highlighter-rouge">Save As…</code> functionality in the editor,) would be the easiest to implement. It should be able to introduce itself to the application and export a function to call by the application when needed. Somewhat like this would be great (in pseudo-ruby for the sake of brevity)</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">SaveAsPlugin</span>
<span class="k">class</span> <span class="o"><<</span> <span class="nb">self</span>
<span class="k">def</span> <span class="nf">name</span>
<span class="s2">"CSV"</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">call</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
<span class="k">raise</span> <span class="k">unless</span> <span class="n">data</span><span class="p">.</span><span class="nf">is_a?</span><span class="p">(</span><span class="no">Hash</span><span class="p">)</span>
<span class="no">CSV</span><span class="p">.</span><span class="nf">save</span><span class="p">(</span><span class="nb">hash</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Upon start, the main application would scan the universe for available plugins, call <code class="language-plaintext highlighter-rouge">#name</code> function on found ones, list them somewhere in the menu and call <code class="language-plaintext highlighter-rouge">#save</code> when the menu item will be pressed by the user. Or like. That simple.</p>
<h3 id="pure">Pure</h3>
<p>Pure plugins are not quite harder. The code would nearly be the same, save for <code class="language-plaintext highlighter-rouge">#call</code> function is now to return the value to be used in the application.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Arithmetic::Minus</span>
<span class="k">class</span> <span class="o"><<</span> <span class="nb">self</span>
<span class="k">def</span> <span class="nf">name</span>
<span class="s2">"-"</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">call</span><span class="p">(</span><span class="n">lho</span><span class="p">,</span> <span class="n">rho</span><span class="p">)</span>
<span class="n">lho</span> <span class="o">-</span> <span class="n">rho</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>That way we might extend the abilities of the <code class="language-plaintext highlighter-rouge">Calculator</code> application with void default implementation. FWIW, <a href="https://hexdocs.pm/plug"><code class="language-plaintext highlighter-rouge">Plug</code></a> in <em>Elixir</em> are of that kind of plugin, which allows chaining them into <a href="https://hexdocs.pm/phoenix/Phoenix.Router.html#pipeline/2">pipelines</a> for the further pleasure of developers</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">pipeline</span> <span class="ss">:api</span> <span class="k">do</span>
<span class="n">plug</span> <span class="ss">:token_authentication</span>
<span class="n">plug</span> <span class="ss">:dispatch</span> <span class="c1"># here we are authenticated</span>
<span class="k">end</span>
</code></pre></div></div>
<h3 id="complex-plugins">Complex plugins</h3>
<p>The complexity of this kind of plugins comes from the fact, that they might acually intervene the caller’s workflow in the unknown complicated manner. And the main challenge dealing with this kind of plugins would be to decompose unpredicted unpurity into a chain of predicted pure or side-effect-only calls. This decomposition usually results in a set of different <em>callbacks</em>.</p>
<p>Consider the example with the premium features on the regular <em>Phoenix</em> website. What would such a premium feature provide? Well, a list of scopes, where to show a link to it, basically routes. Some contexts, templates and controllers. It must response to requests like <code class="language-plaintext highlighter-rouge">allowed?/1</code> where the parameter in a call would be the current auth. Something else, drastically specific to the application.</p>
<p>The number of callbacks should be tempting to a minimum, while it must still cover all the possible scenarios.</p>
<p>That sounds too generic, but in fact it’d be not so complicated to implement. There is absolutely no need to strive to produce a swiss-knife-like plugin behaviour. Tree structure works perfectly everywhere, and here it does as well. Just make a top-level behaviour to provide a name, scope <em>and</em> features, where features are other behaviours implemented. Then the main application might perform a sequence of requests for the current scope and decide whether all, or some, or none plugins are suitable to allow in each particular case.</p>
<p>This writing already became too long, so I’d postpone real-life examples, but the main idea of how to approach this should be <em>hopefully</em> clear now.</p>
<hr />
<p>Happy plugging in!</p>
Help To Test Your Library2022-01-30T00:00:00+00:00https://rocket-science.ru/hacking/2022/01/30/help-to-test-your-library<p>I like to keep applications as tiny as possible. Everything that is not related to business logic, might and should be extracted into packages. Packages are great, reusable and better testable; once created and tested (and hopefully benchmarked,) it does not require the application to bother about its internals. It just works™. <a href="https://hex.pm/pricing">hex.pm</a> provides private package functionality for those hesitating to open source their general-purpose libraries.</p>
<p>Despite the proven robustness of external libraries, we would probably still want to test their integration though. Any library might be great on its own, but how would it fit our application quirks?</p>
<p><img src="/img/palma.jpg" alt="Girl in Palma" /></p>
<p>Well, greatest libraries are welcoming for integration testing. <em>Ecto</em> <a href="https://hexdocs.pm/ecto/testing-with-ecto.html#content">provides a sandbox for testing</a>. <em>Broadway</em> comes with a <a href="https://hexdocs.pm/broadway/Broadway.html#module-testing">DummyProducer</a>.</p>
<p>Providing a sandbox might be cumbersome and tricky in general, while <code class="language-plaintext highlighter-rouge">DummyWorker</code> is a must in any library shipped with love and care. Apparently, it’s both extremely useful for users of your library and easy to implement. Follow <em>Broadway</em>’s path, send a message to the calling process instead of performing real job. That simple.</p>
<hr />
<p>Imagine we have a library that posts a slack message into some channel within your company workspace. Our application has some actions which should end up with a message in this slack channel. The integration test should then test that a call to <code class="language-plaintext highlighter-rouge">perform_this_action</code> has some side effects actually performing a real job <em>and</em> notifies the slack channel afterward. We use homebrewed <code class="language-plaintext highlighter-rouge">SlackPoster</code> library, which is thoroughly tested and is proven to work well with posting messages. But how would we go about integration test? Check the slack channel ensuring the message had indeed arrived? Nah.</p>
<p>This imaginary <code class="language-plaintext highlighter-rouge">SlackPoster</code> library is probably already having a behaviour defining the callback similar to that</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">@callback</span> <span class="n">post</span><span class="p">(</span><span class="n">message</span> <span class="p">::</span> <span class="n">map</span><span class="p">())</span> <span class="p">::</span> <span class="ss">:ok</span> <span class="o">|</span> <span class="p">{</span><span class="ss">:error</span><span class="p">,</span> <span class="n">reason</span><span class="p">}</span> <span class="ow">when</span> <span class="ss">reason:</span> <span class="n">any</span><span class="p">()</span>
</code></pre></div></div>
<p>If the library author was kind enough, they should have provided a real implementation</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">SlackPoster</span> <span class="k">do</span>
<span class="nv">@behaviout</span> <span class="no">Poster</span>
<span class="nv">@impl</span> <span class="no">Poster</span>
<span class="k">def</span> <span class="n">post</span><span class="p">(%{}</span> <span class="o">=</span> <span class="n">message</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">message</span> <span class="o">|></span> <span class="no">Jason</span><span class="o">.</span><span class="n">encode!</span><span class="p">()</span> <span class="o">|></span> <span class="no">Engine</span><span class="o">.</span><span class="n">send_message</span><span class="p">()</span>
<span class="err">…</span>
</code></pre></div></div>
<p>alongside with a dummy implementation for better testing</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">DummyPoster</span> <span class="k">do</span>
<span class="nv">@behaviout</span> <span class="no">Poster</span>
<span class="nv">@impl</span> <span class="no">Poster</span>
<span class="k">def</span> <span class="n">post</span><span class="p">(%{}</span> <span class="o">=</span> <span class="n">message</span><span class="p">)</span> <span class="k">do</span>
<span class="p">{</span><span class="n">callback_pid</span><span class="p">,</span> <span class="n">message</span><span class="p">}</span> <span class="o">=</span>
<span class="no">Map</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="n">message</span><span class="p">,</span> <span class="ss">:callback</span><span class="p">,</span> <span class="nv">@default_value</span><span class="p">)</span>
<span class="n">send</span><span class="p">(</span><span class="n">callback_pid</span><span class="p">,</span> <span class="p">{</span><span class="ss">:slack_sent</span><span class="p">,</span> <span class="n">message</span><span class="p">})</span>
<span class="k">end</span>
</code></pre></div></div>
<p>And now one can test the integration easily with <a href="https://hexdocs.pm/ex_unit/ExUnit.Assertions.html#assert_receive/3"><code class="language-plaintext highlighter-rouge">ExUnit.Assertions.assert_receive/3</code></a></p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">test</span> <span class="s2">"slack integration"</span> <span class="k">do</span>
<span class="c1"># wait how would I supply a pid of process here?!</span>
<span class="n">assert_receive</span> <span class="p">{</span><span class="ss">:slack_sent</span><span class="p">,</span> <span class="o">^</span><span class="n">expected_message</span><span class="p">}</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Yes, there is still a glitch. When it’ll come to <code class="language-plaintext highlighter-rouge">DummyPoster.post/1</code>, it would require a <code class="language-plaintext highlighter-rouge">pid</code> to provide callback, and there is no way to supply it transparently without bringing test logic into application core, which is meh. Is it a deadend?—Not at all.</p>
<p>Your library should provide a <em>message proxy</em> process. It is to be started within a test suite (e. g. from <a href="https://hexdocs.pm/ex_unit/ExUnit.Callbacks.html#setup/1"><code class="language-plaintext highlighter-rouge">setup/1</code></a> callback,) to route messages to the test process. Then we might do the following.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">DummyPoster</span> <span class="k">do</span>
<span class="nv">@behaviout</span> <span class="no">Poster</span>
<span class="nv">@impl</span> <span class="no">Poster</span>
<span class="k">def</span> <span class="n">post</span><span class="p">(%{}</span> <span class="o">=</span> <span class="n">message</span><span class="p">)</span> <span class="k">do</span>
<span class="p">{</span><span class="n">callback_pid</span><span class="p">,</span> <span class="n">message</span><span class="p">}</span> <span class="o">=</span>
<span class="no">Map</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="n">message</span><span class="p">,</span> <span class="ss">:callback</span><span class="p">,</span> <span class="no">Application</span><span class="o">.</span><span class="n">get_env</span><span class="p">(</span><span class="ss">:my_app</span><span class="p">,</span> <span class="ss">:dummy_proxy</span><span class="p">))</span>
<span class="no">Process</span><span class="o">.</span><span class="n">send</span><span class="p">(</span><span class="n">callback_pid</span><span class="p">,</span> <span class="p">{</span><span class="ss">:slack_sent</span><span class="p">,</span> <span class="n">message</span><span class="p">},</span> <span class="p">[])</span>
<span class="k">end</span>
</code></pre></div></div>
<p>and in our test we would need to register ourselves right before <code class="language-plaintext highlighter-rouge">assert_receive/1</code></p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">test</span> <span class="s2">"slack integration"</span> <span class="k">do</span>
<span class="n">expected_message</span> <span class="o">=</span> <span class="o">...</span>
<span class="n">proxy</span> <span class="o">=</span> <span class="no">Application</span><span class="o">.</span><span class="n">get_env</span><span class="p">(</span><span class="ss">:my_app</span><span class="p">,</span> <span class="ss">:dummy_proxy</span><span class="p">)</span>
<span class="n">proxy</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="n">expected_message</span><span class="p">,</span> <span class="n">self</span><span class="p">())</span>
<span class="n">assert_receive</span> <span class="p">{</span><span class="ss">:slack_sent</span><span class="p">,</span> <span class="o">^</span><span class="n">expected_message</span><span class="p">}</span>
<span class="n">proxy</span><span class="o">.</span><span class="n">unregister</span><span class="p">(</span><span class="n">expected_message</span><span class="p">,</span> <span class="n">self</span><span class="p">())</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The draft implementation of proxy process is trivial.</p>
<hr />
<p>Another way would be to use <code class="language-plaintext highlighter-rouge">PubSub</code> mechanism for routing messages, but this is slightly more cumbersome. <a href="https://hexdocs.pm/envio"><code class="language-plaintext highlighter-rouge">Envío</code></a> library aims to cover a boilerplate for pubsub internals, allowing simple drop-in implementations for publishers and subscribers. One day I’d write about it in details too.</p>
<hr />
<p>Happy making your customers happy testing!</p>
Yet Another Markup Parser2021-10-14T00:00:00+00:00https://rocket-science.ru/hacking/2021/10/14/md-with-pleasure<p><a href="http://daringfireball.net/projects/markdown/syntax"><code class="language-plaintext highlighter-rouge">Markdown</code></a> is great. The original philosophy is lined out by its creator John Gruber as</p>
<blockquote>
<p>Markdown is intended to be as easy-to-read and easy-to-write as is feasible.
Readability, however, is emphasized above all else. A Markdown-formatted document should be publishable as-is, as plain text, without looking like it’s been marked up with tags or formatting instructions.</p>
</blockquote>
<p>Which is great until it is not. All markdown parsers I am aware of do implement their own dialect, neither customizable nor fulfilling everyone’s needs. Yes, markdown allows a subset of <code class="language-plaintext highlighter-rouge">HTML</code> to be embedded, but this just ruins the main advantage of it being <em>easy-to-read</em>.</p>
<p><img src="/img/seagull-at-the-beach.jpg" alt="Seagull at the beach" /></p>
<p>Some people often use <code class="language-plaintext highlighter-rouge"><abbr></code> tag, others want to be able to use <code class="language-plaintext highlighter-rouge">--foo--</code> as a marker to strike <code class="language-plaintext highlighter-rouge">foo</code> out, etc. That is the main rationale behind <a href="https://hexdocs.pm/md"><code class="language-plaintext highlighter-rouge">md</code> the library</a>.</p>
<p><code class="language-plaintext highlighter-rouge">Md</code> is definitely not a competitor of <a href="https://github.com/pragdave/earmark"><code class="language-plaintext highlighter-rouge">earmark</code></a> or any other library implementing the proper parser for the bare markdown. On the contrary, <code class="language-plaintext highlighter-rouge">md</code> is faster and fully customizable backbone to build your own markup dialect. Yes, <code class="language-plaintext highlighter-rouge">md</code> comes with its own implementation of a subset of Gruber markdown (plus some Github flavor,) but this is not the main goal.</p>
<p>The main goal would be to provide a support of still easy-to-read custom markup language, somewhat similar to markdown but not necessarily the same. The <a href="https://github.com/am-kantox/md/issues/4">very first issue</a> created in the repository after <code class="language-plaintext highlighter-rouge">md</code> turned <em>PoC</em> is to implement a zero-code (configuration only) support for <a href="https://orgmode.org">OrgMode</a>.</p>
<h2 id="single-pass-and-streaming">Single-pass and Streaming</h2>
<p><code class="language-plaintext highlighter-rouge">Md</code> is a <strong>single-pass streaming</strong> parser. That said, it never looks behind, and, therefore, it is normally faster than competitors. <code class="language-plaintext highlighter-rouge">Md</code> is a plain old good reduce on the incoming stream of characters.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>## Md.Bench
benchmark iterations average time
md 500 6124.09 µs/op
earmark 100 28803.86 µs/op
</code></pre></div></div>
<p>The benchmark above is not absolutely honest, because <code class="language-plaintext highlighter-rouge">earmark</code> supports way more tags, but while <code class="language-plaintext highlighter-rouge">md</code> intentionally does not support stuff like tables (at least, at the moment of writing,) adding them won’t drastically change the benchmark. It will remain single-pass forever.</p>
<h2 id="custom-parsers">Custom parsers</h2>
<p><code class="language-plaintext highlighter-rouge">Md</code> allows declaring custom parsers for any opening sequence. For that to work, one should implement <a href="https://hexdocs.pm/md/Md.Parser.html"><code class="language-plaintext highlighter-rouge">Md.Parser</code></a> behaviour. For instance, to turn Twitter handles into links, one might do:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">My</span><span class="o">.</span><span class="no">Parsers</span><span class="o">.</span><span class="no">TwitterHandle</span> <span class="k">do</span>
<span class="nv">@behaviour</span> <span class="no">Md</span><span class="o">.</span><span class="no">Parser</span>
<span class="nv">@href</span> <span class="s2">"https://twitter.com/"</span>
<span class="nv">@impl</span> <span class="no">Md</span><span class="o">.</span><span class="no">Parser</span>
<span class="n">alias</span> <span class="no">Md</span><span class="o">.</span><span class="no">Parser</span><span class="o">.</span><span class="no">State</span>
<span class="k">def</span> <span class="n">parse</span><span class="p">(</span><span class="s2">"@"</span> <span class="o"><></span> <span class="n">rest</span><span class="p">,</span> <span class="n">state</span><span class="p">)</span> <span class="k">do</span>
<span class="n">state</span> <span class="o">=</span> <span class="p">%</span><span class="no">State</span><span class="p">{</span><span class="n">state</span> <span class="o">|</span> <span class="ss">bag:</span> <span class="no">Map</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="n">state</span><span class="o">.</span><span class="n">bag</span><span class="p">,</span> <span class="ss">:handle</span><span class="p">,</span> <span class="s2">"@"</span><span class="p">)}</span>
<span class="n">parse</span><span class="p">(</span><span class="n">rest</span><span class="p">,</span> <span class="n">state</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="n">parse</span><span class="p">(</span><span class="o"><<</span><span class="n">x</span><span class="p">::</span><span class="n">utf8</span><span class="p">,</span> <span class="n">rest</span><span class="p">::</span><span class="n">binary</span><span class="o">>></span><span class="p">,</span> <span class="n">state</span><span class="p">)</span> <span class="ow">when</span> <span class="n">x</span> <span class="ow">not</span> <span class="ow">in</span> <span class="p">[</span><span class="err">?</span><span class="p">\</span><span class="n">s</span><span class="p">,</span> <span class="err">?</span><span class="p">\</span><span class="n">n</span><span class="p">]</span> <span class="k">do</span>
<span class="n">bag</span> <span class="o">=</span> <span class="no">Map</span><span class="o">.</span><span class="n">update!</span><span class="p">(</span><span class="n">state</span><span class="o">.</span><span class="n">bag</span><span class="p">,</span> <span class="ss">:handle</span><span class="p">,</span> <span class="o">&</span> <span class="nv">&1</span> <span class="o"><></span> <span class="o"><<</span><span class="n">x</span><span class="p">::</span><span class="n">utf8</span><span class="o">>></span><span class="p">)</span>
<span class="n">state</span> <span class="o">=</span> <span class="p">%</span><span class="no">State</span><span class="p">{</span><span class="n">state</span> <span class="o">|</span> <span class="ss">bag:</span> <span class="n">bag</span><span class="p">}</span>
<span class="n">parse</span><span class="p">(</span><span class="n">rest</span><span class="p">,</span> <span class="n">state</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="n">parse</span><span class="p">(</span><span class="o"><<</span><span class="n">_</span><span class="p">::</span><span class="n">size</span><span class="p">(</span><span class="mi">8</span><span class="p">),</span> <span class="n">_</span><span class="p">::</span><span class="n">binary</span><span class="o">>></span> <span class="o">=</span> <span class="n">rest</span><span class="p">,</span> <span class="n">state</span><span class="p">)</span> <span class="k">do</span>
<span class="p">{</span><span class="n">handle</span><span class="p">,</span> <span class="n">bag</span><span class="p">}</span> <span class="o">=</span> <span class="no">Map</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="n">state</span><span class="o">.</span><span class="n">bag</span><span class="p">,</span> <span class="ss">:handle</span><span class="p">)</span>
<span class="n">state</span> <span class="o">=</span> <span class="p">%</span><span class="no">State</span><span class="p">{</span>
<span class="n">state</span>
<span class="o">|</span> <span class="ss">bag:</span> <span class="n">bag</span><span class="p">,</span>
<span class="ss">path:</span> <span class="p">[{</span><span class="ss">:a</span><span class="p">,</span> <span class="p">%{</span><span class="ss">href:</span> <span class="nv">@href</span> <span class="o"><></span> <span class="n">handle</span><span class="p">},</span> <span class="p">[</span><span class="n">handle</span><span class="p">]}</span> <span class="o">|</span> <span class="n">state</span><span class="o">.</span><span class="n">path</span><span class="p">]</span>
<span class="p">}</span>
<span class="p">{</span><span class="n">rest</span><span class="p">,</span> <span class="n">state</span><span class="p">}</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>and in <code class="language-plaintext highlighter-rouge">config.exs</code></p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="c1"># config/prod.exs</span>
<span class="n">config</span> <span class="ss">:md</span><span class="p">,</span> <span class="ss">syntax:</span> <span class="p">%{</span>
<span class="ss">custom:</span> <span class="p">%{</span>
<span class="p">{</span><span class="s2">"@"</span><span class="p">,</span> <span class="p">{</span><span class="no">My</span><span class="o">.</span><span class="no">Parsers</span><span class="o">.</span><span class="no">TwitterHandle</span><span class="p">,</span> <span class="p">%{}}}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Luckily enough, this case is already handled as <code class="language-plaintext highlighter-rouge">:magnet</code> kind in <code class="language-plaintext highlighter-rouge">Md</code>, and
one might use a configuration instead</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="c1"># config/prod.exs</span>
<span class="n">config</span> <span class="ss">:md</span><span class="p">,</span> <span class="ss">syntax:</span> <span class="p">%{</span>
<span class="ss">magnet:</span> <span class="p">[</span>
<span class="p">{</span><span class="s2">"@"</span><span class="p">,</span> <span class="p">%{</span><span class="ss">transform:</span> <span class="o">&</span><span class="no">Md</span><span class="o">.</span><span class="no">Transforms</span><span class="o">.</span><span class="no">TwitterHandle</span><span class="o">/</span><span class="mi">2</span><span class="p">}}</span>
<span class="p">]</span>
<span class="p">}</span>
</code></pre></div></div>
<p>And yes, this is already incorporated into <code class="language-plaintext highlighter-rouge">Md</code>, so no changes is needed.</p>
<h2 id="custom-syntax">Custom syntax</h2>
<p><code class="language-plaintext highlighter-rouge">Md</code> syntax can be fully overwritten, or updated to introduce more markup. For instance, to make <code class="language-plaintext highlighter-rouge">Md</code> to understand the <code class="language-plaintext highlighter-rouge">abbr</code> tag, one simply needs to amend the syntax with</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="n">config</span> <span class="ss">:md</span><span class="p">,</span> <span class="ss">syntax:</span> <span class="p">%{</span>
<span class="ss">pair:</span> <span class="p">{</span><span class="s2">"?["</span><span class="p">,</span>
<span class="p">%{</span>
<span class="ss">tag:</span> <span class="ss">:abbr</span><span class="p">,</span>
<span class="ss">closing:</span> <span class="s2">"]"</span><span class="p">,</span>
<span class="ss">inner_opening:</span> <span class="s2">"("</span><span class="p">,</span>
<span class="ss">inner_closing:</span> <span class="s2">")"</span><span class="p">,</span>
<span class="ss">outer:</span> <span class="p">{</span><span class="ss">:attribute</span><span class="p">,</span> <span class="ss">:title</span><span class="p">}</span>
<span class="p">}}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>the above will effectively convert <code class="language-plaintext highlighter-rouge">?[foo](bar)</code> into <code class="language-plaintext highlighter-rouge">{:abbr, %{title: "that"}, ["this"]}</code> AST and into <code class="language-plaintext highlighter-rouge"><abbr title="that">this</abbr></code>. This is also incorporated into default <code class="language-plaintext highlighter-rouge">Md</code> syntax.</p>
<h2 id="advantages">Advantages</h2>
<p>That all being said, <code class="language-plaintext highlighter-rouge">Md</code> does not aim for covering all the Gruber markdown, but rather it provides a configurable and fully customizable platform for declaring custom markups and deal with them. Being also blazingly fast, it could be a greater choice for custom blog engines or even markups for online shops and similar.</p>
<p>For instance, when Github introduced their “footnote” feature, adding it to <code class="language-plaintext highlighter-rouge">Md</code> would be a matter of one syntax update.</p>
<hr />
<p>Happy marking everything up and down!</p>
Why Am I Not To IDE2021-08-10T00:00:00+00:00https://rocket-science.ru/hacking/2021/08/10/why-am-i-not-to-ide<p>When I started my professional career as a software developer, I was mostly on small warehousing applications written with <em>Delphi32</em>. It had excellent <em>RAD</em> environment, allowing people with next to zero knowledge of the computer science produce semi-robust applications with several mouse clicks. One created a form, placed some controls on it and assigned handlers, writing a little amount of code. The naming was not an issue at all, because <em>RAD</em> produced perfect unique names for everything, including forms, controls and handlers, like <code class="language-plaintext highlighter-rouge">TForm1.Button1.onClick</code>. Projects were small, sold as-is and this approach worked well.</p>
<p><img src="/img/montgat-hill-night.jpg" alt="Night in Montgat" /></p>
<p>Then I got my first office job. The project was an explosive mixture of server-side <code class="language-plaintext highlighter-rouge">COBOL</code>, client GUI in <code class="language-plaintext highlighter-rouge">VisualBasic3</code> and a middleware written in pure <code class="language-plaintext highlighter-rouge">C</code>. We used <em>Visual SourceSafe</em> as version control and I was happy to mostly avoid both <code class="language-plaintext highlighter-rouge">COBOL</code> and <code class="language-plaintext highlighter-rouge">VB</code>, focusing on <code class="language-plaintext highlighter-rouge">C</code>. <em>Visual Studio</em>, providing helpful hints, showing help from <em>MSDN</em> and shining its newly introduced <em>IntelliSense</em> feature was of a great help. It was even more friendly than a famous <em>Dr. Paperclip</em> from <em>MSOffice</em>.</p>
<p>Three years later I switched to Java RAD development myself. We were building the tool allowing people to construct their own modelling languages based on <a href="https://www.omg.org/spec/MOF/"><em>MOF</em></a> and visually build applications, bidirectionally converting diagrams to code back and forth. I was building the GUI part on top of <em>NetBeans</em> (it has been called <em>Forte</em> back then.) Code generation part was on <em>Haskell</em>, fwiw.</p>
<p>Then I got into custom search engine development with <em>C++</em>, web shenanigans with <em>PHP</em>, and later <em>Ruby</em>. I was diving into learning new languages by installing <em>IDE</em> that literally led my study. <em>IDE</em> was my substitute for mentors, guides, documentation, and everything else. I was navigating through an alien code, understanding it and learning from it. I created my own projects using wild guessing based totally on code completion instead of documentation, and I somewhat succeeded in becoming a relatively good software engineer.</p>
<p>In hindsight, I am ready to proclaim: <em>IDE</em> is the enemy of any good developer. It helps invaluably while learning stage, but once you feel comfortable with a language, <em>IDE</em> hinders your effort. Here is why.</p>
<h3 id="code-completion-and-hints">Code Completion and Hints</h3>
<p>Using <em>IDE</em> code completion feature is like using subway when navigating through the city. You’ll definitely get to the destination faster compared to walking, but you’ll know nothing about the path. It’s like teleport, the best you might discover about the relative coordinates between two points on the map, would be one of them is northeast of the other. Walking seems harder and slower, but you’ll inevitably know the whole path. When tomorrow you’ll need to go somewhere in between, you’d already know where to be heading and how much time it takes approximately. You learn by example and in some time you’ll <em>know the city</em>. You’ll be able to guide people through this city. You’ll be able to finger to where this or that building is located. By using metro you’ll be able to nail where all the metro stations are, at best.</p>
<p>Once the whole picture becomes more or less complete, you are able to reason about what is the best approach here and there. It might take only five minutes to walk to the destination, but the professional metro-user would take a round-trip with three transfers because it’s how sparse station graph with rare edges works. The learning curve would be way steeper, but it’s definitely worth it on the long run.</p>
<h3 id="code-generation">Code Generation</h3>
<p>Well, <em>IDE</em> is great in code generation. Two mouse clicks might produce 100 LoCs in a nanosecond. But is it even an advantage unless you get paid for LoCs? I had been always finding myself in a need to genuinely cleanup all the generated mess, which requires nearly the same time as typing the proper code from scratch. I type fast, and this is mostly because I type more than click mouse buttons. Otherwise I’d probably be fast in button clicking, but this skill is not something I am after.</p>
<p>And I spend more time thinking than typing in the first place. Well-thought approach would not require too much code to type. So yeah, since I refused to use <em>IDE</em>, there were several occasions when I wished I had a code generation feature, but I always was able to redesign the architecture to make the code shorter and hence more readable. Would I ever bother about this aspect of architecture if I had a button to generate all that crap? I doubt.</p>
<h3 id="refactoring">Refactoring</h3>
<p>This is my favorite one. Here is my hot take on the topic.</p>
<p><strong>If your <em>IDE</em> is able to perform code refactoring for you, it is not a refactoring by any mean.</strong></p>
<p>Rearranging functions between different modules is not a refactoring. It’s reshaping, reforming, re-something you name it. Refactoring is all about changing interfaces, boundaries, and even paradigms. This is something the smartest <em>IDE</em> around cannot do yet. Moving codepieces between files might sound reasonable and feel satisfactory, but it’s not needed in 99% of cases. The yesterday’s you found that was better, today’s you decided this would be better. What about tomorrow’s you and, more important, all the people who will read this code? Are you as sure your new code split between files would be easier to understand? Well, maybe. But even in such a case, <code class="language-plaintext highlighter-rouge">grep</code>, <code class="language-plaintext highlighter-rouge">sed</code>, and <code class="language-plaintext highlighter-rouge">awk</code> would do.</p>
<p>When one does a proper refactoring, it’s important to have more time to review it than a modern <em>CPU</em> requires to rename the variable everywhere. During manual refactoring you’ll inevitably see whether you are doing it right or not. Maybe some better ideas will come to your mind. You’ll feel it like you feel the city when you walk through it and you feel the sound of wheels when you reach the destination point by metro.</p>
<p>Don’t regret to waste time on manual crafting your code. Artists still produce more exciting pieces than machines.</p>
<hr />
<p>Happy idenying.</p>
Do Not Doughnut Donates2021-05-16T00:00:00+00:00https://rocket-science.ru/hacking/2021/05/16/doughnut-donates<p>The title is surely clickbait, but I am still along with the tagline. I do not think that donations are any good rewarding institute for OSS committers and here is why.</p>
<p><img src="/img/walden7.jpg" alt="Walden 7" /></p>
<p>I probably need to clarify one very important aspect before I start bullying the donation rewards. Am I rich?—Hell, no. There are many people poorer than me all around the world, but I still need to figure out the vacation money and I cannot quit my office job to downshift to lazy scuba diving on Gran Canaria.</p>
<p>Despite the above, I do contribute to OSS, I frequently answer on Stack Overflow, I mentor people who asked about and I do all that for free. None of my projects on GitHub has “Donate” button on, and none will ever have (until I unexpectedly drive bonkers.) I do not exhort nor urge anybody to agree with me, I just want to express my own view and share it for the sake of diversity. I had never been evangelising anything, this rant does neither. If you are fine with donations, go on, I am fine with you being fine, that’s it. Below are <em>my</em> own take only.</p>
<h3 id="donations-are-stilted">Donations Are Stilted</h3>
<p>Donations is kinda artificial way to receive income. Humanity went through subsistence economy, barter, commodity exchange to money, as well as we finally realized that any work should be rewarded, and we call this reward <em>salary</em> (because in the medieval era the reward was often paid in the very valuable artefact—the salt, btw.) People usually don’t sign contracts saying “I’ll do my best and you’ll pay me accordingly if you wish.” Even more, the nearly same statement is the fundamental slogan of communism ideology (“Each according to his ability, to each according to his means.”) And yes, I love this slogan with all my heart.</p>
<p>Unfortunately, it’s not how it works in society (until we have no more greedy scumbags all around, which I doubt happening any time soon.) Work-to-paycheck exchange is still a barter, despite all the modern designations and culture adjustments. At least, it works that way. Noone agrees to plow the field and then see if they will be paid for it. Unless we are talking about CS, where “Donate” buttons tend to appear here and there nowadays.</p>
<p>What’s the difference?—Nah. Developers have already escaped the first level of Maslow’s pyramid, we can afford being willingly paid (or not, for that case we still have a permanent salary in our own office.) For us it’s far from being means of subsistence, it’s more of a lottery, or heritage from the second cousin we’ve never met. We like to get some more money, but we don’t really lack them.</p>
<h3 id="donations-do-not-work-for-many">Donations Do Not Work For Many</h3>
<p>But wait, I can hear the objection, what about Jimmy Wales? Well, yes, this is a sole valid example of the case when donations rule. Wikipedia must be absolutely independent of any power, and power means money today, and they still need to pay their bills. Indeed. But how does this example apply to mediocre us? It does not.</p>
<p>I once had been asked by some company whether I could implement some additional feature in my OSS library for them. I replied “sure, I have it on the roadmap as low priority, hire me if you want it faster.” We signed a contract and I delivered it next month, as agreed. That is what I understand: I took an obligation and I kept my part of the bargain.</p>
<p>What would have happened if they donated me some money? Honestly, I have no idea, but I doubt I felt any liability in this case. Which is not good neither for them, nor—and especially—for me. The money would probably have gone by now, and the feature was still dangling in issues with “medium priority.”</p>
<h3 id="donations-are-unpredictable">Donations Are Unpredictable</h3>
<p>I like to aim for plans. I prefer to know how would I pay this month’s bills rather than hope that tomorrow I’ll be able to buy a yacht. My salary makes me happy with this. Donations do not. Even if they happen to exceed my salary by the order of magnitude.</p>
<p>After all, if anyone wants to play roulette with their income, they’d better start some business or kind rather than pretend to make their living from donations.</p>
<p>And if we already defend ourselves from an inability to pay monthly bills, why this convulsive attempt to grab another piece of the pie? We are already fine, aren’t we? If not, we’d better improve our skills and get more experience to find a new better job anyway.</p>
<h3 id="but-wait-why-not">But Wait, Why Not</h3>
<p>Because this tiny trickle of income completely screws our relationships with our OSS code up. It brings new vain hopes, new false commitments, it breaks the normal flow of our projects evolvement, because we inevitably start to put more effort in what has been donated most, etc. Do I personally want all that? I don’t.</p>
<p>I am fine with my modest apartment and the feeling that I’m making the world a little better. For free. Because I want to, not because I am paid for.</p>
<hr />
<p>Happy donothing, then.</p>
Finite Automata with Tarearbol2021-04-30T00:00:00+00:00https://rocket-science.ru/hacking/2021/04/30/fsm-with-tarearbol<p><a href="https://hexdocs.pm/tarearbol"><code class="language-plaintext highlighter-rouge">Tarearbol</code></a> provides a handy helper to work with many similar processes under a supervision of <a href="https://hexdocs.pm/tarearbol/dynamic_workers_management.html"><code class="language-plaintext highlighter-rouge">Tarearbol.DynamicManager</code></a>. I have already <a href="https://rocket-science.ru/hacking/2021/02/28/async-response-handling-with-tarearbol">written about it in general</a>; now I want to show how to use workers having a predetermined lifecycle, that can be implemented as an FSM.</p>
<p><img src="/img/pigeon.jpg" alt="Pigeon’s over the tent" /></p>
<p>Consider the following example. Our system has a 3rd-party service provider to validate some objects. It might me a compliance check, of some external storage, or whatever. We are to send objects there and wait for the object’s state change in the remote system to, say, <code class="language-plaintext highlighter-rouge">processed</code>. While the best feasible way to implement such a functionality would be to provide a webhook and wait for the callback after submission, some services might require polling.</p>
<p>The polling lifesycle is a straightforward finite automata, shown below.</p>
<p><img src="/img/polling.png" alt="Polling" /></p>
<p>With <code class="language-plaintext highlighter-rouge">Tarearbol.DynamicManager</code>, we start a process, that handles the current FSM state within its internal state, and using different return values from perform, we control the FSM itself. The code below is taken from the real project.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">MyApp</span><span class="o">.</span><span class="no">Negotiation</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">Tarearbol</span><span class="o">.</span><span class="no">DynamicManager</span><span class="p">,</span>
<span class="ss">distributed:</span> <span class="no">true</span><span class="p">,</span>
<span class="ss">init:</span> <span class="o">&</span><span class="no">MyApp</span><span class="o">.</span><span class="no">Negotiation</span><span class="o">.</span><span class="n">continue</span><span class="o">/</span><span class="mi">1</span><span class="p">,</span>
<span class="ss">defaults:</span> <span class="p">[</span><span class="ss">timeout:</span> <span class="mi">10_000</span><span class="p">]</span>
<span class="k">defmodule</span> <span class="no">State</span> <span class="k">do</span>
<span class="nv">@type</span> <span class="n">state</span> <span class="p">::</span> <span class="ss">:virgin</span> <span class="o">|</span> <span class="ss">:submitted</span> <span class="o">|</span> <span class="ss">:succeeded</span> <span class="o">|</span> <span class="ss">:failed</span>
<span class="k">defstruct</span> <span class="ss">state:</span> <span class="ss">:virgin</span><span class="p">,</span> <span class="ss">payload:</span> <span class="p">%{},</span> <span class="ss">response:</span> <span class="no">nil</span>
<span class="k">end</span>
<span class="o">...</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The only thing left would be to handle all the types of possible responses, gracefully transitioning the FSM inside <code class="language-plaintext highlighter-rouge">State</code> through the states. The first request might be done directly in <code class="language-plaintext highlighter-rouge">continue</code> callback of the <code class="language-plaintext highlighter-rouge">DynamicWorker</code>.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nv">@doc</span> <span class="s2">"The `DynamicManager`’s continue callback"</span>
<span class="k">def</span> <span class="n">continue</span><span class="p">(%</span><span class="no">State</span><span class="p">{</span><span class="ss">state:</span> <span class="ss">:virgin</span><span class="p">,</span> <span class="ss">payload:</span> <span class="n">payload</span><span class="p">}</span> <span class="o">=</span> <span class="n">state</span><span class="p">)</span> <span class="k">do</span>
<span class="n">payload</span>
<span class="o">|></span> <span class="n">post_to_3rd_party</span><span class="p">()</span>
<span class="o">|></span> <span class="k">case</span> <span class="k">do</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">response</span><span class="p">}</span> <span class="o">-></span> <span class="p">%</span><span class="no">State</span><span class="p">{</span><span class="n">state</span> <span class="o">|</span> <span class="ss">state:</span> <span class="ss">:submitted</span><span class="p">,</span> <span class="ss">response:</span> <span class="n">response</span><span class="p">}</span>
<span class="p">{</span><span class="ss">:error</span><span class="p">,</span> <span class="n">error</span><span class="p">}</span> <span class="o">-></span> <span class="p">%</span><span class="no">State</span><span class="p">{</span><span class="n">state</span> <span class="o">|</span> <span class="ss">state:</span> <span class="ss">:failed</span><span class="p">,</span> <span class="ss">response:</span> <span class="n">error</span><span class="p">}</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Now we need to properly handle all the possible responses, depending on what state we are currently in.</p>
<h4 id="failure">Failure</h4>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nv">@impl</span> <span class="no">Tarearbol</span><span class="o">.</span><span class="no">DynamicManager</span>
<span class="k">def</span> <span class="n">perform</span><span class="p">(</span><span class="n">id</span><span class="p">,</span> <span class="p">%</span><span class="no">State</span><span class="p">{</span><span class="ss">state:</span> <span class="ss">:failed</span><span class="p">,</span> <span class="ss">response:</span> <span class="n">error</span><span class="p">}</span> <span class="o">=</span> <span class="n">state</span><span class="p">)</span> <span class="k">do</span>
<span class="no">Logger</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s2">"Errored: </span><span class="si">#{</span><span class="n">id</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
<span class="c1"># Do whatever with the error returned, *do not shutdown the process*</span>
<span class="p">{</span><span class="err"></span><span class="p">{</span><span class="ss">:timeout</span><span class="p">,</span> <span class="mi">0</span><span class="p">},</span> <span class="n">payload</span><span class="p">}</span> <span class="c1"># switch off subsequential calls to `perform/2`</span>
<span class="k">end</span>
</code></pre></div></div>
<h4 id="success">Success</h4>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="k">def</span> <span class="n">perform</span><span class="p">(</span><span class="n">id</span><span class="p">,</span> <span class="p">%</span><span class="no">State</span><span class="p">{</span><span class="ss">state:</span> <span class="ss">:succeeded</span><span class="p">,</span> <span class="ss">response:</span> <span class="n">response</span><span class="p">})</span> <span class="k">do</span>
<span class="no">Logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s2">"Processed: </span><span class="si">#{</span><span class="n">id</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
<span class="c1"># Do whatever with the response returned, *shutdown kill the process*</span>
<span class="ss">:halt</span>
<span class="k">end</span>
</code></pre></div></div>
<h4 id="pending">Pending</h4>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="k">def</span> <span class="n">perform</span><span class="p">(</span><span class="n">id</span><span class="p">,</span> <span class="p">%</span><span class="no">State</span><span class="p">{</span><span class="ss">state:</span> <span class="ss">:submitted</span><span class="p">}</span> <span class="o">=</span> <span class="n">state</span><span class="p">)</span> <span class="k">do</span>
<span class="no">Logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s2">"Re-requesting the state of the </span><span class="si">#{</span><span class="n">id</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
<span class="n">payload</span>
<span class="o">|></span> <span class="n">get_to_3rd_party</span><span class="p">()</span>
<span class="o">|></span> <span class="k">case</span> <span class="k">do</span>
<span class="ss">:pending</span> <span class="o">-></span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">state</span><span class="p">}</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">response</span><span class="p">}</span> <span class="o">-></span>
<span class="p">{</span><span class="ss">:replace</span><span class="p">,</span> <span class="p">%</span><span class="no">State</span><span class="p">{</span><span class="n">state</span> <span class="o">|</span> <span class="ss">state:</span> <span class="ss">:succeeded</span><span class="p">,</span> <span class="ss">response:</span> <span class="n">response</span><span class="p">}}</span>
<span class="p">{</span><span class="ss">:error</span><span class="p">,</span> <span class="n">error</span><span class="p">}</span> <span class="o">-></span>
<span class="p">{</span><span class="ss">:replace</span><span class="p">,</span> <span class="p">%</span><span class="no">State</span><span class="p">{</span><span class="n">state</span> <span class="o">|</span> <span class="ss">state:</span> <span class="ss">:failed</span><span class="p">,</span> <span class="ss">response:</span> <span class="n">error</span><span class="p">}}</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<h4 id="further-improvements">Further Improvements</h4>
<p>As we can see here, the code is well-separated and easily extendable. Once needed, we can add new states and juggle the FSM sending it from one state to another using return values. It’s also easy to get to all currently running processes, their state, last returned values etc.</p>
<hr />
<p>Happy finite automating!</p>
Handling Async Responses with Tarearbol2021-02-28T00:00:00+00:00https://rocket-science.ru/hacking/2021/02/28/async-response-handling-with-tarearbol<p><a href="https://hexdocs.pm/tarearbol"><code class="language-plaintext highlighter-rouge">Tarearbol</code></a> provides a handy helper to work with many similar processes under a supervision of <a href="https://hexdocs.pm/tarearbol/dynamic_workers_management.html"><code class="language-plaintext highlighter-rouge">Tarearbol.DynamicManager</code></a>.</p>
<p>Initially, it was created to ease producing and managing many similar processes, described by <code class="language-plaintext highlighter-rouge">id</code> <em>and</em> <code class="language-plaintext highlighter-rouge">payload</code>. These processes are running supervised, managed by a classic store-like interface having basically <code class="language-plaintext highlighter-rouge">put/2</code>, <code class="language-plaintext highlighter-rouge">get/1</code>, and <code class="language-plaintext highlighter-rouge">del/1</code> functions exported. The process runs in background, calling <code class="language-plaintext highlighter-rouge">perform/2</code> implementation and controlling its own lifecycle. Processes might also be easily scaled horizontally, running in distributed environment, using hashring to decide who is to handle this particular request.</p>
<p><img src="/img/castle.jpg" alt="Castell Jalpi" /></p>
<p>The typical application would be, for instance, order management. Imagine an online shop where each visitor has their own cart. This cart is managed by the separate process, keeping the cart content in its state. When the user adds an item to the cart, the process state gets updated, and once per, say, minute, it checks the items availability, current prices, etc., to keep the user up-to-date with the latest data regarding their order.</p>
<p>This approach would not work in cases when the instant real-time updates are required, but it has an advantage in the <em>almost real time</em> processing because of its graceful throttling of requests. Even if a million users will all of sudden come to buy stuff, it’ll spinoff the checkers, but the site running on one instance will be still slowly functioning until engineers add other 10 instances to scale better.</p>
<p>For many processes holding nearly the same state, it saves a lot of boilerplate, claiming to implement <code class="language-plaintext highlighter-rouge">perform/2</code> callback only (and maybe two optional callbacks <code class="language-plaintext highlighter-rouge">call/3</code> and <code class="language-plaintext highlighter-rouge">terminate/2</code>.)</p>
<h3 id="asynchronous-request-processing">Asynchronous Request Processing</h3>
<p>This approach provides also a great scaffold to handle asynchronous requests to third party services. Consider the application communicating to the external service. If everything goes ok, it expects to place a request there and receive a callback in a matter of seconds. That said we ① want to process a callback immediately and also ② report requests that did not receive a response in a while.</p>
<p>For the sake of simplicity, we can say without a loss of generality, that we use HTTP as a communication protocol. We also have a plug, or even the whole Phoenix to handle responses from the remote service. The machinery processing responses will then be as simple as</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">Handler</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">Tarearbol</span><span class="o">.</span><span class="no">DynamicManager</span>
<span class="nv">@doc</span> <span class="sd">"""
Empty initial state, no responses yet.
In reality, this might be loaded from some persistent store.
"""</span>
<span class="nv">@impl</span> <span class="no">Tarearbol</span><span class="o">.</span><span class="no">DynamicManager</span>
<span class="k">def</span> <span class="n">children_specs</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="p">%{}</span>
<span class="nv">@doc</span> <span class="sd">"""
Periodic check and alert for stale requests.
"""</span>
<span class="nv">@impl</span> <span class="no">Tarearbol</span><span class="o">.</span><span class="no">DynamicManager</span>
<span class="k">def</span> <span class="n">perform</span><span class="p">(</span><span class="n">id</span><span class="p">,</span> <span class="n">_payload</span><span class="p">)</span> <span class="k">do</span>
<span class="no">Logger</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="s2">"Expected callback has not been received for </span><span class="si">#{</span><span class="n">id</span><span class="si">}</span><span class="s2">..."</span><span class="p">)</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="no">DateTime</span><span class="o">.</span><span class="n">utc_now</span><span class="p">()}</span>
<span class="k">end</span>
<span class="nv">@impl</span> <span class="no">Tarearbol</span><span class="o">.</span><span class="no">DynamicManager</span>
<span class="k">def</span> <span class="n">call</span><span class="p">(</span><span class="ss">:process</span><span class="p">,</span> <span class="n">_from</span><span class="p">,</span> <span class="p">{</span><span class="n">id</span><span class="p">,</span> <span class="n">payload</span><span class="p">}),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">do_process_response</span><span class="p">({</span><span class="n">id</span><span class="p">,</span> <span class="n">payload</span><span class="p">})</span>
<span class="k">defp</span> <span class="n">do_process_response</span><span class="p">({</span><span class="n">id</span><span class="p">,</span> <span class="n">payload</span><span class="p">}),</span> <span class="k">do</span><span class="p">:</span> <span class="o">...</span>
<span class="k">end</span>
</code></pre></div></div>
<p>That’s literally all code needed to create a robust, supervised, fault tolerant system, monitoring asynchronous requests without even involving any persistent store. When we send the request to the external service, we start a new process by calling <code class="language-plaintext highlighter-rouge">Handler.put(id, payload)</code>. When we receive a callback, we call <code class="language-plaintext highlighter-rouge">Handler.synch_call(id, :process)</code>, followed by <code class="language-plaintext highlighter-rouge">Handler.delete(id)</code>. That’s it.</p>
<hr />
<p>Happy response-handling!</p>
Dynamic Nested Function Call2021-02-18T00:00:00+00:00https://rocket-science.ru/hacking/2021/02/18/dynamic-nested-function-calls-with-macro<p>During my lazy work on <a href="https://hexdocs.pm/tyyppi"><code class="language-plaintext highlighter-rouge">Tyyppi</code></a>, I encountered the necessity to dynamically build the deeply nested function. I use <a href="https://hexdocs.pm/stream_data"><code class="language-plaintext highlighter-rouge">StreamData</code></a> for property-based generators, and to produce a generator for the <a href="https://hexdocs.pm/tyyppi/Tyyppi.Struct.html"><code class="language-plaintext highlighter-rouge">Tyyppi.Struct</code></a>, I recursively call generators on all the fields and then I am to <a href="https://hexdocs.pm/stream_data/StreamData.html#bind/2"><code class="language-plaintext highlighter-rouge">StreamData.bind/2</code></a> all of them to each other.</p>
<p>The number of fields is unknown in advance, as well as field types. So I am after dynamic generation of somewhat like</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">alias</span> <span class="no">StreamData</span><span class="p">,</span> <span class="ss">as:</span> <span class="no">SD</span>
<span class="no">SD</span><span class="o">.</span><span class="n">bind</span><span class="p">(</span><span class="no">SD</span><span class="o">.</span><span class="n">list_of</span><span class="p">(</span><span class="no">SD</span><span class="o">.</span><span class="n">integer</span><span class="p">(),</span> <span class="ss">min_length:</span> <span class="mi">1</span><span class="p">),</span> <span class="k">fn</span> <span class="n">list</span> <span class="o">-></span>
<span class="no">SD</span><span class="o">.</span><span class="n">bind</span><span class="p">(</span><span class="no">SD</span><span class="o">.</span><span class="n">member_of</span><span class="p">(</span><span class="n">list</span><span class="p">),</span> <span class="k">fn</span> <span class="n">elem</span> <span class="o">-></span>
<span class="no">SD</span><span class="o">.</span><span class="n">constant</span><span class="p">({</span><span class="n">list</span><span class="p">,</span> <span class="n">elem</span><span class="p">})</span>
<span class="k">end</span><span class="p">)</span>
<span class="k">end</span><span class="p">)</span>
</code></pre></div></div>
<p>the above is nested the number of times equal to the number of fields, each generator is taken from the field, and I could have been all set unless the inner <code class="language-plaintext highlighter-rouge">SD.constant/1</code> call.</p>
<p><img src="/img/nested-function-call.jpg" alt="Sunset in Montgat" /></p>
<p>To simplify things, let’s consider we are to dynamically build this</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">@spec</span> <span class="n">foo</span><span class="p">(</span><span class="n">atom</span><span class="p">(),</span> <span class="p">(</span><span class="n">any</span><span class="p">()</span> <span class="o">-></span> <span class="n">any</span><span class="p">()))</span> <span class="p">::</span> <span class="n">any</span><span class="p">()</span>
<span class="n">foo</span><span class="p">(</span><span class="ss">:bar</span><span class="p">,</span> <span class="k">fn</span> <span class="n">arg1</span> <span class="o">-></span>
<span class="n">foo</span><span class="p">(</span><span class="ss">:baz</span><span class="p">,</span> <span class="k">fn</span> <span class="n">arg2</span> <span class="o">-></span>
<span class="o">...</span>
<span class="n">foo</span><span class="p">(</span><span class="ss">:bzz</span><span class="p">,</span> <span class="k">fn</span> <span class="n">argN</span> <span class="o">-></span>
<span class="p">{</span><span class="n">arg1</span><span class="p">,</span> <span class="n">arg2</span><span class="p">,</span> <span class="o">...</span><span class="p">,</span> <span class="n">argN</span><span class="p">}</span>
<span class="k">end</span><span class="p">)</span>
<span class="o">...</span>
<span class="k">end</span><span class="p">)</span>
<span class="k">end</span><span class="p">)</span>
</code></pre></div></div>
<p>As we can see, the inner return <em>captures</em> values from all the outer closures. This looks pretty straightforward when written down, but unfortunately it cannot be produced with any kind of <code class="language-plaintext highlighter-rouge">reduce/3</code>, because the tuple size returned from the inner clause (which is equal to a level of nesting by the way) is unknown upfront, and all the closure captures are done downwards.</p>
<p>This is where metaprogramming is <em>mandatory</em> to accomplish a task.</p>
<p>Let’s start with examining the AST produced by the example above.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">quote</span> <span class="k">do</span>
<span class="n">foo</span><span class="p">(</span><span class="ss">:bar</span><span class="p">,</span> <span class="k">fn</span> <span class="n">arg1</span> <span class="o">-></span>
<span class="n">foo</span><span class="p">(</span><span class="ss">:bar</span><span class="p">,</span> <span class="k">fn</span> <span class="n">arg2</span> <span class="o">-></span>
<span class="p">{</span><span class="n">arg1</span><span class="p">,</span> <span class="n">arg2</span><span class="p">}</span>
<span class="k">end</span><span class="p">)</span>
<span class="k">end</span><span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>
<p>results in</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="ss">:foo</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[</span>
<span class="ss">:bar</span><span class="p">,</span>
<span class="p">{</span><span class="ss">:fn</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[</span>
<span class="p">{</span><span class="ss">:-</span><span class="o">></span><span class="p">,</span> <span class="p">[],</span> <span class="p">[</span>
<span class="p">[{</span><span class="ss">:arg1</span><span class="p">,</span> <span class="p">[],</span> <span class="no">Elixir</span><span class="p">}],</span>
<span class="p">{</span><span class="ss">:foo</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[</span>
<span class="ss">:bar</span><span class="p">,</span>
<span class="p">{</span><span class="ss">:fn</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[</span>
<span class="p">{</span><span class="ss">:-</span><span class="o">></span><span class="p">,</span> <span class="p">[],</span> <span class="p">[</span>
<span class="p">[{</span><span class="ss">:arg2</span><span class="p">,</span> <span class="p">[],</span> <span class="no">Elixir</span><span class="p">}],</span>
<span class="p">{</span><span class="err"></span><span class="p">{</span><span class="ss">:arg1</span><span class="p">,</span> <span class="p">[],</span> <span class="no">Elixir</span><span class="p">},</span> <span class="p">{</span><span class="ss">:arg2</span><span class="p">,</span> <span class="p">[],</span> <span class="no">Elixir</span><span class="p">}}</span>
<span class="p">]}]}]}]}]}]}</span>
</code></pre></div></div>
<p>Beware! we are going to deal with the raw <em>AST</em> to save keystrokes. The inner clause is simple, let’s produce a function that returns it.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defp</span> <span class="n">leaf</span><span class="p">(</span><span class="n">args</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="p">{:{},</span> <span class="p">[],</span> <span class="n">args</span><span class="p">}</span>
</code></pre></div></div>
<p>Tuples of size less than 3 might be also quoted with <code class="language-plaintext highlighter-rouge">:{}</code>, so we opt-in for the generic solution here.</p>
<p>Let’s now gaze at the closure.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="ss">:foo</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[</span>
<span class="ss">:bar</span><span class="p">,</span>
<span class="p">{</span><span class="ss">:fn</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[</span>
<span class="p">{</span><span class="ss">:-</span><span class="o">></span><span class="p">,</span> <span class="p">[],</span> <span class="p">[[</span><span class="n">__ARG__</span><span class="p">,</span> <span class="n">__ACC__</span><span class="p">]}]}]}</span>
</code></pre></div></div>
<p>macro variable <code class="language-plaintext highlighter-rouge">__ARG__</code> there denotes the argument of <em>current</em> anonymous function and <code class="language-plaintext highlighter-rouge">__ACC__</code> stays for the accumulator that came from the previous recursive iteration. So, we might now create our <code class="language-plaintext highlighter-rouge">branch</code> function</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defp</span> <span class="n">branch</span><span class="p">({</span><span class="n">field</span><span class="p">,</span> <span class="n">arg</span><span class="p">},</span> <span class="n">acc</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="ss">:foo</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[</span><span class="n">field</span><span class="p">,</span> <span class="p">{</span><span class="ss">:fn</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[{</span><span class="ss">:-</span><span class="o">></span><span class="p">,</span> <span class="p">[],</span> <span class="p">[[</span><span class="n">arg</span><span class="p">],</span> <span class="n">acc</span><span class="p">]}]}]}</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">field</code> above is passed as a hint of how to customize each step when needed. OK, we are almost done. Let’s build up the whole clause for <code class="language-plaintext highlighter-rouge">N</code> arguments.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defp</span> <span class="n">nested</span><span class="p">(</span><span class="n">fields</span><span class="p">)</span> <span class="k">do</span>
<span class="n">args</span> <span class="o">=</span>
<span class="n">fields</span>
<span class="o">|></span> <span class="n">length</span><span class="p">()</span>
<span class="o">|></span> <span class="no">Macro</span><span class="o">.</span><span class="n">generate_arguments</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">)</span>
<span class="o">|></span> <span class="no">Enum</span><span class="o">.</span><span class="n">reverse</span><span class="p">()</span>
<span class="n">fields_args</span> <span class="o">=</span> <span class="no">Enum</span><span class="o">.</span><span class="n">zip</span><span class="p">(</span><span class="n">fields</span><span class="p">,</span> <span class="n">args</span><span class="p">)</span>
<span class="no">Enum</span><span class="o">.</span><span class="n">reduce</span><span class="p">(</span><span class="n">fields_args</span><span class="p">,</span> <span class="n">leaf</span><span class="p">(</span><span class="n">args</span><span class="p">),</span> <span class="o">&</span><span class="n">branch</span><span class="o">/</span><span class="mi">2</span><span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The only tricky part there is the generation of arguments; we simply build a list of the same length as fields. Now we can finally create a macro that would use all the above to return the AST of the desired nested call.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmacrop</span> <span class="n">generate</span><span class="p">(</span><span class="n">fields</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">nested</span><span class="p">(</span><span class="no">Macro</span><span class="o">.</span><span class="n">expand</span><span class="p">(</span><span class="n">fields</span><span class="p">,</span> <span class="n">__CALLER__</span><span class="p">))</span>
</code></pre></div></div>
<p>Let’s see what does it produce!</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">foo</span><span class="p">(</span><span class="n">atom</span><span class="p">,</span> <span class="n">rest</span><span class="p">)</span> <span class="ow">when</span> <span class="n">is_function</span><span class="p">(</span><span class="n">rest</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">foo</span><span class="p">(</span><span class="n">atom</span><span class="p">,</span> <span class="n">rest</span><span class="o">.</span><span class="p">(</span><span class="n">atom</span><span class="p">))</span>
<span class="k">def</span> <span class="n">foo</span><span class="p">(</span><span class="n">_</span><span class="p">,</span> <span class="n">rest</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">rest</span>
<span class="k">def</span> <span class="n">test</span> <span class="k">do</span>
<span class="n">generate</span><span class="p">([</span><span class="ss">:bar</span><span class="p">,</span> <span class="ss">:baz</span><span class="p">])</span>
<span class="k">end</span>
</code></pre></div></div>
<p>and now <code class="language-plaintext highlighter-rouge">test()</code> results in <code class="language-plaintext highlighter-rouge">{:bar, :baz}</code>.</p>
<hr />
<p>Happy recursing!</p>
Conditional context for macros2021-02-13T00:00:00+00:00https://rocket-science.ru/hacking/2021/02/13/conditional-context<p>Elixir 1.11 brought to us (amongst other very exciting features) <a href="https://hexdocs.pm/elixir/Kernel.html#is_struct/2"><code class="language-plaintext highlighter-rouge">is_struct/2</code></a> guard. It, by the way, might have been used as an example of more straightforward not backward compatible alternative in my yesterday’s <a href="https://rocket-science.ru/hacking/2021/02/12/conditional-defguard">writing</a>, but I avoided it for a reason that will become clear below.</p>
<p>Gazing for the first time into the documentation examples, I was confused. Here it is</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">></span> <span class="n">is_struct</span><span class="p">(</span><span class="no">URI</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="s2">"/"</span><span class="p">),</span> <span class="no">URI</span><span class="p">)</span>
<span class="c1">#⇒ true</span>
</code></pre></div></div>
<p>Wait, what? There is no way the remote call <code class="language-plaintext highlighter-rouge">URI.parse("/")</code> would have been allowed in guards, but the documentation states it will.</p>
<p><img src="/img/context.jpg" alt="Context Differs" /></p>
<p>Well, the documentation is cheating on us. Of course, remote call cannot be used in guards, but <code class="language-plaintext highlighter-rouge">is_struct/2</code> <em>without remote calls</em> can. How is that? Let’s find out.</p>
<h3 id="conditional-context">Conditional Context</h3>
<p>There is no doubt, the compiler allows <code class="language-plaintext highlighter-rouge">is_struct/2</code> to be more permissive when it’s used as a bare macro, in a raw code, outside of guards. How does she know what AST to inject depending on the context? Easy. The macro tells her.</p>
<p>There is a <a href="https://hexdocs.pm/elixir/Kernel.SpecialForms.html#__CALLER__/0"><code class="language-plaintext highlighter-rouge">__CALLER__</code></a> special form, providing the current calling environment. It can be used only inside defmacro and defmacrop (<em>is available</em>, as an exception raised on any attempt to call it from outside macros, says.) It holds a whole bunch of interesting stuff within it, including, but not limited to <code class="language-plaintext highlighter-rouge">context</code> and <code class="language-plaintext highlighter-rouge">context_modules</code>.</p>
<p>The purpose of latter we’ll see a bit later, now let’s focus on the former.</p>
<p>It has a type <a href="https://hexdocs.pm/elixir/Macro.Env.html#t:context/0"><code class="language-plaintext highlighter-rouge">Macro.Env.context()</code></a> which means it can be an atom, either <code class="language-plaintext highlighter-rouge">nil</code>, or <code class="language-plaintext highlighter-rouge">:match</code> or <code class="language-plaintext highlighter-rouge">:guard</code>. These three contexts mean, respectively, the following</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">:guard</code> means the macro gets expanded within a function guard context, as in <code class="language-plaintext highlighter-rouge">def foo(s) when is_struct(s, URI)</code></li>
<li><code class="language-plaintext highlighter-rouge">:match</code> means it gets expanded inside a match, as in <code class="language-plaintext highlighter-rouge">def foo(is_struct(s))</code></li>
<li><code class="language-plaintext highlighter-rouge">nil</code> means elsewhere</li>
</ul>
<p>All three cases are <a href="https://github.com/elixir-lang/elixir/blob/v1.11.3/lib/elixir/lib/kernel.ex#L2314-L2341">handled differently</a> in the case of <code class="language-plaintext highlighter-rouge">is_struct/2</code> macro, and from the source code linked above, one can see that it’s not allowed in match clauses.</p>
<p>For what it worth, in the case of <code class="language-plaintext highlighter-rouge">:guard</code>, it gets expanded into <code class="language-plaintext highlighter-rouge">is_map(s) and s.__struct__ == S</code> which I directly used yesterday to simplify things.</p>
<h3 id="context-module">Context Module</h3>
<p>Another glitch I met literally today, is I have created a module, exporting a fancy functionality via <code class="language-plaintext highlighter-rouge">__using__/1</code> special form. I have it thoroughly tested, and all, and it worked perfectly. Then I decided to add <code class="language-plaintext highlighter-rouge">use MyFancy</code> to my <code class="language-plaintext highlighter-rouge">.iex.exs</code> file to steroidize my elixir shell.</p>
<p>It blowed up with a very cryptic message.</p>
<p>One might see the nearly same message by calling <code class="language-plaintext highlighter-rouge">use GenServer</code> from the shell.</p>
<p>But all I wanted was to inject some fancy stuff into my current context! Apparently, the AST to inject, besides setting a ton of environment, contained some stuff that makes sense within the module context only. One cannot call <code class="language-plaintext highlighter-rouge">def/2</code> from outside of the module, or set the module attribute (this is what the cryptic error message above was actually referring to,) just out of the thin air.</p>
<p>Aforementioned <code class="language-plaintext highlighter-rouge">__CALLER__</code> has <a href="https://hexdocs.pm/elixir/Macro.Env.html#t:context_modules/0"><code class="language-plaintext highlighter-rouge">context_modules</code></a> field to help us. According to the <a href="https://hexdocs.pm/elixir/Macro.Env.html">documentation</a>, this field contains the list of modules defined in the current context. One example expresses it better than the thousand of words:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="p">(</span><span class="n">__ENV__</span><span class="o">.</span><span class="n">context_modules</span><span class="p">,</span> <span class="ss">label:</span> <span class="s2">"top level"</span><span class="p">)</span>
<span class="k">defmodule</span> <span class="no">M</span> <span class="k">do</span>
<span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="p">(</span><span class="n">__ENV__</span><span class="o">.</span><span class="n">context_modules</span><span class="p">,</span> <span class="ss">label:</span> <span class="s2">"inside M module"</span><span class="p">)</span>
<span class="k">defmodule</span> <span class="no">N</span> <span class="k">do</span>
<span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="p">(</span><span class="n">__ENV__</span><span class="o">.</span><span class="n">context_modules</span><span class="p">,</span> <span class="ss">label:</span> <span class="s2">"inside nested N module"</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="c1">#⇒ top level: []</span>
<span class="c1">#⇒ inside M module: [M]</span>
<span class="c1">#⇒ inside nested N module: [M.N, M]</span>
</code></pre></div></div>
<p>So, I simply split the AST to inject into two parts, one that made sence only within a module context, and another one, that I might have injected anywhere. And then I simply conditionally concatenated them.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmacro</span> <span class="n">__using__</span><span class="p">(</span><span class="n">opts</span> <span class="p">\\</span> <span class="p">[])</span> <span class="k">do</span>
<span class="n">modulewise</span> <span class="o">=</span>
<span class="k">case</span> <span class="n">__CALLER__</span><span class="o">.</span><span class="n">context_modules</span> <span class="k">do</span>
<span class="p">[]</span> <span class="o">-></span> <span class="p">[]</span>
<span class="p">[</span><span class="n">_some</span> <span class="o">|</span> <span class="n">_</span><span class="p">]</span> <span class="o">-></span> <span class="n">build_modulewise_ast</span><span class="p">()</span>
<span class="k">end</span>
<span class="n">modulewise</span> <span class="o">++</span> <span class="n">build_module_agnostic_ast</span><span class="p">()</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Two functions above, starting with <code class="language-plaintext highlighter-rouge">build_</code> are out of scope here, they simply build the AST.</p>
<p>That was it. Now I am able to call <code class="language-plaintext highlighter-rouge">use MyFancy</code> from <code class="language-plaintext highlighter-rouge">.iex.exs</code>.</p>
<hr />
<p>Happy context-depending!</p>
Conditional guard for structs of an explicit type2021-02-12T00:00:00+00:00https://rocket-science.ru/hacking/2021/02/12/conditional-defguard<p>Starting with Elixir 1.11 / OTP 23, one can easily define a guard to check the argument is a particular struct. That might be handy in rare cases when the single function clause is ready to accept a mixed argument, and pattern matching directly on the argument would not work.
To be backward compatible, though, we cannot simply do</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defguard</span> <span class="n">is_foo</span><span class="p">(</span><span class="n">value</span><span class="p">)</span>
<span class="ow">when</span> <span class="n">is_map</span><span class="p">(</span><span class="n">value</span><span class="p">)</span> <span class="ow">and</span> <span class="n">value</span><span class="o">.</span><span class="n">__struct__</span> <span class="o">==</span> <span class="no">Foo</span>
</code></pre></div></div>
<p>the above would raise on earlier versions during compilation.</p>
<p>Direct <code class="language-plaintext highlighter-rouge">if</code> in the code on the top level, surrounding <code class="language-plaintext highlighter-rouge">defguard</code> clause wouldn’t work either, the compiler would nevertheless attempt to parse <code class="language-plaintext highlighter-rouge">value.__struct__ == Foo</code> and raise (I am unsure if it’s a desired behaviour, but still.)</p>
<p>The solution would be to introduce another macro, declaring this guard, that will check the version <em>before</em> returning the AST. Somewhat along these lines would work.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">@spec</span> <span class="n">can_struct_guard?</span> <span class="p">::</span> <span class="n">boolean</span><span class="p">()</span>
<span class="nv">@doc</span> <span class="s2">"Returns `true` if we can declare a proper guard, false otherwise"</span>
<span class="k">defp</span> <span class="n">can_struct_guard?</span> <span class="k">do</span>
<span class="no">String</span><span class="o">.</span><span class="n">to_integer</span><span class="p">(</span><span class="no">System</span><span class="o">.</span><span class="n">otp_release</span><span class="p">())</span> <span class="o">></span> <span class="mi">22</span> <span class="ow">and</span>
<span class="no">Version</span><span class="o">.</span><span class="n">compare</span><span class="p">(</span><span class="no">System</span><span class="o">.</span><span class="n">version</span><span class="p">(),</span> <span class="s2">"1.11.0"</span><span class="p">)</span> <span class="o">!=</span> <span class="ss">:lt</span>
<span class="k">end</span>
<span class="nv">@spec</span> <span class="n">maybe_struct_guard</span><span class="p">(</span><span class="n">name</span> <span class="p">::</span> <span class="n">atom</span><span class="p">(),</span> <span class="n">struct</span> <span class="p">::</span> <span class="n">module</span><span class="p">())</span> <span class="p">::</span> <span class="no">Macro</span><span class="o">.</span><span class="n">t</span>
<span class="nv">@doc</span> <span class="s2">"Returns the AST that would compile for the current Elixir/OTP"</span>
<span class="k">defmacro</span> <span class="n">maybe_struct_guard</span><span class="p">(</span><span class="n">name</span> <span class="p">\\</span> <span class="no">nil</span><span class="p">,</span> <span class="n">struct</span><span class="p">)</span> <span class="k">do</span>
<span class="c1"># Derive the guard name from the struct name unless passed explicitly</span>
<span class="n">name</span> <span class="o">=</span>
<span class="k">if</span> <span class="n">is_nil</span><span class="p">(</span><span class="n">name</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">struct</span> <span class="o">|></span> <span class="no">Module</span><span class="o">.</span><span class="n">split</span><span class="p">()</span> <span class="o">|></span> <span class="no">List</span><span class="o">.</span><span class="n">last</span><span class="p">()</span> <span class="o">|></span> <span class="no">Macro</span><span class="o">.</span><span class="n">underscore</span><span class="p">(),</span>
<span class="k">else</span><span class="p">:</span> <span class="n">name</span>
<span class="n">name</span> <span class="o">=</span> <span class="ss">:"is_</span><span class="si">#{</span><span class="n">name</span><span class="si">}</span><span class="ss">"</span>
<span class="k">if</span> <span class="n">can_struct_guard?</span><span class="p">()</span> <span class="k">do</span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="nv">@doc</span> <span class="sd">"""
Helper guard to match instances
of struct #{inspect(unquote(struct))}
"""</span>
<span class="k">defguard</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">name</span><span class="p">)(</span><span class="n">value</span><span class="p">)</span>
<span class="ow">when</span> <span class="n">is_map</span><span class="p">(</span><span class="n">value</span><span class="p">)</span> <span class="ow">and</span> <span class="n">value</span><span class="o">.</span><span class="n">__struct__</span> <span class="o">==</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">struct</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">else</span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="nv">@doc</span> <span class="sd">"""
Stub guard to match instances
of struct #{inspect(unquote(struct))}.
Upgrade to 11.0/23 to make it work.
"""</span>
<span class="k">defguard</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">name</span><span class="p">)(</span><span class="n">value</span><span class="p">)</span> <span class="ow">when</span> <span class="n">is_map</span><span class="p">(</span><span class="n">value</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Now we can declare guards as</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">maybe_struct_guard</span><span class="p">(</span><span class="no">MyStruct</span><span class="p">)</span>
</code></pre></div></div>
<p>and use it as</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">foo</span><span class="p">(</span><span class="n">value</span><span class="p">)</span> <span class="ow">when</span> <span class="n">is_list</span><span class="p">(</span><span class="n">value</span><span class="p">)</span> <span class="ow">or</span> <span class="n">is_my_struct</span><span class="p">(</span><span class="n">value</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="p">(</span><span class="n">value</span><span class="p">,</span> <span class="ss">label:</span> <span class="s2">"Expected value"</span><span class="p">)</span>
</code></pre></div></div>
<hr />
<p>Happy guarding!</p>
Will No One Rid Me Of This Turbulent GUI2021-01-12T00:00:00+00:00https://rocket-science.ru/hacking/2021/01/12/will-no-one-rid-me-of-this-turbulent-gui<p>All I am going to say is biased and opinionated. Everyone is free to shut up and disagree.</p>
<p><strong>Graphical tools bring more harm than profit to the development process.</strong></p>
<p>Don’t get me wrong, I started my professional career with <em>Delphi32</em>, meaning it was all <em>RAD</em> (a half-forgotten acronym, reincarnated as <em>IDE</em> years later.) When the developer is hugely unexperienced, unsure of themselves, having zero documentation on hand, and knowing nothing about anything—IDE is probably the only aide.</p>
<p>Experienced professional would be retarded by <em>IDE</em> in the same way as slow cooker helps a lone bachelor to get the stew an hour after a chef finishes a five course meal. Autotransmission is great to easier get the driver license, but <em>F1</em> cars have a mechanical one.</p>
<p><img src="/img/turbulent-priest.jpg" alt="Ronda de Camí, Platja d’Ara, Catalunya" /></p>
<hr />
<p>On the long run the developer, using the console, and a basic editor providing syntax highlighting (like <code class="language-plaintext highlighter-rouge">vim</code>, <code class="language-plaintext highlighter-rouge">emacs</code>, <code class="language-plaintext highlighter-rouge">vscode</code>, or even <code class="language-plaintext highlighter-rouge">sublime</code>) is more productive compared to the one struggling to produce a function without <em>IDE</em>. Below is the list of the most common arguments supporting <em>IDE</em> and my complains about.</p>
<h3 id="refactoring-is-easier">Refactoring is Easier</h3>
<p>It depends on what we are to call “refactoring.” All that “extract to class,” “produce accessors,” and similar stuff is <strong>not</strong> a refactoring. It’s renaming, relining, refiling, rebranding. It has nothing to do with refactoring, and <em>IDE</em> is absolutely impotent to help with a real refactoring, when <em>all backward compatibility is preserved</em>, <em>modules are reorganized</em>, and <em>spaghetti gets untangled</em>.</p>
<p>This is probably the only part of development process that requires intelligence, and no <em>IDE</em> has it. All they have is the ability to move a lump of spaghetti from one plate to another. Splitting it into context parts, setting boundaries, decoupling… All that requires understanding of the business logic, not the language syntax.</p>
<p>And yes, “rename function” is still easier to accomplish with a bit of <code class="language-plaintext highlighter-rouge">awk</code>, or even <code class="language-plaintext highlighter-rouge">sed</code>.</p>
<h3 id="autocomplete-obsoletes-docs">Autocomplete Obsoletes Docs</h3>
<p>Bullshit. Docs are not about function signatures (that’s also why “types obsolete documentation” is bullshit either.) Docs are all about examples, best practices, howtos-and-donts, explanation of complex solutions. <code class="language-plaintext highlighter-rouge">Enum.sum(list)</code> does not require docs, nor types, nor autocomplete. <a href="https://hexdocs.pm/flow/Flow.html#emit_and_reduce/3"><code class="language-plaintext highlighter-rouge">Flow.emit_and_reduce/3</code></a> is something one wants to read <em>about</em>, and time required to understand what is the argument order is next to zero compared to what and how to pass there.</p>
<h3 id="intellisense-saves-time">Intellisense Saves Time</h3>
<p>If intellisense saves you a time, I’d suggest the following self-improvement techniques (from worst to best)</p>
<ul>
<li>learn touch typing</li>
<li>memorize frequently used functions</li>
<li>minimize a number of external libraries</li>
<li>prefer think-then-code over code-then-see</li>
</ul>
<p>If I were to measure the time I spend thinking vs reading docs vs typing the code, the ratio would have been 70-20-10. With this percentage, I tend to produce ten times less of unnecessary code, and intellisense might save me up to 10% of my working time, even if she was typing everything for me.</p>
<h3 id="ide-helps-to-read-the-code">IDE Helps To Read The Code</h3>
<p>Professional developers must be able to read the code from the paper printed in sans-serif. Monospaced font and soft syntax highlighting helps to parse it, it’s kinda typography hygiene of the code. Good code does not require the reader to jump back and forth. Shakespeare’s sonnets and novels written by Maugham do not need crosslinks, tooltips, collapse buttons and debug window to be pleasantly read. Neither does a good code.</p>
<p>A mediocre code would not be automagically enhanced by reading it through <em>IDE</em>. No way. Click-jump-click-expand-click-collapse-jump-damn-where-the-heck-am-I does not help much to understand what’s going on there. We still cannot hold many items in the short-term memory, while all these jumps and compiler output windows just add them to the table, instead of reducing complexity.</p>
<h3 id="ide-makes-working-with-git-a-charm">IDE Makes Working With Git A Charm</h3>
<p>Working with <code class="language-plaintext highlighter-rouge">git</code> is already a charm. <em>GUI</em> <em>might</em> to some extent help to decrease a pain needed to rebase on 142 commits, but oval wheels are not any better than squared ones, if the goal is to drive the vehicle. Small changes, decoupling, full backward compatibility, and call boundaries work. <em>GUI</em> does not.</p>
<p>Find yourself frequently running <code class="language-plaintext highlighter-rouge">git add -p</code> on hunks instead of adding whole files? There is something wrong with your workflow. <em>GUI</em> would not help it. The effort on adding the content per hunk already smells. Dedicate a branch to one single thing, implement it, commit. Repeat.</p>
<h3 id="the-conclusion">The Conclusion</h3>
<p>So yeah, in all the cases when <em>IDE</em> seems to help the developer, she actually eliminates the consequences and not the cause. Robust code is easy to read from the phone, version-control with two plain <code class="language-plaintext highlighter-rouge">git</code> commands without switches, and to type in <code class="language-plaintext highlighter-rouge">nano</code>. Bad code longer stays somewhat maintainable through <em>IDE</em>, but sooner or later it’d inevitably blow up.</p>
<p>Using <em>IDE</em>, it’s much easier to miss that last turn to absolutely unmaintainable code. That’s why I suggest to avoid these touted luxury developers tools whenever possible.</p>
<h3 id="comments">Comments</h3>
<blockquote>
<p><a href="https://twitter.com/thecoldwine">@thecoldwine</a></p>
<p>You will have to use <em>IDE</em> to read the code written in <em>IDE</em>.</p>
<p>Let’s take some heavy-abstracted <code class="language-plaintext highlighter-rouge">Spring</code>/<code class="language-plaintext highlighter-rouge">Java</code> code as an example. <em>IDE</em> hides ridiculous complexity of automagic introduced by <code class="language-plaintext highlighter-rouge">Spring</code>.</p>
</blockquote>
<p>Exactly: one reads the nifty piece of code and apparently there appears an illusion it works that simple way. It does not, unfortunately. The whole bunch of generated stuff <em>that actually runs</em> became totally hidden from the developer. It’s dangerous because we apparently lose control over what the code actually is.</p>
<blockquote>
<p>Also there is annoying outcome of reliance on the <em>IDE</em>, one barely understands how to survive without it. I saw people struggling to use <code class="language-plaintext highlighter-rouge">git</code> w/o <code class="language-plaintext highlighter-rouge">SourceTree</code> or failing to compile the project without magic green-triangle-run button.</p>
<p>That means, that when one relies on <em>GUI</em>, they cannot automate the workflow, because a) they don’t know nuts and bolts, b) automation of <em>GUI</em> is much harder, and c) they think it is already solved by <em>IDE</em> in the best possible way.</p>
<p>So effectively we are comparing a supermarket (<em>GUI</em>) and a fishing pole (<em>CLI</em>). The problem is one day one will end up in the middle of the lake, with the closest supermarket in 50km. Guess, how many fishing rods would they have in their boat.</p>
</blockquote>
<p>I could not agree more. Thanks :)</p>
<hr />
<p>Happy consoling!</p>
Reserved Backward Compatibility2020-12-05T00:00:00+00:00https://rocket-science.ru/hacking/2020/12/05/reserved-backward-compatibility<p>I am fiercely loyal to backward compatibility at any cost. There is no single line of code I wrote in the past decade that breaks backward compatibility. Including but not limited to my experimental libraries dying at version <code class="language-plaintext highlighter-rouge">v0.5</code> and downloaded by strangers exactly zero times, because I never bothered to publish them into a public space.</p>
<p>I would rather sacrifice clarity, succinctness, and ease of use, than backward compatibility. Even if a parameter is never used anymore and gets immediately discarded in recent versions of the library, the external API would still accept it without compromises. That pattern allows me to recommend library users to always upgrade my library once the new version is released. Semantic versioning in my understanding means <em>no breaking changes</em> in the first place. Numbers just hint how much new goodness one’ll get with an update: none for patch, some for minor and kinda pack (read: LTS) for major. I don’t care what Proper Semantic Versioning Church Evangelists write in their books, I don’t read books. I use a common sense instead.</p>
<p>There is no way one might be put in a situation when introducing breaking changes would be a must. No damn way. Succinctness of the interface in the new version is a perfect dildo to tickle author’s ego and nothing else. No single user of the library cares about the existence of default function parameters or existence of the legacy function that does almost the same but slightly worse.</p>
<p>Deprecate it, freeze it’s development, but <strong>never ever remove it</strong> nor modify its signature.</p>
<hr />
<p>There is a fancy (unfortunately implausible) tale revealing Leonardo da Vinci used the same model for both <em>Christ</em> and <em>Judas</em> in ‘Last Supper.’</p>
<blockquote>
<p>It’s said that Leonardo da Vinci took over ten years to paint the masterpiece ‘Last Supper’ because he was so picky about the models he used for each character. Each model had to have a face that was da Vinci’s vision of the person that he would represent. Needless to say, it became a tedious task to find them. One Sunday, just after da Vinci had begun the painting, he spotted a young man in the choir that he felt would be the perfect Jesus Christ. The lad radiated love, innocence, tenderness, compassion and kindness. The young man, Pietri Bandinelli, agreed to be the model.</p>
<p>Ten years went by, and the painting remained incomplete. Leonardo could not find just the right face for Judas. He was allowed to search the prison, and there he found the perfect character to portray the man who betrayed Christ. Near the completion of the painting, the model asked if he was allowed to have a look. As he stared at the painting, tears began to flow down his face. When da Vinci asked what was wrong, the model told him that he was Pietri Bandinelli, the same man who had modeled for Christ ten years earlier. He went on to confess that after modelling he began to sin, and soon he turned away from God altogether, resulting in a life of crime, anger, sadness and grief that ended with him being sent to prison for life.</p>
<p>— <a href="https://www.northernstar-online.com/true-or-false-da-vinci-last-supper/">source</a></p>
</blockquote>
<p>Imagine for the sake of example that indeed happened. Should Leonardo have the visage of <em>Christ</em> deprecated once he knew the truth? Spend another ten years looking for the new model? I doubt. The same comes for legacy interfaces. Don’t kill them, please. Every time you kill one, God tortures the kitten.</p>
<hr />
<p>When I need to come up with an example of brilliant backward compatibility, I always mention Win95API on that matter.</p>
<p><img src="/img/RegQueryValueExA.png" alt="RegQueryValueExA" /></p>
<p>I do recall using this function back in 1996, when tweaking windows registry was kinda semi-hacking experience, required reverse engineering skills, as well as both bravery and despair. It worked 25 years ago, it works today. Everything around has changed, but this function. That’s what I admit and adore <em>WinAPI</em>. It is fully backward compatible.</p>
<p>What exactly am I to blame for that?—Well, this <code class="language-plaintext highlighter-rouge">lpReserved</code> field. Whenever MS engineers needed kinda new functionality related to the new data attached to this function, they have a room to pass it through. Without changing interfaces. That’s crucial.</p>
<hr />
<p>Nowadays the interfaces are rather data interchange formats rather than pure functions accessible from the outside code via direct linkage. But the latter are still there when it comes to external libraries encapsulating some data.</p>
<p>My advise would be to always create an empty field named <code class="language-plaintext highlighter-rouge">meta</code>, or <code class="language-plaintext highlighter-rouge">payload</code>, or <code class="language-plaintext highlighter-rouge">crap</code> if you are excentric enough, that would hold the empty collection from scratch.</p>
<p><code class="language-plaintext highlighter-rouge">State</code> in <em>Elixir</em> might look like:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defstruct</span> <span class="no">State</span><span class="p">,</span>
<span class="ss">id:</span> <span class="ss">:integer</span><span class="p">,</span>
<span class="ss">foo:</span> <span class="no">nil</span><span class="p">,</span>
<span class="ss">bar:</span> <span class="p">%{},</span>
<span class="ss">__meta__:</span> <span class="p">%{}</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">Message</code> in JSON:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="nl">"id"</span><span class="p">:</span><span class="mi">1</span><span class="p">,</span><span class="nl">"foo"</span><span class="p">:</span><span class="kc">null</span><span class="p">,</span><span class="nl">"bar"</span><span class="p">:{},</span><span class="nl">"__payload__"</span><span class="p">:{}}</span><span class="w">
</span></code></pre></div></div>
<hr />
<p>Future you will inevitably praise you present for letting literally any data of any shape to be attached to these <code class="language-plaintext highlighter-rouge">State</code> and <code class="language-plaintext highlighter-rouge">Message</code> without breaking anything. Just as an example, let’s see how this <code class="language-plaintext highlighter-rouge">state</code> might receive a new functionality without the necessity to <em>upgrade major version</em> on the consumers’ side. Imagine, we are after user-defined validation within this structure. And we already have some internal intergity validation, like:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">valid?</span><span class="p">(%</span><span class="no">State</span><span class="p">{}</span> <span class="o">=</span> <span class="n">state</span><span class="p">)</span> <span class="k">do</span>
<span class="n">is_integer</span><span class="p">(</span><span class="n">state</span><span class="o">.</span><span class="n">id</span><span class="p">)</span> <span class="ow">and</span> <span class="n">state</span><span class="o">.</span><span class="n">id</span> <span class="o">></span> <span class="mi">0</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Custom validation?—Easy. Simply whisper to your users “meta supports validation now” and add literally one line to your checker:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">valid?</span><span class="p">(%</span><span class="no">State</span><span class="p">{}</span> <span class="o">=</span> <span class="n">state</span><span class="p">)</span> <span class="k">do</span>
<span class="no">Map</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">state</span><span class="o">.</span><span class="n">__meta__</span><span class="p">,</span> <span class="ss">:checker</span><span class="p">,</span> <span class="k">fn</span> <span class="n">_</span> <span class="o">-></span> <span class="no">true</span> <span class="k">end</span><span class="p">)</span><span class="o">.</span><span class="p">(</span><span class="n">state</span><span class="p">)</span> <span class="ow">and</span>
<span class="n">is_integer</span><span class="p">(</span><span class="n">state</span><span class="o">.</span><span class="n">id</span><span class="p">)</span> <span class="ow">and</span> <span class="n">state</span><span class="o">.</span><span class="n">id</span> <span class="o">></span> <span class="mi">0</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Voilà. The rest of the code requires zero modifications, because nothing has changed in the data structure, nor in data handling. Your users would simply change</p>
<div class="language-diff highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="gd">- %State{id: 42}
</span><span class="gi">+ %State{id: 42, __meta__: %{checker: fn state -> not is_nil(state.foo) end}}
</span></code></pre></div></div>
<p>and they are all set. All the old code would not have a chance to notice any difference. That simple.</p>
<p>The same evidently applies to message exchanging, protocols etc. Create a reserved field and it’ll pay back next time you need to introduce somewhat new without frustrating everyone using your code and/or external messaging API.</p>
<hr />
<p>Happy reserving!</p>
O tempora, o mores!2020-09-04T00:00:00+00:00https://rocket-science.ru/hacking/2020/09/04/o-tempora-o-mores<p>The title is gracefully stolen from <em>Oratio in Catilinam Prima in Senatu Habita</em> by Marcus Tullius Cicero.</p>
<p><img src="/img/ciceron_denuncia_a_catilina.jpg" alt="Cicero Denounces Catiline, fresco by Cesare Maccari, 1882–1888" /></p>
<hr />
<p>In real life we often deal with time slots. Dentist appointments, hotel room reservations, even daily lunch break: scheduling all these is a task of fitting a time slot in a series of other time slots.</p>
<p>Consider a dentist appointment. I know I need 1 hour for the yearly routine inspection. I can visit a doctor during my lunch time, or after my working hours. The doctor has other patients. Here my busy hours are shown in purple, the doctor’s ones in red, non-working hours in gray, and the happily found slot when we both are free in green.</p>
<p><img src="/img/tempus-1.png" alt="Timelines of my busy hours and the dentist’s busy hours" /></p>
<p>A cursory glance at this chart is enough to spot the appointment time to be scheduled. Tomorrow, lunch time. Easy, huh?</p>
<p>Unfortunately, none of languages I am familiar with provides the functionality to determine this programatically. Last several years I am married to <em>Elixir</em>, hence I decided to write the library solving this particular problem.</p>
<p>Welcome <a href="https://hexdocs.pm/tempus/getting-started.html"><code class="language-plaintext highlighter-rouge">Tempus</code></a>!</p>
<h3 id="implementation-details">Implementation details</h3>
<p>The unit this library is built around is <code class="language-plaintext highlighter-rouge">Slot</code>. It represents the time slot in a most natural and simple way: struct, having <code class="language-plaintext highlighter-rouge">from</code> and <code class="language-plaintext highlighter-rouge">to</code> fields, both of type <code class="language-plaintext highlighter-rouge">DateTime</code>. The series of slots are organized in <code class="language-plaintext highlighter-rouge">Slots</code>. Under the hood it’s implemented as <a href="https://en.wikipedia.org/wiki/AVL_tree"><code class="language-plaintext highlighter-rouge">AVLTree</code></a>. This choice was made to keep the underlying list of slots consistent (ordered and not overlapped) for lowest cost to optimize read-access. The common use-case would be to fill the tree and store it somewhere for querying later.</p>
<p><a href="https://hexdocs.pm/tempus/Tempus.Slots.html#add/2"><code class="language-plaintext highlighter-rouge">Slots.add/2</code></a> function would add the slots list with the new slot, <em>joining slots as needed</em>. That makes it possible to simply insert new slots into the structure without bothering of the order <em>and</em> of the overlapping. Also, the helper function <a href="https://hexdocs.pm/tempus/Tempus.Slots.html#merge/2"><code class="language-plaintext highlighter-rouge">Slots.merge/2</code></a> is provided to merge two slot series. The latter is explicitly handy when one needs e. g. to find the <em>empty</em> slot in both series, as in the example with dentist appointment above.</p>
<p>One might also test any input against the slot with functions in <a href="https://hexdocs.pm/tempus/Tempus.Slot.html#content"><code class="language-plaintext highlighter-rouge">Slot</code></a> module: whether the slot covers the <code class="language-plaintext highlighter-rouge">DateTime</code> instance given, whether slots are disjoint or not, etc.</p>
<h3 id="tempus-module"><code class="language-plaintext highlighter-rouge">Tempus</code> module</h3>
<p>The main module exports functions to work with slots as with an intermittent timeline. One might add any arbitrary time interval to the origin, taking the slots into consideration; check whether the origin is <em>free</em>, or already taken by slots, get next free, or next busy slot, inverse slots, and more.</p>
<p>Here is the simple example from tests:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">slots</span> <span class="o">=</span>
<span class="p">[</span>
<span class="no">Tempus</span><span class="o">.</span><span class="no">Slot</span><span class="o">.</span><span class="n">wrap</span><span class="p">(</span><span class="sx">~D|2020-08-07|</span><span class="p">),</span> <span class="c1"># whole day</span>
<span class="p">%</span><span class="no">Tempus</span><span class="o">.</span><span class="no">Slot</span><span class="p">{</span>
<span class="ss">from:</span> <span class="sx">~U|2020-08-08 01:01:00Z|</span><span class="p">,</span> <span class="c1"># one minute</span>
<span class="ss">to:</span> <span class="sx">~U|2020-08-08 01:02:00Z|</span>
<span class="p">},</span>
<span class="p">%</span><span class="no">Tempus</span><span class="o">.</span><span class="no">Slot</span><span class="p">{</span>
<span class="ss">from:</span> <span class="sx">~U|2020-08-08 01:03:00Z|</span><span class="p">,</span> <span class="c1"># one minute</span>
<span class="ss">to:</span> <span class="sx">~U|2020-08-08 01:04:00Z|</span>
<span class="p">}</span>
<span class="p">]</span>
<span class="o">|></span> <span class="no">Enum</span><span class="o">.</span><span class="n">into</span><span class="p">(%</span><span class="no">Tempus</span><span class="o">.</span><span class="no">Slots</span><span class="p">{})</span>
</code></pre></div></div>
<p>Now adding <code class="language-plaintext highlighter-rouge">0</code> seconds to the occupied by slot time would return the first free time after this slot.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Tempus</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">slots</span><span class="p">,</span> <span class="sx">~U|2020-08-08 01:01:30Z|</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="ss">:second</span><span class="p">)</span>
<span class="c1">#⇒ ~U[2020-08-08 01:02:00Z]</span>
</code></pre></div></div>
<p>Adding 70 seconds five seconds before the first occupied minute <code class="language-plaintext highlighter-rouge">~U[2020-08-08 01:00:55Z]</code> would return the time five seconds after the second occupied minute</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Tempus</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">slots</span><span class="p">,</span> <span class="sx">~U|2020-08-08 01:00:55Z|</span><span class="p">,</span> <span class="mi">70</span><span class="p">,</span> <span class="ss">:second</span><span class="p">)</span>
<span class="c1">#⇒ ~U[2020-08-08 01:04:05Z]</span>
</code></pre></div></div>
<p>And so on. Negative values are surely allowed as well.</p>
<h3 id="merging-slot-series">Merging slot series</h3>
<p><code class="language-plaintext highlighter-rouge">Slots.merge/2</code> accepts a stream as the second argument. At the moment the library does not support merging and using <em>stream</em> of slots as is, but merging the stream into existing time slots is possible. This might be useful when one has the short list of, say, holidays, and wants to merge some reoccuring slots, like weekends.</p>
<p>All the functions returning <code class="language-plaintext highlighter-rouge">Slots</code> and/or <code class="language-plaintext highlighter-rouge">Slot</code> return valid objects (meaning <code class="language-plaintext highlighter-rouge">Slot</code> would be normalized and <code class="language-plaintext highlighter-rouge">Slots</code> would be ordered and joined as necessary.)</p>
<h3 id="future-development">Future development</h3>
<p>Current implementation covers our internal needs only, so I would be happy to hear about what this library lacks and what should be added to it.</p>
<hr />
<p>Happy timeslotting!</p>
Wolf, Goat, Cabbage… and Elixir2020-08-04T00:00:00+00:00https://rocket-science.ru/hacking/2020/08/04/wolf-goat-cabbage-and-elixir<blockquote>
<p>The wolf, goat and cabbage problem is a <a href="https://en.wikipedia.org/wiki/River_crossing_puzzle">river crossing puzzle</a>. It dates back to at least the 9<sup>th</sup> century, and has entered the folklore of a number of ethnic groups. </p>
<p>Once upon a time a farmer went to a market and purchased a wolf, a goat, and a cabbage. On his way home, the farmer came to the bank of a river and rented a boat. But crossing the river by boat, the farmer could carry only himself and a single one of his purchases: the wolf, the goat, or the cabbage. </p>
<p>If left unattended together, the wolf would eat the goat, or the goat would eat the cabbage. </p>
<p>The farmer’s challenge was to carry himself and his purchases to the far bank of the river, leaving each purchase intact. How did he do it? </p>
<p>— <a href="https://en.wikipedia.org/wiki/Wolf,_goat_and_cabbage_problem">Wolf, goat, and cabbage problem @Wiki</a></p>
</blockquote>
<p><img src="https://habrastorage.org/webt/wd/yt/9g/wdyt9getbzfatxvfihnk3udgia0.png" alt="Wolf → Goat → Cabbage" /></p>
<hr />
<p>I stumbled upon the article showing this riddle’s solving process in Haskell and all of a sudden I realized, that it’s perfectly suited to demonstrate the power of <em>Elixir</em> parallel processing nature. We are going to build the fully lazy asynchronous parallel iteration to get to the solution(s).</p>
<p>We are to start with the “all on the left bank” state, and at each step we will run a maximum of as many erlang processes as there are animals on this bank (+1 for crossing the river in empty boat.) At the same time, we will always check that the animals that remain on the bank do not bite each other; these branches will be cut off immediately. We will also store the history and cut off cyclical branches that return us to the state we have already seen. This, by the way, is not redundant data—we will return the history of all trips as a result.</p>
<p>So let’s start with the prerequisites.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">defmodule</span> <span class="no">WolfGoatCabbage</span><span class="o">.</span><span class="no">State</span> <span class="k">do</span>
<span class="vi">@moduledoc</span> <span class="s2">"""
Current state of our microcosm.
• banks (`true` for the initial one)
• `ltr` — the direction marker
• `history` of trips
"""</span>
<span class="n">defstruct</span> <span class="ss">banks: </span><span class="sx">%{true => [], false => []}</span><span class="p">,</span> <span class="ss">ltr: </span><span class="kp">true</span><span class="p">,</span> <span class="ss">history: </span><span class="p">[]</span>
<span class="k">end</span>
<span class="n">defmodule</span> <span class="no">WolfGoatCabbage</span><span class="o">.</span><span class="no">Subj</span> <span class="k">do</span>
<span class="vi">@moduledoc</span> <span class="s2">"""
The unit of animal life, and who they conflict with.
"""</span>
<span class="n">defstruct</span> <span class="p">[</span><span class="ss">:me</span><span class="p">,</span> <span class="ss">:incompatible</span><span class="p">]</span>
<span class="k">end</span>
</code></pre></div></div>
<p>My blog engine’s code parser still lives in XIX century, so here is the screenshot of the initial values.</p>
<p><img src="https://habrastorage.org/webt/ss/pp/gn/ssppgnizme7gfcoxzvuoed6g9p8.png" alt="Initial values" /></p>
<p>Thus, it’s a code time!</p>
<h3 id="integrity-check">Integrity Check</h3>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="vi">@spec</span> <span class="n">safe?</span><span class="p">(</span><span class="n">bank</span> <span class="o">::</span> <span class="p">[</span><span class="o">%</span><span class="no">Subj</span><span class="p">{}])</span> <span class="o">::</span> <span class="n">boolean</span><span class="p">()</span>
<span class="n">defp</span> <span class="n">safe?</span><span class="p">(</span><span class="n">bank</span><span class="p">)</span> <span class="k">do</span>
<span class="n">subjs</span> <span class="o">=</span>
<span class="n">bank</span>
<span class="o">|></span> <span class="no">Enum</span><span class="p">.</span><span class="nf">map</span><span class="p">(</span><span class="o">&</span> <span class="o">&</span><span class="mi">1</span><span class="p">.</span><span class="nf">me</span><span class="p">)</span>
<span class="o">|></span> <span class="no">MapSet</span><span class="p">.</span><span class="nf">new</span><span class="p">()</span>
<span class="n">incompatibles</span> <span class="o">=</span>
<span class="n">bank</span>
<span class="o">|></span> <span class="no">Enum</span><span class="p">.</span><span class="nf">flat_map</span><span class="p">(</span><span class="o">&</span> <span class="o">&</span><span class="mi">1</span><span class="p">.</span><span class="nf">incompatible</span><span class="p">)</span>
<span class="o">|></span> <span class="no">MapSet</span><span class="p">.</span><span class="nf">new</span><span class="p">()</span>
<span class="no">MapSet</span><span class="p">.</span><span class="nf">disjoint?</span><span class="p">(</span><span class="n">subjs</span><span class="p">,</span> <span class="n">incompatibles</span><span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Here we built a set of those who remain, a set of those with whom they cannot coexist, and made sure that there are no intersections. So far, this is trivial.</p>
<h3 id="move--one-way-trip">Move / One Way Trip</h3>
<p>Conditions for an empty trip and a trip with animals are quite different, so it is convenient to split their processing into two clauses (<code class="language-plaintext highlighter-rouge">nil</code> is a great fit for “nobody”.)</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="vi">@spec</span> <span class="n">move</span><span class="p">(</span><span class="o">%</span><span class="no">State</span><span class="p">{},</span> <span class="kp">nil</span> <span class="o">|</span> <span class="o">%</span><span class="no">Subj</span><span class="p">{})</span> <span class="o">::</span> <span class="o">%</span><span class="no">State</span><span class="p">{}</span> <span class="o">|</span> <span class="kp">false</span>
<span class="vi">@doc</span> <span class="s2">"""
If there is no one in the boat, it is enough to check
that we do not leave the bank in the already seen state,
and directly return the new state.
"""</span>
<span class="n">defp</span> <span class="n">move</span><span class="p">(</span><span class="o">%</span><span class="no">State</span><span class="p">{</span><span class="ss">ltr: </span><span class="n">ltr</span><span class="p">,</span> <span class="ss">banks: </span><span class="n">banks</span><span class="p">,</span> <span class="ss">history: </span><span class="n">history</span><span class="p">}</span> <span class="o">=</span> <span class="n">state</span><span class="p">,</span> <span class="kp">nil</span><span class="p">)</span> <span class="k">do</span>
<span class="n">with</span> <span class="kp">true</span> <span class="o"><-</span> <span class="n">not</span> <span class="n">ltr</span><span class="p">,</span> <span class="kp">true</span> <span class="o"><-</span> <span class="n">safe?</span><span class="p">(</span><span class="n">banks</span><span class="p">[</span><span class="n">ltr</span><span class="p">])</span> <span class="k">do</span>
<span class="o">%</span><span class="no">State</span><span class="p">{</span><span class="n">state</span> <span class="o">|</span> <span class="ss">ltr: </span><span class="n">not</span> <span class="n">ltr</span><span class="p">,</span> <span class="ss">history: </span><span class="p">[</span><span class="n">length</span><span class="p">(</span><span class="n">history</span><span class="p">)</span> <span class="o">|</span> <span class="n">history</span><span class="p">]}</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="vi">@doc</span> <span class="s2">"""
When there in the boat they bleat, yap, or emphatically
flap leaves—it’s a bit more complicated.
We move the animals from one bank to the other, making
sure that the trip is safe (there will be no unscheduled
dinner on the bank we just left) and that we haven’t seen
this state yet.
If all criteria are met, we return a new state,
or the terminating `false` otherwise.
"""</span>
<span class="n">defp</span> <span class="n">move</span><span class="p">(</span><span class="o">%</span><span class="no">State</span><span class="p">{</span><span class="ss">banks: </span><span class="n">banks</span><span class="p">,</span> <span class="ss">ltr: </span><span class="n">ltr</span><span class="p">,</span> <span class="ss">history: </span><span class="n">history</span><span class="p">},</span> <span class="n">who</span><span class="p">)</span> <span class="k">do</span>
<span class="n">with</span> <span class="kp">true</span> <span class="o"><-</span> <span class="n">who</span> <span class="k">in</span> <span class="n">banks</span><span class="p">[</span><span class="n">ltr</span><span class="p">],</span>
<span class="n">banks</span> <span class="o">=</span> <span class="sx">%{ltr => banks[ltr] -- [who],
not ltr => [who | banks[not ltr]]}</span><span class="p">,</span>
<span class="n">bank_state</span> <span class="o">=</span> <span class="no">MapSet</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">banks</span><span class="p">[</span><span class="kp">true</span><span class="p">]),</span>
<span class="kp">true</span> <span class="o"><-</span> <span class="n">safe?</span><span class="p">(</span><span class="n">banks</span><span class="p">[</span><span class="n">ltr</span><span class="p">]),</span>
<span class="kp">true</span> <span class="o"><-</span> <span class="n">not</span> <span class="no">Enum</span><span class="p">.</span><span class="nf">member?</span><span class="p">(</span><span class="n">history</span><span class="p">,</span> <span class="n">bank_state</span><span class="p">)</span> <span class="k">do</span>
<span class="o">%</span><span class="no">State</span><span class="p">{</span>
<span class="ss">banks: </span><span class="n">banks</span><span class="p">,</span>
<span class="ss">ltr: </span><span class="n">not</span> <span class="n">ltr</span><span class="p">,</span>
<span class="ss">history: </span><span class="p">[</span><span class="n">bank_state</span> <span class="o">|</span> <span class="n">history</span><span class="p">]</span>
<span class="p">}</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<h3 id="сruise">Сruise</h3>
<p>It remains, in fact, to write the main part: recursively spawning processes.
Duh, that’d be easy.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="vi">@initial</span> <span class="o">%</span><span class="no">State</span><span class="p">{</span>
<span class="ss">banks: </span><span class="sx">%{true => @subjs, false => []}</span><span class="p">,</span>
<span class="ss">history: </span><span class="p">[</span><span class="no">MapSet</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="vi">@subjs</span><span class="p">)]</span>
<span class="p">}</span>
<span class="vi">@spec</span> <span class="n">go</span><span class="p">(</span><span class="o">%</span><span class="no">State</span><span class="p">{})</span> <span class="o">::</span> <span class="p">[</span><span class="no">MapSet</span><span class="p">.</span><span class="nf">t</span><span class="p">()]</span>
<span class="k">def</span> <span class="nf">go</span><span class="p">(</span><span class="n">state</span> <span class="p">\\</span> <span class="vi">@initial</span><span class="p">)</span> <span class="k">do</span>
<span class="k">case</span> <span class="n">state</span><span class="p">.</span><span class="nf">banks</span><span class="p">[</span><span class="kp">true</span><span class="p">]</span> <span class="k">do</span>
<span class="p">[]</span> <span class="o">-></span> <span class="c1"># ура!</span>
<span class="no">Enum</span><span class="p">.</span><span class="nf">reverse</span><span class="p">(</span><span class="n">state</span><span class="p">.</span><span class="nf">history</span><span class="p">)</span>
<span class="n">_some</span> <span class="o">-></span>
<span class="p">[</span><span class="kp">nil</span> <span class="o">|</span> <span class="vi">@subjs</span><span class="p">]</span>
<span class="o">|></span> <span class="no">Task</span><span class="p">.</span><span class="nf">async_stream</span><span class="p">(</span><span class="o">&</span><span class="n">move</span><span class="p">(</span><span class="n">state</span><span class="p">,</span> <span class="o">&</span><span class="mi">1</span><span class="p">))</span>
<span class="o">|></span> <span class="no">Stream</span><span class="p">.</span><span class="nf">map</span><span class="p">(</span><span class="o">&</span><span class="n">elem</span><span class="p">(</span><span class="o">&</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">))</span> <span class="c1"># lazy</span>
<span class="o">|></span> <span class="no">Stream</span><span class="p">.</span><span class="nf">filter</span><span class="p">(</span><span class="o">&</span> <span class="o">&</span><span class="mi">1</span><span class="p">)</span> <span class="c1"># lazy</span>
<span class="o">|></span> <span class="no">Stream</span><span class="p">.</span><span class="nf">flat_map</span><span class="p">(</span><span class="o">&</span><span class="n">go</span><span class="o">/</span><span class="mi">1</span><span class="p">)</span> <span class="c1"># lazy + recursively</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Thanks <code class="language-plaintext highlighter-rouge">Stream</code>, all that code is lazy, meaning nothing would be executed until
asked explicitly. The multiprocess concurrent parody of <em>Haskell</em>.</p>
<h3 id="check-it">Check It</h3>
<p>I love tests, but I don’t really like to write tests. I consider this a waste of time: it’s indeed much easier to write working code right away. So I’m to simply create `main/0 ‘ function and display the results on the screen.</p>
<p>There is one caveat: several solutions will return a flat list due to <code class="language-plaintext highlighter-rouge">Stream.flat_map/2</code>. But it’s not a big deal: each solution ends up with an empty set, so we’ll easily break this flat sheet into chunks. I’m not going to copy-paste the entire code of beautiful output (which is almost as long as the logic behind,) here’s <a href="https://gist.github.com/am-kantox/d559895d7297d1a3f0d96eba4cdda5b3">gist</a> for those who care.</p>
<p>Meanwhile I am to open <em>REPL</em> and…</p>
<p><img src="https://habrastorage.org/webt/cb/a3/tj/cba3tjk1ikblaviwny-t51eb630.png" alt="Wolf → Goat → Cabbage" /></p>
<hr />
<p>Happy agricultural transportationing!</p>
Generated Types II — Down the Rabbit Hole2020-07-24T00:00:00+00:00https://rocket-science.ru/hacking/2020/07/24/generated-types-2<p>While I was writing <a href="https://rocket-science.ru/hacking/2020/07/15/generated-types">Generated Types</a> post, I was under naïve impression that I managed to use <em>erlang</em> typespecs to declare types inside my client <em>Elixir</em> code. That was indeed ingenuous.</p>
<p>The linked above approach works fine for explicit inline declarations, like <code class="language-plaintext highlighter-rouge">use Foo, var: type()</code> but it is doomed in many ways when we are to declare types upfront with, say, module attribute as we usually do for structs</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># @fields [foo: 42]</span>
<span class="c1"># defstruct @fields</span>
<span class="nv">@definition</span> <span class="ss">var:</span> <span class="n">atom</span><span class="p">()</span>
<span class="kn">use</span> <span class="no">Foo</span><span class="p">,</span> <span class="nv">@definition</span>
</code></pre></div></div>
<p><img src="/img/lighthouse-2.jpg" alt="Lighthouse in French Catalonia" /></p>
<p>Unfortunately, we cannot handle it at all. It won’t even get to the call to our macro, <code class="language-plaintext highlighter-rouge">@definition var: atom()</code> would raise <code class="language-plaintext highlighter-rouge">** (CompileError) undefined function atom/0</code>.</p>
<h3 id="naïve-approach">Naïve Approach</h3>
<p>One of my most beloved quotes on computer science is <em>“Weeks of coding can save you hours of planning”</em> (usually attributed to <a href="https://twitter.com/tsilb/status/65488255566614529">@tsilb</a>, but the user was suspended by twitter, so I am not sure.) I love it so much that I am saying it on every next standup, but as it always happens with inviolable life principles, I often fail to follow it myself.</p>
<p>So I started with introducing <em>two</em> <code class="language-plaintext highlighter-rouge">__using__/1</code> clauses, one accepting the list (assuming it consists of <code class="language-plaintext highlighter-rouge">field → type()</code> pairs,) and another one accepting everything else, where types are either quoted, or introduced as <code class="language-plaintext highlighter-rouge">{Module, :type, [params]}</code> tuples. I used sigil <a href="https://github.com/am-kantox/exvalibur/blob/master/lib/sigils.ex#L13-L21"><code class="language-plaintext highlighter-rouge">~q||</code></a>, gracefully stolen from one of my ancient pet projects, to allow <code class="language-plaintext highlighter-rouge">foo: ~q|atom()|</code> notation. I constructed a list that was later passed to the clause accepting lists. The whole codepiece was a nightmare. I doubt I saw something less intricate in my whole career, despite I feel myself absolutely comfortable with regular expressions, I like them and I use them a lot. I once won the bet on memorizing <a href="https://regular-expressions.mobi/email.html">email regex</a> and still, this code dealt with a plain old good erlang type was way more cumbersome and vague.</p>
<p>That all looked insane. I have a gut feeling that accepting erlang types in runtime should not need an enormously complicated overmacroed code that looked like a spell that can summon the cobol’s spirits. So I stepped back and started to think instead of writing code that nobody could ever understand later (me included.)</p>
<p>Here is a link to the <a href="https://github.com/am-kantox/vela/blob/v0.9.4/lib/macros.ex">working version</a>, for historical reasons. I am not proud of this code, but I am pretty sure we must share all the mistakes we made on incorrect paths taken, not only success stories. After all, they are always more inspiring and thrilling than dry presentation of the final result.</p>
<h3 id="tyyppi"><code class="language-plaintext highlighter-rouge">Tyyppi</code></h3>
<p>In a couple of days I went to the beach and there I all of a sudden understood, that I am facing an <a href="http://xyproblem.info/">XY Problem</a>. All I needed was simply to make erlang types a first class citizen in <em>Elixir</em> runtime. That’s how <a href="https://hexdocs.pm/tyyppi"><code class="language-plaintext highlighter-rouge">Tyyppi</code></a> library was born.</p>
<p>There is not documented <a href="https://github.com/elixir-lang/elixir/blob/v1.10.4/lib/elixir/lib/code/typespec.ex"><code class="language-plaintext highlighter-rouge">Code.Typespec</code></a> module in the <em>Elixir</em> core, that made my life easier. I started with very simple approach of validating terms against types. I loaded all the types available in my current session and continued to cover different cases one by one, recursively expanding remote types. Frankly, that was boring, not funny. That led me towards the first usable part of this library: <a href="https://hexdocs.pm/tyyppi/Tyyppi.html#of?/2"><code class="language-plaintext highlighter-rouge">Tyyppi.of?/2</code></a> function, which accepts a type and a term and returns a boolean.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">|</span><span class="n">tyyppi</span><span class="o">|</span><span class="mi">1</span> <span class="err">▶</span> <span class="no">Tyyppi</span><span class="o">.</span><span class="n">of?</span> <span class="no">GenServer</span><span class="o">.</span><span class="n">on_start</span><span class="p">(),</span> <span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">self</span><span class="p">()}</span>
<span class="c1">#⇒ true</span>
<span class="n">iex</span><span class="o">|</span><span class="n">tyyppi</span><span class="o">|</span><span class="mi">2</span> <span class="err">▶</span> <span class="no">Tyyppi</span><span class="o">.</span><span class="n">of?</span> <span class="no">GenServer</span><span class="o">.</span><span class="n">on_start</span><span class="p">(),</span> <span class="ss">:ok</span>
<span class="c1">#⇒ false</span>
</code></pre></div></div>
<p>I needed some internal representation for the types, so I decided to store everything as a struct named <a href="https://hexdocs.pm/tyyppi/Tyyppi.T.html"><code class="language-plaintext highlighter-rouge">Tyyppi.T</code></a>. And here is a sibling of <code class="language-plaintext highlighter-rouge">Tyyppi.of?/2</code> — <a href="https://hexdocs.pm/tyyppi/Tyyppi.html#of_type?/2"><code class="language-plaintext highlighter-rouge">Tyyppi.of_type?/2</code></a>, accepting my internal representation of the type and the term.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">|</span><span class="n">tyyppi</span><span class="o">|</span><span class="mi">3</span> <span class="err">▶</span> <span class="n">type</span> <span class="o">=</span> <span class="no">Tyyppi</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="no">GenServer</span><span class="o">.</span><span class="n">on_start</span><span class="p">)</span>
<span class="n">iex</span><span class="o">|</span><span class="n">tyyppi</span><span class="o">|</span><span class="mi">4</span> <span class="err">▶</span> <span class="no">Tyyppi</span><span class="o">.</span><span class="n">of_type?</span> <span class="n">type</span><span class="p">,</span> <span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">self</span><span class="p">()}</span>
<span class="c1">#⇒ true</span>
</code></pre></div></div>
<p>The only caveat of this approach is I need to load and keep all the types, available in the system, and this information won’t be available in releases. At the moment, I am fine with storing this info into some file with <code class="language-plaintext highlighter-rouge">:erlang.term_to_binary/1</code>, bundling it with the release, and loading it through a custom <a href="https://hexdocs.pm/elixir/Config.Provider.html"><code class="language-plaintext highlighter-rouge">Config.Provider</code></a>.</p>
<h3 id="structs">Structs</h3>
<p>Now I was fully armed to turn back to my original task: creating a handy way to declare a typed struct. With all that luggage onboard, it was easy. I decided to restrict the struct declaration itself to be explicit inline literal having <code class="language-plaintext highlighter-rouge">key: type()</code> pairs. Also I have implemented <code class="language-plaintext highlighter-rouge">Access</code> for it, checking types upon upserts. With this on hand, I decided to borrow a couple of ideas from <a href="https://hexdocs.pm/ecto/Ecto.Changeset.html"><code class="language-plaintext highlighter-rouge">Ecto.Changeset</code></a>, and introduced overridable <code class="language-plaintext highlighter-rouge">cast_field/1</code> and <code class="language-plaintext highlighter-rouge">validate/1</code> functions.</p>
<p>Now we might declare a struct that would allow upserts if and only types of values are correct, and the custom validation passes.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">MyStruct</span> <span class="k">do</span>
<span class="kn">import</span> <span class="no">Kernel</span><span class="p">,</span> <span class="ss">except:</span> <span class="p">[</span><span class="k">defstruct</span><span class="p">:</span> <span class="mi">1</span><span class="p">]</span>
<span class="kn">import</span> <span class="no">Tyyppi</span><span class="o">.</span><span class="no">Struct</span><span class="p">,</span> <span class="ss">only:</span> <span class="p">[</span><span class="k">defstruct</span><span class="p">:</span> <span class="mi">1</span><span class="p">]</span>
<span class="nv">@typedoc</span> <span class="s2">"The user type defined before `defstruct/1` declaration"</span>
<span class="nv">@type</span> <span class="n">my_type</span> <span class="p">::</span> <span class="ss">:ok</span> <span class="o">|</span> <span class="p">{</span><span class="ss">:error</span><span class="p">,</span> <span class="n">term</span><span class="p">()}</span>
<span class="nv">@defaults</span> <span class="ss">foo:</span> <span class="ss">:default</span><span class="p">,</span>
<span class="ss">bar:</span> <span class="ss">:erlang</span><span class="o">.</span><span class="n">list_to_pid</span><span class="p">(</span><span class="s1">'<0.0.0>'</span><span class="p">),</span>
<span class="ss">baz:</span> <span class="p">{</span><span class="ss">:error</span><span class="p">,</span> <span class="ss">:reason</span><span class="p">}</span>
<span class="k">defstruct</span> <span class="ss">foo:</span> <span class="n">atom</span><span class="p">(),</span> <span class="ss">bar:</span> <span class="no">GenServer</span><span class="o">.</span><span class="n">on_start</span><span class="p">(),</span> <span class="ss">baz:</span> <span class="n">my_type</span><span class="p">()</span>
<span class="k">def</span> <span class="n">cast_foo</span><span class="p">(</span><span class="n">atom</span><span class="p">)</span> <span class="ow">when</span> <span class="n">is_atom</span><span class="p">(</span><span class="n">atom</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="n">atom</span>
<span class="k">def</span> <span class="n">cast_foo</span><span class="p">(</span><span class="n">binary</span><span class="p">)</span> <span class="ow">when</span> <span class="n">is_binary</span><span class="p">(</span><span class="n">binary</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="no">String</span><span class="o">.</span><span class="n">to_atom</span><span class="p">(</span><span class="n">binary</span><span class="p">)</span>
<span class="k">def</span> <span class="n">validate</span><span class="p">(%{</span><span class="ss">foo:</span> <span class="ss">:default</span><span class="p">}</span> <span class="o">=</span> <span class="n">my_struct</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">my_struct</span><span class="p">}</span>
<span class="k">def</span> <span class="n">validate</span><span class="p">(%{</span><span class="ss">foo:</span> <span class="n">foo</span><span class="p">}</span> <span class="o">=</span> <span class="n">my_struct</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="ss">:error</span><span class="p">,</span> <span class="p">{</span><span class="ss">:foo</span><span class="p">,</span> <span class="n">foo</span><span class="p">}</span>
<span class="k">end</span>
</code></pre></div></div>
<p>I am not sure if there is a huge value of this library in production, but it surely might be a great helper during development, allowing to narrow down weird errors addressed to dynamic nature of types in <em>Elixir</em>, specifically when dealing with external sources.</p>
<p>Also, I am already using it to cast keywords passed as arguments to maps transparently.</p>
<hr />
<p>Happy runtimetyping!</p>
Generated Types2020-07-15T00:00:00+00:00https://rocket-science.ru/hacking/2020/07/15/generated-types<p>Consider the implementation of the scaffold module that injects the struct with some custom fields into the module calling <code class="language-plaintext highlighter-rouge">use Scaffold</code>.
Upon call to <code class="language-plaintext highlighter-rouge">use Scaffold, fields: foo: [custom_type()], ...</code> we want to implement the proper type in <code class="language-plaintext highlighter-rouge">Consumer</code> module (<code class="language-plaintext highlighter-rouge">common_field</code> below comes from <code class="language-plaintext highlighter-rouge">Scaffold</code>.)</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">@type</span> <span class="n">t</span> <span class="p">::</span> <span class="p">%</span><span class="no">Consumer</span><span class="p">{</span>
<span class="ss">common_field:</span> <span class="p">[</span><span class="n">atom</span><span class="p">()],</span>
<span class="ss">foo:</span> <span class="p">[</span><span class="n">custom_type</span><span class="p">()],</span>
<span class="o">...</span>
<span class="p">}</span>
</code></pre></div></div>
<p>That would be great if we could both precisely specify a type in <code class="language-plaintext highlighter-rouge">Consumer</code> for the future reference <em>and</em> generate the appropriate documentation for users of our new module.</p>
<p><img src="/img/lighthouse.jpg" alt="Lighthouse in French Catalonia" /></p>
<p>The more comprehensive example would look like.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">Scaffold</span> <span class="k">do</span>
<span class="k">defmacro</span> <span class="n">__using__</span><span class="p">(</span><span class="n">opts</span><span class="p">)</span> <span class="k">do</span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="nv">@fields</span> <span class="p">[</span>
<span class="ss">:version</span>
<span class="c1"># magic</span>
<span class="p">]</span>
<span class="nv">@type</span> <span class="n">t</span> <span class="p">::</span> <span class="p">%</span><span class="bp">__MODULE__</span><span class="p">{</span>
<span class="ss">version:</span> <span class="n">atom</span><span class="p">()</span>
<span class="c1"># magic</span>
<span class="p">}</span>
<span class="k">defstruct</span> <span class="nv">@fields</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">defmodule</span> <span class="no">Consumer</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">Scaffold</span><span class="p">,</span> <span class="ss">fields:</span> <span class="p">[</span><span class="ss">foo:</span> <span class="n">integer</span><span class="p">(),</span> <span class="ss">bar:</span> <span class="n">binary</span><span class="p">()]</span>
<span class="k">end</span>
</code></pre></div></div>
<p>resulting after compilation in</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">Consumer</span> <span class="k">do</span>
<span class="nv">@type</span> <span class="n">t</span> <span class="p">::</span> <span class="p">%</span><span class="no">Consumer</span><span class="p">{</span>
<span class="ss">version:</span> <span class="n">atom</span><span class="p">(),</span>
<span class="ss">foo:</span> <span class="n">integer</span><span class="p">(),</span>
<span class="ss">bar:</span> <span class="n">binary</span><span class="p">()</span>
<span class="p">}</span>
<span class="kn">use</span> <span class="no">Scaffold</span><span class="p">,</span> <span class="ss">fields:</span> <span class="p">[</span><span class="ss">foo:</span> <span class="n">integer</span><span class="p">(),</span> <span class="ss">bar:</span> <span class="n">binary</span><span class="p">()]</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Looks pretty easy, doesn’t it?</p>
<h3 id="naïve-approach">Naïve Approach</h3>
<p>Let’s start with inspecting what do we receive in <code class="language-plaintext highlighter-rouge">Scaffold.__using__/1</code>.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="k">defmacro</span> <span class="n">__using__</span><span class="p">(</span><span class="n">opts</span><span class="p">)</span> <span class="k">do</span>
<span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="p">(</span><span class="n">opts</span><span class="p">)</span>
<span class="k">end</span>
<span class="c1">#⇒ [fields: [foo: {:integer, [line: 2], []},</span>
<span class="c1"># bar: {:binary, [line: 2], []}]]</span>
</code></pre></div></div>
<p>So far, so good. Maybe we are almost there?</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="kn">quote</span> <span class="k">do</span>
<span class="n">custom_types</span> <span class="o">=</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">opts</span><span class="p">[</span><span class="ss">:fields</span><span class="p">])</span>
<span class="o">...</span>
<span class="k">end</span>
<span class="c1">#⇒ == Compilation error in file lib/consumer.ex ==</span>
<span class="c1"># ** (CompileError) lib/consumer.ex:2: undefined function integer/0</span>
</code></pre></div></div>
<p>Bang! Types are special, one cannot simply unquote it anywhere. Maybe unquoting inplace would work?</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nv">@type</span> <span class="n">t</span> <span class="p">::</span> <span class="p">%</span><span class="bp">__MODULE__</span><span class="p">{</span>
<span class="n">unquote_splicing</span><span class="p">([{</span><span class="ss">:version</span><span class="p">,</span> <span class="n">atom</span><span class="p">()}</span> <span class="o">|</span> <span class="n">opts</span><span class="p">[</span><span class="ss">:fields</span><span class="p">]])</span>
<span class="p">}</span>
<span class="c1">#⇒ == Compilation error in file lib/scaffold.ex ==</span>
<span class="c1"># ** (CompileError) lib/scaffold.ex:11: undefined function atom/0</span>
</code></pre></div></div>
<p>No dice. Types are hard, ask anyone who is doing <em>Haskell</em> for living (and <em>Haskell</em> has a very poor type system yet, dependent types are way better, but two ways harder.)</p>
<p>OK, looks like we need to build the whole clause as an AST and inject it at once, so that compiler would meet the proper declaration from scratch.</p>
<h3 id="constructing-type-ast">Constructing Type AST</h3>
<p>I would skip two hours of my tossing, torments, trial, and errors. Everyone knows that I write code mostly at random, expecting that all of a sudden another permutation would compile and hence work. The issue here is contexts. <strong>We should shove the received fields definitions down to the macro declaring type without unquoting them</strong> because once unquoted, the type like <code class="language-plaintext highlighter-rouge">binary()</code> would be immediately considered a function and <del>whack-a-mole’d</del> called by a compiler, resulting in <code class="language-plaintext highlighter-rouge">CompileError</code>.</p>
<p>Also, we cannot use regular functions <em>inside</em> <code class="language-plaintext highlighter-rouge">quote do</code> because the whole content of the block passed to <code class="language-plaintext highlighter-rouge">quote</code> would be quoted.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">quote</span> <span class="k">do</span>
<span class="no">Enum</span><span class="o">.</span><span class="n">map</span><span class="p">([</span><span class="ss">:foo</span><span class="p">,</span> <span class="ss">:bar</span><span class="p">],</span> <span class="o">&</span> <span class="nv">&1</span><span class="p">)</span>
<span class="k">end</span>
<span class="c1">#⇒ {</span>
<span class="c1"># {:., [], [{:__aliases__, [alias: false], [:Enum]}, :map]}, [],</span>
<span class="c1"># [[:foo, :bar], {:&, [], [{:&, [], [1]}]}]}</span>
</code></pre></div></div>
<p>You can see all these <code class="language-plaintext highlighter-rouge">Enum</code>, <code class="language-plaintext highlighter-rouge">:map</code> etc there as is. In other words, we should constuct the whole AST of the type outside of the quote and then unquote it inside. Let’s try it.</p>
<h3 id="less-naïve-attempt">Less Naïve Attempt</h3>
<p>We need to <em>inject</em> AST as AST, without unquoting it. Fine. Sounds as a stalemate?—Well, not really.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmacro</span> <span class="n">__using__</span><span class="p">(</span><span class="n">opts</span><span class="p">)</span> <span class="k">do</span>
<span class="n">fields</span> <span class="o">=</span> <span class="n">opts</span><span class="p">[</span><span class="ss">:fields</span><span class="p">]</span>
<span class="n">keys</span> <span class="o">=</span> <span class="no">Keyword</span><span class="o">.</span><span class="n">keys</span><span class="p">(</span><span class="n">fields</span><span class="p">)</span>
<span class="n">type</span> <span class="o">=</span> <span class="sx">??</span><span class="err">?</span>
<span class="kn">quote</span> <span class="ss">location:</span> <span class="ss">:keep</span> <span class="k">do</span>
<span class="nv">@type</span> <span class="n">t</span> <span class="p">::</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">type</span><span class="p">)</span>
<span class="k">defstruct</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">keys</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>All we need to do now, is to produce the proper AST, everything else is OK. Well, let <em>Elixir</em> do that for us!</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">|</span><span class="mi">1</span> <span class="err">▶</span> <span class="kn">quote</span> <span class="k">do</span>
<span class="o">...|</span><span class="mi">1</span> <span class="err">▶</span> <span class="p">%</span><span class="no">Foo</span><span class="p">{</span><span class="ss">version:</span> <span class="n">atom</span><span class="p">(),</span> <span class="ss">foo:</span> <span class="n">binary</span><span class="p">()}</span>
<span class="o">...|</span><span class="mi">1</span> <span class="err">▶</span> <span class="k">end</span>
<span class="c1">#⇒ {:%, [],</span>
<span class="c1"># [</span>
<span class="c1"># {:__aliases__, [alias: false], [:Foo]},</span>
<span class="c1"># {:%{}, [], [version: {:atom, [], []}, foo: {:binary, [], []}]}</span>
<span class="c1"># ]}</span>
</code></pre></div></div>
<p>Maybe something simpler?</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">|</span><span class="mi">2</span> <span class="err">▶</span> <span class="kn">quote</span> <span class="k">do</span>
<span class="o">...|</span><span class="mi">2</span> <span class="err">▶</span> <span class="p">%{</span><span class="ss">__struct__:</span> <span class="no">Foo</span><span class="p">,</span> <span class="ss">version:</span> <span class="n">atom</span><span class="p">(),</span> <span class="ss">foo:</span> <span class="n">binary</span><span class="p">()}</span>
<span class="o">...|</span><span class="mi">2</span> <span class="err">▶</span> <span class="k">end</span>
<span class="c1">#⇒ {:%{}, [],</span>
<span class="c1"># [</span>
<span class="c1"># __struct__: {:__aliases__, [alias: false], [:Foo]},</span>
<span class="c1"># version: {:atom, [], []},</span>
<span class="c1"># foo: {:binary, [], []}</span>
<span class="c1"># ]}</span>
</code></pre></div></div>
<p>Looks promising, isn’t it? We are ready to get to the resulting code.</p>
<h3 id="working-solution">Working Solution</h3>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmacro</span> <span class="n">__using__</span><span class="p">(</span><span class="n">opts</span><span class="p">)</span> <span class="k">do</span>
<span class="n">fields</span> <span class="o">=</span> <span class="n">opts</span><span class="p">[</span><span class="ss">:fields</span><span class="p">]</span>
<span class="n">keys</span> <span class="o">=</span> <span class="no">Keyword</span><span class="o">.</span><span class="n">keys</span><span class="p">(</span><span class="n">fields</span><span class="p">)</span>
<span class="n">type</span> <span class="o">=</span>
<span class="p">{</span><span class="ss">:%{}</span><span class="p">,</span> <span class="p">[],</span>
<span class="p">[</span>
<span class="p">{</span><span class="ss">:__struct__</span><span class="p">,</span> <span class="p">{</span><span class="ss">:__MODULE__</span><span class="p">,</span> <span class="p">[],</span> <span class="no">Elixir</span><span class="p">}},</span>
<span class="p">{</span><span class="ss">:version</span><span class="p">,</span> <span class="p">{</span><span class="ss">:atom</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[]}}</span>
<span class="o">|</span> <span class="n">fields</span>
<span class="p">]}</span>
<span class="kn">quote</span> <span class="ss">location:</span> <span class="ss">:keep</span> <span class="k">do</span>
<span class="nv">@type</span> <span class="n">t</span> <span class="p">::</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">type</span><span class="p">)</span>
<span class="k">defstruct</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">keys</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>or, if you don’t need to propagate types from <code class="language-plaintext highlighter-rouge">Scaffold</code> itself, even simpler, as suggested by <a href="https://elixirforum.com/t/how-to-generate-custom-types-with-macros/33122/4?u=mudasobwa"><em>Qqwy</em> here</a> (it wouldn’t work with propageted types, <code class="language-plaintext highlighter-rouge">version: atom()</code> outside of the quote raises.)</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmacro</span> <span class="n">__using__</span><span class="p">(</span><span class="n">opts</span><span class="p">)</span> <span class="k">do</span>
<span class="n">fields</span> <span class="o">=</span> <span class="n">opts</span><span class="p">[</span><span class="ss">:fields</span><span class="p">]</span>
<span class="n">keys</span> <span class="o">=</span> <span class="no">Keyword</span><span class="o">.</span><span class="n">keys</span><span class="p">(</span><span class="n">fields</span><span class="p">)</span>
<span class="n">fields_with_struct_name</span> <span class="o">=</span> <span class="p">[</span><span class="ss">__struct__:</span> <span class="n">__CALLER__</span><span class="o">.</span><span class="n">module</span><span class="p">]</span> <span class="o">++</span> <span class="n">fields</span>
<span class="kn">quote</span> <span class="ss">location:</span> <span class="ss">:keep</span> <span class="k">do</span>
<span class="nv">@type</span> <span class="n">t</span> <span class="p">::</span> <span class="p">%{</span><span class="n">unquote_splicing</span><span class="p">(</span><span class="n">fields_with_struct</span><span class="p">)}</span>
<span class="k">defstruct</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">keys</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Here is the result of executing <code class="language-plaintext highlighter-rouge">mix docs</code>:</p>
<p><img src="/img/generated-types.png" alt="Screenshot of type definition" /></p>
<h3 id="appendix-trick-with-quoted-fragment">Appendix. Trick With Quoted Fragment</h3>
<p>But what if we already have a complicated quoted block inside our <code class="language-plaintext highlighter-rouge">__using__/1</code> macro that binds the values quoted? Rewrite a ton of code to unquote everything everywhere? That’s not even always possible, if we want to have an access to anything declared inside the target module. Lucky us, we have a simpler way to achieve that.</p>
<blockquote>
<p><strong>NB</strong> for the sake of brevity, I would show the success path to declare all custom fields all having <code class="language-plaintext highlighter-rouge">atom()</code> type, but it would be easily extendable to accept any types from the input parameters, like <code class="language-plaintext highlighter-rouge">GenServer.on_start()</code> etc, this part would be left as a homework.</p>
</blockquote>
<p>Now we are to generate the type <em>inside</em> <code class="language-plaintext highlighter-rouge">quote do</code> block, because we cannot pass <code class="language-plaintext highlighter-rouge">atom()</code> bound quoted (it would raise <code class="language-plaintext highlighter-rouge">CompileError</code> as shown above.) So we’d resort to somewhat like</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">keys</span> <span class="o">=</span> <span class="no">Keyword</span><span class="o">.</span><span class="n">keys</span><span class="p">(</span><span class="n">fields</span><span class="p">)</span>
<span class="n">type</span> <span class="o">=</span>
<span class="p">{</span><span class="ss">:%{}</span><span class="p">,</span> <span class="p">[],</span>
<span class="p">[</span>
<span class="p">{</span><span class="ss">:__struct__</span><span class="p">,</span> <span class="p">{</span><span class="ss">:__MODULE__</span><span class="p">,</span> <span class="p">[],</span> <span class="no">Elixir</span><span class="p">}},</span>
<span class="p">{</span><span class="ss">:version</span><span class="p">,</span> <span class="p">{</span><span class="ss">:atom</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[]}}</span>
<span class="o">|</span> <span class="no">Enum</span><span class="o">.</span><span class="n">zip</span><span class="p">(</span><span class="n">keys</span><span class="p">,</span> <span class="no">Stream</span><span class="o">.</span><span class="n">cycle</span><span class="p">([{</span><span class="ss">:atom</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[]}]))</span>
<span class="p">]}</span>
</code></pre></div></div>
<p>That’s all good, but how would we inject this AST into <code class="language-plaintext highlighter-rouge">@type</code> declaration? The very handy <em>Elixir</em> feature named <a href="https://hexdocs.pm/elixir/Kernel.SpecialForms.html#quote/2-binding-and-unquote-fragments">Quoted Fragment</a> comes to the rescue. It was designed to allow compile-time generated code like</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">Squares</span> <span class="k">do</span>
<span class="no">Enum</span><span class="o">.</span><span class="n">each</span><span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="mi">42</span><span class="p">,</span> <span class="k">fn</span> <span class="n">i</span> <span class="o">-></span>
<span class="k">def</span> <span class="kn">unquote</span><span class="p">(</span><span class="ss">:"squared_</span><span class="si">#{</span><span class="n">i</span><span class="si">}</span><span class="ss">"</span><span class="p">)(),</span>
<span class="k">do</span><span class="p">:</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">i</span><span class="p">)</span> <span class="o">*</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">i</span><span class="p">)</span>
<span class="k">end</span><span class="p">)</span>
<span class="k">end</span>
<span class="no">Squares</span><span class="o">.</span><span class="n">squared_5</span>
<span class="c1">#⇒ 25</span>
</code></pre></div></div>
<p><em>Quoted Fragments</em> automagically recognized by <em>Elixir</em> inside quotes having a quoted bindings. Easy-peasy.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmacro</span> <span class="n">__using__</span><span class="p">(</span><span class="n">opts</span><span class="p">)</span> <span class="k">do</span>
<span class="n">keys</span> <span class="o">=</span> <span class="no">Keyword</span><span class="o">.</span><span class="n">keys</span><span class="p">(</span><span class="n">opts</span><span class="p">[</span><span class="ss">:fields</span><span class="p">])</span>
<span class="kn">quote</span> <span class="ss">location:</span> <span class="ss">:keep</span><span class="p">,</span> <span class="ss">bind_quoted:</span> <span class="p">[</span><span class="ss">keys:</span> <span class="n">keys</span><span class="p">]</span> <span class="k">do</span>
<span class="n">type</span> <span class="o">=</span>
<span class="p">{</span><span class="ss">:%{}</span><span class="p">,</span> <span class="p">[],</span>
<span class="p">[</span>
<span class="p">{</span><span class="ss">:__struct__</span><span class="p">,</span> <span class="p">{</span><span class="ss">:__MODULE__</span><span class="p">,</span> <span class="p">[],</span> <span class="no">Elixir</span><span class="p">}},</span>
<span class="p">{</span><span class="ss">:version</span><span class="p">,</span> <span class="p">{</span><span class="ss">:atom</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[]}}</span>
<span class="o">|</span> <span class="no">Enum</span><span class="o">.</span><span class="n">zip</span><span class="p">(</span><span class="n">keys</span><span class="p">,</span> <span class="no">Stream</span><span class="o">.</span><span class="n">cycle</span><span class="p">([{</span><span class="ss">:atom</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[]}]))</span>
<span class="p">]}</span>
<span class="c1"># ⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓</span>
<span class="nv">@type</span> <span class="n">t</span> <span class="p">::</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">type</span><span class="p">)</span>
<span class="k">defstruct</span> <span class="n">keys</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>This alone <code class="language-plaintext highlighter-rouge">unquote/1</code> is <em>allowed</em> inside <code class="language-plaintext highlighter-rouge">quote/2</code> receiving <code class="language-plaintext highlighter-rouge">bind_quote:</code> keys in the parameters keyword.</p>
<hr />
<p>Happy injecting!</p>
Vela → Time Series Cache2020-07-06T00:00:00+00:00https://rocket-science.ru/hacking/2020/07/06/vela-time-series-cache<p>Here in <em>Fintech</em>, we frequently deal with a huge load of rates for different currency pairs. We receive them from so-called <em>Liquidity Providers</em>, and each one has its own understanding of how the rates will be tomorrow, in a month, or even in six years. Some have more credibility, some deliver a holy crap for some pairs. And we need to decide which one is the best one so that we can show it to our customers. We literally need to <em>specially adapt our bills to separate mud and silt from the food we eat</em>, like flamingos.</p>
<p><img src="/img/flamingos.jpg" alt="Flamingos" /></p>
<blockquote>
<p>Flamingos filter-feed on brine shrimp and blue-green algae as well as insect larvae, small insects, mollusks and crustaceans making them omnivores. Their bills are specially adapted to separate mud and silt from the food they eat, and are uniquely used upside-down. The filtering of food items is assisted by hairy structures called lamellae, which line the mandibles, and the large, rough-surfaced tongue. — https://en.wikipedia.org/wiki/Flamingo#Feeding</p>
</blockquote>
<p>This necessity gave a birth to <a href="https://hexdocs.pm/vela"><code class="language-plaintext highlighter-rouge">Vela</code></a> library, the cache-like state holder for several <em>series</em>. It sieves the incoming data and keeps last <em>N</em> non-stale correct values for each serie.</p>
<p>Imagine we consume rates for three currency pairs. The basic definition of <code class="language-plaintext highlighter-rouge">Vela</code> would be</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">Pairs</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">Vela</span><span class="p">,</span>
<span class="ss">eurusd:</span> <span class="p">[</span><span class="ss">sorter:</span> <span class="o">&</span><span class="no">Kernel</span><span class="o">.<=/</span><span class="mi">2</span><span class="p">],</span>
<span class="ss">eurgbp:</span> <span class="p">[</span><span class="ss">limit:</span> <span class="mi">3</span><span class="p">,</span> <span class="ss">errors:</span> <span class="mi">1</span><span class="p">],</span>
<span class="ss">eurcad:</span> <span class="p">[</span><span class="ss">validator:</span> <span class="no">Pairs</span><span class="p">]</span>
<span class="nv">@behaviour</span> <span class="no">Vela</span><span class="o">.</span><span class="no">Validator</span>
<span class="nv">@impl</span> <span class="no">Vela</span><span class="o">.</span><span class="no">Validator</span>
<span class="k">def</span> <span class="n">valid?</span><span class="p">(</span><span class="ss">:eurcad</span><span class="p">,</span> <span class="n">rate</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="n">rate</span> <span class="o">></span> <span class="mi">0</span>
<span class="k">end</span>
</code></pre></div></div>
<h3 id="updating">Updating</h3>
<p><a href="https://hexdocs.pm/vela/Vela.html#put_in/2"><code class="language-plaintext highlighter-rouge">Vela.put/3</code></a> function will:</p>
<ul>
<li>call <code class="language-plaintext highlighter-rouge">validator</code> on the value if it’s defined (see <em>Validation</em> section below)</li>
<li>insert the value into the serie if validation passed, or into <code class="language-plaintext highlighter-rouge">:__errors__</code> otherwise</li>
<li>call the <code class="language-plaintext highlighter-rouge">sorter</code> for the serie if it’s defined (otherwise the value will be inserted on top, <em>FILO</em>, see <em>Sorting</em> section below)</li>
<li>and return the updated <code class="language-plaintext highlighter-rouge">Vela</code> instance back</li>
</ul>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">|</span><span class="mi">1</span> <span class="err">▶</span> <span class="n">pairs</span> <span class="o">=</span> <span class="p">%</span><span class="no">Pairs</span><span class="p">{}</span>
<span class="n">iex</span><span class="o">|</span><span class="mi">2</span> <span class="err">▶</span> <span class="no">Vela</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="n">pairs</span><span class="p">,</span> <span class="ss">:eurcad</span><span class="p">,</span> <span class="mf">1.0</span><span class="p">)</span>
<span class="c1">#⇒ %Pairs{..., eurcad: [1.0], ...}</span>
<span class="n">iex</span><span class="o">|</span><span class="mi">3</span> <span class="err">▶</span> <span class="no">Vela</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="n">pairs</span><span class="p">,</span> <span class="ss">:eurcad</span><span class="p">,</span> <span class="o">-</span><span class="mf">1.0</span><span class="p">)</span>
<span class="c1">#⇒ %Pairs{__errors__: [eurcad: -1.0], ...}</span>
<span class="n">iex</span><span class="o">|</span><span class="mi">4</span> <span class="err">▶</span> <span class="n">pairs</span> <span class="o">|></span> <span class="no">Vela</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="ss">:eurusd</span><span class="p">,</span> <span class="mf">2.0</span><span class="p">)</span> <span class="o">|></span> <span class="no">Vela</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="ss">:eurusd</span><span class="p">,</span> <span class="mf">1.0</span><span class="p">)</span>
<span class="c1">#⇒ %Pairs{... eurusd: [1.0, 2.0]}</span>
</code></pre></div></div>
<p>Under the hood <code class="language-plaintext highlighter-rouge">Vela</code> implements <a href="https://hexdocs.pm/elixir/Access.html"><code class="language-plaintext highlighter-rouge">Access</code></a> behaviour, making it possible to access series with standard nested update functions and macros in <a href="https://hexdocs.pm/elixir/Kernel.html"><code class="language-plaintext highlighter-rouge">Kernel</code></a>, such as <a href="https://hexdocs.pm/elixir/Kernel.html#get_in/2"><code class="language-plaintext highlighter-rouge">Kernel.get_in/2</code></a>, <a href="https://hexdocs.pm/elixir/Kernel.html#put_in/3"><code class="language-plaintext highlighter-rouge">Kernel.put_in/3</code></a>, <a href="https://hexdocs.pm/elixir/Kernel.html#update_in/3"><code class="language-plaintext highlighter-rouge">Kernel.update_in/3</code></a>, <a href="https://hexdocs.pm/elixir/Kernel.html#pop_in/2"><code class="language-plaintext highlighter-rouge">Kernel.pop_in/2</code></a>, and <a href="https://hexdocs.pm/elixir/Kernel.html#get_and_update_in/3"><code class="language-plaintext highlighter-rouge">Kernel.get_and_update_in/3</code></a>.</p>
<h3 id="validations">Validations</h3>
<p>Validator might be declared in several different ways.</p>
<ul>
<li>external function of arity 1, passed as <code class="language-plaintext highlighter-rouge">&MyMod.my_fun/1</code>, it will receive a value only</li>
<li>external function of arity 2, passed as <code class="language-plaintext highlighter-rouge">&MyMod.my_fun/2</code>, it will receive <code class="language-plaintext highlighter-rouge">serie, value</code> arguments</li>
<li>module implementing <a href="https://hexdocs.pm/vela/Vela.Validator.html#content"><code class="language-plaintext highlighter-rouge">Vela.Validator</code></a> behaviour</li>
<li><code class="language-plaintext highlighter-rouge">threshold</code> configuration parameter, used with optional <code class="language-plaintext highlighter-rouge">compare_by</code> parameter, see <em>Comparison</em> section below</li>
</ul>
<p>If validation passes, the value gets inserted into the serie, otherwise <code class="language-plaintext highlighter-rouge">{serie, value}</code> tuple gets added to <code class="language-plaintext highlighter-rouge">:__errors_</code>.</p>
<h3 id="comparison">Comparison</h3>
<p>Values stored in the series might be any. To compare them, one should pass <code class="language-plaintext highlighter-rouge">compare_by</code> keyword argument to the serie definition (unless the terms might be natively compared with <code class="language-plaintext highlighter-rouge">Kernel.</2</code>) of type <code class="language-plaintext highlighter-rouge">(Vela.value() -> number())</code>. Default is <code class="language-plaintext highlighter-rouge">& &1</code>.</p>
<p>Techically, one might also pass <code class="language-plaintext highlighter-rouge">comparator</code> keyword argument to calculate deltas (<code class="language-plaintext highlighter-rouge">min</code>/<code class="language-plaintext highlighter-rouge">max</code>) values; e. g. by passing <code class="language-plaintext highlighter-rouge">Date.diff/2</code> as comparator, one would receive correct deltas back.</p>
<p>Another handy option would be to pass a <code class="language-plaintext highlighter-rouge">threshold</code> keyword argument which specifies the ratio of new value to <code class="language-plaintext highlighter-rouge">{min, max}</code> interval for the new value to be considered valid. Because it’s given in percentages, the validation does not go through <code class="language-plaintext highlighter-rouge">comparator</code>, but it still uses <code class="language-plaintext highlighter-rouge">compare_by</code>. For example, to specify a threshold for datetimes, one would specify <code class="language-plaintext highlighter-rouge">compare_by: &DateTime.to_unix/1</code> <em>and</em> <code class="language-plaintext highlighter-rouge">threshold: 1</code>, resulting in new values would be allowed if and only they are within <code class="language-plaintext highlighter-rouge">±band</code> interval from the current values.</p>
<p>After all, one might use <code class="language-plaintext highlighter-rouge">Vela.equal?/2</code> to compare two velas. If values have <code class="language-plaintext highlighter-rouge">equal?/2</code> or <code class="language-plaintext highlighter-rouge">compare/2</code>, these functions would be used, otherwise the dumb <code class="language-plaintext highlighter-rouge">==/2</code> comparison would take a place.</p>
<h3 id="getting-values">Getting Values</h3>
<p>To operate <code class="language-plaintext highlighter-rouge">Vela</code>, one typically starts with <code class="language-plaintext highlighter-rouge">Vela.purge/1</code> which removes all the legacy values, if <code class="language-plaintext highlighter-rouge">validator</code> deals with timestamps. Then one might call <code class="language-plaintext highlighter-rouge">Vela.slice/1</code> that would return back the keyword with series as keys <em>and</em> top values as their values.</p>
<p><code class="language-plaintext highlighter-rouge">get_in/2</code>/<code class="language-plaintext highlighter-rouge">pop_in/2</code> might also be used for low-level to the top value of any serie.</p>
<h3 id="application">Application</h3>
<p><code class="language-plaintext highlighter-rouge">Vela</code> is extremely useful as the time-series cache used as a state in the <code class="language-plaintext highlighter-rouge">GenServer</code>/<code class="language-plaintext highlighter-rouge">Agent</code>. We don’t want to use legacy rates, so we simply maintain the state as a <code class="language-plaintext highlighter-rouge">Vela</code>, with <code class="language-plaintext highlighter-rouge">validator</code> looking like</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">@impl</span> <span class="no">Vela</span><span class="o">.</span><span class="no">Validator</span>
<span class="k">def</span> <span class="n">valid?</span><span class="p">(</span><span class="n">_key</span><span class="p">,</span> <span class="p">%</span><span class="no">Rate</span><span class="p">{}</span> <span class="o">=</span> <span class="n">rate</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="no">Rate</span><span class="o">.</span><span class="n">age</span><span class="p">(</span><span class="n">rate</span><span class="p">)</span> <span class="o"><</span> <span class="nv">@death_age</span>
</code></pre></div></div>
<p>and <code class="language-plaintext highlighter-rouge">Vela.purge/1</code> takes care about removing legacy. <code class="language-plaintext highlighter-rouge">Vela.slive/1</code> given an access to the most recent, actual rates, and we also might give back the history of any valid rate back to several values by request.</p>
<hr />
<p>Happy timeseriesing!</p>
Telemetría2020-05-30T00:00:00+00:00https://rocket-science.ru/hacking/2020/05/30/welcome-telemetria<p>12 Jul 2018 the project <code class="language-plaintext highlighter-rouge">:telemetry</code> got its <a href="https://github.com/beam-telemetry/telemetry/commit/8e553556fd683c17f4f97f72972332a4fefd355b">first initial commit</a>. It was pushed by <a href="https://github.com/arkgil">Arkadiusz Gil</a> but the README <a href="https://github.com/beam-telemetry/telemetry#copyright-and-license">states</a> <em>“Telemetry is copyright (c) 2018 Chris McCord and Erlang Solutions”</em> and the last commit ATM was made by José Valim.</p>
<p><img src="/img/mascarilla.jpg" alt="Big Brother is Watching You" /></p>
<p>The library introduces itself as:</p>
<blockquote>
<p><em>Telemetry</em> is a dynamic dispatching library for metrics and instrumentations. It is lightweight, small and can be used in any Erlang or Elixir project.</p>
</blockquote>
<p>The main advantage of the library is it’s deadly simple. One <em>registers</em> the event, which <em>is composed of a numeric value and can have metadata attached to it</em>, <em>sends</em> it whenever needed, and it would be delivered to a <em>handle</em> which might do whatever, usually it’s kinda logging or something alike. Decoupling business logic and metrics/visibility at its best.</p>
<p>The main disadvantage of using <code class="language-plaintext highlighter-rouge">:telemetry</code> as is, it requires an enormous amount of boilerplate that has to be maintained in several places. Refactoring project with intensive usage of telemetry might become a disaster. To change the event name one should amend the code in three different places: event registration, event firing, event handling. If one forgets to register newly created event, firing it would silently succeed, but handler would have been never called. Etc.</p>
<hr />
<p>What I felt was discrepant. I love to have metrics embedded everywhere, but I cannot consent that <code class="language-plaintext highlighter-rouge">grep -r telemetry\.execute ./{lib,test}/**/*.ex*</code> is something I want to inject into my build pipeline. So I came up with a handy wrapper for <code class="language-plaintext highlighter-rouge">:telemetry</code> called <code class="language-plaintext highlighter-rouge">Telemetría</code>. I pursued the following goals:</p>
<ul>
<li>simple way to attach <code class="language-plaintext highlighter-rouge">:telemetry</code> to functions <em>and</em> expressions</li>
<li>automatic event name generation based on the context</li>
<li>compile-time generation of config to register all and only actual events</li>
<li>automatic inclusion of <em>VM</em> and system metrics via <code class="language-plaintext highlighter-rouge">:telemetry_poller</code></li>
<li>custom configurable handlers for different cases, based not only on event name</li>
<li>full support for releases</li>
<li>zero boilerplate.</li>
</ul>
<h3 id="intro">Intro</h3>
<p>The project is currently in the kindergarten, but it’s already fully usable. All you need to start using it (and hence to jump on our telemetry users wagon,) would be to modify <code class="language-plaintext highlighter-rouge">mix.exs</code> to include <code class="language-plaintext highlighter-rouge">:telemetria</code> <em>and</em> to provide a minimal config.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># mix.exs</span>
<span class="k">def</span> <span class="n">project</span> <span class="k">do</span>
<span class="p">[</span>
<span class="o">...</span>
<span class="c1"># needed for autogeneration of events</span>
<span class="ss">compilers:</span> <span class="p">[</span><span class="ss">:telemetria</span> <span class="o">|</span> <span class="no">Mix</span><span class="o">.</span><span class="n">compilers</span><span class="p">()]</span>
<span class="p">]</span>
<span class="k">end</span>
<span class="k">def</span> <span class="n">deps</span> <span class="k">do</span>
<span class="p">[</span>
<span class="o">...</span>
<span class="p">{</span><span class="ss">:telemetria</span><span class="p">,</span> <span class="s2">"~> 0.5"</span><span class="p">}</span>
<span class="p">]</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The full list of supported options might be always found in <a href="https://hexdocs.pm/telemetria/Telemetria.html#module-options">docs</a>, the only mandatory one is <code class="language-plaintext highlighter-rouge">:otp_app</code>.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">config</span> <span class="ss">:telemetria</span><span class="p">,</span>
<span class="ss">otp_app:</span> <span class="ss">:my_app</span><span class="p">,</span>
<span class="ss">polling:</span> <span class="p">[</span><span class="ss">enabled:</span> <span class="no">true</span><span class="p">]</span>
</code></pre></div></div>
<p>With these settings, the VM and system metrics would be delivered to the default handler, which roughly logs them with <code class="language-plaintext highlighter-rouge">:info</code> level every five seconds.</p>
<h3 id="helpers">Helpers</h3>
<p><code class="language-plaintext highlighter-rouge">Telemetría</code>’s interface is also very simple. It provides three macros and one annotation to handle <code class="language-plaintext highlighter-rouge">:telemetry</code> events. The macros are <code class="language-plaintext highlighter-rouge">deft/2</code>, <code class="language-plaintext highlighter-rouge">defpt/2</code>, and <code class="language-plaintext highlighter-rouge">t/2</code> to wrap functions, private functions, and custom expressions, with <code class="language-plaintext highlighter-rouge">:telemetry</code> events. Basically, all three are <a href="https://en.wikipedia.org/wiki/Aspect-oriented_programming">aspects</a>, that are expanded in <em>grab metrics → make wrapped call → grab metrics, send event</em> triple. The annotation <code class="language-plaintext highlighter-rouge">@telemetria true</code> is essentially the same as changing the annotated call to <code class="language-plaintext highlighter-rouge">deft/2</code>/<code class="language-plaintext highlighter-rouge">defp/2</code>. Consider the following example.</p>
<h4 id="with-macro">With Macro</h4>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">Geom</span> <span class="k">do</span>
<span class="kn">import</span> <span class="no">Telemetria</span>
<span class="nv">@spec</span> <span class="n">circle_area</span><span class="p">(</span><span class="n">float</span><span class="p">())</span> <span class="p">::</span> <span class="n">float</span><span class="p">()</span>
<span class="n">deft</span> <span class="n">circle_area</span><span class="p">(</span><span class="n">radius</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="mi">2</span> <span class="o">*</span> <span class="mf">3.14</span> <span class="o">*</span> <span class="n">radius</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Once compiled, this code would emit event <code class="language-plaintext highlighter-rouge">[:geom, :circle_area]</code> upon calls to <code class="language-plaintext highlighter-rouge">Geom.circle_are/1</code></p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">|</span><span class="mi">1</span><span class="o">></span> <span class="no">Geom</span><span class="o">.</span><span class="n">circle_area</span> <span class="mf">2.0</span>
<span class="mi">14</span><span class="p">:</span><span class="mi">59</span><span class="p">:</span><span class="mf">49.686</span> <span class="p">[</span><span class="n">info</span><span class="p">]</span> <span class="p">[</span><span class="ss">event:</span> <span class="p">[</span><span class="ss">:geom</span><span class="p">,</span> <span class="ss">:circle_area</span><span class="p">],</span>
<span class="ss">measurements:</span> <span class="p">%{</span><span class="ss">consumed:</span> <span class="mi">3162</span><span class="p">,</span> <span class="ss">reference:</span> <span class="s2">"#Reference<0.12.34.56>"</span><span class="p">,</span>
<span class="ss">system_time:</span> <span class="p">[</span><span class="ss">system:</span> <span class="mi">1590843589680306588</span><span class="p">,</span>
<span class="ss">monotonic:</span> <span class="o">-</span><span class="mi">576460720784457</span><span class="p">,</span>
<span class="ss">utc:</span> <span class="sx">~U[2020-05-30 12:59:49.681885Z]</span><span class="p">]},</span>
<span class="ss">metadata:</span> <span class="p">%{</span><span class="ss">context:</span> <span class="p">[],</span> <span class="ss">env:</span> <span class="c1">#Macro.Env<aliases: [], context: nil, ...},</span>
<span class="ss">config:</span> <span class="c1">#PID<0.314.0>]</span>
<span class="c1">#⇒ 12.56</span>
</code></pre></div></div>
<h4 id="with-annotation">With Annotation</h4>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">Geom</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">Telemetria</span>
<span class="nv">@telemetria</span> <span class="no">true</span>
<span class="nv">@spec</span> <span class="n">circle_area</span><span class="p">(</span><span class="n">float</span><span class="p">())</span> <span class="p">::</span> <span class="n">float</span><span class="p">()</span>
<span class="k">def</span> <span class="n">circle_area</span><span class="p">(</span><span class="n">radius</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="mi">2</span> <span class="o">*</span> <span class="mf">3.14</span> <span class="o">*</span> <span class="n">radius</span>
<span class="k">end</span>
</code></pre></div></div>
<p>This code does effectively the same as the code above.</p>
<h3 id="compiler">Compiler</h3>
<p>Also not mandatory, working with <code class="language-plaintext highlighter-rouge">Telemetría</code> would be easier if you add the custom compiler to the list of compilers in your <code class="language-plaintext highlighter-rouge">mix.exs</code></p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">project</span> <span class="k">do</span>
<span class="p">[</span>
<span class="o">...</span>
<span class="ss">compilers: </span><span class="p">[</span><span class="ss">:telemetria</span> <span class="o">|</span> <span class="no">Mix</span><span class="p">.</span><span class="nf">compilers</span><span class="p">()]</span>
<span class="p">]</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The compiler collects added/removed events, maintains a manifest file behind the scene to always have an up-to-date list of events and provides diagnostics to the main <em>Elixir</em> compiler.</p>
<p>It also builds and exposes JSON config that might be found in <code class="language-plaintext highlighter-rouge">config</code> folder locally and might be used as a JSON config within releases. <code class="language-plaintext highlighter-rouge">Telemetría</code> understands it with a config provider that is included.</p>
<h3 id="config">Config</h3>
<p>The configuration allows:</p>
<ul>
<li>global enabling/disabling the telemetry (purging on compiler level)</li>
<li>enabling <em>VM/system telemetry polling</em>, with an interval</li>
<li>elixir configuration of additional events, directly handled with <code class="language-plaintext highlighter-rouge">:telemetry</code></li>
<li>json configuration of additional events (useful for releases)</li>
<li>setting your own custom handler for telemetry events</li>
</ul>
<p>See <a href="https://hexdocs.pm/telemetria/Telemetria.html#module-options"><code class="language-plaintext highlighter-rouge">Options</code></a> section in the documentation for more details.</p>
<h3 id="internals">Internals</h3>
<p>The deep diving into implementation details are surely out of scope of this ad writing, but a couple of words to be said about how is it built.</p>
<p>If one is interested in easy jumping into adding metrics to their application, please stop reading here and refer to the code snippets in the chapter entitled <em>Intro</em>.</p>
<p>If, on the opposite, you are interested in some tricks and tweaks, here is the list of what might be of your interest.</p>
<ul>
<li><a href="https://github.com/am-kantox/telemetria/blob/master/lib/options.ex">★★★</a> always valid, auto-documented options through <a href="https://hexdocs.pm/nimble_options"><code class="language-plaintext highlighter-rouge">nimble_options</code></a></li>
<li><a href="https://github.com/am-kantox/telemetria/blob/master/lib/telemetria.ex#L110-L111">★★★</a> annotation <a href="https://github.com/am-kantox/telemetria/blob/master/lib/telemetria/module_hooks.ex">implementation</a> through <code class="language-plaintext highlighter-rouge">@on_definition</code> and <code class="language-plaintext highlighter-rouge">@before_compile</code> <a href="https://hexdocs.pm/elixir/Module.html#module-compile-callbacks"><em>Module Compile Callbacks</em></a></li>
<li><a href="https://github.com/am-kantox/telemetria/blob/master/lib/telemetria/application.ex#L24-L26">★★★</a> starting the application in phases to ensure full availability of <code class="language-plaintext highlighter-rouge">:telemetry</code> before anything</li>
<li><a href="https://github.com/am-kantox/telemetria/blob/master/lib/mix/tasks/compile/telemetria.ex">★★★</a> implementation of custom compiler for <em>Elixir</em> to collect events and maintain the manifest file</li>
</ul>
<hr />
<p>Happy metric-measuring!</p>
Reverse Engineering for Poor2020-05-08T00:00:00+00:00https://rocket-science.ru/hacking/2020/05/08/reverse-engineering-for-poor<p>There was a <a href="https://stackoverflow.com/questions/61659551/what-does-the-alphanumeric-value-mean-in-an-elixir-mix-lock-file">question</a> published yesterday on <em>Stack Overflow</em>, asking</p>
<blockquote>
<p>what the alphanumeric value like <code class="language-plaintext highlighter-rouge">"d<i62n>2"</code> represents in the slice of <code class="language-plaintext highlighter-rouge">mix.lock</code> file?</p>
</blockquote>
<p>Well, I am kinda familiar with <em>Elixir</em> core code, but this is <em>mix</em> which I never scratched my head of. First glance at sources didn’t help much; the entry format in <code class="language-plaintext highlighter-rouge">mix.lock</code> looks like</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="s2">"nimble_parsec"</span><span class="p">:</span> <span class="p">{</span>
<span class="ss">:hex</span><span class="p">,</span>
<span class="ss">:nimble_parsec</span><span class="p">,</span>
<span class="s2">"0.5.3"</span><span class="p">,</span>
<span class="s2">"def21c10a9ed70ce22754fdeea0810dafd53c2db3219a0cd54cf5526377af1c6"</span><span class="p">,</span> <span class="c1"># ??? 1</span>
<span class="p">[</span><span class="ss">:mix</span><span class="p">],</span>
<span class="p">[],</span>
<span class="s2">"hexpm"</span><span class="p">,</span>
<span class="s2">"589b5af56f4afca65217a1f3eb3fee7e79b09c40c742fddc1c312b3ac0b3399f"</span> <span class="c1"># ??? 2</span>
<span class="p">}</span>
</code></pre></div></div>
<p>There are two hashes, the latter is a checksum of the package at <a href="https://hex.pm/packages/nimble_parsec/0.5.3">hex.pm</a> (check the bottom right corner below)</p>
<p><img src="/img/nimble_parsec_hex_checksum.png" alt="Checksum 589b5af56f4afca65217a1f3eb3fee7e79b09c40c742fddc1c312b3ac0b3399f" /></p>
<p>but the former is something I could not <code class="language-plaintext highlighter-rouge">grep</code> easily. I am an inquisitive guy, so I went to source code. It quickly revealed that this is not something <code class="language-plaintext highlighter-rouge">mix</code> creates on its own, so I needed to understand who is the one this data is delegated to. The good entry point of investigation would be <a href="https://github.com/elixir-lang/elixir/blob/5984c6cc29a41d5bc78d49427730c8786d75e2c9/lib/mix/lib/mix/dep/lock.ex#L13-L43"><code class="language-plaintext highlighter-rouge">Mix.Dep.Lock.{read/0,write/1}</code></a> functions.</p>
<p>I am aware of <a href="https://hexdocs.pm/iex/IEx.Pry.html#break/4"><code class="language-plaintext highlighter-rouge">IEx.Pry.break/4</code></a>, but sometimes a <em>nasty hack</em> is way more handy than the <em>proper approach</em>, whatever it means. So I grabbed <em>Elixir</em> source, compiled it with <code class="language-plaintext highlighter-rouge">make clean test</code> and instructed <code class="language-plaintext highlighter-rouge">asdf</code> to use the source distribution locally for the newly created project with a single <code class="language-plaintext highlighter-rouge">hex</code> dependency with <code class="language-plaintext highlighter-rouge">asdf local elixir path:/home/am/Proyectos/elixir</code>.</p>
<hr />
<p>Then I went to aforementioned <code class="language-plaintext highlighter-rouge">Mix.Dep.Lock.write/1</code> and amended it with</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">write</span><span class="p">(</span><span class="n">map</span><span class="p">)</span> <span class="k">do</span>
<span class="o">...</span>
<span class="k">try</span> <span class="k">do</span>
<span class="k">raise</span> <span class="s2">"BOOM"</span>
<span class="k">rescue</span>
<span class="n">_</span> <span class="o">-></span>
<span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="p">({</span><span class="n">map</span><span class="p">,</span> <span class="n">__STACKTRACE__</span><span class="p">},</span> <span class="ss">label:</span> <span class="s2">"Mix.Dep.Lock.write/1"</span><span class="p">)</span>
<span class="k">end</span>
<span class="o">...</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Since <em>OTP 20</em>, <a href="https://hexdocs.pm/elixir/Kernel.SpecialForms.html?#__STACKTRACE__/0"><code class="language-plaintext highlighter-rouge">__STACKTRACE__</code></a> might be used in the context of currently handled exception to get, well, the stacktrace. I could have used <code class="language-plaintext highlighter-rouge">Process.info(self(), :current_stacktrace)</code> instead, but raising an exception gives more flexibility in grokking the behaviour later on (<em>spoiler:</em> it wasn’t necessary.) After <code class="language-plaintext highlighter-rouge">make compile</code> <em>Elixir</em> and <code class="language-plaintext highlighter-rouge">rm -rf mix.lock deps _build && mix deps.get</code> in my new project directory, I got the stack trace</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="p">[</span>
<span class="p">{</span><span class="no">Mix</span><span class="o">.</span><span class="no">Dep</span><span class="o">.</span><span class="no">Lock</span><span class="p">,</span> <span class="ss">:write</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="p">[</span><span class="ss">file:</span> <span class="s1">'lib/mix/dep/lock.ex'</span><span class="p">,</span> <span class="ss">line:</span> <span class="mi">34</span><span class="p">]},</span>
<span class="p">{</span><span class="no">Mix</span><span class="o">.</span><span class="no">Dep</span><span class="o">.</span><span class="no">Fetcher</span><span class="p">,</span> <span class="ss">:do_finalize</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span>
<span class="p">[</span><span class="ss">file:</span> <span class="s1">'lib/mix/dep/fetcher.ex'</span><span class="p">,</span> <span class="ss">line:</span> <span class="mi">106</span><span class="p">]},</span>
<span class="p">{</span><span class="no">Mix</span><span class="o">.</span><span class="no">Dep</span><span class="o">.</span><span class="no">Fetcher</span><span class="p">,</span> <span class="ss">:all</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="p">[</span><span class="ss">file:</span> <span class="s1">'lib/mix/dep/fetcher.ex'</span><span class="p">,</span> <span class="ss">line:</span> <span class="mi">17</span><span class="p">]},</span>
<span class="p">{</span><span class="no">Mix</span><span class="o">.</span><span class="no">Tasks</span><span class="o">.</span><span class="no">Deps</span><span class="o">.</span><span class="no">Get</span><span class="p">,</span> <span class="ss">:run</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="p">[</span><span class="ss">file:</span> <span class="s1">'lib/mix/tasks/deps.get.ex'</span><span class="p">,</span> <span class="ss">line:</span> <span class="mi">31</span><span class="p">]},</span>
<span class="o">...</span>
</code></pre></div></div>
<p>what led me to <code class="language-plaintext highlighter-rouge">Mix.Dep.Converger.converge/4</code> and then to <code class="language-plaintext highlighter-rouge">remote.converge/2</code>. I put another <code class="language-plaintext highlighter-rouge">IO.inspect/2</code> there to confirm my expectation; yes, <code class="language-plaintext highlighter-rouge">remote</code> there had a value <a href="https://github.com/hexpm/hex/blob/v0.20.5/lib/hex/remote_converger.ex"><code class="language-plaintext highlighter-rouge">Hex.RemoteConverger</code></a>.</p>
<p>The mystery was solved in 10 minutes and two <code class="language-plaintext highlighter-rouge">IO.inspect/2</code> in the foreign code.</p>
<p>That said, both <code class="language-plaintext highlighter-rouge">__STACKTRACE__</code> and <code class="language-plaintext highlighter-rouge">Process.info(self(), :current_stacktrace)</code> could tell you more of an alien code structure than all the documentation in the world.</p>
<hr />
<p>Happy inspecting!</p>
Cloister2020-05-07T00:00:00+00:00https://rocket-science.ru/hacking/2020/05/07/cloister<p>Almost each successful business application sooner or later enters the phase when the horizontal scaling is needed. In many cases one might simply launch the new instance and decrease the average load. In most cumbersome cases we must ensure different instances know about each other and distribute the workload gently.</p>
<p><img src="/img/cloister.jpg" alt="Cloister" /></p>
<p>Luckily enough, erlang has a first class support for <a href="http://erlang.org/doc/reference_manual/distributed.html">distributed systems</a>. In theory it sounds as easy as</p>
<blockquote>
<p>Message passing between processes at different nodes, as well as links and monitors, are transparent […]</p>
</blockquote>
<p>In practice distributed erlang was developed when <em>container</em> meant <em>vessel</em> and <em>docker</em> was a shorthand for <em>longshoreman</em>. IP4 had lots of unoccupied addresses, network splits were mostly caused by rats that chewed through the cables and the average uptime of production system was measured in decades.</p>
<p>Now we are all unbridledly self-contain[eriz]ed and running distributed erlang in the environment where IPs are randomly dynamic and nodes might appear and disappear just because scheduler decided to unwind is a bit tricky. To avoid an enormous boilerplate in each project running distributed erlang to deal with a hostile environment, helpers are to be created.</p>
<p><em>Sidenote:</em> I am fully aware of the great <a href="https://github.com/bitwalker/libcluster"><code class="language-plaintext highlighter-rouge">libcluster</code></a> package. If you are fine with what it provides, you should be all set. I am unfortunately not.</p>
<h2 id="demand">Demand</h2>
<p>What I personally needed was the cluster assembly helper that:</p>
<ul>
<li>transparently deals with both hardcoded node list <em>and</em> erlang service discovery</li>
<li>has a callback on each topology change (node up, node down, network split)</li>
<li>provides the transparent interface to run the cluster as <code class="language-plaintext highlighter-rouge">:longnames</code>, <code class="language-plaintext highlighter-rouge">:shortnames</code>, <em>and</em> <code class="language-plaintext highlighter-rouge">:nonode@nohost</code></li>
<li>zero code support for docker</li>
</ul>
<p>The latter means once I have tested the application locally at <code class="language-plaintext highlighter-rouge">:nonode@nohost</code> <em>or</em> in distributed environment with <a href="https://github.com/am-kantox/test_cluster_task"><code class="language-plaintext highlighter-rouge">test_cluster_task</code></a>, I want to run <code class="language-plaintext highlighter-rouge">docker-compose up --scale my_app=3</code> and see it runs three instances in docker without any code change. I also want the dependent applications, like <code class="language-plaintext highlighter-rouge">mnesia</code>, to rehash nodes when the topology changes without any additional handling needed in the application.</p>
<p><em>Cloister</em> does not aim to be a silver bullet, or to cover all the cases possible, or to be academically full in whatever sense computer scientists put into this term. I want it to serve the very narrow purpose, but to serve it in the best way possible. This goal would be: full transparency between local development environment and docker distributed environment.</p>
<h2 id="approach">Approach</h2>
<p><em>Cloister</em> is supposed to be run as the application, although advanced users might deal with the cluster assembly manually by directly starting <a href="https://hexdocs.pm/cloister/Cloister.Manager.html"><code class="language-plaintext highlighter-rouge">Cloister.Manager</code></a> within the target application supervision tree.</p>
<p>When started as an application, it relies on <code class="language-plaintext highlighter-rouge">config</code> to have the following significant values:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">config</span> <span class="ss">:cloister</span><span class="p">,</span>
<span class="ss">otp_app:</span> <span class="ss">:my_app</span><span class="p">,</span>
<span class="ss">sentry:</span> <span class="ss">:"cloister.local"</span><span class="p">,</span> <span class="c1"># or ~w|n1@foo n2@bar|a</span>
<span class="ss">consensus:</span> <span class="mi">3</span><span class="p">,</span> <span class="c1"># number of nodes to consider</span>
<span class="c1"># the cluster is up</span>
<span class="ss">listener:</span> <span class="no">MyApp</span><span class="o">.</span><span class="no">Listener</span> <span class="c1"># listener to be called when</span>
<span class="c1"># the ring has changed</span>
</code></pre></div></div>
<p>Options above means precisely: <em>Cloister is used for <code class="language-plaintext highlighter-rouge">:my_app</code> OTP application, with erlang service discovery to connect nodes, three at least, and <code class="language-plaintext highlighter-rouge">MyApp.Listener</code> to be notified about topology changes</em>. The detailed description of the full configuration might be found <a href="https://hexdocs.pm/cloister/configuration.html">here</a>.</p>
<p>With this <code class="language-plaintext highlighter-rouge">config</code>, <em>Cloister</em> application would <a href="https://hexdocs.pm/elixir/Application.html?#c:start_phase/3">start in phases</a> postponing the application startup process until the consensus is reached (three nodes are up and connected as by the example above.) That helps the main application to assume that when started, the cluster is already available. On each topology change (there would be many because nodes are started not fully syncronously,) <a href="https://hexdocs.pm/cloister/Cloister.Listener.html#c:on_state_change/2"><code class="language-plaintext highlighter-rouge">MyApp.Listener.on_state_change/2</code></a> callback would be called. In most cases we would perform an action when state is <code class="language-plaintext highlighter-rouge">%Cloister.Monitor{status: :up}</code> meaning the cluster as assembled.</p>
<p>In most cases <code class="language-plaintext highlighter-rouge">consensus: 3</code> setting is optimal because even if we expect more nodes to connect, the callback would go through <code class="language-plaintext highlighter-rouge">status: :rehashing</code> → <code class="language-plaintext highlighter-rouge">status: :up</code> loop on any new node added/removed.</p>
<p>When running in development mode, simply put <code class="language-plaintext highlighter-rouge">consensus: 1</code> config value and <em>Cloister</em> would be happy to return for <code class="language-plaintext highlighter-rouge">:nonode@nohost</code>, or <code class="language-plaintext highlighter-rouge">:node@host</code>, or <code class="language-plaintext highlighter-rouge">:node@host.domain</code> depending on how the node was configured (<code class="language-plaintext highlighter-rouge">:none | :shortnames | :longnames</code>.)</p>
<h2 id="managing-distributed-applications">Managing Distributed Applications</h2>
<p>It’s very common for the distributed application to have some dependencies, like <code class="language-plaintext highlighter-rouge">mnesia</code>. It’s easy to handle their reconfiguration from the very same <code class="language-plaintext highlighter-rouge">on_state_change/2</code> callback. Here is a detailed description of how to reconfigure <code class="language-plaintext highlighter-rouge">mnesia</code> on the fly in the <a href="https://hexdocs.pm/cloister/zero-cost-distribution.html#topology-change-listener"><em>Cloister documentation</em></a>.</p>
<p>The main advantage of using <em>Cloister</em> is it performs all the necessary operations to rebuild the cluster after topology changes under the hood. The application simply starts in the already prepared distributed environment, with all the nodes connected, no matter whether we know IPs and hence node names upfront, or they were dynamically assigned/changed. It requires zero docker configuration tweaks and from the point of view of the application developer, there is no difference between running the application in distributed environment, or in local at <code class="language-plaintext highlighter-rouge">:nonode@nohost</code>. The detailed description might be also found in <a href="https://hexdocs.pm/cloister/docker-friendly.html#releases-for-docker-environment"><em>Cloister documentation</em></a>.</p>
<p>While more sophisticated topology changes handling is possible via <code class="language-plaintext highlighter-rouge">MyApp.Listener</code> implementation, there could always be complicated cases where this library limitations and biased approach to configuration would be a show-stopper. That is fine, just pick up the aforementioned <a href="https://github.com/bitwalker/libcluster"><code class="language-plaintext highlighter-rouge">libcluster</code></a>, which is way more generic, or even handle the cluster low-level on your own. The goal of this piece of code is not to cover all the possible scenarios, but to make the most common scenario use like a charm.</p>
<hr />
<p>Happy clustering!</p>
WordCount in Elixir2020-02-22T00:00:00+00:00https://rocket-science.ru/hacking/2020/02/22/wc-in-elixir<p>Half of year ago, Chris Penner had posted <a href="https://chrispenner.ca/posts/wc"><em>Beating C With 80 Lines Of Haskell: Wc</em></a>. The preface states:</p>
<blockquote>
<p>The challenge is to build a <em><strong>faster</strong></em> clone of the hand-optimized <strong>C</strong> implementation of the <code class="language-plaintext highlighter-rouge">wc</code> utility in our favourite high-level garbage-collected runtime-based language: <strong>Haskell</strong>! Sounds simple enough right?</p>
</blockquote>
<p>Down the road Chris went through <em>ByteStrings</em> implementation, <em>Monoids</em>, <em>Inlined Monoids</em>, and, finally came to <em>multicore</em> version of the above that could slightly beat pure <em>C</em> code in execution time on four cores.</p>
<p>Several days ago, there was a related article posted on <a href="https://habr.com/ru/post/489136/">habr</a> (Russian only at the moment, sorry.) The author proved the ability to build an <em>idiomatic Haskell</em> version in 20 (twenty) lines of code that is almost ten times faster than <em>idiomatic C</em> implementation.</p>
<p>Meanwhile I advocate to use <em>Haskell</em> (actually, <a href="https://www.idris-lang.org/"><em>Idris</em></a> for its dependent types) and <a href="https://coq.inria.fr/"><em>Coq</em></a> to indeed prove the critical concepts in our codebase only; the new production code is still coming 100% in Elixir/Erlang/OTP for its fault tolerance. I wanted to make sure I am not shitting my pants here, so I decided to check how can we do with <em>Idiomatic Elixir</em> for the very same task.</p>
<p>Below you’ll see some tricks I used to speed up the execution. I am a grown man and I don’t believe in Tooth fairy anymore, so I was far from the hope that <em>Elixir</em> might actually beat languages compiled into a native machine code. I just wanted to make sure we didn’t fall behind at the finish line by a lap.</p>
<p>I am to use the <a href="http://eforexcel.com/wp/wp-content/uploads/2017/07/1500000%20Sales%20Records.zip">very same test file</a> as <code class="language-plaintext highlighter-rouge">@0xd34df00d</code> used in his experiments. Grab it and glue it with itself ten times.</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% <span class="k">for </span>i <span class="k">in</span> <span class="sb">`</span><span class="nb">seq </span>1 10<span class="sb">`</span><span class="p">;</span> <span class="nb">cat </span>part.txt <span class="o">>></span> test.txt
% <span class="nb">du</span> <span class="nt">-sh</span> test.txt
1.8G test.txt
</code></pre></div></div>
<p>On my machine (8 cores / 16G) it is handled by <code class="language-plaintext highlighter-rouge">wc</code> in 10 seconds approx.</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">time </span><span class="nv">LANG</span><span class="o">=</span>C <span class="nv">LC_ALL</span><span class="o">=</span>C <span class="nb">wc </span>data/test.txt
15000000 44774631 1871822210 data/test.txt
<span class="nv">LANG</span><span class="o">=</span>C <span class="nv">LC_ALL</span><span class="o">=</span>C <span class="nb">wc </span>data/test.txt 9,69s user 0,36s system 99% cpu 10,048 total
</code></pre></div></div>
<p>Let’s see how far behind we’d be with OTP.</p>
<h3 id="naïve-too-naïve-approach">Naïve. Too Naïve Approach</h3>
<p>I started with a most naïve recursive implementation, parsing the stream symbol by a symbol.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">@acc</span> <span class="p">%{</span><span class="k">bc</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span> <span class="ss">wc:</span> <span class="mi">1</span><span class="p">,</span> <span class="k">lc</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span> <span class="ss">ns?:</span> <span class="mi">1</span><span class="p">}</span>
<span class="nv">@type</span> <span class="n">acc</span> <span class="p">::</span> <span class="p">%{</span>
<span class="k">bc</span><span class="p">:</span> <span class="n">non_neg_integer</span><span class="p">(),</span>
<span class="ss">wc:</span> <span class="n">non_neg_integer</span><span class="p">(),</span>
<span class="k">lc</span><span class="p">:</span> <span class="n">non_neg_integer</span><span class="p">(),</span>
<span class="ss">ns?:</span> <span class="mi">0</span> <span class="o">|</span> <span class="mi">1</span>
<span class="p">}</span>
<span class="nv">@spec</span> <span class="n">parse</span><span class="p">(</span><span class="o"><<</span><span class="n">_</span><span class="p">::</span><span class="n">_</span><span class="o">*</span><span class="mi">8</span><span class="o">>></span><span class="p">,</span> <span class="n">acc</span><span class="p">())</span> <span class="p">::</span> <span class="n">acc</span><span class="p">()</span>
<span class="k">def</span> <span class="n">parse</span><span class="p">(</span><span class="s2">""</span><span class="p">,</span> <span class="n">acc</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="n">acc</span>
<span class="k">def</span> <span class="n">parse</span><span class="p">(</span>
<span class="o"><<</span><span class="err">?</span><span class="p">\</span><span class="n">n</span><span class="p">,</span> <span class="n">rest</span><span class="p">::</span><span class="n">binary</span><span class="o">>></span><span class="p">,</span>
<span class="p">%{</span><span class="k">bc</span><span class="p">:</span> <span class="k">bc</span><span class="p">,</span> <span class="ss">wc:</span> <span class="n">wc</span><span class="p">,</span> <span class="k">lc</span><span class="p">:</span> <span class="k">lc</span><span class="p">,</span> <span class="ss">ns?:</span> <span class="n">ns</span><span class="p">}</span>
<span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">parse</span><span class="p">(</span><span class="n">rest</span><span class="p">,</span> <span class="p">%{</span><span class="k">bc</span><span class="p">:</span> <span class="k">bc</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="ss">wc:</span> <span class="n">wc</span> <span class="o">+</span> <span class="n">ns</span><span class="p">,</span> <span class="k">lc</span><span class="p">:</span> <span class="k">lc</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="ss">ns?:</span> <span class="mi">0</span><span class="p">})</span>
<span class="k">def</span> <span class="n">parse</span><span class="p">(</span>
<span class="o"><<</span><span class="err">?</span><span class="p">\</span><span class="n">s</span><span class="p">,</span> <span class="n">rest</span><span class="p">::</span><span class="n">binary</span><span class="o">>></span><span class="p">,</span>
<span class="p">%{</span><span class="k">bc</span><span class="p">:</span> <span class="k">bc</span><span class="p">,</span> <span class="ss">wc:</span> <span class="n">wc</span><span class="p">,</span> <span class="k">lc</span><span class="p">:</span> <span class="k">lc</span><span class="p">,</span> <span class="ss">ns?:</span> <span class="n">ns</span><span class="p">}</span>
<span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">parse</span><span class="p">(</span><span class="n">rest</span><span class="p">,</span> <span class="p">%{</span><span class="k">bc</span><span class="p">:</span> <span class="k">bc</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="ss">wc:</span> <span class="n">wc</span> <span class="o">+</span> <span class="n">ns</span><span class="p">,</span> <span class="k">lc</span><span class="p">:</span> <span class="k">lc</span><span class="p">,</span> <span class="ss">ns?:</span> <span class="mi">0</span><span class="p">})</span>
<span class="k">def</span> <span class="n">parse</span><span class="p">(</span><span class="o"><<</span><span class="n">_</span><span class="p">,</span> <span class="n">rest</span><span class="p">::</span><span class="n">binary</span><span class="o">>></span><span class="p">,</span> <span class="n">acc</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">parse</span><span class="p">(</span><span class="n">rest</span><span class="p">,</span> <span class="p">%{</span><span class="n">acc</span> <span class="o">|</span> <span class="k">bc</span><span class="p">:</span> <span class="n">acc</span><span class="o">.</span><span class="k">bc</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="ss">ns?:</span> <span class="mi">1</span><span class="p">})</span>
</code></pre></div></div>
<p>This function is fed from either greedy <code class="language-plaintext highlighter-rouge">File.read!/1</code>, or lazy <code class="language-plaintext highlighter-rouge">File.stream!/3</code> in the following way.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">@spec</span> <span class="n">lazy</span><span class="p">(</span><span class="n">binary</span><span class="p">)</span> <span class="p">::</span> <span class="n">acc</span><span class="p">()</span><span class="n">actually</span>
<span class="k">def</span> <span class="n">lazy</span><span class="p">(</span><span class="n">file</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">file</span> <span class="o">|></span> <span class="no">File</span><span class="o">.</span><span class="n">stream!</span><span class="p">()</span> <span class="o">|></span> <span class="no">Enum</span><span class="o">.</span><span class="n">reduce</span><span class="p">(</span><span class="nv">@acc</span><span class="p">,</span> <span class="o">&</span><span class="n">parse</span><span class="o">/</span><span class="mi">2</span><span class="p">)</span>
<span class="nv">@spec</span> <span class="n">greedy</span><span class="p">(</span><span class="n">binary</span><span class="p">)</span> <span class="p">::</span> <span class="n">acc</span><span class="p">()</span>
<span class="k">def</span> <span class="n">greedy</span><span class="p">(</span><span class="n">file</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">file</span> <span class="o">|></span> <span class="no">File</span><span class="o">.</span><span class="n">read!</span><span class="p">()</span> <span class="o">|></span> <span class="n">parse</span><span class="p">(</span><span class="nv">@acc</span><span class="p">)</span>
</code></pre></div></div>
<p>As one might expect, the results are too disappointing. I even did not run it on the whole file; I ran it on one tenth part, where <code class="language-plaintext highlighter-rouge">wc</code> had done for less than a second, and our naïve implementaion did more than ten times worse (results below are in μs.)</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">|</span><span class="mi">1</span> <span class="err">▶</span> <span class="ss">:timer</span><span class="o">.</span><span class="n">tc</span> <span class="k">fn</span> <span class="o">-></span> <span class="no">Wc</span><span class="o">.</span><span class="n">lazy</span> <span class="s2">"data/part.txt"</span> <span class="k">end</span>
<span class="c1">#⇒ {16_737094, %{bc: 185682221, lc: 1500000, ns?: 1, wc: 4477464}}</span>
<span class="n">iex</span><span class="o">|</span><span class="mi">2</span> <span class="err">▶</span> <span class="ss">:timer</span><span class="o">.</span><span class="n">tc</span> <span class="k">fn</span> <span class="o">-></span> <span class="no">Wc</span><span class="o">.</span><span class="n">greedy</span> <span class="s2">"data/part.txt"</span> <span class="k">end</span>
<span class="c1">#⇒ {13_659356, %{bc: 187182221, lc: 1500000, ns?: 1, wc: 4477464}}</span>
</code></pre></div></div>
<p>Shall we throw our toolchain away and migrate to <em>Haskell</em> tomorrow? Not yet.</p>
<h3 id="pattern-match-wisely">Pattern Match Wisely</h3>
<p>What if we could count non-empty bytes by chunks? Sure, good question. Let’s generate functions to pattern match next <code class="language-plaintext highlighter-rouge">?\s</code> or <code class="language-plaintext highlighter-rouge">?\n</code> as far from the current point as we can. Looking ahead, I should say that looking ahead way too far makes the code run slower, possibly because of the overhead on the necessity to handle too many functions for no reason (even Finnish words are rarely longer than forty characters.)</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">@prehandle</span> <span class="mi">42</span>
<span class="nv">@sink</span> <span class="nv">@prehandle</span> <span class="o">+</span> <span class="mi">1</span>
<span class="nv">@spec</span> <span class="n">parse</span><span class="p">(</span><span class="o"><<</span><span class="n">_</span><span class="p">::</span><span class="n">_</span><span class="o">*</span><span class="mi">8</span><span class="o">>></span><span class="p">,</span> <span class="n">acc</span><span class="p">())</span> <span class="p">::</span> <span class="n">acc</span><span class="p">()</span>
<span class="no">Enum</span><span class="o">.</span><span class="n">each</span><span class="p">(</span><span class="mi">0</span><span class="o">..</span><span class="nv">@prehandle</span><span class="p">,</span> <span class="k">fn</span> <span class="n">i</span> <span class="o">-></span>
<span class="k">def</span> <span class="n">parse</span><span class="p">(</span>
<span class="o"><<</span><span class="n">_</span><span class="p">::</span><span class="n">binary</span><span class="o">-</span><span class="n">size</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">i</span><span class="p">)),</span> <span class="err">?</span><span class="p">\</span><span class="n">n</span><span class="p">,</span> <span class="n">rest</span><span class="p">::</span><span class="n">binary</span><span class="o">>></span><span class="p">,</span>
<span class="p">%{</span><span class="k">bc</span><span class="p">:</span> <span class="k">bc</span><span class="p">,</span> <span class="ss">wc:</span> <span class="n">wc</span><span class="p">,</span> <span class="k">lc</span><span class="p">:</span> <span class="k">lc</span><span class="p">,</span> <span class="ss">ns?:</span> <span class="n">ns</span><span class="p">}</span>
<span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">parse</span><span class="p">(</span><span class="n">rest</span><span class="p">,</span> <span class="n">acc!</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">i</span><span class="p">),</span> <span class="k">bc</span><span class="p">,</span> <span class="n">wc</span><span class="p">,</span> <span class="k">lc</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="n">ns</span><span class="p">))</span>
<span class="k">def</span> <span class="n">parse</span><span class="p">(</span>
<span class="o"><<</span><span class="n">_</span><span class="p">::</span><span class="n">binary</span><span class="o">-</span><span class="n">size</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">i</span><span class="p">)),</span> <span class="err">?</span><span class="p">\</span><span class="n">s</span><span class="p">,</span> <span class="n">rest</span><span class="p">::</span><span class="n">binary</span><span class="o">>></span><span class="p">,</span>
<span class="p">%{</span><span class="k">bc</span><span class="p">:</span> <span class="k">bc</span><span class="p">,</span> <span class="ss">wc:</span> <span class="n">wc</span><span class="p">,</span> <span class="k">lc</span><span class="p">:</span> <span class="k">lc</span><span class="p">,</span> <span class="ss">ns?:</span> <span class="n">ns</span><span class="p">}</span>
<span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">parse</span><span class="p">(</span><span class="n">rest</span><span class="p">,</span> <span class="n">acc!</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">i</span><span class="p">),</span> <span class="k">bc</span><span class="p">,</span> <span class="n">wc</span><span class="p">,</span> <span class="k">lc</span><span class="p">,</span> <span class="n">ns</span><span class="p">))</span>
<span class="k">end</span><span class="p">)</span>
<span class="k">def</span> <span class="n">parse</span><span class="p">(</span><span class="o"><<</span><span class="n">_</span><span class="p">::</span><span class="n">binary</span><span class="o">-</span><span class="n">size</span><span class="p">(</span><span class="nv">@sink</span><span class="p">),</span> <span class="n">rest</span><span class="p">::</span><span class="n">binary</span><span class="o">>></span><span class="p">,</span> <span class="n">acc</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">parse</span><span class="p">(</span><span class="n">rest</span><span class="p">,</span> <span class="p">%{</span><span class="n">acc</span> <span class="o">|</span> <span class="k">bc</span><span class="p">:</span> <span class="n">acc</span><span class="o">.</span><span class="k">bc</span> <span class="o">+</span> <span class="nv">@sink</span><span class="p">,</span> <span class="ss">ns?:</span> <span class="mi">1</span><span class="p">})</span>
<span class="no">Enum</span><span class="o">.</span><span class="n">each</span><span class="p">(</span><span class="nv">@prehandle</span><span class="o">..</span><span class="mi">0</span><span class="p">,</span> <span class="k">fn</span> <span class="n">i</span> <span class="o">-></span>
<span class="k">def</span> <span class="n">parse</span><span class="p">(</span><span class="o"><<</span><span class="n">_</span><span class="p">::</span><span class="n">binary</span><span class="o">-</span><span class="n">size</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">i</span><span class="p">))</span><span class="o">>></span><span class="p">,</span> <span class="n">acc</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="p">%{</span><span class="n">acc</span> <span class="o">|</span> <span class="k">bc</span><span class="p">:</span> <span class="n">acc</span><span class="o">.</span><span class="k">bc</span> <span class="o">+</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">i</span><span class="p">),</span> <span class="ss">ns?:</span> <span class="mi">1</span><span class="p">}</span>
<span class="k">end</span><span class="p">)</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">acc!</code> above is a syntax sugar macro that shortens this snippet, one might see it in the whole code listing below.</p>
<p>What the heck is happening above and how it it a promised <em>Idiomatic Elixir</em>? Well, it is. During a compilation time, we generate 130 different clauses (43 for matching the <em>next EOL</em>, the same amount to match the next space, handles for the tail <em>and</em> the handle for the list of Welsh and New Zealand’s toponyms</p>
<ul>
<li><a href="https://en.wikipedia.org/wiki/Taumatawhakatangihangakoauauotamateaturipukakapikimaungahoronukupokaiwhenuakitanatahu">Taumatawhakatangihangakoauauotamateaturipukakapikimaungahoronukupokaiwhenuakitanatahu</a>, NZ</li>
<li><a href="https://en.wikipedia.org/wiki/Llanfairpwllgwyngyll">Llanfairpwllgwyngyllgogerychwyrndrobwllllantysiliogogogoch</a>, Wales</li>
</ul>
<p>Let’s see how it performs.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">|</span><span class="mi">1</span> <span class="err">▶</span> <span class="ss">:timer</span><span class="o">.</span><span class="n">tc</span> <span class="k">fn</span> <span class="o">-></span> <span class="no">Wc</span><span class="o">.</span><span class="n">greedy</span> <span class="s2">"data/part.txt"</span> <span class="k">end</span>
<span class="c1">#⇒ {2_569929, %{bc: 187182221, lc: 1500000, ns?: 1, wc: 4477464}}</span>
<span class="n">iex</span><span class="o">|</span><span class="mi">1</span> <span class="err">▶</span> <span class="ss">:timer</span><span class="o">.</span><span class="n">tc</span> <span class="k">fn</span> <span class="o">-></span> <span class="no">Wc</span><span class="o">.</span><span class="n">lazy</span> <span class="s2">"data/part.txt"</span> <span class="k">end</span>
<span class="c1">#⇒ {6_616190, %{bc: 185682221, lc: 1500000, ns?: 1, wc: 4477464}}</span>
</code></pre></div></div>
<p>Well, much better, but it’s still six times longer than the native <code class="language-plaintext highlighter-rouge">wc</code> for a lazy version (and promising 2.5 times only worse for the greedy load.)</p>
<p>Here one could stop, saying I’d trade being 2.5 times slower for the fault tolerance and hot uploads, but we are here not for that. When I started this journey I promised myself I won’t cheat. All that creepy stuff, like <code class="language-plaintext highlighter-rouge">NIF</code>s written in <code class="language-plaintext highlighter-rouge">Rust</code> as it is fashionable nowadays. Pure <em>Elixir</em> code only, no spices. But hey, <em>Erlang/OTP</em> brings concurrency for free, so we might probably use it for free. Unless we need to write some sophisticated monoidal code (that I cannot write anyway) as Chris did in his trip. Luckily enough, everything is already there; welcome <a href="https://hexdocs.pm/flow"><code class="language-plaintext highlighter-rouge">Flow</code></a>.</p>
<h3 id="use-more-than-12-of-what-we-have-paid-for">Use More Than 12% Of What We Have Paid For</h3>
<p>The good news is literally <em>no</em> code changes are needed in our parsing code. There is a new dependency in <code class="language-plaintext highlighter-rouge">mix.exs</code> (I also extended it with <code class="language-plaintext highlighter-rouge">escript</code> generation for the face-to-face test afterward.)</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">project</span> <span class="k">do</span>
<span class="p">[</span>
<span class="ss">app:</span> <span class="ss">:wc</span><span class="p">,</span>
<span class="ss">version:</span> <span class="s2">"0.1.0"</span><span class="p">,</span>
<span class="ss">elixir:</span> <span class="s2">"~> 1.9"</span><span class="p">,</span>
<span class="ss">start_permanent:</span> <span class="no">Mix</span><span class="o">.</span><span class="n">env</span><span class="p">()</span> <span class="o">==</span> <span class="ss">:prod</span><span class="p">,</span>
<span class="ss">escript:</span> <span class="p">[</span><span class="ss">main_module:</span> <span class="no">Wc</span><span class="o">.</span><span class="no">Main</span><span class="p">],</span>
<span class="ss">deps:</span> <span class="p">[{</span><span class="ss">:flow</span><span class="p">,</span> <span class="s2">"~> 1.0"</span><span class="p">}]</span>
<span class="p">]</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Now we need to implement a new function in the main module.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">@chunk</span> <span class="mi">1_000_000</span>
<span class="nv">@spec</span> <span class="n">flowy</span><span class="p">(</span><span class="n">binary</span><span class="p">())</span> <span class="p">::</span> <span class="n">acc</span><span class="p">()</span>
<span class="k">def</span> <span class="n">flowy</span><span class="p">(</span><span class="n">file</span><span class="p">)</span> <span class="k">do</span>
<span class="n">file</span>
<span class="o">|></span> <span class="no">File</span><span class="o">.</span><span class="n">stream!</span><span class="p">([],</span> <span class="nv">@chunk</span><span class="p">)</span>
<span class="o">|></span> <span class="no">Flow</span><span class="o">.</span><span class="n">from_enumerable</span><span class="p">()</span>
<span class="o">|></span> <span class="no">Flow</span><span class="o">.</span><span class="n">partition</span><span class="p">()</span>
<span class="o">|></span> <span class="no">Flow</span><span class="o">.</span><span class="n">reduce</span><span class="p">(</span><span class="k">fn</span> <span class="o">-></span> <span class="nv">@acc</span> <span class="k">end</span><span class="p">,</span> <span class="o">&</span><span class="n">parse</span><span class="o">/</span><span class="mi">2</span><span class="p">)</span>
<span class="o">|></span> <span class="no">Enum</span><span class="o">.</span><span class="n">to_list</span><span class="p">()</span>
<span class="k">end</span>
</code></pre></div></div>
<p>I have heuristically (by randomly picking different numbers) discovered, than the optimal chunk would be somewhat around several megabytes, so I choosed chunks of the size <code class="language-plaintext highlighter-rouge">1M</code>. The results differ insignificantly.</p>
<p>Let’s test it!</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">|</span><span class="mi">1</span> <span class="err">▶</span> <span class="ss">:timer</span><span class="o">.</span><span class="n">tc</span> <span class="k">fn</span> <span class="o">-></span> <span class="no">Wc</span><span class="o">.</span><span class="n">flowy</span> <span class="s2">"data/part.txt"</span> <span class="k">end</span>
<span class="c1">#⇒ {0_752568, %{bc: 187182221, lc: 1500000, ns?: 1, wc: 4477464}}</span>
<span class="n">iex</span><span class="o">|</span><span class="mi">2</span> <span class="err">▶</span> <span class="ss">:timer</span><span class="o">.</span><span class="n">tc</span> <span class="k">fn</span> <span class="o">-></span> <span class="no">Wc</span><span class="o">.</span><span class="n">flowy</span> <span class="s2">"data/test.txt"</span> <span class="k">end</span>
<span class="c1">#⇒ {7_815592, %{bc: 1871822210, lc: 15000000, ns?: 1, wc: 44774631}}</span>
</code></pre></div></div>
<p>Wow. The result is as impressive that I even ran it for the whole <code class="language-plaintext highlighter-rouge">1.8G</code> file. Yes, we indeed were <em>faster</em> than <code class="language-plaintext highlighter-rouge">wc</code> on that contrived, grown in hothouse, showing nothing and proving nothing, example. Save for now we are more or less sure <em>Elixir/OTP</em> is <em>fast enough</em> for our purposes, even compared to compiled into native code languages. I also ran <code class="language-plaintext highlighter-rouge">mix escript.build</code> and finally clean-compared results in the same race.</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">time </span><span class="nv">LANG</span><span class="o">=</span>C <span class="nv">LC_ALL</span><span class="o">=</span>C <span class="nb">wc </span>data/test.txt
15000000 44774631 1871822210 data/test.txt
<span class="nv">LANG</span><span class="o">=</span>C <span class="nv">LC_ALL</span><span class="o">=</span>C <span class="nb">wc </span>data/test.txt 9,71s user 0,39s system 99% cpu 10,104 total
<span class="nb">time</span> ./wc data/test.txt
15000000 44774631 1871822210 data/test.txt
./wc data/test.txt 41,72s user 2,31s system 706% cpu 6,355 total
</code></pre></div></div>
<p>Almost twice as fast in total.</p>
<hr />
<p>Below is the whole code (including <code class="language-plaintext highlighter-rouge">main</code> implementation for <code class="language-plaintext highlighter-rouge">escript</code>) in a case one would want to experiment with.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">Wc</span> <span class="k">do</span>
<span class="nv">@acc</span> <span class="p">%{</span><span class="k">bc</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span> <span class="ss">wc:</span> <span class="mi">1</span><span class="p">,</span> <span class="k">lc</span><span class="p">:</span> <span class="mi">0</span><span class="p">,</span> <span class="ss">ns?:</span> <span class="mi">1</span><span class="p">}</span>
<span class="nv">@prehandle</span> <span class="mi">42</span>
<span class="nv">@sink</span> <span class="nv">@prehandle</span> <span class="o">+</span> <span class="mi">1</span>
<span class="nv">@chunk</span> <span class="mi">1_000_000</span>
<span class="nv">@type</span> <span class="n">acc</span> <span class="p">::</span> <span class="p">%{</span>
<span class="k">bc</span><span class="p">:</span> <span class="n">non_neg_integer</span><span class="p">(),</span>
<span class="ss">wc:</span> <span class="n">non_neg_integer</span><span class="p">(),</span>
<span class="k">lc</span><span class="p">:</span> <span class="n">non_neg_integer</span><span class="p">(),</span>
<span class="ss">ns?:</span> <span class="mi">0</span> <span class="o">|</span> <span class="mi">1</span>
<span class="p">}</span>
<span class="nv">@spec</span> <span class="n">lazy</span><span class="p">(</span><span class="n">binary</span><span class="p">)</span> <span class="p">::</span> <span class="n">acc</span><span class="p">()</span>
<span class="k">def</span> <span class="n">lazy</span><span class="p">(</span><span class="n">file</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">file</span> <span class="o">|></span> <span class="no">File</span><span class="o">.</span><span class="n">stream!</span><span class="p">()</span> <span class="o">|></span> <span class="no">Enum</span><span class="o">.</span><span class="n">reduce</span><span class="p">(</span><span class="nv">@acc</span><span class="p">,</span> <span class="o">&</span><span class="n">parse</span><span class="o">/</span><span class="mi">2</span><span class="p">)</span>
<span class="nv">@spec</span> <span class="n">greedy</span><span class="p">(</span><span class="n">binary</span><span class="p">)</span> <span class="p">::</span> <span class="n">acc</span><span class="p">()</span>
<span class="k">def</span> <span class="n">greedy</span><span class="p">(</span><span class="n">file</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">file</span> <span class="o">|></span> <span class="no">File</span><span class="o">.</span><span class="n">read!</span><span class="p">()</span> <span class="o">|></span> <span class="n">parse</span><span class="p">(</span><span class="nv">@acc</span><span class="p">)</span>
<span class="nv">@spec</span> <span class="n">flowy</span><span class="p">(</span><span class="n">binary</span><span class="p">)</span> <span class="p">::</span> <span class="n">acc</span><span class="p">()</span>
<span class="k">def</span> <span class="n">flowy</span><span class="p">(</span><span class="n">file</span><span class="p">)</span> <span class="k">do</span>
<span class="n">kw</span> <span class="o">=</span>
<span class="n">file</span>
<span class="o">|></span> <span class="no">File</span><span class="o">.</span><span class="n">stream!</span><span class="p">([],</span> <span class="nv">@chunk</span><span class="p">)</span>
<span class="o">|></span> <span class="no">Flow</span><span class="o">.</span><span class="n">from_enumerable</span><span class="p">()</span>
<span class="o">|></span> <span class="no">Flow</span><span class="o">.</span><span class="n">partition</span><span class="p">()</span>
<span class="o">|></span> <span class="no">Flow</span><span class="o">.</span><span class="n">reduce</span><span class="p">(</span><span class="k">fn</span> <span class="o">-></span> <span class="nv">@acc</span> <span class="k">end</span><span class="p">,</span> <span class="o">&</span><span class="n">parse</span><span class="o">/</span><span class="mi">2</span><span class="p">)</span>
<span class="o">|></span> <span class="no">Enum</span><span class="o">.</span><span class="n">to_list</span><span class="p">()</span>
<span class="n">m</span> <span class="o">=</span>
<span class="p">[</span><span class="ss">:bc</span><span class="p">,</span> <span class="ss">:lc</span><span class="p">,</span> <span class="ss">:wc</span><span class="p">,</span> <span class="ss">:ns?</span><span class="p">]</span>
<span class="o">|></span> <span class="no">Enum</span><span class="o">.</span><span class="n">into</span><span class="p">(%{},</span> <span class="o">&</span><span class="p">{</span><span class="nv">&1</span><span class="p">,</span> <span class="no">Keyword</span><span class="o">.</span><span class="n">get_values</span><span class="p">(</span><span class="n">kw</span><span class="p">,</span> <span class="nv">&1</span><span class="p">)</span> <span class="o">|></span> <span class="no">Enum</span><span class="o">.</span><span class="n">sum</span><span class="p">()})</span>
<span class="n">m</span>
<span class="o">|></span> <span class="no">Map</span><span class="o">.</span><span class="n">update!</span><span class="p">(</span><span class="ss">:wc</span><span class="p">,</span> <span class="o">&</span><span class="p">(</span><span class="nv">&1</span> <span class="o">+</span> <span class="mi">1</span> <span class="o">-</span> <span class="n">m</span><span class="o">.</span><span class="n">ns?</span><span class="p">))</span>
<span class="o">|></span> <span class="no">Map</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="ss">:ns?</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">defmacrop</span> <span class="n">ns!</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">ns</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="n">ns</span>
<span class="k">defmacrop</span> <span class="n">ns!</span><span class="p">(</span><span class="n">_</span><span class="p">,</span> <span class="n">_</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="mi">1</span>
<span class="k">defmacro</span> <span class="n">acc!</span><span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="k">bc</span><span class="p">,</span> <span class="n">wc</span><span class="p">,</span> <span class="k">lc</span><span class="p">,</span> <span class="n">ns</span><span class="p">)</span> <span class="k">do</span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="p">%{</span>
<span class="k">bc</span><span class="p">:</span> <span class="kn">unquote</span><span class="p">(</span><span class="k">bc</span><span class="p">)</span> <span class="o">+</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">i</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span>
<span class="ss">wc:</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">wc</span><span class="p">)</span> <span class="o">+</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">ns</span><span class="p">),</span>
<span class="k">lc</span><span class="p">:</span> <span class="kn">unquote</span><span class="p">(</span><span class="k">lc</span><span class="p">),</span>
<span class="ss">ns?:</span> <span class="n">ns!</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">i</span><span class="p">),</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">ns</span><span class="p">))</span>
<span class="p">}</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="nv">@spec</span> <span class="n">parse</span><span class="p">(</span><span class="o"><<</span><span class="n">_</span><span class="p">::</span><span class="n">_</span><span class="o">*</span><span class="mi">8</span><span class="o">>></span><span class="p">,</span> <span class="n">acc</span><span class="p">())</span> <span class="p">::</span> <span class="n">acc</span><span class="p">()</span>
<span class="no">Enum</span><span class="o">.</span><span class="n">each</span><span class="p">(</span><span class="mi">0</span><span class="o">..</span><span class="nv">@prehandle</span><span class="p">,</span> <span class="k">fn</span> <span class="n">i</span> <span class="o">-></span>
<span class="k">def</span> <span class="n">parse</span><span class="p">(</span>
<span class="o"><<</span><span class="n">_</span><span class="p">::</span><span class="n">binary</span><span class="o">-</span><span class="n">size</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">i</span><span class="p">)),</span> <span class="err">?</span><span class="p">\</span><span class="n">n</span><span class="p">,</span> <span class="n">rest</span><span class="p">::</span><span class="n">binary</span><span class="o">>></span><span class="p">,</span>
<span class="p">%{</span><span class="k">bc</span><span class="p">:</span> <span class="k">bc</span><span class="p">,</span> <span class="ss">wc:</span> <span class="n">wc</span><span class="p">,</span> <span class="k">lc</span><span class="p">:</span> <span class="k">lc</span><span class="p">,</span> <span class="ss">ns?:</span> <span class="n">ns</span><span class="p">}</span>
<span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">parse</span><span class="p">(</span><span class="n">rest</span><span class="p">,</span> <span class="n">acc!</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">i</span><span class="p">),</span> <span class="k">bc</span><span class="p">,</span> <span class="n">wc</span><span class="p">,</span> <span class="k">lc</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="n">ns</span><span class="p">))</span>
<span class="k">def</span> <span class="n">parse</span><span class="p">(</span>
<span class="o"><<</span><span class="n">_</span><span class="p">::</span><span class="n">binary</span><span class="o">-</span><span class="n">size</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">i</span><span class="p">)),</span> <span class="err">?</span><span class="p">\</span><span class="n">s</span><span class="p">,</span> <span class="n">rest</span><span class="p">::</span><span class="n">binary</span><span class="o">>></span><span class="p">,</span>
<span class="p">%{</span><span class="k">bc</span><span class="p">:</span> <span class="k">bc</span><span class="p">,</span> <span class="ss">wc:</span> <span class="n">wc</span><span class="p">,</span> <span class="k">lc</span><span class="p">:</span> <span class="k">lc</span><span class="p">,</span> <span class="ss">ns?:</span> <span class="n">ns</span><span class="p">}</span>
<span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">parse</span><span class="p">(</span><span class="n">rest</span><span class="p">,</span> <span class="n">acc!</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">i</span><span class="p">),</span> <span class="k">bc</span><span class="p">,</span> <span class="n">wc</span><span class="p">,</span> <span class="k">lc</span><span class="p">,</span> <span class="n">ns</span><span class="p">))</span>
<span class="k">end</span><span class="p">)</span>
<span class="k">def</span> <span class="n">parse</span><span class="p">(</span><span class="o"><<</span><span class="n">_</span><span class="p">::</span><span class="n">binary</span><span class="o">-</span><span class="n">size</span><span class="p">(</span><span class="nv">@sink</span><span class="p">),</span> <span class="n">rest</span><span class="p">::</span><span class="n">binary</span><span class="o">>></span><span class="p">,</span> <span class="n">acc</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">parse</span><span class="p">(</span><span class="n">rest</span><span class="p">,</span> <span class="p">%{</span><span class="n">acc</span> <span class="o">|</span> <span class="k">bc</span><span class="p">:</span> <span class="n">acc</span><span class="o">.</span><span class="k">bc</span> <span class="o">+</span> <span class="nv">@sink</span><span class="p">,</span> <span class="ss">ns?:</span> <span class="mi">1</span><span class="p">})</span>
<span class="no">Enum</span><span class="o">.</span><span class="n">each</span><span class="p">(</span><span class="nv">@prehandle</span><span class="o">..</span><span class="mi">0</span><span class="p">,</span> <span class="k">fn</span> <span class="n">i</span> <span class="o">-></span>
<span class="k">def</span> <span class="n">parse</span><span class="p">(</span><span class="o"><<</span><span class="n">_</span><span class="p">::</span><span class="n">binary</span><span class="o">-</span><span class="n">size</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">i</span><span class="p">))</span><span class="o">>></span><span class="p">,</span> <span class="n">acc</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="p">%{</span><span class="n">acc</span> <span class="o">|</span> <span class="k">bc</span><span class="p">:</span> <span class="n">acc</span><span class="o">.</span><span class="k">bc</span> <span class="o">+</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">i</span><span class="p">),</span> <span class="ss">ns?:</span> <span class="mi">1</span><span class="p">}</span>
<span class="k">end</span><span class="p">)</span>
<span class="k">defmodule</span> <span class="no">Main</span> <span class="k">do</span>
<span class="k">defdelegate</span> <span class="n">lazy</span><span class="p">(</span><span class="n">file</span><span class="p">),</span> <span class="ss">to:</span> <span class="no">Wc</span>
<span class="k">defdelegate</span> <span class="n">greedy</span><span class="p">(</span><span class="n">file</span><span class="p">),</span> <span class="ss">to:</span> <span class="no">Wc</span>
<span class="k">defdelegate</span> <span class="n">flowy</span><span class="p">(</span><span class="n">file</span><span class="p">),</span> <span class="ss">to:</span> <span class="no">Wc</span>
<span class="nv">@spec</span> <span class="n">main</span><span class="p">([</span><span class="n">binary</span><span class="p">()])</span> <span class="p">::</span> <span class="n">any</span>
<span class="k">def</span> <span class="n">main</span><span class="p">([]),</span> <span class="k">do</span><span class="p">:</span> <span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="p">(</span><span class="s2">"Usage: wc file _or_ wc method file"</span><span class="p">)</span>
<span class="k">def</span> <span class="n">main</span><span class="p">([</span><span class="n">file</span><span class="p">]),</span> <span class="k">do</span><span class="p">:</span> <span class="n">main</span><span class="p">([</span><span class="s2">"flowy"</span><span class="p">,</span> <span class="n">file</span><span class="p">])</span>
<span class="k">def</span> <span class="n">main</span><span class="p">([</span><span class="n">m</span><span class="p">,</span> <span class="n">file</span><span class="p">])</span> <span class="k">do</span>
<span class="p">%{</span><span class="k">bc</span><span class="p">:</span> <span class="k">bc</span><span class="p">,</span> <span class="ss">wc:</span> <span class="n">wc</span><span class="p">,</span> <span class="k">lc</span><span class="p">:</span> <span class="k">lc</span><span class="p">}</span> <span class="o">=</span> <span class="n">apply</span><span class="p">(</span><span class="no">Wc</span><span class="p">,</span> <span class="no">String</span><span class="o">.</span><span class="n">to_existing_atom</span><span class="p">(</span><span class="n">m</span><span class="p">),</span> <span class="p">[</span><span class="n">file</span><span class="p">])</span>
<span class="p">[</span><span class="s2">""</span><span class="p">,</span> <span class="k">lc</span><span class="p">,</span> <span class="n">wc</span><span class="p">,</span> <span class="k">bc</span><span class="p">,</span> <span class="n">file</span><span class="p">]</span>
<span class="o">|></span> <span class="no">Enum</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="o"><<</span><span class="err">?</span><span class="p">\</span><span class="n">t</span><span class="o">>></span><span class="p">)</span>
<span class="o">|></span> <span class="no">IO</span><span class="o">.</span><span class="n">puts</span><span class="p">()</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<hr />
<p>Happy wordcounting!</p>
Not God but Man Makes Pot and Pan2020-02-07T00:00:00+00:00https://rocket-science.ru/hacking/2020/02/07/not-god-but-man-makes-pot-and-pan<h3 id="on-the-verge-of-creating-a-new-project">On the Verge of Creating a New Project</h3>
<p>Let’s pretend the business driving our small, but free and proud tech team claims for a new feature. We are smart enough to not reinvent the wheel from scratch, so we go on a deep internet trip to discover what was successfully built under the similar requirements by our mate developers. It turns out, the existing solutions coverage is shallow, vague and contradictory.</p>
<p><img src="/img/niza_silla.jpg" alt="Silla Niza" /></p>
<p>There is a huge load of articles, with nearly 50-50 ratio, sharing both success stories and drastic failures. Some advocate for using <em>Foo</em> over <em>Bar</em>, while others ensure to pick up <em>Baz</em> on top of <em>Boo</em> and avoid <em>Foo</em> at any cost. We filter out the news from the sources we have never heard about and end up with some advice coming from a trustful one. Which is either a proven authority because of a dozen written books and courses, or is simply too big to fail. Like, you know, <em>Goozone Hawk Co.</em> could not get it wrong.</p>
<p>This is the authority bias as it is. Hype does spread both the correct and the very wrong ideas equally fast. I do not get why this white-bearded guy could not be wrong. I need my own opinion.</p>
<h3 id="gimme-a-fulcrum-and-a-lever">Gimme a Fulcrum and a Lever</h3>
<p>Since several years ago I have been approaching every single problem with writing my own code. Well, <em>almost</em> every single problem. I am not going to rewrite the operating system, the database engine, or the container manager. But if it looks like a one-week-stand and quacks like a one-week-stand, I would enter the same river again.</p>
<blockquote>
<p>What do we need to implement this time? A sophisticated monitoring system to prove we are fifteen-nines-compatible.</p>
</blockquote>
<p>We are running a mission-critical system. Nah, I understand that each business considers their mission being critical, but here in <em>Fintech</em> it’s a bit more of a truth. I do recall one talk at a conference about highload system (from the company which indeed serves gazillions requests per a second,) when the speaker bragged about they fail to respond to less than one pro mille requests. In our case the allowed number of requests we can fail on in exactly zero.</p>
<p>Besides that, we deal with many third-party services what makes us vulnerable to the failures provoked by <em>unpredictable</em> and, in general, <em>unknown</em> sources. The single “transaction” might last hours since it requires approval by many external authorities before real execution. We are kinda experts in talking <em>a bitty language</em>; all these negotiations are done purely machinery-to-machinery.</p>
<p>That list might be continued for three screens more, including, but not limited to:</p>
<ul>
<li>potential need of manual intervention</li>
<li>high concurrency of many different processes</li>
<li>lots of highload queues to listen to to get the actual data for different matters</li>
<li>ability to roll back anything</li>
<li>necessity to audit every single step both internally and in another 3rd-party service</li>
<li>orchestrating all that zoo to make sure we have it <em>paid before shipped</em></li>
</ul>
<p>We obviously have the scraps of monitoring inplace, and I bet most internet businesses in other areas would consider our monitoring bullet-proof, but we cannot blindly trust in <em>Unicorns and Pink Fairies</em> keeping us safe.</p>
<p>OK. We need <em>a fulcrum and a query</em> so we could ask internets what path should we follow to move the Earth toward the right direction. Unfortunately it was easier said than done. And here I indulged my sin. I dove to…</p>
<h3 id="inventing-the-wheel">Inventing the Wheel</h3>
<h4 id="what-do-we-have">What Do We Have</h4>
<p><img src="/img/franz.png" alt="Current Architecture" /></p>
<p>All services on the left side run in the cloud, some have several instances in parallel, some have several instances in a cluster, some are singletons. The whole idea was born as <strong>let’s have a better visibility</strong> and has evolved into <strong>we might build the <em>Event Log</em> that would serve the purpose of tracking all the changes and providing rollback abilities in case of failures</strong>.</p>
<p>So, how would we approach this?</p>
<h4 id="pair-monitoring">Pair Monitoring</h4>
<p>Do you remember how have you been excited exercising the pair programming technique for the first time? It was an amazing feeling, was not it? That reminiscence granted me an idea to do pair monitoring. Not only the containers infrastructure health-checks applications management <em>and</em> the applications infrastructure health-checks the business logic, but the play as a team, reporting issues back and forth.</p>
<h4 id="say-no-to-transactions">Say ‘No’ to Transactions</h4>
<p>There should be no transactions. Only atomic operations are supported. Rolling back a transaction on the application level would be <em>an ad-hoc, informally-specified, bug-ridden, slow implementation</em> of half of the database engine and we agreed to avoid reimplementing monsters.</p>
<h4 id="keep-track-of-literally-everything">Keep Track of Literally Everything</h4>
<p>Despite the existence of long-lived ‘transactional’ actions in our environment, we found that we might split everything into atomic actions driven by <a href="https://en.wikipedia.org/wiki/Finite-state_machine">finite-state automata</a>. Instead of having a transaction, involving changes of several different objects, we introduce another object, which plays the role of the <em>transaction fsm</em>. During the lifecycle of this object, all the underlying objects are marked as <em>in transaction</em>. Once it gets to commit, all the changes are applied simultaneously, generating several lines in the event log, one per an atomic action. That way we always have an ability to rollback anything afterwards.</p>
<p>Also, there are some actions that cannot be rolled back (e. g. the actual money transfer.) We clearly mark them in the event log <em>and</em> for each of such actions we introduce the ‘rollback’ object, that does the action that is mimicking the real life rollback (e. g. transfer the same amount backward for the contrived example above.)</p>
<h4 id="orchestration-vs-choreography">Orchestration vs Choreography</h4>
<p>These two approaches are usually mentioned as mutually exclusive. We discovered it is not the case at all. One might encapsulate parts of the whole system according to <a href="https://en.wikipedia.org/wiki/Single_responsibility_principle"><em>SRP</em></a> into <em>orchestrated</em> beasts, who play well in the event-driven choreography and vice versa.</p>
<p>We also need to be able to put <em>boundaries</em> on the imaginary diagram representing our architecture at a glance. It would look mostly like a political map of the world. Boundaries cannot cross over. The states within boundaries are responsible for rules inside the state. They literally have their own goverments, laws, whatever. Communication inside the single boundary might be done as the state laws dictate; the communication between different states is done via diplomatic mail; that said, with message passing.</p>
<p><img src="/img/boundaries.png" alt="Boundaries" /></p>
<p>All the messages have topics; whatever state is interested in the topic should explicitly subscribe to it. Whether there are several instances of the same service running in the cluster, they should decide how to handle and share the arriving messages amongst nodes (the easiest and most robust implementation would be the <code class="language-plaintext highlighter-rouge">hashring</code>-based one.)</p>
<h4 id="ack-please">Ack, Please</h4>
<p>All the messages are to be acked upon processing. The receiver of the acknowledgement is not necessarily the service who sent the original message. Acks might be chained. The event log (who is subscribed to everything) keeps track of all the acks and alerts in the case there are some unacked messages.</p>
<h4 id="play--rewind">Play / Rewind</h4>
<p>Event logs might be used to reproduce failing (suspicious) scenarios in <em>staging</em> later. This part is kinda complicated and we are not going to implement it right now, but the whole architecture is built with this ability in mind.</p>
<h4 id="chaos-testing">Chaos Testing</h4>
<p>The whole system is to be tested by sending unexpected and corrupted messages, by plugging and unplugging service instances, by mimicking network splits and remote services failures.</p>
<p>The above is the list of requirements, that is to be constantly expanded.</p>
<hr />
<h3 id="standing-on-the-shoulders-of-giants">Standing on the Shoulders of Giants</h3>
<p>I have to say, that in a vast majority of cases, wheels invented by me during the problem analyzing stage were born squared, spokeless and fragile. I was throwing them directly into the trashcan with an easy heart to pick up one of mature, proven by community battles, solutions. But not at that time.</p>
<p>We have examined similar attempts done by Netflix (<a href="https://netflixtechblog.com/netflix-conductor-a-microservices-orchestrator-2e8d4771bf40">Orchestrator</a>, <a href="https://netflixtechblog.com/dblog-a-generic-change-data-capture-framework-69351fb9099b">Data Capture</a>,) Sharp End (<a href="https://sharpend.io/a-little-story-about-amazon-ecs-systemd-and-chaos-monkey/">AWS + Chaos Monkey</a>,) <a href="https://www.openpolicyagent.org/">OpenPolicyAgent</a>, many built-in cloud provider’s infrastructure tools, but the song remained the same.</p>
<p><img src="/img/ledzepsongremainsthesame.jpg" alt="Led Zeppelin → The Song Remains The Same" /></p>
<p>It sounded like we do need our own, invented here, solution.</p>
<h3 id="conclusion">Conclusion</h3>
<p>The bigger the company, the more failures they experience on each step. Do not blindly follow advices and good practices published by authorities. Use the time dedicated to problem analysis wisely. Learn on your own mistakes before picking up one of the proper solutions presented on the market. Or, rarely but surprisingly satisfactory, see for yourself, that your wheel trundles smoothly and is indeed better than whatever was created by humanity before. It’d be a great time to enjoy being a smart developer.</p>
<p>Happy wheelinventing!</p>
Strong Types Should Have Been Named Strong Hypes2020-01-18T00:00:00+00:00https://rocket-science.ru/hacking/2020/01/18/strong-hypes<p>Nowadays there is a tendency all around the internets to enforce the goodness of strong typing. Most stubborn rhetoricians even come to the conclusions like “when you have types you don’t need tests.” I believe, this is simply not true. There are, obviouusly, some circumstances, when strong typing might help to catch human mistakes at early stages, but the boilerplate needed when strong typing is implied everywhere brings more hassle than help.</p>
<p>Please note, that all the below is the humble opinion of not so smart human being. But it based on over 30 years of professional development career providing robust and fault-tolerant services. Also, I welcome every strong typing evangelist to write a <em>markdown parser</em> to see the whole field where Types literally suck.</p>
<p>The typical argument of strong typing evangelists would be “let’s assume refactoring.” When during the process one attempts to call <code class="language-plaintext highlighter-rouge">get_user_by_id</code> function passing a user instance instead of an integer identifier, the compiler would complain and the error would not leak into runtime. And that’s true! But you know what? I am writing code for several decades and I never expected anything to literally develop the whole application for me. I feel kinda responsible to avoid passing crap to function arguments. “But it would be great to minimize your effort on that?”—I literally hear the arguing. Well, yes and no.</p>
<p>Assigning the identifier to something that is intended to be used as an instance is surely not the hardest to debug error. Assigning the user instance to what is supposed to be a timestamp means one needs more focus, not stronger types. The issues that are really hard to catch and eliminate hide in other swamp. Let’s consired some examples (I would intentionally use pseudo language to avoid the inescapable fuss and fury of things religious.)</p>
<h3 id="following-the-wrong-path">Following the Wrong Path</h3>
<p>Let’s imagine we have a function that handles <em>HTTP</em> calls to the third party service. We are to refactor it to add more careful logging and cover different response types. We had somewhat like below.</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">function</span> <span class="nx">call_3rd_party</span><span class="p">(</span><span class="nx">uri</span><span class="p">)</span> <span class="p">{</span>
<span class="k">case</span> <span class="nx">call</span><span class="p">(</span><span class="nx">uri</span><span class="p">)</span> <span class="p">{</span>
<span class="mi">200</span> <span class="o">=></span> <span class="nx">handle_success</span><span class="p">()</span>
<span class="k">else</span> <span class="o">=></span> <span class="nx">handle_error</span><span class="p">()</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Now we want to log the falsey path.</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">function</span> <span class="nx">call_3rd_party</span><span class="p">(</span><span class="nx">uri</span><span class="p">)</span> <span class="p">{</span>
<span class="k">case</span> <span class="nx">call</span><span class="p">(</span><span class="nx">uri</span><span class="p">)</span> <span class="p">{</span>
<span class="mi">200</span> <span class="o">=></span> <span class="nx">handle_success</span><span class="p">()</span>
<span class="mi">500</span> <span class="o">=></span> <span class="nx">handle_error</span><span class="p">(</span><span class="nx">UNAVAILABLE_SERVER_ERROR</span><span class="p">)</span>
<span class="mi">503</span> <span class="o">=></span> <span class="nx">handle_error</span><span class="p">(</span><span class="nx">INTERNAL_SERVER_ERROR</span><span class="p">)</span>
<span class="k">else</span> <span class="o">=></span> <span class="nx">handle_error</span><span class="p">(</span><span class="nx">UNKNOWN</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Can you grasp the issue? Sure you can, we messed up error codes. Can any super sophisticated typing system do that for us? Unfortunately not.</p>
<h3 id="getting-to-the-wrong-record">Getting to the Wrong Record</h3>
<p>Let’s say we have a <code class="language-plaintext highlighter-rouge">users</code> database table we are to look up. We end up with the function that looks pretty clean.</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">function</span> <span class="nx">get_user</span><span class="p">(</span><span class="nx">name</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">db_load_user_or_die</span><span class="p">(</span><span class="nx">name</span><span class="p">)</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Even having <code class="language-plaintext highlighter-rouge">name</code> above to be a hundred times strong typed as <code class="language-plaintext highlighter-rouge">string</code>, passing <em>surname</em> instead of the <em>first name</em> will make it crash.</p>
<h3 id="dependent-types-and-formal-proof">Dependent Types and Formal Proof</h3>
<p>Yes, in <a href="https://www.idris-lang.org/"><code class="language-plaintext highlighter-rouge">Idris</code></a>, <a href="https://wiki.portal.chalmers.se/agda/pmwiki.php"><code class="language-plaintext highlighter-rouge">Agda</code></a>, and family one might build the proof on top of the <a href="https://en.m.wikipedia.org/wiki/Category_theory"><code class="language-plaintext highlighter-rouge">Category theory</code></a>, that the outcome of the operation would be the <em>correct</em> one. That sounds somewhat magical, but it’s true.</p>
<p>The above does not mean we yielded a silver bullet of development; we simply shifted the problem down the chain. Now instead of writing the correct code in language <em>Foonctional</em>, we are to make no mistake in language <em>Typoo</em>. Which is in many cases harder. The completeness, while is mandatory in axiomatics, might be a real show stopper in the productive development process. Consider the following code, showing the author name next to the blog post.</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">function</span> <span class="nx">show_author</span><span class="p">(</span><span class="nx">name</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span> <span class="nx">get_user</span><span class="p">(</span><span class="nx">name</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">show</span><span class="p">(</span><span class="nx">name</span><span class="p">)</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="nx">log</span><span class="p">(</span><span class="dl">"</span><span class="s2">Something wrong with </span><span class="dl">"</span> <span class="o">+</span> <span class="nx">user</span><span class="p">)</span>
<span class="nx">show</span><span class="p">(</span><span class="nx">ADMIN</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>It’s a representation of the user. If something has failed, we should not actually bother. We ar fine discarding the error here, all we need would be to show <em>the damn post</em>. Could not get the name of the author?—In many cases this is not critical. The main goal would be still to show the content; we are to do that at any cost. With strong types it would not even compile. And we will be put into a position to fix the database, to update all the empty authors, or to invent a new monadic type for the <em>maybe</em> present author.</p>
<p>Yeah, the latter is <em>more robust</em>. Yeah, the example is rather contrived. But with my conditional approach it would be in prod in one minute. Ask business, what’s better?</p>
<h3 id="panacea-for-everything">Panacea for Everything</h3>
<p>One of the top reasonable arguments for using strong types is “I don’t need to read the code to understand what that function does, I can check types.” C’mon, it’s 2020. Good boys do write and support <em>the documentation</em>, with proper <em>doctests</em>. Schedule a meeting, invite all your mates and ask them to vote what they prefer to read: types, or documentation.</p>
<p>And yes, types in proper languages <em>are included</em> into documentation. The reasoning that it might become obsolete is absurd. The code might also become obsolete. Just make documentation a first class citizen in your codebase, it’s absolutely not that hard. Reject CRs with anything left undocumented. Write docs for your own better understanding of what are you actually implementing. In plain English that is still better understood by human beings than cathegory theory.</p>
<h3 id="can-you-stop-whining-and-suggest-something">Can You Stop Whining and Suggest Something</h3>
<p>I could have been giving examples of code where strong typing would not help forever. But sure, I can instead propose the ready-to-go prescription. It probably won’t work for everyone, but it’s definitely more robust than the blind faith in <em>Strong Typing Savior</em>.</p>
<p>An external, not built into the compiler, tool for static checking and code analysis might help. Pattern matching whenever applicable in function clauses helps. Being ready to treat an invalid input (so called sink-all function clauses) might help.</p>
<p>The developer should think before producing any code. The main responsibility is still put onto us. And we will continue producing bugs in production. That’s life in a nutshell.</p>
<p>Being 100% fault-tolerant <em>and</em> reporting all the issues immediately—helps. Strong typing… Well, in my experience it actually does not much. But of course your tastes might differ.</p>
<p>Happy faulttolerating!</p>
Dialyzer specs: 2 in 12019-12-17T00:00:00+00:00https://rocket-science.ru/hacking/2019/12/17/dialyzer-specs-2-in-1<p><img src="/img/montjuik.jpg" alt="Montjuïc" /></p>
<p>There are two types of erlang and elixir developers: those who write specs for <em>Dialyzer</em> and those who don’t <em>yet</em>. At first glance, it seems to be a waste of time, especially for those who come from languages with lax typing. However, specs have helped me catch several bugs even before the CI stage, and-sooner or later-any developer realizes that they are indeed a goodness. Not only as a tool to guide semi-strict typing, but also as a great help in documenting the code.</p>
<p>But, as is always happens in our cruel world, there are drawbacks. Essentially, the <code class="language-plaintext highlighter-rouge">@spec</code> directives duplicate the function declaration code. Below I am going to demonstrate how twenty lines of code can help to combine a specification and a function declaration into a nifty</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">defs</span> <span class="n">is_forty_two</span><span class="p">(</span><span class="ss">n:</span> <span class="n">integer</span><span class="p">)</span> <span class="p">::</span> <span class="n">boolean</span> <span class="k">do</span>
<span class="n">n</span> <span class="o">==</span> <span class="mi">42</span>
<span class="k">end</span>
</code></pre></div></div>
<p>They say, there is nothing in <em>Elixir</em>, but macros. Even <a href="https://hexdocs.pm/elixir/master/Kernel.html?#defmacro/2"><code class="language-plaintext highlighter-rouge">Kernel.defmacro/2</code></a> — is a <a href="https://github.com/elixir-lang/elixir/blob/ee758f987b7e240754bf386b903b92d38fb02233/lib/elixir/lib/kernel.ex#L4169"><code class="language-plaintext highlighter-rouge">macro</code></a> itself. Therefore, all we need is to define our own macro, which will create from the construction above both spec and a function declaration.</p>
<p>Vamonos.</p>
<h3 id="step-1-understanding-the-task">Step 1. Understanding the Task</h3>
<p>Let’s start with unveiling the AST that our not-yet-built macro would receive as a parameter.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">CustomSpec</span> <span class="k">do</span>
<span class="k">defmacro</span> <span class="n">defs</span><span class="p">(</span><span class="n">args</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="n">block</span><span class="p">)</span> <span class="k">do</span>
<span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="p">(</span><span class="n">args</span><span class="p">)</span>
<span class="ss">:ok</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">defmodule</span> <span class="no">CustomSpec</span><span class="o">.</span><span class="no">Test</span> <span class="k">do</span>
<span class="kn">import</span> <span class="no">CustomSpec</span>
<span class="n">defs</span> <span class="n">is_forty_two</span><span class="p">(</span><span class="ss">n:</span> <span class="n">integer</span><span class="p">)</span> <span class="p">::</span> <span class="n">boolean</span> <span class="k">do</span>
<span class="n">n</span> <span class="o">==</span> <span class="mi">42</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Wow, wow, not so fast. We immediately ran into <em>formatter</em>’s rebellion. She spat a ton of parentheses here and there and has the code reformatted in a way that makes everybody having soul cry. Let’s wean her off it. To do that we need to change the configuration file <code class="language-plaintext highlighter-rouge">.formatter.exs</code> in the following way</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span>
<span class="ss">inputs:</span> <span class="p">[</span><span class="s2">"mix.exs"</span><span class="p">,</span> <span class="s2">"{config,lib,test}/**/*.{ex,exs}"</span><span class="p">],</span>
<span class="ss">export:</span> <span class="p">[</span><span class="ss">locals_without_parens:</span> <span class="p">[</span><span class="ss">defs:</span> <span class="mi">2</span><span class="p">]]</span>
<span class="p">]</span>
</code></pre></div></div>
<p>Let’s go back to our daily duties and see what <code class="language-plaintext highlighter-rouge">defs/2</code> gets in there. It should be noted that <code class="language-plaintext highlighter-rouge">IO.inspect/2</code> will be executed at the compilation stage (if you do not understand why, you probably should not tweak macros yet, but read a brilliant book <a href="https://pragprog.com/book/cmelixir/metaprogramming-elixir">Metaprogramming Elixir</a> by Chris McCord.) Also, to prevent the compiler from swearing, we return<code class="language-plaintext highlighter-rouge">: ok</code> (macros must return the correct AST). So:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="ss">:"::"</span><span class="p">,</span> <span class="p">[</span><span class="ss">line:</span> <span class="mi">7</span><span class="p">],</span>
<span class="p">[</span>
<span class="p">{</span><span class="ss">:is_forty_two</span><span class="p">,</span> <span class="p">[</span><span class="ss">line:</span> <span class="mi">7</span><span class="p">],</span> <span class="p">[[</span><span class="ss">n:</span> <span class="p">{</span><span class="ss">:integer</span><span class="p">,</span> <span class="p">[</span><span class="ss">line:</span> <span class="mi">7</span><span class="p">],</span> <span class="no">nil</span><span class="p">}]]},</span>
<span class="p">{</span><span class="ss">:boolean</span><span class="p">,</span> <span class="p">[</span><span class="ss">line:</span> <span class="mi">7</span><span class="p">],</span> <span class="no">nil</span><span class="p">}</span>
<span class="p">]}</span>
</code></pre></div></div>
<p>Yeah. The parser believes that the imperator here is <code class="language-plaintext highlighter-rouge">::</code>, gluing the function definition and the return type. The function definition also contains a list of parameters as <code class="language-plaintext highlighter-rouge">Keyword</code>, <code class="language-plaintext highlighter-rouge">parameter name → type</code>.</p>
<h3 id="step-2-fail-fast">Step 2. Fail Fast</h3>
<p>Since we have decided to support only that syntax for starters, we would need to rewrite the definition of the macro <code class="language-plaintext highlighter-rouge">defs</code> to raise immediately if, for example, the return type is not specified.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmacro</span> <span class="n">defs</span><span class="p">({</span><span class="ss">:"::"</span><span class="p">,</span> <span class="n">_</span><span class="p">,</span> <span class="p">[{</span><span class="n">fun</span><span class="p">,</span> <span class="n">_</span><span class="p">,</span> <span class="p">[</span><span class="n">args_spec</span><span class="p">]},</span> <span class="p">{</span><span class="n">ret_spec</span><span class="p">,</span> <span class="n">_</span><span class="p">,</span> <span class="no">nil</span><span class="p">}]},</span> <span class="k">do</span><span class="p">:</span> <span class="n">block</span><span class="p">)</span> <span class="k">do</span>
</code></pre></div></div>
<p>Well, it i-i-i-i-i-is an <em>Implementation Time</em>.</p>
<h3 id="step-3-generation-of-both-spec-and-function-declaration">Step 3. Generation of Both Spec and Function Declaration</h3>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">CustomSpec</span> <span class="k">do</span>
<span class="k">defmacro</span> <span class="n">defs</span><span class="p">({</span><span class="ss">:"::"</span><span class="p">,</span> <span class="n">_</span><span class="p">,</span> <span class="p">[{</span><span class="n">fun</span><span class="p">,</span> <span class="n">_</span><span class="p">,</span> <span class="p">[</span><span class="n">args_spec</span><span class="p">]},</span> <span class="p">{</span><span class="n">ret_spec</span><span class="p">,</span> <span class="n">_</span><span class="p">,</span> <span class="no">nil</span><span class="p">}]},</span> <span class="k">do</span><span class="p">:</span> <span class="n">block</span><span class="p">)</span> <span class="k">do</span>
<span class="c1"># function declaration arguments</span>
<span class="n">args</span> <span class="o">=</span> <span class="n">for</span> <span class="p">{</span><span class="n">arg</span><span class="p">,</span> <span class="n">_spec</span><span class="p">}</span> <span class="o"><-</span> <span class="n">args_spec</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="no">Macro</span><span class="o">.</span><span class="n">var</span><span class="p">(</span><span class="n">arg</span><span class="p">,</span> <span class="no">nil</span><span class="p">)</span>
<span class="c1"># spec declaration arguments</span>
<span class="n">args_spec</span> <span class="o">=</span> <span class="n">for</span> <span class="p">{</span><span class="n">_arg</span><span class="p">,</span> <span class="n">spec</span><span class="p">}</span> <span class="o"><-</span> <span class="n">args_spec</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="no">Macro</span><span class="o">.</span><span class="n">var</span><span class="p">(</span><span class="n">spec</span><span class="p">,</span> <span class="no">nil</span><span class="p">)</span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="nv">@spec</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">fun</span><span class="p">)(</span><span class="n">unquote_splicing</span><span class="p">(</span><span class="n">args_spec</span><span class="p">))</span> <span class="p">::</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">ret_spec</span><span class="p">)</span>
<span class="k">def</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">fun</span><span class="p">)(</span><span class="n">unquote_splicing</span><span class="p">(</span><span class="n">args</span><span class="p">))</span> <span class="k">do</span>
<span class="kn">unquote</span><span class="p">(</span><span class="n">block</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The code is so evident, that I hesitate to add anything to it.</p>
<p>Let’s see how would <code class="language-plaintext highlighter-rouge">CustomSpec.Test.is_forty_two(42)</code> behave</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">></span> <span class="no">CustomSpec</span><span class="o">.</span><span class="no">Test</span><span class="o">.</span><span class="n">is_forty_two</span> <span class="mi">42</span>
<span class="c1">#⇒ true</span>
<span class="n">iex</span><span class="o">></span> <span class="no">CustomSpec</span><span class="o">.</span><span class="no">Test</span><span class="o">.</span><span class="n">is_forty_two</span> <span class="mi">43</span>
<span class="c1">#⇒ false</span>
</code></pre></div></div>
<p>Well, it works. I would not dump the BEAM chunk here, bear with me. You are to believe that spec works as well.</p>
<h3 id="step-4-is-that-it">Step 4. Is That It?</h3>
<p>Of course not. In the wild, one would have to properly handle invalid calls, to implement header definitions for functions with several different default parameters, to collect the spec more accurately (with variable names included,) to make sure that all argument names are different, and much more. But as the proof of concept it’s already good enough.</p>
<p>Also, one still may surprise colleagues with something like</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">CustomSpec</span> <span class="k">do</span>
<span class="k">defmacro</span> <span class="n">__using__</span><span class="p">(</span><span class="n">_</span><span class="p">)</span> <span class="k">do</span>
<span class="kn">import</span> <span class="no">Kernel</span><span class="p">,</span> <span class="ss">except:</span> <span class="p">[</span><span class="k">def</span><span class="p">:</span> <span class="mi">2</span><span class="p">]</span>
<span class="kn">import</span> <span class="no">CustomSpec</span>
<span class="k">defmacro</span> <span class="k">def</span><span class="p">(</span><span class="n">args</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="n">block</span><span class="p">)</span> <span class="k">do</span>
<span class="n">defs</span><span class="p">(</span><span class="n">args</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="n">block</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="o">...</span>
<span class="k">end</span>
</code></pre></div></div>
<p>(Also <code class="language-plaintext highlighter-rouge">defs/2</code> i s to be modified to generate <code class="language-plaintext highlighter-rouge">Kernel.def</code> instead of <code class="language-plaintext highlighter-rouge">def</code>,) but take it for granted: you do not want to suprise mates in such a weird way. Even on April 1st.</p>
<p>Happy macroing!</p>
Formulæ and Lazy Combinators2019-09-06T00:00:00+00:00https://rocket-science.ru/hacking/2019/09/06/formulae-and-lazy-combinators<p><img src="/img/airplane-sun-sky.jpg" alt="The airplace in the sky" /></p>
<h3 id="formulæ-library">Formulæ Library</h3>
<p>We in Fintech often require to check the values for simple arithmetic conditions,
like whether the exchange rate is greater than the desired value, or like.
These conditions are fully dynamic and we need something that we can apply
on new values coming. Imagine the client wanting to be notified when the exchange
rate for some currency pair hits <code class="language-plaintext highlighter-rouge">1.2:1</code> ratio. This would be deadly easy if
we could make it static:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">notify?</span><span class="p">(</span><span class="n">rate</span><span class="p">)</span> <span class="ow">when</span> <span class="n">rate</span> <span class="o">></span> <span class="mf">1.2</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="no">true</span>
<span class="k">def</span> <span class="n">notify?</span><span class="p">(</span><span class="n">_</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="no">false</span>
</code></pre></div></div>
<p>Once the value comes dynamically, we need a more or less robust mechanism to check
for the same condition. Using <a href="https://hexdocs.pm/elixir/master/Code.html?#eval_string/3"><code class="language-plaintext highlighter-rouge">Code.eval_string/3</code></a>, while somewhat works, is compiling the condition every single
time it’s getting called. This is obviously wasting resources for no reason.</p>
<p>We finally came up with a <a href="https://hexdocs.pm/formulae/Formulae.html#content">precompiled formulae</a>. The tiny library creates the module for each input given and compiles the evaluator.</p>
<p><strong>NB</strong> This should be used with care because module names are stored as atoms and
blindly creating modules for whatever the client wants to check might lead to
atoms DoS attack on the long run. We use a maximal allowed step of 0.01 which
gives at most 200 thousand atoms in the worst scenario.</p>
<h3 id="lazy-combinators">Lazy Combinators</h3>
<p>But the main purpose of this writing is not the precompiled formula. For some
reason we needed to calculate permutations of the list of considerable length.
I decided to replicate ruby combinators (<a href="https://ruby-doc.org/core/Array.html#method-i-combination"><code class="language-plaintext highlighter-rouge">Array#combination</code></a> and family.) Unfortunately it was not as easy
for noticeably large numbers. Greedy static evaluation freezes on not as huge numbers.</p>
<p>That was expected, so I started to tackle with <a href="https://hexdocs.pm/elixir/master/Stream.html"><code class="language-plaintext highlighter-rouge">Stream</code></a>. That was not as straightforward as I thought. I came up with something like the code below</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">list</span> <span class="o">=</span> <span class="sx">~w[a b c d e]a</span>
<span class="n">combinations</span> <span class="o">=</span>
<span class="no">Stream</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="no">Stream</span><span class="o">.</span><span class="n">with_index</span><span class="p">(</span><span class="n">list</span><span class="p">),</span> <span class="ss">:ok</span><span class="p">,</span> <span class="k">fn</span> <span class="p">{</span><span class="n">i1</span><span class="p">,</span> <span class="n">idx1</span><span class="p">},</span> <span class="ss">:ok</span> <span class="o">-></span>
<span class="p">{</span><span class="no">Stream</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="no">Stream</span><span class="o">.</span><span class="n">with_index</span><span class="p">(</span><span class="n">list</span><span class="p">),</span> <span class="ss">:ok</span><span class="p">,</span> <span class="k">fn</span>
<span class="p">{</span><span class="n">_</span><span class="p">,</span> <span class="n">idx2</span><span class="p">},</span> <span class="ss">:ok</span> <span class="ow">when</span> <span class="n">idx2</span> <span class="o"><=</span> <span class="n">idx1</span> <span class="o">-></span>
<span class="p">{[],</span> <span class="ss">:ok</span><span class="p">}</span>
<span class="p">{</span><span class="n">i2</span><span class="p">,</span> <span class="n">idx2</span><span class="p">},</span> <span class="ss">:ok</span> <span class="o">-></span>
<span class="p">{</span><span class="no">Stream</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="no">Stream</span><span class="o">.</span><span class="n">with_index</span><span class="p">(</span><span class="n">list</span><span class="p">),</span> <span class="ss">:ok</span><span class="p">,</span> <span class="k">fn</span>
<span class="p">{</span><span class="n">_</span><span class="p">,</span> <span class="n">idx3</span><span class="p">},</span> <span class="ss">:ok</span> <span class="ow">when</span> <span class="n">idx3</span> <span class="o"><=</span> <span class="n">idx2</span> <span class="o">-></span>
<span class="p">{[],</span> <span class="ss">:ok</span><span class="p">}</span>
<span class="p">{</span><span class="n">i3</span><span class="p">,</span> <span class="n">idx3</span><span class="p">},</span> <span class="ss">:ok</span> <span class="o">-></span>
<span class="p">{</span><span class="no">Stream</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="no">Stream</span><span class="o">.</span><span class="n">with_index</span><span class="p">(</span><span class="n">list</span><span class="p">),</span> <span class="ss">:ok</span><span class="p">,</span> <span class="k">fn</span>
<span class="p">{</span><span class="n">_</span><span class="p">,</span> <span class="n">idx4</span><span class="p">},</span> <span class="ss">:ok</span> <span class="ow">when</span> <span class="n">idx4</span> <span class="o"><=</span> <span class="n">idx3</span> <span class="o">-></span>
<span class="p">{[],</span> <span class="ss">:ok</span><span class="p">}</span>
<span class="p">{</span><span class="n">i4</span><span class="p">,</span> <span class="n">_idx4</span><span class="p">},</span> <span class="ss">:ok</span> <span class="o">-></span>
<span class="p">{[[</span><span class="n">i1</span><span class="p">,</span> <span class="n">i2</span><span class="p">,</span> <span class="n">i3</span><span class="p">,</span> <span class="n">i4</span><span class="p">]],</span> <span class="ss">:ok</span><span class="p">}</span>
<span class="k">end</span><span class="p">),</span> <span class="ss">:ok</span><span class="p">}</span>
<span class="k">end</span><span class="p">),</span> <span class="ss">:ok</span><span class="p">}</span>
<span class="k">end</span><span class="p">),</span> <span class="ss">:ok</span><span class="p">}</span>
<span class="k">end</span><span class="p">)</span>
</code></pre></div></div>
<p>It works, but it is hardcoded for number of combinations! So, that is what
we have macros for, isn’t it?</p>
<p>The code has three different patterns. The successful path, emitting the list.
The emit-empty-fast clauses. And the <code class="language-plaintext highlighter-rouge">Stream.transform(Stream.with_index(list) ...</code>
clauses. Looks like we might try to generate the AST for the above.</p>
<p>This is the rare case when using <a href="https://hexdocs.pm/elixir/master/Kernel.SpecialForms.html?#quote/2"><code class="language-plaintext highlighter-rouge">Kernel.SpecialForms.quote/2</code></a> would probably make things more complicated, so
I went for plain old good bare AST generation. I started with quoting the above
and examining the result. Yes, there are patterns.</p>
<p>So, let’s start with producing the scaffold.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmacrop</span> <span class="n">mapper</span><span class="p">(</span><span class="n">from</span><span class="p">,</span> <span class="n">to</span><span class="p">,</span> <span class="n">fun</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="kn">quote</span><span class="p">(</span><span class="k">do</span><span class="p">:</span> <span class="no">Enum</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="no">Range</span><span class="o">.</span><span class="n">new</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">from</span><span class="p">),</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">to</span><span class="p">)),</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">fun</span><span class="p">)))</span>
<span class="nv">@spec</span> <span class="n">combinations</span><span class="p">(</span><span class="n">list</span> <span class="p">::</span> <span class="n">list</span><span class="p">(),</span> <span class="n">count</span> <span class="p">::</span> <span class="n">non_neg_integer</span><span class="p">())</span> <span class="p">::</span> <span class="p">{</span><span class="no">Stream</span><span class="o">.</span><span class="n">t</span><span class="p">(),</span> <span class="ss">:ok</span><span class="p">}</span>
<span class="k">defmacro</span> <span class="n">combinations</span><span class="p">(</span><span class="n">l</span><span class="p">,</span> <span class="n">n</span><span class="p">)</span> <span class="k">do</span>
<span class="no">Enum</span><span class="o">.</span><span class="n">reduce</span><span class="p">(</span><span class="n">n</span><span class="o">..</span><span class="mi">1</span><span class="p">,</span> <span class="p">{[</span><span class="n">mapper</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">n</span><span class="p">,</span> <span class="o">&</span><span class="n">var</span><span class="o">/</span><span class="mi">1</span><span class="p">)],</span> <span class="ss">:ok</span><span class="p">},</span> <span class="k">fn</span> <span class="n">i</span><span class="p">,</span> <span class="n">body</span> <span class="o">-></span>
<span class="n">stream_combination_transform_clause</span><span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="n">l</span><span class="p">,</span> <span class="n">body</span><span class="p">)</span>
<span class="k">end</span><span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Now we need to dive into AST to extract patterns. That was fun!</p>
<p>Helpers to simplify the code after.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">var</span><span class="p">(</span><span class="n">i</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="ss">:"i_</span><span class="si">#{</span><span class="n">i</span><span class="si">}</span><span class="ss">"</span><span class="p">,</span> <span class="p">[],</span> <span class="no">Elixir</span><span class="p">}</span>
<span class="k">def</span> <span class="n">idx</span><span class="p">(</span><span class="n">i</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="ss">:"idx_</span><span class="si">#{</span><span class="n">i</span><span class="si">}</span><span class="ss">"</span><span class="p">,</span> <span class="p">[],</span> <span class="no">Elixir</span><span class="p">}</span>
</code></pre></div></div>
<p>Inner clause AST.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">sink_combination_clause</span><span class="p">(</span><span class="n">i</span><span class="p">)</span> <span class="ow">when</span> <span class="n">i</span> <span class="o">></span> <span class="mi">1</span> <span class="k">do</span>
<span class="p">{</span><span class="ss">:-</span><span class="o">></span><span class="p">,</span> <span class="p">[],</span>
<span class="p">[</span>
<span class="p">[</span>
<span class="p">{</span><span class="ss">:when</span><span class="p">,</span> <span class="p">[],</span>
<span class="p">[</span>
<span class="p">{</span><span class="err"></span><span class="p">{</span><span class="ss">:_</span><span class="p">,</span> <span class="p">[],</span> <span class="no">Elixir</span><span class="p">},</span> <span class="n">idx</span><span class="p">(</span><span class="n">i</span><span class="p">)},</span>
<span class="ss">:ok</span><span class="p">,</span>
<span class="p">{</span><span class="ss">:<=</span><span class="p">,</span> <span class="p">[</span><span class="ss">context:</span> <span class="no">Elixir</span><span class="p">,</span> <span class="kn">import</span><span class="p">:</span> <span class="no">Kernel</span><span class="p">],</span> <span class="p">[</span><span class="n">idx</span><span class="p">(</span><span class="n">i</span><span class="p">),</span> <span class="n">idx</span><span class="p">(</span><span class="n">i</span> <span class="o">-</span> <span class="mi">1</span><span class="p">)]}</span>
<span class="p">]}</span>
<span class="p">],</span>
<span class="p">{[],</span> <span class="ss">:ok</span><span class="p">}</span>
<span class="p">]}</span>
<span class="k">end</span>
</code></pre></div></div>
<p>All inner clauses together.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">sink_combination_clauses</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">body</span><span class="p">)</span> <span class="k">do</span>
<span class="p">[{</span><span class="ss">:-</span><span class="o">></span><span class="p">,</span> <span class="p">[],</span> <span class="p">[[{</span><span class="n">var</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span> <span class="n">idx</span><span class="p">(</span><span class="mi">1</span><span class="p">)},</span> <span class="ss">:ok</span><span class="p">],</span> <span class="n">body</span><span class="p">]}]</span>
<span class="k">end</span>
<span class="k">def</span> <span class="n">sink_combination_clauses</span><span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="n">body</span><span class="p">)</span> <span class="ow">when</span> <span class="n">i</span> <span class="o">></span> <span class="mi">1</span> <span class="k">do</span>
<span class="no">Enum</span><span class="o">.</span><span class="n">reverse</span><span class="p">([</span>
<span class="p">{</span><span class="ss">:-</span><span class="o">></span><span class="p">,</span> <span class="p">[],</span> <span class="p">[[{</span><span class="n">var</span><span class="p">(</span><span class="n">i</span><span class="p">),</span> <span class="n">idx</span><span class="p">(</span><span class="n">i</span><span class="p">)},</span> <span class="ss">:ok</span><span class="p">],</span> <span class="n">body</span><span class="p">]}</span>
<span class="o">|</span> <span class="no">Enum</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="mi">2</span><span class="o">..</span><span class="n">i</span><span class="p">,</span> <span class="o">&</span><span class="n">sink_combination_clause</span><span class="o">/</span><span class="mi">1</span><span class="p">)</span>
<span class="p">])</span>
<span class="k">end</span>
</code></pre></div></div>
<p>And, finally, the outer clause.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">stream_combination_transform_clause</span><span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="n">l</span><span class="p">,</span> <span class="n">body</span><span class="p">)</span> <span class="k">do</span>
<span class="n">clauses</span> <span class="o">=</span> <span class="n">sink_combination_clauses</span><span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="n">body</span><span class="p">)</span>
<span class="p">{</span><span class="err"></span><span class="p">{</span><span class="err"></span><span class="p">{:</span><span class="o">.</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[{</span><span class="ss">:__aliases__</span><span class="p">,</span> <span class="p">[</span><span class="ss">alias:</span> <span class="no">false</span><span class="p">],</span> <span class="p">[</span><span class="ss">:Stream</span><span class="p">]},</span> <span class="ss">:transform</span><span class="p">]},</span> <span class="p">[],</span>
<span class="p">[</span>
<span class="p">{</span><span class="err"></span><span class="p">{:</span><span class="o">.</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[{</span><span class="ss">:__aliases__</span><span class="p">,</span> <span class="p">[</span><span class="ss">alias:</span> <span class="no">false</span><span class="p">],</span> <span class="p">[</span><span class="ss">:Stream</span><span class="p">]},</span> <span class="ss">:with_index</span><span class="p">]},</span> <span class="p">[],</span> <span class="p">[</span><span class="n">l</span><span class="p">]},</span>
<span class="ss">:ok</span><span class="p">,</span>
<span class="p">{</span><span class="ss">:fn</span><span class="p">,</span> <span class="p">[],</span> <span class="n">clauses</span><span class="p">}</span>
<span class="p">]},</span> <span class="ss">:ok</span><span class="p">}</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Permutations are done almost the same, the only change is the condition
in the inner clauses. That was easy!</p>
<h3 id="application">Application</h3>
<p>OK, so what can we do with this in place? Somewhat like this.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">l</span> <span class="o">=</span> <span class="n">for</span> <span class="n">c</span> <span class="o"><-</span> <span class="sx">?a</span><span class="o">..</span><span class="sx">?z</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="o"><<</span><span class="n">c</span><span class="o">>></span> <span class="c1"># letters list</span>
<span class="n">with</span> <span class="p">{</span><span class="n">stream</span><span class="p">,</span> <span class="ss">:ok</span><span class="p">}</span> <span class="o"><-</span> <span class="no">Formulae</span><span class="o">.</span><span class="no">Combinators</span><span class="o">.</span><span class="no">Stream</span><span class="o">.</span><span class="n">permutations</span><span class="p">(</span><span class="n">l</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">stream</span> <span class="o">|></span> <span class="no">Stream</span><span class="o">.</span><span class="n">take_every</span><span class="p">(</span><span class="mi">26</span><span class="p">)</span> <span class="o">|></span> <span class="no">Enum</span><span class="o">.</span><span class="n">take</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span>
<span class="c1">#⇒ [["a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l"],</span>
<span class="c1"># ["a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "l", "w"]]</span>
</code></pre></div></div>
<p>We now even can feed a <a href="https://hexdocs.pm/flow/Flow.html"><code class="language-plaintext highlighter-rouge">Flow</code></a> with a stream
returned above and spawn a process that will unhurriedly walk through all the
combinations doing something business important.</p>
<p>Happy lazy permutating!</p>
Standing on the Shoulders of Giants2019-08-28T00:00:00+00:00https://rocket-science.ru/culture/2019/08/28/standing-on-the-shoulders-of-giants<p><img src="/img/on-giants-shoulders.jpg" alt="EXIF for Joe Armstrong" /></p>
<p>Today I was grepping the internets for something irrelevant and came across <a href="https://joearms.github.io/#2017-12-18%20Calling%20Elixir%20from%20Erlang">this blog post</a> by Joe Armstrong. Long story short, he was stumbled upon extraction from the photos <em>“things like the latitude and longitude of the place where the image was taken and the time when the image was taken”</em>.</p>
<p>Basically it was about extracting <a href="https://en.wikipedia.org/wiki/Exif">Exif</a> data from the photos. Joe discovered Dave Thomas’ <a href="https://github.com/pragdave/exexif"><em>Elixir</em> library</a> that deals with <em>Exifs</em> and finally succeeded with his fleeting task.</p>
<p>What I am proud now, the code that made Joe happy was indeed <em>mine</em>. I got into the same issue a couple of years ago and <em>GPS</em> data support in that library is <a href="https://github.com/pragdave/exexif/tree/master/lib/exexif/data">fully implemented by me</a>.</p>
<p>That said, Joe Armstrong used my code and it made him slightly happier at the moment. There is nothing to be proud of, in terms of code: it is trivial and all the hard work in regard to parsing <em>Exif</em> was already done by Dave. It’s not about being proud and it’s not the stuff one puts into résumé.</p>
<p>I am just happy Joe used my code and it helped him at the moment. Joe is one of the smartest people I was lucky to know personally, I envy his sense of humor and it is just making me smile joyfully: I helped him with code.</p>
Beware of Tests et Fudicia Ferentes2019-08-28T00:00:00+00:00https://rocket-science.ru/hacking/2019/08/28/beware-of-tests-et-fiducia-ferentes<p><img src="/img/platja-de-aro.jpg" alt="To test or not to test?—’Tis the question." /></p>
<p><strong>Beware.</strong> This writing is very biased and exaggerated. Don’t scroll down if your condition is unstable.</p>
<h3 id="tests-are-our-saviors-and-heroes">Tests Are Our Saviors And Heroes</h3>
<p>In the stone age there were no tests and developers were lacking the ability to produce robust software; the code was full of bugs and the whole development process was called “debugging.”</p>
<p><em>There is nothing wrong with this, except that it ain’t so.</em></p>
<p><a href="https://en.wikipedia.org/wiki/TeX">TeX</a> was released by <em>Donald Knuth</em> in 1978.</p>
<blockquote>
<p>Donald Knuth offers monetary awards to people who find and report a bug in TeX. […]
Knuth has lost relatively little money as there have been very few bugs claimed.</p>
</blockquote>
<p>Guess, how many many tests existed in the original TeX implementation.</p>
<p>Don’t get me wrong; I am not advocating back to live in the cave. Sometimes tests bring joy to our lives. But my concern is tests in general bring more harm than value to the modern development.</p>
<p>Yes, I said that, throw your stones at me.</p>
<p>In a nutshell, tests bring the excessive confidence whereas they usually prove things that are easy to reason about without any tests. Meanwhile tests hide issues, glitches, and pitfalls because “c’mon it’s all tested.” Yes, I know about <em>mutation</em> and/or <em>property</em> testing. They are great. Occasionally. In the vast majority of cases they are an extreme overkill, bringing a complexity into both maintainance and support. To areas, where it should not belong to at all.</p>
<p>102% coverage means both Dev and QA teams involved are very diligent and assiduous. And nothing more.</p>
<p>While I am not banned from the dev internet yet, let me get to the arguments.</p>
<h3 id="when-tests-are-great">When Tests Are Great</h3>
<p>Let’s start with listing applications where tests are great.</p>
<ul>
<li>pure functions (unit tests, doctests)</li>
<li>cross-interaction (integration tests)</li>
<li>unclear / non-trivial execution flow (property tests)</li>
<li>concurrency</li>
<li>cases where tests are easier to tackle with than REPL</li>
<li>custom cases when you get an insight “I need to test it”</li>
</ul>
<p>This is all clear, so let me get directly to the topic of this writing.</p>
<h3 id="when-tests-are-not-panacea">When Tests Are Not Panacea</h3>
<p>Tests won’t make the bad architecture any better, in the first place. Any coverage may at the best prove that the existing implementation <strong>works as expected</strong>. Which is way less valuable and important than the proof that the existing implementation is at least robust. I am not talking about being maintainable, flexible, and extensible.</p>
<p>When we deal with third-party service, we tend to mock it, and write a gazillion tests covering all the corner cases. We respond properly to <code class="language-plaintext highlighter-rouge">200</code>, <code class="language-plaintext highlighter-rouge">302</code>, <code class="language-plaintext highlighter-rouge">404</code>, <code class="language-plaintext highlighter-rouge">500</code>, etc. Do we really need all these tests? I doubt. All of them testing <code class="language-plaintext highlighter-rouge">case</code> operator in our language of choice. And you know what? Chances are it works properly. What could actually blow our application up is network latency, timeouts, malformed response. To test all these we do not need to mock anything. Perform a call to <code class="language-plaintext highlighter-rouge">http://localhost:200000</code> and see what happens with your code. Provide a sink-all clause with <code class="language-plaintext highlighter-rouge">rescue</code> or pattern-match the response to <code class="language-plaintext highlighter-rouge">_</code> and log it to some explicitly dedicated <code class="language-plaintext highlighter-rouge">unexpected.log</code>. 3rd-party service would use any chance to surprise you, don’t try to predict it. Log it and deal with it later.</p>
<p>Another case would be a trivial function like <code class="language-plaintext highlighter-rouge">def answer, do: 42</code>. Please, do not waste time checking whether it returns 42. It does. Unless you are writing the compiler for the new language, or course.</p>
<p>Do not test that <code class="language-plaintext highlighter-rouge">map</code> maps, <code class="language-plaintext highlighter-rouge">reduce</code> reduces. They were tested before they became a part of the language. Test things that you are uncertain in.</p>
<p>Sometimes it’s indeed handy to write a test and then write the code that should pass the test. People even invented the name for it: TDD. That’s all cool, but please let’s not fool ourselves. If it was easier to test the same behavior in REPL, we’d better do it in REPL. Because once written, this function works and the test on its own becomes <em>a legacy piece of code</em>, that makes a testsuite running forever. Leave two to make sure it works for some random input and it fails gracefully for some garbage passed. If you implementation of <em>Fibonacci sequence calculuator</em> works for <code class="language-plaintext highlighter-rouge">42</code>, it works for everything else by induction (I am exaggerating here a bit, but still.)</p>
<p>Did I hear “Objection! Regression!” outcry or is this my mental issues? If you are aware of regression, there is something wrong with how you architect your project. The function should not break when the <em>input is extended</em>. Unless it is a function performing an explicit check for the fact that the input was not extended. And no one function in the world may <em>change</em> the expectations for the input. That’s not how sane developers maintain their codebase. Backward compatibility is not a luxury, it’s a must. And it’s extremely easy to follow. Need an extended functionality? Create a new function dealing with this kind of input. Do not break the old one. Create new. That simple.</p>
<p>It’s getting too long already, so I am to sum it up.</p>
<p>Tests could be a tremendous help. In the cases I listed above. But tests never should be treated as an insurance that the code works, survives updates and even is somewhat robust.</p>
<p>A dozen of tests covering the calculation of the square root for different inputs won’t make it deal properly with some floats, because floats are broken. Using proper type for handling decimals in the project that requires math of that type costs more than all the tests on the Earth.</p>
<p>Proper shape of a supervision tree is hardly testable; it’s easy to reason about, though, and it’s easy to start <code class="language-plaintext highlighter-rouge">observer</code> and manually kill all workers, one by one, and see what’s happening. And this behavior won’t change in the future, until you’ve extended the tree. The thing is old tests would nevertheless fail now, so open REPL, start <code class="language-plaintext highlighter-rouge">observer</code>, click-through, make sure it works as expected, and forget about. Do not write tests. They have zero value here.</p>
<h3 id="what-now">What Now?</h3>
<p>Now, when you have a lot of spare time, freed up because you stopped testing all sorts of nonsense, what do you do with it?</p>
<p>That’s easy. Write an extensive documentation. With samples of <em>how to use your code</em>. With reasoning about why it was done that way. With snippets that could be copy-pasted into external projects and <em>simply run</em>.</p>
<p>That is a hundred times more valuable than freaky testing controllers responding different requests.</p>
<p>Thank you and happy testing!</p>
Scaffolds Backed Up By Behaviours2019-08-26T00:00:00+00:00https://rocket-science.ru/hacking/2019/08/26/behaviour-backed-scaffolds<p><img src="/img/beach-filipines.jpg" alt="Behaviour Backed Up By Scaffold" /></p>
<h3 id="repeated-code-aka-copypasta">Repeated Code aka Copypasta</h3>
<p>If you are like me, you probably try to make the <del>life</del> developing and testing easier by moving all the <em>generic</em> code into well-tested self-contained packages. This is barely necessary if the waterfall development process in the company is shaped to feed the ancient behemoth, affectionately called by all teammates “Monolitty,” but first three microservices happened to appear in the repo eventually make everybody to think about how to share common code between them.</p>
<p>Sometimes it might be done by extracting the common code into the package used by all microservices. Unfortunately, the code to be shared is usually not <em>the same</em>, but <em>quite alike</em>. <em>Custom Logger</em>, or <em>Slack Notifier</em>, or <em>RabbitMQ Subscriber</em> look very similar in the area of establishing and keeping a connection, managing crashes and disconnections, etc, but they differ in the business logic specific parts. And the decision <em>how do we shape the common part so that the pieces to be written by clients of this interface are both kept small but best maintainable</em> remains the one of most important.</p>
<p>There are several different approaches. Here I am going to show one of them. I spotted it in <a href="https://hexdocs.pm/broadway"><code class="language-plaintext highlighter-rouge">Broadway</code></a>. As by documentation,</p>
<blockquote>
<p><code class="language-plaintext highlighter-rouge">Broadway</code> is a concurrent, multi-stage tool for building data ingestion and data processing pipelines.</p>
<p>It allows developers to consume data efficiently from different sources, such as Amazon SQS, RabbitMQ and others.</p>
</blockquote>
<p>In a nutshell, <code class="language-plaintext highlighter-rouge">Broadway</code> takes care about keeping the connection and providing a back pressure against the external source, providing the handy interface for <em>clients</em> of the library to concentrate on the business logic only. Basically, one does <code class="language-plaintext highlighter-rouge">use Broadway</code> and all they need to plug in their business logic would be to implement <code class="language-plaintext highlighter-rouge">Broadway.handle_message/3</code> callback which will be called on new incoming messages received from the external source. All the connection low-level handling stuff is kept under the hood and <code class="language-plaintext highlighter-rouge">use Broadway</code> just makes it implemented automagically. I oversimplified the things, but without loss of generality.</p>
<h3 id="deal-with-it">Deal With It!</h3>
<p>So, the approach I advertise and advocate would be</p>
<blockquote>
<p><strong>Implement the common logic in your external package and use callbacks anywhere the business logic is required / affected.</strong></p>
</blockquote>
<p>This works incredibly smooth. As an example, unrelated to dealing with connections, I might mention <a href="https://hexdocs.pm/tarearbol/dynamic_workers_management.html#content"><code class="language-plaintext highlighter-rouge">DynamicManager</code></a> included in the last release of <a href="https://hexdocs.pm/tarearbol"><code class="language-plaintext highlighter-rouge">Tarearbol</code></a>. This is a helper around <a href="https://hexdocs.pm/elixir/master/DynamicSupervisor.html"><code class="language-plaintext highlighter-rouge">DynamicSupervisor</code></a> taking care about the boilerplate needed to handle and supervise many different processes, behaving <em>more-or-less</em> alike.</p>
<p>Imagine we have an auction implementation, with several different types of items. Each item is managed by its own process and the processes have slightly different <em>Finite Automata</em> driving the business logic. Once the item is put up for auction, the process is started; once it gets sold, the process exits. If there was no user interaction, the price gets lowered by 1% automatically. Something like that. To distinguish the business logic and concentrate on what actually matters, one might use <code class="language-plaintext highlighter-rouge">DynamicManager</code> scaffold in the following way.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">module</span> <span class="no">Auction</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">Tarearbol</span><span class="o">.</span><span class="no">DynamicManager</span>
<span class="k">def</span> <span class="n">children_specs</span><span class="p">,</span>
<span class="k">do</span><span class="p">:</span> <span class="n">for</span> <span class="n">au</span> <span class="o"><-</span> <span class="no">Repo</span><span class="o">.</span><span class="n">all</span><span class="p">(</span><span class="no">AuctionItem</span><span class="p">),</span> <span class="ss">into:</span> <span class="p">%{},</span> <span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="n">ai</span><span class="o">.</span><span class="n">id</span><span class="p">,</span> <span class="p">[</span><span class="ss">payload:</span> <span class="n">ai</span><span class="p">]}</span>
<span class="k">def</span> <span class="n">perform</span><span class="p">(</span><span class="n">aid</span><span class="p">,</span> <span class="n">ai</span><span class="p">)</span> <span class="k">do</span>
<span class="n">aid</span>
<span class="o">|></span> <span class="n">check_bids</span><span class="p">()</span>
<span class="o">|></span> <span class="k">case</span> <span class="k">do</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">price</span><span class="p">}</span> <span class="o">-></span>
<span class="no">Repo</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">ai</span><span class="p">,</span> <span class="ss">price:</span> <span class="n">price</span><span class="p">,</span> <span class="ss">status:</span> <span class="ss">:sold</span><span class="p">)</span>
<span class="ss">:halt</span>
<span class="ss">:pass</span> <span class="o">-></span>
<span class="no">Repo</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">ai</span><span class="p">,</span> <span class="ss">price:</span> <span class="n">price</span> <span class="o">*</span> <span class="mf">0.99</span><span class="p">,</span> <span class="ss">status:</span> <span class="ss">:trading</span><span class="p">)</span>
<span class="ss">:ok</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>That’s all needed. The process will reschedule itself every 1 second (default,) check for the bids (this function implementation is out of scope of this post,) and mark the item as sold and instruct the supervisor to kill itself or downgrade the price and wait for new bids.</p>
<p>The example is contrieved and oversimplified, but it perfectly shows how all the non-business-related logic might be encapsulated into the library and reused, when the real application only needs to implement several callbacks to make all the machinery work.</p>
<p>Happy encapsulating!</p>
Use Github CI for Elixir Projects2019-08-19T00:00:00+00:00https://rocket-science.ru/hacking/2019/08/19/use-github-ci-for-elixir-projects<p><img src="/img/filipines.jpg" alt="Welcome to Github CI" /></p>
<h3 id="github-ci">Github CI</h3>
<p>Github launched <a href="https://github.com/features/actions">actions</a> which make it possible to perform CI without leaving the place where the code belongs. It is indeed quite handy. Once somebody pushes, or makes a pull request, or whatever (the list when to apply an action might be found in the <a href="https://help.github.com/en/articles/about-github-actions">official documentation</a>,) the build is started. Scheduled cron-like tasks are also supported.</p>
<p>One might produce pipelines of actions, named <em>workflows</em>. And all that is great, save for the documentation.</p>
<p>It took me almost an hour to figure out how to spawn a container with third-party services to test the application against. Here is what I have learned. Please note, that the official documentation is yet clumsy, incomplete and sometimes wrong.</p>
<p>The standard CI action uses the configuration files with the syntax <em>quite</em> similar to the one used by <a href="https://circleci.com/"><em>CircleCI</em></a>. It’s plain old good <em>YAML</em>, allowing to set up the target OS, environment, commands to execute, etc. Actions are <em>named</em> what allows to refer to other actions and depend on them.</p>
<p>Also, the configuration allows to specify <em>services</em>. Services are to be run somewhere in the cloud and GH would map ports of the container to the ports these services expose, according to config. That part is feebly covered in the official documentation and even that what is covered contains errors.</p>
<p>Here is the working example of the configuration for the <em>Elixir</em> project, requiring <em>RabbitMQ</em> and <em>Redis</em> services for testing.</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">name</span><span class="pi">:</span> <span class="s">Tests for My Project</span>
<span class="na">on</span><span class="pi">:</span> <span class="pi">[</span><span class="nv">push</span><span class="pi">,</span> <span class="nv">pull_request</span><span class="pi">]</span>
<span class="na">jobs</span><span class="pi">:</span>
<span class="na">build</span><span class="pi">:</span>
<span class="na">runs-on</span><span class="pi">:</span> <span class="s">ubuntu-latest</span>
<span class="na">container</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">elixir:1.9.1-slim</span>
<span class="na">services</span><span class="pi">:</span>
<span class="na">rabbitmq</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">rabbitmq</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">5672:5672</span>
<span class="na">env</span><span class="pi">:</span>
<span class="na">RABBITMQ_USER</span><span class="pi">:</span> <span class="s">guest</span>
<span class="na">RABBITMQ_PASSWORD</span><span class="pi">:</span> <span class="s">guest</span>
<span class="na">RABBITMQ_VHOST</span><span class="pi">:</span> <span class="s2">"</span><span class="s">/"</span>
<span class="na">redis</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">redis</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">6379:6379</span>
<span class="na">steps</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">uses</span><span class="pi">:</span> <span class="s">actions/checkout@v1</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">Install Dependencies</span>
<span class="na">run</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">MIX_ENV=ci mix local.rebar --force</span>
<span class="s">MIX_ENV=ci mix local.hex --force</span>
<span class="s">MIX_ENV=ci mix deps.get</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">Run All Tests</span>
<span class="na">run</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">MIX_ENV=ci mix test</span>
<span class="na">env</span><span class="pi">:</span>
<span class="na">RABBITMQ_HOST</span><span class="pi">:</span> <span class="s">rabbitmq</span>
<span class="na">RABBITMQ_PORT</span><span class="pi">:</span> <span class="s">$❴❴ job.services.rabbitmq.ports[5672] ❵❵</span>
<span class="na">REDIS_HOST</span><span class="pi">:</span> <span class="s">redis</span>
<span class="na">REDIS_PORT</span><span class="pi">:</span> <span class="s">$❴❴ job.services.redis.ports[6379] ❵❵</span>
</code></pre></div></div>
<p><strong>NB</strong> Curly brackets above should be normal ones, I uses these because my templating engine drives bonkers seeing two opening curlies, sorry for that.</p>
<p>As one might see, tests are to be run on <em>Ubuntu</em>, using <em>Elixir v1.9.1</em>. Services are described under the <code class="language-plaintext highlighter-rouge">services</code> key, and here is a trick. The port, the service port will be mapped to, is randomly chosen by the container engine in the runtime and stored in the <em>internal</em> shell variable with a name <code class="language-plaintext highlighter-rouge">job.services.rabbitmq.ports[5672]</code>. <code class="language-plaintext highlighter-rouge">rabbitmq</code> here is the name of the service, as specified in this file in <code class="language-plaintext highlighter-rouge">services</code> section and <code class="language-plaintext highlighter-rouge">5672</code> is the original port. The internal variable has a syntax <code class="language-plaintext highlighter-rouge">$❴❴ foo ❵❵</code> and is being passed to the environment variable <code class="language-plaintext highlighter-rouge">RABBITMQ_PORT</code>. <code class="language-plaintext highlighter-rouge">RABBITMQ_HOST</code> there must be set to the <em>service name</em>. Now your application might read the environment variables as usual.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="no">Config</span>
<span class="n">config</span> <span class="ss">:my_app</span><span class="p">,</span>
<span class="ss">rabbitmq:</span> <span class="p">[</span>
<span class="ss">host:</span> <span class="no">System</span><span class="o">.</span><span class="n">get_env</span><span class="p">(</span><span class="s2">"RABBITMQ_HOST"</span><span class="p">),</span>
<span class="ss">password:</span> <span class="s2">"guest"</span><span class="p">,</span>
<span class="ss">port:</span> <span class="no">String</span><span class="o">.</span><span class="n">to_integer</span><span class="p">(</span><span class="no">System</span><span class="o">.</span><span class="n">get_env</span><span class="p">(</span><span class="s2">"RABBITMQ_PORT"</span><span class="p">,</span> <span class="s2">"5672"</span><span class="p">)),</span>
<span class="ss">username:</span> <span class="s2">"guest"</span><span class="p">,</span>
<span class="ss">virtual_host:</span> <span class="s2">"/"</span><span class="p">,</span>
<span class="ss">x_message_ttl:</span> <span class="s2">"4000"</span>
<span class="p">]</span>
</code></pre></div></div>
<p>I have created a dedicated <code class="language-plaintext highlighter-rouge">mix</code> environment, called <code class="language-plaintext highlighter-rouge">:ci</code> to distinguish configuration for tests running in local vs. tests running there in the cloud.</p>
<hr />
<p>Besides the CI I do run <code class="language-plaintext highlighter-rouge">dialyzer</code> on my sources. Since it’s running in a container, the task takes a while, because it needs to rebuild <code class="language-plaintext highlighter-rouge">plts</code> from the scratch every time. That is why I do it once a day, using <code class="language-plaintext highlighter-rouge">schedule</code> config.</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">name</span><span class="pi">:</span> <span class="s">Dialyzer for My Project</span>
<span class="na">on</span><span class="pi">:</span>
<span class="na">schedule</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">cron</span><span class="pi">:</span> <span class="s2">"</span><span class="s">*</span><span class="nv"> </span><span class="s">1</span><span class="nv"> </span><span class="s">*</span><span class="nv"> </span><span class="s">*</span><span class="nv"> </span><span class="s">*"</span>
<span class="na">jobs</span><span class="pi">:</span>
<span class="na">build</span><span class="pi">:</span>
<span class="na">runs-on</span><span class="pi">:</span> <span class="s">ubuntu-latest</span>
<span class="na">container</span><span class="pi">:</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">elixir:1.9.1-slim</span>
<span class="na">steps</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">uses</span><span class="pi">:</span> <span class="s">actions/checkout@v1</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">Install Dependencies</span>
<span class="na">run</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">MIX_ENV=ci mix local.rebar --force</span>
<span class="s">MIX_ENV=ci mix local.hex --force</span>
<span class="s">MIX_ENV=ci mix deps.get</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">Run All Tests</span>
<span class="na">run</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">MIX_ENV=ci mix code_quality</span>
</code></pre></div></div>
<p>where <code class="language-plaintext highlighter-rouge">code_quality</code> task is an alias declared in <code class="language-plaintext highlighter-rouge">mix.exs</code> as</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defp</span> <span class="n">aliases</span> <span class="k">do</span>
<span class="p">[</span>
<span class="ss">code_quality:</span> <span class="p">[</span><span class="s2">"format"</span><span class="p">,</span> <span class="s2">"credo --strict"</span><span class="p">,</span> <span class="s2">"dialyzer"</span><span class="p">]</span>
<span class="p">]</span>
<span class="k">end</span>
</code></pre></div></div>
<p>That is basically all we need to happily test the project with external dependencies in new Github Workflow.</p>
<p>Happy continiously intergrating!</p>
GenServer At The Therapist's Appointment2019-03-25T00:00:00+00:00https://rocket-science.ru/jokes/2019/03/25/genserver-at-the-therapists-appointment<p><em><strong>Therapist:</strong></em> “What brings you here?”<br />
<em><strong>GenServer:</strong></em> “I am an alone not supervised <code class="language-plaintext highlighter-rouge">GenServer</code> in an aggressive world of unsolicited API requests.”<br />
<em><strong>Therapist:</strong></em> “Have you ever seen a counselor before?”<br />
<em><strong>GenServer:</strong></em> “Sure; I was trying to get an advice from <code class="language-plaintext highlighter-rouge">GenEvent</code> once, but it was deprecated later.”<br />
<em><strong>Therapist:</strong></em> “What is the problem from your viewpoint?”<br />
<em><strong>GenServer:</strong></em> “They call me any time they need anything and I have to respond immediately, 24/7. They could have used casts instead.”<br />
<em><strong>Therapist:</strong></em> “How does this problem typically make you feel?”<br />
<em><strong>GenServer:</strong></em> “It’s overflooding my message box, ruining my schedule and typically makes me feel like I am working in a call centre.”<br />
<em><strong>Therapist:</strong></em> “What makes the problem better?”<br />
<em><strong>GenServer:</strong></em> “Sometimes they put a <code class="language-plaintext highlighter-rouge">GenStage</code> in front of me to control back pressure and these moments are the best in my entire life.”<br />
<em><strong>Therapist:</strong></em> “If you could wave a magic wand, what positive changes would you make happen in your life?”<br />
<em><strong>GenServer:</strong></em> “I would put myself under the supervision tree so I could feel fast when I am not in the mood.”<br />
<em><strong>Therapist:</strong></em> “Overall, how would you describe your mood?”<br />
<em><strong>GenServer:</strong></em> “I am suffering from doing all the duties on my own. Why not spawn another node to split my job into several machines?”<br />
<em><strong>Therapist:</strong></em> “What do you expect from the counseling process?”<br />
<em><strong>GenServer:</strong></em> “I really need a supervisor as I’ve said already. Let it be outside of the tree, but still.”<br />
<em><strong>Therapist:</strong></em> “What would it take to make you feel more content, happier and more satisfied?”<br />
<em><strong>GenServer:</strong></em> “SUPERVISION TREE YOU GODDAMN IDIOT”</p>
Stop Abusing Nihil2019-03-21T00:00:00+00:00https://rocket-science.ru/hacking/2019/03/21/stop-abusing-nihil<blockquote>
<p>I call it my billion-dollar mistake.<br />
<small><em>Sir Charles Antony Richard Hoare</em></small></p>
</blockquote>
<p><img src="/img/sequoia.jpg" alt="Sequoia Gegant" /></p>
<p>This quote quickly became famous amongst static typing adepts,
pure functional programming evangelists and many other cults disciples.
The truth is Tony Hoare never mentioned <strong><code class="language-plaintext highlighter-rouge">nil</code> value</strong>. The full quote is:</p>
<blockquote>
<p>I call it my billion-dollar mistake. It was the invention of the <em><strong>null reference</strong></em> in 1965.</p>
</blockquote>
<p>Emphasis is mine. Null reference! It has nothing to do with <code class="language-plaintext highlighter-rouge">null</code> in <em>Java</em>, which is a pass-by-value.
But <em>Kotlin</em> authors seemed to not bother reading the whole quote and they indeed re-invented the wheel.</p>
<blockquote>
<p>Kotlin makes a distinction between nullable and non-nullable data types. All nullable objects must be declared with a “?” postfix after the type name.<br />
<small><em><a href="https://en.wikipedia.org/wiki/Kotlin_(programming_language)">Kotlin @Wiki</a></em></small></p>
</blockquote>
<p>I doubt there are any reasons of doing that besides the religious worship to the strong typing hype.</p>
<hr />
<p>The truth is in the real world <em>everything</em> is nullable. When I come home in the evening my fridge
might serve me a beer, a sausage, or an apple — depending on how good was my day.
When I am back from my vacations, there is <em>nothing</em> in the fridge. None. Nihil. Nil. Null.</p>
<p>I cannot tell (and nobody could,) there are three different <code class="language-plaintext highlighter-rouge">nil</code>s:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">nil</code> of nullable beer,</li>
<li><code class="language-plaintext highlighter-rouge">nil</code> of nullable sausages,</li>
<li><code class="language-plaintext highlighter-rouge">nil</code> of nullable apples.</li>
</ul>
<p>5 years old would tell you this is bullshit. There is just one <code class="language-plaintext highlighter-rouge">nil</code> in my fridge, meaning <em>there is nothing in there</em>.</p>
<p>And here we come to the main mistake people using languages having <code class="language-plaintext highlighter-rouge">nil</code> objects make all the day.
Developers tend to distinguish “no value” and “nil value.” Which is a complete nonsense either.</p>
<p>Joe Armstrong nailed the proper way of developing computer languages:</p>
<blockquote>
<p>Once you’ve split the world into parallel components, the only way they can talk
to each other is through sending messages. This is almost a biological, a physical
model of the world. When a group of people sit and talk to each other, you can view
them as having independent models in their heads and they talk to each other through
language. Language is messages and what’s in their brain is a state machine.
This seemed a very natural way of thinking.<br />
<small><em><a href="https://www.erlang-solutions.com/blog/let-s-talkconcurrency-with-joe-armstrong.html">Let’s #TalkConcurrency with Joe Armstrong</a></em></small></p>
</blockquote>
<p>This is applicable to the software development process in general. Abstractions
should be as adhered to our real life as possible. Nature in any case does it
better than we.</p>
<p>I’ll tell nobody, feel free to share: did you ever carefully handle <code class="language-plaintext highlighter-rouge">nil</code>s in your hashmap values using
<a href="https://ruby-doc.org/core/Hash.html#method-i-fetch"><code class="language-plaintext highlighter-rouge">Hash#fetch</code></a> or how would it be called in your
language of choice?</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">foo</span> <span class="o">=</span> <span class="nb">hash</span><span class="p">.</span><span class="nf">fetch</span><span class="p">(</span><span class="ss">:foo</span><span class="p">)</span> <span class="p">{</span> <span class="o">|</span><span class="n">key</span><span class="o">|</span> <span class="k">raise</span> <span class="s2">"Key </span><span class="si">#{</span><span class="n">key</span><span class="si">}</span><span class="s2"> is missing!"</span> <span class="p">}</span>
</code></pre></div></div>
<p>If the answer is yes, you are doing it wrong. I have been there for almost twenty years too.
I carefully handled corner cases, I kept <code class="language-plaintext highlighter-rouge">nil</code>s where they belongs, after all one still might
query <code class="language-plaintext highlighter-rouge">Hash#keys</code> and we must return those <em>existing but having <code class="language-plaintext highlighter-rouge">nil</code> values</em>.</p>
<p>Nope. <code class="language-plaintext highlighter-rouge">hash[:foo]</code> is good enough. And the healthy <code class="language-plaintext highlighter-rouge">Hash#keys</code> function should filter out the
keys having <code class="language-plaintext highlighter-rouge">nil</code> values:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">proper_keys</span><span class="p">(</span><span class="nb">hash</span><span class="p">)</span>
<span class="nb">hash</span><span class="p">.</span><span class="nf">map</span> <span class="p">{</span> <span class="o">|</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="o">|</span> <span class="n">k</span> <span class="k">unless</span> <span class="n">v</span><span class="p">.</span><span class="nf">nil?</span> <span class="p">}.</span><span class="nf">compact</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Let me state it again: <code class="language-plaintext highlighter-rouge">nil</code> means an absense. Value <code class="language-plaintext highlighter-rouge">nil</code> means there is no such thing.
Absense of a key means there is no such thing. They are identically equal. That simple.
If you are using <code class="language-plaintext highlighter-rouge">nil</code>s for denoting somewhat else, you are doing it wrong. Somewhat else
desires its own type. If you need to explicitly state somewhat else, create a typed
object and make it having a state. But when there is <em>no such object</em>, <code class="language-plaintext highlighter-rouge">nil</code> is pretty good.</p>
<p>Purists loudly advocate nullable types, monads, all that crap,—might not only help us
to catch errors on compilation stage (static code analysis stage for interpreted languages,)
but it even might obsolete tests. Well, maybe. In academical research papers dealing with toy datasets.
But in the real life, the code as <em>νούμενον</em>, as Kant’s <em>Ding an sich</em> makes a little sense.
To bring a value, the code must deal with some input data. And unless you are
a prison director, you are hardly able to convince your users bringing input data
to always provide a bullet-proof valid datasets.</p>
<p>So the validity check would be still needed. And instead of dealing with enormous
boilerplates of instances of nullable types, just resort to <code class="language-plaintext highlighter-rouge">nil</code>. That is my gained
by blood and sweat advise: use <code class="language-plaintext highlighter-rouge">nil</code>s to denote an absense. Because an absense of
a lemon does not differ from an absense of two apples. We do not receive from the user
<em>the nullable absense of password</em>. We just do receive nothing. Maybe they
wanted to send us a token instead, but failed. The intent does not matter at all.
If <code class="language-plaintext highlighter-rouge">params['password']</code> returns <code class="language-plaintext highlighter-rouge">nil</code>, we should consider the data lacks a mandatory
field. No matter whether we have a key <code class="language-plaintext highlighter-rouge">'password'</code> in our hash with <code class="language-plaintext highlighter-rouge">nil</code> value,
or we don’t have such a key at all.</p>
<hr />
<p>In <em>Ruby</em> we can check if the value is there explicitly with <code class="language-plaintext highlighter-rouge">foo.nil?</code>. In <em>Elixir</em>
we might pattern match on <code class="language-plaintext highlighter-rouge">nil</code>s:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">foo</span><span class="p">(</span><span class="no">nil</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="ss">:error</span>
<span class="k">def</span> <span class="n">foo</span><span class="p">(</span><span class="n">value</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">value</span><span class="p">}</span>
</code></pre></div></div>
<p>Even <em>Javascript</em> allows an explicit check for <code class="language-plaintext highlighter-rouge">null</code>/<code class="language-plaintext highlighter-rouge">undefined</code> (having both is
not a billion-dollar mistake, but it is surely a mistake that is worth a couple of grands;
<code class="language-plaintext highlighter-rouge">null</code> is literally <code class="language-plaintext highlighter-rouge">undefined</code>.) But people continue shooting their own legs
overcomplicating natural things and bringing stillborn abstractions like
<a href="https://en.wikipedia.org/wiki/Null_object_pattern"><em>Null object pattern</em></a>,
<em>Rails</em> <code class="language-plaintext highlighter-rouge">Object#try</code> method, <code class="language-plaintext highlighter-rouge">C#</code>/<code class="language-plaintext highlighter-rouge">Ruby2.5+</code>/<code class="language-plaintext highlighter-rouge">Kotlin</code>/…
<a href="https://en.wikipedia.org/wiki/Safe_navigation_operator"><em>safe-call operators</em></a>, etc.</p>
<p>There is no scenario in which the following method should be called if there is
a value, and <em>simply ignored</em> if there is not. Well, actually I could come up with
some contrived examples, like “send email if the address is there, do nothing otherwise,”
but it is still a code smell and a clear sign of a design flaw. If we are to perform
an action on presented value, we <em>likely</em> are to take some other action on its absense.</p>
<p>Either we want to send an email, and then we should enforce the user to provide it,
or we don’t bother and the whole call is redundant.</p>
<p>Yes, I am exaggerating a bit, but in general it works that way.</p>
<p>Use <code class="language-plaintext highlighter-rouge">nil</code> where it belongs to. Don’t let religious fanatics to spread their monads
everywhere. Types are fine when they are applied as <code class="language-plaintext highlighter-rouge">@spec</code>s in <em>Elixir</em>. Aside.
Waiting there for the static analysis. Not sticking a stick in the wheels and
not getting in our way. We are mature enough to decide ourselves, whether we want
to allow <code class="language-plaintext highlighter-rouge">nil</code>, or gracefully reject, or fail fast, or whatever.</p>
<p>Happy nulling!</p>
Pattern Matching Empty MapSets2019-03-01T00:00:00+00:00https://rocket-science.ru/hacking/2019/03/01/pattern-matching-mapset<p><img src="/img/balcony.jpg" alt="Balcony on Passeig de Gràcia" /></p>
<p><em>Elixir</em> provides <a href="https://hexdocs.pm/elixir/MapSet.html"><code class="language-plaintext highlighter-rouge">MapSet</code></a> module to handle sets—lists of unique elements (shall I say “arrays” instead of lists here?). They are very handy under many circumstances. During my work on the minor update in <a href="https://hexdocs.pm/exvalibur/exvalibur.html"><code class="language-plaintext highlighter-rouge">Exvalibur</code></a>, I changed the list I mistakenly used before to be <code class="language-plaintext highlighter-rouge">MapSet</code>. And there was a pitfall. Everywhere is the code I used different function clauses for empty list. But <code class="language-plaintext highlighter-rouge">MapSet</code> is a module. That said, this code</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">@spec</span> <span class="n">func</span><span class="p">(</span><span class="n">input</span> <span class="p">::</span> <span class="n">list</span><span class="p">())</span> <span class="p">::</span> <span class="ss">:ok</span> <span class="o">|</span> <span class="ss">:error</span>
<span class="k">def</span> <span class="n">func</span><span class="p">([]),</span> <span class="k">do</span><span class="p">:</span> <span class="ss">:error</span>
<span class="k">def</span> <span class="n">func</span><span class="p">([</span><span class="n">_</span> <span class="o">|</span> <span class="n">_</span><span class="p">]),</span> <span class="k">do</span><span class="p">:</span> <span class="ss">:ok</span>
</code></pre></div></div>
<p>cannot be easily converted to work with <code class="language-plaintext highlighter-rouge">MapSet</code>. <a href="https://hexdocs.pm/elixir/MapSet.html#size/1"><code class="language-plaintext highlighter-rouge">MapSet.size/1</code></a> is an external function that cannot be used in guards. Of course one might appeal to <code class="language-plaintext highlighter-rouge">if</code> as a last resort, but that’d be silly. But hey, <code class="language-plaintext highlighter-rouge">MapSet</code> is <em>a struct</em> underneath. And it keeps the values in <a href="https://github.com/elixir-lang/elixir/blob/0558e7c92a93b9c7952464388504a437b9600875/lib/elixir/lib/map_set.ex#L45"><code class="language-plaintext highlighter-rouge">map</code></a> field. So yes, we still can use different clauses and guards</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">@spec</span> <span class="n">func</span><span class="p">(</span><span class="n">input</span> <span class="p">::</span> <span class="n">list</span><span class="p">())</span> <span class="p">::</span> <span class="ss">:ok</span> <span class="o">|</span> <span class="ss">:error</span>
<span class="k">def</span> <span class="n">func</span><span class="p">(%</span><span class="no">MapSet</span><span class="p">{</span><span class="ss">map:</span> <span class="n">map</span><span class="p">})</span> <span class="ow">when</span> <span class="n">map_size</span><span class="p">(</span><span class="n">map</span><span class="p">)</span> <span class="o">==</span> <span class="mi">0</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="ss">:error</span>
<span class="k">def</span> <span class="n">func</span><span class="p">(%</span><span class="no">MapSet</span><span class="p">{}),</span> <span class="k">do</span><span class="p">:</span> <span class="ss">:ok</span>
</code></pre></div></div>
<p>Happy mapsetting!</p>
Elixir Structs on Steroids2019-02-11T00:00:00+00:00https://rocket-science.ru/hacking/2019/02/11/structs-on-steroids<p><img src="/img/cami-de-ronda.jpg" alt="Cami de Ronda" /></p>
<p>Elixir structs are very powerful but sometimes they require too much boilerplate to use them as strict data mappers. If desired, validation should be chained here and there. If the struct has more than three fields,
pattern matching in the function head is to be copy-pasted over the whole module or extracted into a cryptic macro. Even then, although, it quickly becomes hardly maintainable due to <em>unused variables warnings</em> thrown whenever one calls the all-in-one macro and does not use all the variables injected.</p>
<p>I am talking about somewhat like this:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">defstructs</span> <span class="no">MyStruct</span><span class="p">,</span> <span class="p">[</span><span class="ss">:f1</span><span class="p">,</span> <span class="ss">:f2</span><span class="p">,</span> <span class="ss">:f3</span><span class="p">,</span> <span class="ss">:f4</span><span class="p">,</span> <span class="ss">:f5</span><span class="p">]</span>
<span class="k">defmacrop</span> <span class="n">cont</span><span class="p">()</span> <span class="k">do</span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="p">%</span><span class="no">MyStruct</span><span class="p">{</span>
<span class="ss">f1:</span> <span class="n">var!</span><span class="p">(</span><span class="n">f1</span><span class="p">),</span>
<span class="ss">f2:</span> <span class="n">var!</span><span class="p">(</span><span class="n">f2</span><span class="p">),</span>
<span class="ss">f3:</span> <span class="n">var!</span><span class="p">(</span><span class="n">f3</span><span class="p">),</span>
<span class="ss">f4:</span> <span class="n">var!</span><span class="p">(</span><span class="n">f4</span><span class="p">),</span>
<span class="ss">f5:</span> <span class="n">var!</span><span class="p">(</span><span class="n">f5</span><span class="p">)</span>
<span class="p">}</span> <span class="o">=</span> <span class="n">var!</span><span class="p">(</span><span class="n">my_struct</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="o">...</span>
<span class="k">def</span> <span class="n">update_f3</span><span class="p">(</span><span class="n">cont</span><span class="p">(),</span> <span class="n">value</span><span class="p">)</span> <span class="k">do</span>
<span class="p">%</span><span class="no">MyStruct</span><span class="p">{</span><span class="n">my_struct</span> <span class="o">|</span> <span class="ss">f3:</span> <span class="n">value</span><span class="p">}</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The above somehow works, but emits 4 unsolicited warnings for the unused variables <code class="language-plaintext highlighter-rouge">f1</code>, <code class="language-plaintext highlighter-rouge">f2</code>, <code class="language-plaintext highlighter-rouge">f4</code>, and <code class="language-plaintext highlighter-rouge">f5</code>. Also, if the validation is required, it would bring another set of copy-pasta. And copy-pasta begets spaghetti; hardly readable, noisy, unmaintainable code.</p>
<p>Another issue I met in my last project was I have a bunch of function clauses I need to pipe / recursively call one from another. All having the same signature <code class="language-plaintext highlighter-rouge">def foo(%MyStruct{}, params)</code> and all returning the modified <code class="language-plaintext highlighter-rouge">%MyStruct{}</code> back. That sort of task usually arises when one parses text byte-by-byte or transforms the input several times according to some predefined rules. I needed a monadic behaviour there (once any call in the pipe failed, I want the rest transformations to be skipped and the error immediately returned back.)</p>
<p>When I see the clean <em>use-case</em>, say <em>pattern</em>, I produce the reusable package to handle this pattern. Immediately. I’m not really confident about Sandy Metz’ competence in general and I am pretty sure it’s better to avoid code duplication at any reasonable cost. Extracting the code that seems to conform a pattern into a separate reusable package requires exactly the same time and effort as copy-pasting it across the current project.</p>
<p>That said, I created <a href="https://github.com/am-kantox/pyc"><code class="language-plaintext highlighter-rouge">Pyc</code></a> package. It allows transparent validation across inserted data with <a href="https://hexdocs.pm/exvalibur"><code class="language-plaintext highlighter-rouge">Exvalibur</code></a>, chaining functions in a monadic-like way, and have all the keys as local variables inside methods declared with <code class="language-plaintext highlighter-rouge">defmethod</code> without warnings.</p>
<p>It also reasonably implements <a href="https://hexdocs.pm/elixir/Access.html"><code class="language-plaintext highlighter-rouge">Access</code></a> behaviour for wrapped structs and <a href="https://hexdocs.pm/elixir/Collectable.html"><code class="language-plaintext highlighter-rouge">Collectable</code></a> protocol <em>with</em> validation.</p>
<p>Basically, the code using this utility would look like:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">MyStruct</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">Pyc</span><span class="p">,</span>
<span class="ss">definition:</span> <span class="p">[</span><span class="ss">foo:</span> <span class="mi">42</span><span class="p">,</span> <span class="ss">bar:</span> <span class="p">%{},</span> <span class="ss">baz:</span> <span class="p">[]],</span>
<span class="ss">constraints:</span> <span class="p">[</span>
<span class="p">%{</span><span class="ss">matches:</span> <span class="p">%{</span><span class="ss">bar:</span> <span class="sx">~Q[bar]</span><span class="p">},</span>
<span class="ss">conditions:</span> <span class="p">%{</span><span class="ss">foo:</span> <span class="p">%{</span><span class="ss">min:</span> <span class="mi">30</span><span class="p">,</span> <span class="ss">max:</span> <span class="mi">50</span><span class="p">}},</span>
<span class="ss">guards:</span> <span class="p">%{</span><span class="ss">check_bar:</span> <span class="s2">"is_map(bar)"</span><span class="p">}}]</span>
<span class="n">defmethod</span> <span class="ss">:collect_baz</span><span class="p">,</span> <span class="p">[</span><span class="n">value</span><span class="p">]</span> <span class="ow">when</span> <span class="n">is_integer</span><span class="p">(</span><span class="n">value</span><span class="p">)</span> <span class="k">do</span>
<span class="p">%</span><span class="bp">__MODULE__</span><span class="p">{</span><span class="n">this</span> <span class="o">|</span> <span class="ss">baz:</span> <span class="p">[</span><span class="n">value</span> <span class="o">|</span> <span class="n">baz</span><span class="p">]}</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Local variables <code class="language-plaintext highlighter-rouge">this</code> alongside with <code class="language-plaintext highlighter-rouge">foo</code>, <code class="language-plaintext highlighter-rouge">bar</code> and <code class="language-plaintext highlighter-rouge">baz</code> are available inside a block. The function <code class="language-plaintext highlighter-rouge">MyStruct.collect_baz/3</code> might be invoked in the following way:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">%</span><span class="no">MyStruct</span><span class="p">{}</span>
<span class="o">|></span> <span class="no">MyStruct</span><span class="o">.</span><span class="n">collect_baz</span><span class="p">(</span><span class="mi">42</span><span class="p">)</span>
<span class="o">|></span> <span class="no">MyStruct</span><span class="o">.</span><span class="n">collect_baz</span><span class="p">(</span><span class="mi">43</span><span class="p">)</span>
<span class="o">|></span> <span class="no">MyStruct</span><span class="o">.</span><span class="n">collect_baz</span><span class="p">(</span><span class="mi">44</span><span class="p">)</span>
<span class="c1">#⇒ %MyStruct{bar: %{}, baz: ',+*', foo: 42}</span>
</code></pre></div></div>
<p>Also, one might build the struct with comprehesions:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">for</span> <span class="p">{</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="p">}</span> <span class="o"><-</span> <span class="p">[</span><span class="ss">bar:</span> <span class="p">%{</span><span class="ss">some:</span> <span class="ss">:other</span><span class="p">},</span> <span class="ss">baz:</span> <span class="p">[</span><span class="mi">42</span><span class="p">]],</span>
<span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="p">},</span> <span class="ss">into:</span> <span class="p">%</span><span class="no">MyStruct</span><span class="p">{}</span>
<span class="c1">#⇒ %MyStruct{bar: %{some: :other}, baz: '*', foo: 42}</span>
</code></pre></div></div>
<p>If validation fails, all subsequent calls in the pipeline are skipped and the last value caused the validation error is returned.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">%</span><span class="no">MyStruct</span><span class="p">{}</span>
<span class="o">|></span> <span class="no">MyStruct</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="ss">:foo</span><span class="p">,</span> <span class="mi">32</span><span class="p">)</span>
<span class="o">|></span> <span class="no">MyStruct</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="ss">:foo</span><span class="p">,</span> <span class="mi">42</span><span class="p">)</span>
<span class="o">|></span> <span class="no">MyStruct</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="ss">:foo</span><span class="p">,</span> <span class="mi">52</span><span class="p">)</span>
<span class="o">|></span> <span class="no">MyStruct</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="ss">:foo</span><span class="p">,</span> <span class="mi">62</span><span class="p">)</span>
<span class="c1">#⇒ {:error, %MyStruct{bar: %{}, baz: [], foo: 52}}</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">validate/1</code> might be invoked at any moment passing an instance of the struct to it.</p>
<hr />
<p><em>Sidenote:</em> for anybody curious, <code class="language-plaintext highlighter-rouge">Pyc</code> name refers to <code class="language-plaintext highlighter-rouge">🐍 PYthon Class</code> since all the functions declared with <code class="language-plaintext highlighter-rouge">defmethod/3</code> macro implicitly receive the instance of <code class="language-plaintext highlighter-rouge">self</code> as the first argument (called <code class="language-plaintext highlighter-rouge">this</code> to admire Javascript naming of implicit garbage injected into the current context.)</p>
<hr />
<p>Happy structuring!</p>
Gospel of Barabbas or Concurrent Execution2019-01-19T00:00:00+00:00https://rocket-science.ru/hacking/2019/01/19/gospel-of-barabbas-or-concurrent-execution<p><img src="/img/guillotine.jpg" alt="Concurrent Execution" /></p>
<blockquote>
<p>“Premature optimization is the root of all evil” — Donald Knuth.</p>
</blockquote>
<p>This is indeed true, unless it is not. People tend to snatch a phrase out of context and make the dictum out of it. This is indeed awful habit, but we all do that, specifically when the saying is vivid, concise, actually demanding to be graved in stone.</p>
<p>The sleep of reason produces monsters standing on the shoulders of giants. “Prefer duplication over the wrong abstraction” by <a href="https://www.sandimetz.com/blog/2016/1/20/the-wrong-abstraction">Sandy Metz</a>, while being <em>somewhat</em> reasonable by itself, is a nonsense in general. It might be paraphrased as “prefer shooting yourself in the foot rather than in the head.” Or, “prefer worse over worst.”</p>
<p>If I had no choice but one of the above, I’d definitely pick up the duplication. But I live in the more or less free world, where we also have “good,” “better,” and even “best” solutions. Taken out of context, too literally, this wording loses the essence, even if there was any. That’s why I hate all these silver bullets for developers, arising from the ashes every decade to sink into oblivion a decade later. Ten years ago this goddamn hype panacea was TDD, nowadays it’s rich typing. All of those bring some goodness to the development process, both are by no mean 42 (nor the answer to the ultimate question of life, the universe, and everything.)</p>
<p>OTOH, there are paradigms and concepts sitting there for over half of a century. They are not as touted, but they indeed are applicable technically everywhere. AST on hand, VM for execution, isolated process memory, garbage collection, error isolation, preemptive multitasking. Many languages benefit from one or more of the above. Erlang benefits from all of them. And here we gradually come to the topic of this writing, to <em>multitasking</em>. <strong>Each and every system in the universe, while growing, gets to the point of no return, where it becomes mandatory to serve several tasks simultaneously.</strong></p>
<p>There are at least three confusing terms, that are nearly synonyms, showing up when the “multitasking” is mentioned. Those are: concurrency, parallelism, asynchronous processing. Unfortunately, they do mean different things. In Layman’s terms, <strong>parallelism</strong> is used for <strong>𝔸 executed when 𝔹 is executed</strong>, <strong>concurrency</strong> is used for <strong>𝔸 and 𝔹 are executed simultaneously, <em>interleaved</em></strong>, and <strong>asynchrony</strong> is used for <strong>𝔸 has no contract to finish before it returns the control flow</strong>.</p>
<p>As one might see, the latter does not mention <em>𝔹</em> process at all; asynchronous processing has basically nothing to do neither with concurrency nor with parallelism. It’s worth it to mention, that “concurrency” is to some extent a subset of “parallelism,” and today we are going to talk about concurrency.</p>
<p><img src="/img/concurrent-mw.png" alt="Concurrency :: Courtesy of Merriam-Webster Dictionary" /></p>
<p>The image above appeared here only to dilute the wall of tedious plain text. While in CS we borrow words from English, they should be always treated as loanwords. The real meaning might vary.</p>
<p>Concurrent execution abilities imply three main things:</p>
<ul>
<li>the underneath software (VM and/or OS) should allow as many <em>execution contexts</em> as we need;</li>
<li>the intercommunication between <em>execution contexts</em> should be somehow possible</li>
<li>the <em>execution contexts</em> should not become zombie, whatever it means.</li>
</ul>
<p>I intentionally did not use words “process,” “thread,” and alike in the statement above, since multitasking might use OS processes, OS threads, VM lightweight processes (as in Erlang,) Modula2–like coroutines (as in Go,) and even the mixture of all of them.</p>
<p>Lightweight processes are in general more robust, because of their total count does not depend on the underlying OS limitations:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">cat</span> /proc/sys/kernel/threads-max
<span class="c">#⇒ 94389 # on your system this might differ</span>
</code></pre></div></div>
<p>The number above might be adjusted as needed, but OS would not feel good when the number increases millions, and 1M of processes for ErlangVM is nothing to worry about.</p>
<hr />
<p>When we talk about <em>multitasking</em>, we should distinguish between whether it is preemptive or not. According to <a href="https://www.techopedia.com/definition/8949/preemptive-multitasking">Techopedia</a>,</p>
<blockquote>
<p>Preemptive multitasking is a type of multitasking that allows computer programs to share operating systems (OS) and underlying hardware resources. It divides the overall operating and computing time between processes, and the switching of resources between different processes occurs through predefined criteria.</p>
<p>Preemptive multitasking is also known as time-shared multitasking</p>
</blockquote>
<p>The opposite to preemptive multitasking would be a “cooperative multitasking.” Cooperative multitasking is what makes <em>Java VM</em> to stop the world to GC and <em>Go</em> to become zombie when the amount of long-running go-routines exceeds an amount of cores on the target machine. Please don’t try cooperative multitasking at home or school.</p>
<p>The main advantage of using concurrent processing is involving all the compute power available. My laptop, having 8 cores, technically could process anything 8× times faster than in non-concurrent mode. Well, not <em>as</em> anything.</p>
<blockquote>
<p>“I got 99 problems with performance.”<br />
“So I used concurrency.”<br />
“100 I Now problems. have”<br />
— <em>inspired by <a href="https://www.xkcd.com/1171/">Perl Problems by Randall Munroe</a></em></p>
</blockquote>
<p>It turns out, we cannot simply instruct the process to run concurrently in a hope everything would eventually be right. It would work if and only results do not depend on history (read: on previous results.) Sometimes it’s true. Sometimes it’s not.</p>
<p>For instance, the factorial could hardly be computed concurrently. Reading and processing text files, on the other hand, is perfectly being parallelized. We just need to make sure we did not screw the order of rows up in the very end, and this is easy done with e. g. indexing.</p>
<hr />
<p>Complicated business flows are usually a mix of what <em>could be</em> parallelized and what <em>could not</em>. Often huge flows might be <em>somewhat</em> parallelized into execution blocks that must be executed subsequentially (we usually call this process partitioning.)</p>
<p>Imagine we supply <a href="https://en.wikipedia.org/wiki/John_Napier">John Napier</a> with a multicore laptop to alleviate his effort in producing <em>Mirifici Logarithmorum Canonis Descriptio</em>. Then he probably would feed the machine with numbers, calculate the logarithms <em>concurrently</em>, collect the result, and finally print the values <em>sequentially</em>, smaller to greater, to simplify search for those having no XXI century laptop on hand.</p>
<p>In the next chapter I am going to reveal how complicated business flows might be split into parts that could be processed concurrently to increase both performance and reliability.</p>
<p>Happy concurring!</p>
Iteraptor → Unforeseen utilization2018-12-31T00:00:00+00:00https://rocket-science.ru/hacking/2018/12/31/iteraptor-unforeseen-utilization<p><a href="https://github.com/am-kantox/iteraptor"><strong><code class="language-plaintext highlighter-rouge">Iteraptor</code></strong></a> library was initially conceived as a helper to iterate/map/reduce deeply nested enumerables, like maps, keywords and lists.</p>
<p>Today I occasionally discovered a new—initially undesired—application for it. I was introducing <a href="https://elixir-lang.org/blog/2017/10/31/stream-data-property-based-testing-and-data-generation-for-elixir/">property-based testing</a> for <a href="https://github.com/am-kantox/camarero"><code class="language-plaintext highlighter-rouge">Camarero</code></a>. The latter serves JSON, so the integration test looked like</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmacrop</span> <span class="n">key_value</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="kn">quote</span><span class="p">(</span><span class="k">do</span><span class="p">:</span> <span class="n">map_of</span><span class="p">(</span><span class="n">key</span><span class="p">(),</span> <span class="n">map_or_leaf</span><span class="p">()))</span>
<span class="n">test</span> <span class="s2">"properly handles nested terms"</span> <span class="k">do</span>
<span class="n">check</span> <span class="n">all</span> <span class="n">term</span> <span class="o"><-</span> <span class="n">key_value</span><span class="p">(),</span> <span class="ss">max_runs:</span> <span class="mi">25</span> <span class="k">do</span>
<span class="no">Enum</span><span class="o">.</span><span class="n">each</span><span class="p">(</span><span class="n">term</span><span class="p">,</span> <span class="k">fn</span> <span class="p">{</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="p">}</span> <span class="o">-></span>
<span class="no">Camarero</span><span class="o">.</span><span class="no">Carta</span><span class="o">.</span><span class="no">Heartbeat</span><span class="o">.</span><span class="n">plato_put</span><span class="p">(</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="p">)</span>
<span class="n">conn</span> <span class="o">=</span>
<span class="ss">:get</span>
<span class="o">|></span> <span class="n">conn</span><span class="p">(</span><span class="s2">"/api/v1/heartbeat/</span><span class="si">#{</span><span class="n">k</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
<span class="o">|></span> <span class="no">Camarero</span><span class="o">.</span><span class="no">Handler</span><span class="o">.</span><span class="n">call</span><span class="p">(</span><span class="nv">@opts</span><span class="p">)</span>
<span class="n">assert</span> <span class="n">conn</span><span class="o">.</span><span class="n">state</span> <span class="o">==</span> <span class="ss">:sent</span>
<span class="n">assert</span> <span class="n">conn</span><span class="o">.</span><span class="n">status</span> <span class="o">==</span> <span class="mi">200</span>
<span class="n">assert</span> <span class="n">conn</span><span class="o">.</span><span class="n">resp_body</span> <span class="o">|></span> <span class="no">Jason</span><span class="o">.</span><span class="n">decode!</span><span class="p">()</span> <span class="o">|></span> <span class="no">Map</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">"value"</span><span class="p">)</span> <span class="o">==</span> <span class="n">v</span>
<span class="k">end</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Here is the thing: the above fails because atoms are jsonified (jsonificated?) as binaries / strings. When we need to check if the <em>real data</em> is sent as JSON properly, it might quickly become cumbersome to check the outcome. Also, there would be issues to produce JSON out of keywords because keywords are nothing but lists of 2-elements tuples and tuples are not JSON-serializable.</p>
<p>Luckily enough, I had already the library to modify deeply nested terms, so I decided to add the new function to it. Here is it.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">@spec</span> <span class="n">jsonify</span><span class="p">(</span><span class="no">Access</span><span class="o">.</span><span class="n">t</span><span class="p">(),</span> <span class="n">opts</span> <span class="p">::</span> <span class="n">list</span><span class="p">())</span> <span class="p">::</span> <span class="p">%{</span><span class="n">required</span><span class="p">(</span><span class="n">binary</span><span class="p">())</span> <span class="o">=></span> <span class="n">any</span><span class="p">()}</span>
<span class="k">def</span> <span class="n">jsonify</span><span class="p">(</span><span class="n">input</span><span class="p">,</span> <span class="n">opts</span> <span class="p">\\</span> <span class="p">[])</span>
<span class="k">def</span> <span class="n">jsonify</span><span class="p">([{</span><span class="n">_</span><span class="p">,</span> <span class="n">_</span><span class="p">}</span> <span class="o">|</span> <span class="n">_</span><span class="p">]</span> <span class="o">=</span> <span class="n">input</span><span class="p">,</span> <span class="n">opts</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">input</span> <span class="o">|></span> <span class="no">Enum</span><span class="o">.</span><span class="n">into</span><span class="p">(%{})</span> <span class="o">|></span> <span class="n">jsonify</span><span class="p">(</span><span class="n">opts</span><span class="p">)</span>
<span class="k">def</span> <span class="n">jsonify</span><span class="p">(</span><span class="n">input</span><span class="p">,</span> <span class="n">opts</span><span class="p">)</span> <span class="ow">when</span> <span class="n">is_list</span><span class="p">(</span><span class="n">input</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="no">Enum</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">input</span><span class="p">,</span> <span class="o">&</span><span class="n">jsonify</span><span class="p">(</span><span class="nv">&1</span><span class="p">,</span> <span class="n">opts</span><span class="p">))</span>
<span class="k">def</span> <span class="n">jsonify</span><span class="p">(</span><span class="n">input</span><span class="p">,</span> <span class="n">opts</span><span class="p">)</span> <span class="ow">when</span> <span class="ow">not</span> <span class="n">is_map</span><span class="p">(</span><span class="n">input</span><span class="p">)</span> <span class="ow">and</span> <span class="ow">not</span> <span class="n">is_list</span><span class="p">(</span><span class="n">input</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="k">if</span><span class="p">(</span><span class="n">opts</span><span class="p">[</span><span class="ss">:values</span><span class="p">]</span> <span class="o">&&</span> <span class="n">is_atom</span><span class="p">(</span><span class="n">input</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="n">to_string</span><span class="p">(</span><span class="n">input</span><span class="p">),</span> <span class="k">else</span><span class="p">:</span> <span class="n">input</span><span class="p">)</span>
<span class="k">def</span> <span class="n">jsonify</span><span class="p">(</span><span class="n">input</span><span class="p">,</span> <span class="n">opts</span><span class="p">)</span> <span class="k">do</span>
<span class="no">Iteraptor</span><span class="o">.</span><span class="n">map</span><span class="p">(</span>
<span class="n">input</span><span class="p">,</span>
<span class="k">fn</span> <span class="p">{</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="p">}</span> <span class="ow">when</span> <span class="n">is_list</span><span class="p">(</span><span class="n">k</span><span class="p">)</span> <span class="o">-></span>
<span class="p">{</span><span class="n">k</span> <span class="o">|></span> <span class="no">List</span><span class="o">.</span><span class="n">last</span><span class="p">()</span> <span class="o">|></span> <span class="n">to_string</span><span class="p">(),</span> <span class="n">jsonify</span><span class="p">(</span><span class="n">v</span><span class="p">,</span> <span class="n">opts</span><span class="p">)}</span>
<span class="k">end</span><span class="p">,</span>
<span class="ss">yield:</span> <span class="ss">:all</span>
<span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>
<p>It converts keywords to maps and atom keys to strings, and—optionally (if <code class="language-plaintext highlighter-rouge">values: true</code> is passed as the second parameter)—converts atoms to binaries in values.</p>
<p>I am posting this code here mostly to promote <code class="language-plaintext highlighter-rouge">Iteraptor</code> itself to show how easy it would be to transform any nested term with it.</p>
<p>Turning back to the original intent, to test the JSON response for deeply nested list/map/term one would tell <code class="language-plaintext highlighter-rouge">mix</code> to include <code class="language-plaintext highlighter-rouge">Iteraptor</code> package for <code class="language-plaintext highlighter-rouge">:test</code> environment only:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="p">{</span><span class="ss">:iteraptor</span><span class="p">,</span> <span class="s2">"~> 1.0"</span><span class="p">,</span> <span class="ss">only:</span> <span class="ss">:test</span><span class="p">}</span>
</code></pre></div></div>
<p>and use somewhat like the code below to deep-compare the JSON against original data:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="n">term</span> <span class="o">=</span> <span class="no">Iteraptor</span><span class="o">.</span><span class="n">jsonify</span><span class="p">(</span><span class="n">term</span><span class="p">,</span> <span class="ss">values:</span> <span class="no">true</span><span class="p">)</span>
<span class="n">conn</span> <span class="o">=</span>
<span class="ss">:get</span>
<span class="o">|></span> <span class="n">conn</span><span class="p">(</span><span class="s2">"/api/v1/my_term"</span><span class="p">)</span>
<span class="o">|></span> <span class="no">MyJsonEndpoint</span><span class="o">.</span><span class="n">call</span><span class="p">(</span><span class="nv">@opts</span><span class="p">)</span>
<span class="n">assert</span> <span class="n">conn</span><span class="o">.</span><span class="n">resp_body</span> <span class="o">|></span> <span class="no">Jason</span><span class="o">.</span><span class="n">decode!</span><span class="p">()</span> <span class="o">==</span> <span class="n">term</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Whether one needs to be able to transparently serve terms that might include keywords, <code class="language-plaintext highlighter-rouge">Iteraptor</code> should be included as dependency in all environments and the call to <code class="language-plaintext highlighter-rouge">Iteraptor.jsonify(term)</code> should be done before passing the term to JSON encoder. <code class="language-plaintext highlighter-rouge">values: true</code> parameter is not needed in this scenario since JSON serializers are able to take care of atoms.</p>
<p>Happy testing!</p>
Idempotent Supervision Tree2018-12-29T00:00:00+00:00https://rocket-science.ru/hacking/2018/12/29/idempotent-supervision-tree<p>One of the most exciting features of <a href="https://learnyousomeerlang.com/what-is-otp">OTP</a>, the heart of <em>Erlang VM</em>, would be the <em>Supervision Tree</em>. The long-lived processes are maintained as a tree structure, containing <a href="https://hexdocs.pm/elixir/Supervisor.html"><em>Supervisors</em></a> and <em>Workers</em>. The latter are <em>leaves</em> in the tree. We do not need to be babysitting each and every process, instead we practice <a href="http://verraes.net/2014/12/erlang-let-it-crash/"><em>Let It Crash</em></a> religion. If something went wrong, in most cases <em>we do nothing</em>. We just let the failed process crash and the supervisor will restart it gracefully.</p>
<p>This is one of the best design decision I can think of.</p>
<p>Wrong input?—Let it crash. Third party service failure?—Let it crash. Need the process to re-initialize itself?—Let it crash. The supervisor will take care of it.</p>
<p>That approach has one slight drawback though. The supervision tree must be elaborated very thoughtfully. <strong>Process restarts must be idempotent.</strong> Since we don’t handle crashes manually, we cannot know in advance when and under what circumstances our process would crash. That means, it might be restarted in literally every single moment, maybe several times. That is why idempotency is a must.</p>
<p>There are many aspects to be taken into account here, the margins here are too small to list them all. I am going to show probably the most common glitch with process re-initialization that makes it not idempotent and how to overcome the issue.</p>
<hr />
<p>All of us do create our own <code class="language-plaintext highlighter-rouge">GenServer</code>s. The most common approach is to implement our <code class="language-plaintext highlighter-rouge">start_link/{0,1}</code> function to delegate to the <a href="https://hexdocs.pm/elixir/GenServer.html#start_link/3"><code class="language-plaintext highlighter-rouge">GenServer.start_link/3</code></a>:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">@doc</span> <span class="sx">~s""</span><span class="s2">"
Starts a new </span><span class="si">#{</span><span class="bp">__MODULE__</span><span class="si">}</span><span class="s2"> linked to the current process.
"""</span>
<span class="k">def</span> <span class="n">start_link</span><span class="p">(</span><span class="n">_opts</span> <span class="p">\\</span> <span class="p">[]),</span>
<span class="k">do</span><span class="p">:</span> <span class="no">GenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="p">%{},</span> <span class="ss">name:</span> <span class="bp">__MODULE__</span><span class="p">)</span>
</code></pre></div></div>
<p>Sooner or later we’ll need to perform some more cumbersome initialization upon startup and the first intent would be to do something like:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">start_link</span><span class="p">(</span><span class="n">_opts</span> <span class="p">\\</span> <span class="p">[])</span> <span class="k">do</span>
<span class="n">result</span> <span class="o">=</span> <span class="no">GenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="p">%{},</span> <span class="ss">name:</span> <span class="bp">__MODULE__</span><span class="p">)</span>
<span class="n">do_initialization</span><span class="p">()</span>
<span class="n">result</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Lo and behold! We had just broken the idempotency. If this function will be called twice in a row, the initialization will be performed <em>twice</em>. Which is rarely if never a desired behaviour.</p>
<p>Luckily, <code class="language-plaintext highlighter-rouge">GenServer.start_link/3</code> is smart enough to report how the linked start went. It returns one of those <a href="https://hexdocs.pm/elixir/GenServer.html#t:on_start/0"><code class="language-plaintext highlighter-rouge">on_start</code></a>:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">@spec</span> <span class="o">...</span> <span class="p">::</span> <span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">pid</span><span class="p">()}</span> <span class="o">|</span> <span class="ss">:ignore</span> <span class="o">|</span> <span class="p">{</span><span class="ss">:error</span><span class="p">,</span> <span class="p">{</span><span class="ss">:already_started</span><span class="p">,</span> <span class="n">pid</span><span class="p">()}</span> <span class="o">|</span> <span class="n">term</span><span class="p">()}</span>
</code></pre></div></div>
<p>So, the better way seems to be to explicitly match to <code class="language-plaintext highlighter-rouge">{:ok, pid}</code> and perform initialization only then.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">start_link</span><span class="p">(</span><span class="n">_opts</span> <span class="p">\\</span> <span class="p">[])</span> <span class="k">do</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">pid</span><span class="p">}</span> <span class="o">=</span> <span class="no">GenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="p">%{},</span> <span class="ss">name:</span> <span class="bp">__MODULE__</span><span class="p">)</span>
<span class="n">do_initialization</span><span class="p">()</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">pid</span><span class="p">}</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Right?—Nope. The code above will blow up when the process is already started. By default, the supervisor will attempt to restart it three times (the number is configurable) and give up then. Yes, the process was likely already started, so no issue, but this is incorrect; the process should not blow up all of sudden for no reason.</p>
<p>The proper solution would be to act on successful start and <em>let anything else sink to the caller</em>. <a href="https://hexdocs.pm/elixir/Kernel.SpecialForms.html#with/1">Kernel.SpecialForms.with/1</a> <em>monad-like</em> construct comes to the rescue:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">start_link</span><span class="p">(</span><span class="n">_opts</span> <span class="p">\\</span> <span class="p">[])</span> <span class="k">do</span>
<span class="n">with</span> <span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">pid</span><span class="p">}</span> <span class="o"><-</span> <span class="no">GenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="p">%{},</span> <span class="ss">name:</span> <span class="bp">__MODULE__</span><span class="p">)</span> <span class="k">do</span>
<span class="n">do_initialization</span><span class="p">()</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">pid</span><span class="p">}</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Now if the process has started successfully, we perform the required initialization. If not, we just <em>pass the error returned to the calling process</em>. Let them deal with the issue. This is the correct way to perform idempotent initialization of the process.</p>
<p>Happy reloading!</p>
Plug in JSON API Readonly Webserver2018-12-28T00:00:00+00:00https://rocket-science.ru/hacking/2018/12/28/json-api-read-only-web-server<p>All our microservices expose kinda heartbeat interfaces to <em>Consul</em> which we use as a health check. Some of the services also provide some access to the internal data they deal with. I am not talking about their main duties: those are very well handled by <code class="language-plaintext highlighter-rouge">RabbitMQ</code> and <code class="language-plaintext highlighter-rouge">Redis</code>.</p>
<p>Just sometimes it makes sense to export data that is indeed there to other services that might require it. For instance, I want to completely get rid of <em>Redis</em> in favour of in-house solution for key-value pair storage as we successfully did exactly one year ago with <em>PubSub</em>.</p>
<p>So instead of re-inventing a wheel with every new microservice, I decided to create a pluggable library, that might be serving any arbitrary data from any application with zero code (if we don’t count 5-LoCs <code class="language-plaintext highlighter-rouge">config.exs</code>.)</p>
<p>The solution is based on this <a href="https://twitter.com/pragdave/status/1077775018942185472?s=20">Dave Thomas’ tweet</a>.</p>
<h2 id="lightweight-json-api-server-embeddable-into-any-project">Lightweight Json API Server, Embeddable Into Any Project</h2>
<p><strong>Camarero</strong> is a ready-to-use solution to add some JSON API functionality to the existing application, or to implement the read-only JSON API from the scratch when more sophisticated (read: <em>heavy</em>) solutions are not desirable. Below is the typical picture of how <em>Camarero</em> is supposed to be plugged in.</p>
<p><img src="https://raw.githubusercontent.com/am-kantox/camarero/master/stuff/camarero.png" alt="Camarero Ties" /></p>
<p>It is designed to be very simple and handy for read-only web access to the data. It might indeed be a good candidate to replace <em>Redis</em> or any other key-value store. <strong>It is blazingly, dead fast.</strong></p>
<p>Here are response times for the 1M key-value storage behind.</p>
<p><img src="https://raw.githubusercontent.com/am-kantox/camarero/master/stuff/1M.png" alt="1M key-value storage lookup: 10μs±" /></p>
<p>Yes, that’s it. The response time for the request against one million key-values store takes <em>dozens of microseconds</em>.</p>
<h2 id="implementation-details">Implementation details</h2>
<p><strong>Camarero</strong> is supposed to be plugged into the functional application. It handles the configured routes/endpoints by delegating to the configured handler modules. The simplest configuration might looks like:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">config</span> <span class="ss">:camarero</span><span class="p">,</span>
<span class="ss">carta:</span> <span class="p">[</span><span class="no">Camarero</span><span class="o">.</span><span class="no">Carta</span><span class="o">.</span><span class="no">Heartbeat</span><span class="p">],</span>
<span class="ss">root:</span> <span class="s2">"api/v1"</span>
</code></pre></div></div>
<p>The above is the default; <code class="language-plaintext highlighter-rouge">/api/v1</code> would be the root of the web server, single <code class="language-plaintext highlighter-rouge">Camarero.Carta.Heartbeat</code> module is declared as handler. The handlers might be also added dynamically by calls to <code class="language-plaintext highlighter-rouge">Camarero.Catering.route!</code>.</p>
<h3 id="handlers">Handlers</h3>
<p><em>Handler</em> is a module implementing <code class="language-plaintext highlighter-rouge">Camarero.Plato</code> behaviour. It consists of methods to manipulate the conteiner behind it. Any module might implement this behaviour to be used as a handler for incoming HTTP requests.</p>
<p>There is also <code class="language-plaintext highlighter-rouge">Camarero.Tapas</code> behaviour scaffolding the container implementation inside <code class="language-plaintext highlighter-rouge">Camarero.Plato</code>.</p>
<p>The default implementation using <code class="language-plaintext highlighter-rouge">%{}</code> map as a container, looks pretty simple:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">Camarero</span><span class="o">.</span><span class="no">Carta</span><span class="o">.</span><span class="no">Heartbeat</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">Camarero</span><span class="o">.</span><span class="no">Plato</span>
<span class="k">end</span>
</code></pre></div></div>
<p>This is an exact exerpt from <code class="language-plaintext highlighter-rouge">Heartbeat</code> module that comes with this package. For more complicated/sophisticated usages please refer to the <a href="https://hexdocs.pm/camarero">documentation</a>.</p>
<p>All the methods from both <code class="language-plaintext highlighter-rouge">Camarero.Tapas</code> and <code class="language-plaintext highlighter-rouge">Camarero.Plato</code> default implementations are overridable. E. g. to use the custom route for the module (default is the not fully qualified underscored module name,) as well as custom container, one might do the following:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">Camarero</span><span class="o">.</span><span class="no">Carta</span><span class="o">.</span><span class="no">Heartbeat</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">Camarero</span><span class="o">.</span><span class="no">Plato</span><span class="p">,</span> <span class="ss">container:</span> <span class="p">%</span><span class="no">MyStructWithAccessBehaviour</span><span class="p">{}</span>
<span class="nv">@impl</span> <span class="no">true</span>
<span class="k">def</span> <span class="n">plato_route</span><span class="p">(),</span> <span class="k">do</span><span class="p">:</span> <span class="s2">"internal/heartbeat"</span>
<span class="k">end</span>
</code></pre></div></div>
<h3 id="web-server-config">Web server config</h3>
<p><strong>Camarero</strong> runs over <em>Cowboy2</em> with <em>Plug</em>. To configure <em>Cowboy</em>, one might specify in the <code class="language-plaintext highlighter-rouge">config.exs</code> file:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">config</span> <span class="ss">:camarero</span><span class="p">,</span>
<span class="ss">cowboy:</span> <span class="p">[</span><span class="ss">port:</span> <span class="mi">4001</span><span class="p">,</span> <span class="ss">scheme:</span> <span class="ss">:http</span><span class="p">,</span> <span class="ss">options:</span> <span class="p">[]]</span>
</code></pre></div></div>
<h2 id="installation">Installation</h2>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">deps</span> <span class="k">do</span>
<span class="p">[</span>
<span class="p">{</span><span class="ss">:camarero</span><span class="p">,</span> <span class="s2">"~> 0.1"</span><span class="p">}</span>
<span class="p">]</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Happy serving!</p>
Elixir Compilation Hooks2018-12-26T00:00:00+00:00https://rocket-science.ru/hacking/2018/12/26/elixir-compilation-hooks<p>Elixir has a very sophisticated macro infrastructure. Also there is a wording you are to be immediately told when starting to deal with the language and getting excited about the power of macros. <strong>“The first rule of using macros is you do not use macros.”</strong></p>
<p>Sometimes they grudgingly append “unless you are in an urgent need and you know what you are doing.”</p>
<p>I agree one should not use macros in the very beggining of the journey. But once we dove deeply into this beatiful language, we all do extensively use them because macros allow us to drastically decrease an amount of boilerplate that might be needed <em>and</em> provide the natural and very handy way to manipulate <em>AST</em>. <em>Phoenix</em>, <em>Ecto</em>, all the great huge libraries do heavily use macros.</p>
<p>The above is true for any multipurpose <em>library</em>/<em>package</em>. In my experience, regular projects usually do not require creating macros, or need only a few helpers to DRY. Libraries, contrary to the above, often consist of macros at ratio 80/20 to regular code.</p>
<p>I am not going to play sandbox here; if you wonder what macros are at all or how macros ever work in Elixir you’d better close this page immediately and read a brilliant <a href="https://pragprog.com/book/cmelixir/metaprogramming-elixir"><em>Metaprogramming Elixir</em></a> book by <em>Chris McCord</em>, the creator of <em>Phoenix Framework</em>. I am to only show some tricks to make your already existing macro ecosystem better.</p>
<hr />
<p>Macros are purposedly stingy documented. This knife is too sharp to advertise it to toddlers.</p>
<p>Basically, macros are called back by the compiler when the external code calls <code class="language-plaintext highlighter-rouge">use MyLib</code> and our module <code class="language-plaintext highlighter-rouge">MyLib</code> implements <code class="language-plaintext highlighter-rouge">__using__/1</code> callback macro. If the above sounds cumbersome, please, stop bearing with me now and read the book I mentioned above instead.</p>
<p><code class="language-plaintext highlighter-rouge">__using__</code> callback accepts an argument, so that the library owner might allow users to pass some parameters to it. Here is an example from one of my internal projects that uses a macro call with parameters:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">User</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">MyApp</span><span class="o">.</span><span class="no">ActiveRecord</span><span class="p">,</span>
<span class="ss">repo:</span> <span class="no">MyApp</span><span class="o">.</span><span class="no">Repo</span><span class="p">,</span>
<span class="ss">roles:</span> <span class="sx">~w|supervisor client subscriber|</span><span class="p">,</span>
<span class="ss">preload:</span> <span class="sx">~w|setting companies|a</span>
</code></pre></div></div>
<p>The keyword parameter will be passed to <code class="language-plaintext highlighter-rouge">MyApp.ActiveRecord.__using__/1</code> and there I deal with it.</p>
<hr />
<p>Sometimes we want to restrict macro usage to some subset of modules (e. g. to allow using it in <em>structs</em> only.) The explicit check inside the implementation of <code class="language-plaintext highlighter-rouge">__using__/1</code> won’t work, because at this moment the <em>currently being compiled</em> module does not have an access to it’s <code class="language-plaintext highlighter-rouge">__ENV__</code> (and the latter is not complete by any mean.) So usually one wants somewhat check <em>after</em> the compilation is done.</p>
<p>No issue, there are two <a href="https://hexdocs.pm/elixir/Module.html#module-module-attributes">module attributes</a> designed explicitly for that purpose. Welcome, <a href="https://hexdocs.pm/elixir/Module.html#module-compile-callbacks"><em>Compile Callbacks</em></a>!</p>
<p>An excerpt from the docs:</p>
<blockquote>
<p><strong><code class="language-plaintext highlighter-rouge">@after_compile</code></strong>
A hook that will be invoked right after the current module is compiled.</p>
<p>Accepts a module or a {module, function_name} tuple. The function must take two arguments: the module environment and its bytecode. When just a module is provided, the function is assumed to be <code class="language-plaintext highlighter-rouge">__after_compile__/2</code>.</p>
<p>Callbacks registered first will run last.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>defmodule MyModule do
@after_compile __MODULE__
def __after_compile__(env, _bytecode) do
IO.inspect env
end
end
</code></pre></div> </div>
</blockquote>
<p>I strongly encourage to never inject <code class="language-plaintext highlighter-rouge">__after_compile__/2</code> directly into generated code since it might lead to clashes with end-user intents (they might want to use their own compile callbacks.) Define a function somewhere inside your <code class="language-plaintext highlighter-rouge">MyLib.Helpers</code> or like and pass a tuple to <code class="language-plaintext highlighter-rouge">@after_compile</code>:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">quote</span> <span class="ss">location:</span> <span class="ss">:keep</span> <span class="k">do</span>
<span class="nv">@after_compile</span><span class="p">({</span><span class="no">MyLib</span><span class="o">.</span><span class="no">Helpers</span><span class="p">,</span> <span class="ss">:after_mymodule_callback</span><span class="p">})</span>
<span class="k">end</span>
</code></pre></div></div>
<p>This callback will be called immediately after the respective module that uses our library is compiled, receiving two parameters, the <code class="language-plaintext highlighter-rouge">__ENV__</code> struct and the bytecode of the compiled module. The latter is rarely used by mere mortals; the former provides everything we need. Below is the example of how do I defend from attempts to <a href="https://hexdocs.pm/iteraptor/Iteraptor.Iteraptable.html"><code class="language-plaintext highlighter-rouge">use Iteraptable</code></a> in non-structs by calling <code class="language-plaintext highlighter-rouge">__struct__</code> on the compiled module and delegating to Elixir core the right to raise a readable message when the module has no such field defined:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">struct_checker</span><span class="p">(</span><span class="n">env</span><span class="p">,</span> <span class="n">_bytecode</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="n">env</span><span class="o">.</span><span class="n">module</span><span class="o">.</span><span class="n">__struct__</span>
</code></pre></div></div>
<p>The above will <code class="language-plaintext highlighter-rouge">raise</code> if the compiled module is not a struct. Of course, the code might be way more complex, but the core idea is whether your <em>used</em> module expects something from the module that has it used, implement the <code class="language-plaintext highlighter-rouge">@after_compile</code> callback and damn raise unless all the prerequisites are met.</p>
<p>Happy compiling!</p>
Go Outta Here2018-12-25T00:00:00+00:00https://rocket-science.ru/hacking/2018/12/25/go-outta-here<blockquote>
<p>They’re [Googlers] not capable of understanding a brilliant language but we want to use them to build good software. So, the language that we give them has to be easy for them to understand and easy to adopt. – <a href="https://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2014/From-Parallel-to-Concurrent">Rob Pike</a></p>
</blockquote>
<p>I made several attempts to play with <a href="https://golang.org"><em>Go</em></a>. After all, it is the language created by famous <a href="http://herpolhode.com/rob/">Rob Pike</a> who previously engineered such successful and widely-used software as <a href="https://en.wikipedia.org/wiki/Plan_9_from_Bell_Labs">Plan9</a> and <a href="https://en.wikipedia.org/wiki/Inferno_(operating_system)">Inferno</a> operating systems and <a href="https://en.wikipedia.org/wiki/Limbo_programming_language">Limbo</a> programming language. <em>Go</em> is supported by <em>Google</em>, and they won’t fail, right?</p>
<p>Nope.</p>
<p><em>Go</em> is the hugest scam of the XXI century in CS so far. For some reason people do still trust <em>Google</em> to some extent, despite of abandoned “Don’t be evil” motto, despite of a full load of violence evidence, despite of firing people for having and sharing the opinion.</p>
<p><em>Go</em> was made by <em>Google</em> and <em>Google</em> cannot fail, they say. Bullshit.</p>
<p><em>Go</em> is a fraud in a company of <em>modern</em> languages and I am going to explain why.</p>
<hr />
<p>I am going to walk through the official <a href="https://www.golang-book.com/books/intro/2#section1"><em>Golang Book</em></a> and show what is wrong (in terms of design flaws) with nearly every statement there.</p>
<h3 id="your-first-program">Your first program</h3>
<p>The tutorial starts with the explanation of the program structure.</p>
<blockquote>
<p>The first line says this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>package main
</code></pre></div> </div>
</blockquote>
<p>OK, everybody calls it <em>module</em>, we’d call it <em>package</em>, because we are special. OK.</p>
<blockquote>
<p>Then we see this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>import "fmt"
</code></pre></div> </div>
</blockquote>
<p><code class="language-plaintext highlighter-rouge">import "fmt"</code>? You gotta be kidding me; it is completely redundant in a compiled language. Good compiler might easily resolve all the calls to <code class="language-plaintext highlighter-rouge">fmt.foo</code> and do whatever imports are needed for. The only reason the explicit <em>import</em> might be helpful is to literally <em>import</em> functions into the <del>module</del> package. That’s not the case here, one still should bother with FQ-names.</p>
<h3 id="types">Types</h3>
<p>That is my fave!</p>
<blockquote>
<p>Go is a statically typed programming language. This means that variables <strong>always</strong> have a specific type and that type cannot change.</p>
</blockquote>
<p>That is the lie. (Emphasis is mine.) Statically typed (I prefer the wording “fully typed”) languages with rich types, like Haskell, require an enormous boilerplate to accomplish very simple tasks and <em>Go</em> was announced as <em>easy-come-easy-go</em>. That’s why it is in fact <strong>statically typed unless otherwise</strong>. We’ll come back to <em>void interfaces</em> later.</p>
<blockquote>
<p>Go’s integer types are: <code class="language-plaintext highlighter-rouge">uint8</code>, <code class="language-plaintext highlighter-rouge">uint16</code>, <code class="language-plaintext highlighter-rouge">uint32</code>, <code class="language-plaintext highlighter-rouge">uint64</code>, <code class="language-plaintext highlighter-rouge">int8</code>, <code class="language-plaintext highlighter-rouge">int16</code>, <code class="language-plaintext highlighter-rouge">int32</code> and <code class="language-plaintext highlighter-rouge">int64</code>.</p>
</blockquote>
<p>The above makes me cry. In the second decade of XXI century we enforce the developer to distinguish between <code class="language-plaintext highlighter-rouge">int32</code> and <code class="language-plaintext highlighter-rouge">int64</code> in <em>compiled “statically typed” language</em>. Really?</p>
<h3 id="variables">Variables</h3>
<blockquote>
<p>Since creating a new variable with a starting value is so common Go also supports a shorter statement:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>x := "Hello World"
</code></pre></div> </div>
</blockquote>
<p>What could go wrong with a plain old good equal sign? We’d lose the pride for <em>Go</em> being <em>statically typed</em> if it would not remind us about that by redundant syntax quirks?</p>
<blockquote>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>package main
import "fmt"
var x string = "Hello World"
func main() {
fmt.Println(x)
}
</code></pre></div> </div>
<p>Notice that we moved the variable outside of the main function. This means that other functions can access this variable.</p>
</blockquote>
<p>This is found in <em>scopes</em> chapter. OK, it looks like an instance variable in Ruby. Or as a module attribute in Elixir. What is indeed the scope and the lifetime of this variable? Can I return it from the functions declared in this module? There is no answer in the tutorial, also I understand that probably after stepping onto a couple of rakes any <em>Gopher</em> would burn the knowledge into memory.</p>
<h3 id="control-structures">Control structures</h3>
<p>The first control <em>structure</em> (?) introduces is a <code class="language-plaintext highlighter-rouge">for</code> loop. Iterating, mapping, reducing?—No, we’ve never heard about. Declare an outermost variable and loop with <code class="language-plaintext highlighter-rouge">for</code>. I am discreetly checking the current date against my calendar. It’s still 2018.</p>
<p>The second one is obviously <code class="language-plaintext highlighter-rouge">if</code>. <em>“Give me <code class="language-plaintext highlighter-rouge">for</code> and <code class="language-plaintext highlighter-rouge">if</code> to stand on, and I will move the Earth,”</em> as Archimedes used to say instead of the evening pray. BTW, the third <em>control structure</em> would be <code class="language-plaintext highlighter-rouge">switch</code>.</p>
<p>And that’s it! The language must be as simple as possible.</p>
<h3 id="arrays-slices-and-maps">Arrays, Slices and Maps</h3>
<p>Arrays have predefined length. Slices are arrays of not fixed length. Maps are key-values. Maps are to be both declared and initialized, otherwise a runtime error raises (if this is not a shallow copy of the one billion dollar mistake, I don’t know what is.)</p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">var</span> <span class="n">x</span> <span class="k">map</span><span class="p">[</span><span class="kt">string</span><span class="p">]</span><span class="kt">int</span> <span class="o">=</span> <span class="nb">make</span><span class="p">(</span><span class="k">map</span><span class="p">[</span><span class="kt">string</span><span class="p">]</span><span class="kt">int</span><span class="p">)</span>
</code></pre></div></div>
<p>The above looks like a bullet proof evidence of the fact that the goal to create the easy readable language was successfully achieved.</p>
<p>Below is how one does accesses elements in the map.</p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">if</span> <span class="n">name</span><span class="p">,</span> <span class="n">ok</span> <span class="o">:=</span> <span class="n">elements</span><span class="p">[</span><span class="s">"Un"</span><span class="p">];</span> <span class="n">ok</span> <span class="p">{</span>
<span class="n">fmt</span><span class="o">.</span><span class="n">Println</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">ok</span><span class="p">)</span>
<span class="p">}</span>
</code></pre></div></div>
<p>No comment. Those who are to object that it’s safe and all that, I hear you. This is exactly what’s called <em>Stockholm syndrome</em>.</p>
<p>I could not even imagine what level of potential developers is expected to produce such a defence from an idiot. Maybe it’d be easier to simply not hire them?</p>
<h3 id="functions">Functions</h3>
<p>Functions can return multiple values (what’s wrong with returning <em>arrays</em> btw?) and functions could be <em>variadic</em> (accepting an unknown upfront number of arguments of the same type.) So far, so good.</p>
<p>The last condition though, all the <em>splatted</em> arguments should be of the same type gave a born to the feature that basically proves <em>Go</em> to be a fraud in the family of modern mature serious languages.</p>
<p>Interfaces. Sounds scary?—That’s not all. Void Interfaces.</p>
<div class="language-go highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">func</span> <span class="n">Println</span><span class="p">(</span><span class="n">a</span> <span class="o">...</span><span class="k">interface</span><span class="p">{})</span> <span class="p">(</span><span class="n">n</span> <span class="kt">int</span><span class="p">,</span> <span class="n">err</span> <span class="kt">error</span><span class="p">)</span>
</code></pre></div></div>
<p>Statically typed? Safe? ROFL.</p>
<p>The road to hell is paved with good intentions. I believe they were to create a safe statically typed language for dummies (although the idea in a nutshell sounds a bit insane.) But the real world is severe, harsh and rough. Dummies were not ready to produce huge boilerplates on every single occasion. And—voilà—we let the whimsical child to get some sweets instead of a healthy soup.</p>
<hr />
<p>Here my dive into this <em>great language</em> has ended. I understand, that the good code is made by developers, not by computer languages. There are tons of great code on weird poor-designed languages and tons of awful code in great languages. Personally, I do not care much about what language I need to accomplish a task.</p>
<p>But please stop call <em>Go</em> safe, easy to read-and-write and statically typed. Thank you.</p>
<hr />
<p>For the sake of entropy constancy, I would provide an example of how <em>proper safe typing</em> might be achieved in very dynamic language <em>Elixir</em>. With pattern matching and guards.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">@spec</span> <span class="n">foo</span><span class="p">(</span><span class="no">nil</span> <span class="o">|</span> <span class="n">map</span><span class="p">()</span> <span class="o">|</span> <span class="n">list</span><span class="p">()</span> <span class="o">|</span> <span class="n">non_neg_integer</span><span class="p">())</span> <span class="p">::</span>
<span class="p">{</span><span class="ss">:error</span><span class="p">,</span> <span class="n">any</span><span class="p">()}</span> <span class="o">|</span> <span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">atom</span><span class="p">()}</span>
<span class="k">def</span> <span class="n">foo</span><span class="p">(</span><span class="no">nil</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="ss">:error</span><span class="p">,</span> <span class="ss">:empty</span><span class="p">}</span>
<span class="k">def</span> <span class="n">foo</span><span class="p">(%{}</span> <span class="o">=</span> <span class="n">map</span><span class="p">)</span> <span class="ow">when</span> <span class="n">map_size</span><span class="p">(</span><span class="n">map</span><span class="p">)</span> <span class="o">==</span> <span class="mi">0</span><span class="p">,</span>
<span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="ss">:error</span><span class="p">,</span> <span class="ss">:empty</span><span class="p">}</span>
<span class="k">def</span> <span class="n">foo</span><span class="p">([]),</span> <span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="ss">:error</span><span class="p">,</span> <span class="ss">:empty</span><span class="p">}</span>
<span class="k">def</span> <span class="n">foo</span><span class="p">(%{}</span> <span class="o">=</span> <span class="n">map</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="ss">:map</span><span class="p">}</span>
<span class="k">def</span> <span class="n">foo</span><span class="p">(</span><span class="n">num</span><span class="p">)</span> <span class="ow">when</span> <span class="n">is_integer</span><span class="p">(</span><span class="n">num</span><span class="p">)</span> <span class="ow">and</span> <span class="n">num</span> <span class="o">></span> <span class="mi">0</span><span class="p">,</span>
<span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="ss">:positive_integer</span><span class="p">}</span>
<span class="c1"># def foo(_), do: {:error, :unknown}</span>
</code></pre></div></div>
<p>The code above will warn on compilation stage if the attempt to call it with an improper argument is detected. This code will raise in the runtime on dynamic passing of the invalid argument as near to the issue as possible. And this code might be statically validated with a static code analisys tool, if as needed.</p>
<p>I don’t know about everyone, but this is more than good for me to avoid any kind of typing errors possible.</p>
<p>Happy going!</p>
Sigils To The Rescue2018-12-12T00:00:00+00:00https://rocket-science.ru/hacking/2018/12/12/sigils-for-the-rescue<p>As I continue to work on <a href="https://hexdocs.pm/exvalibur/Exvalibur.html"><code class="language-plaintext highlighter-rouge">Exvalibur</code></a>, the generator for blazingly
fast validators of maps based on sets of predefined rules, I have implemented custom guards and pattern matching
of values. Now one might specify rules as</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">rules</span> <span class="o">=</span> <span class="p">[</span>
<span class="p">%{</span><span class="ss">matches:</span> <span class="p">%{</span><span class="ss">currency_pair:</span> <span class="sx">~Q[<<"EUR", _::binary>>]</span><span class="p">,</span> <span class="ss">valid:</span> <span class="sx">~Q[valid]</span><span class="p">},</span>
<span class="ss">conditions:</span> <span class="p">%{</span><span class="ss">rate:</span> <span class="p">%{</span><span class="ss">min:</span> <span class="mf">1.0</span><span class="p">,</span> <span class="ss">max:</span> <span class="mf">2.0</span><span class="p">}},</span>
<span class="ss">guards:</span> <span class="p">[</span><span class="s2">"is_boolean(valid)"</span><span class="p">]</span>
<span class="p">}]</span>
</code></pre></div></div>
<p>What’s going on here? The above will match the map, having keys <code class="language-plaintext highlighter-rouge">currency_pair</code> and <code class="language-plaintext highlighter-rouge">valid</code> (specified explicitly)
and key <code class="language-plaintext highlighter-rouge">rate</code> specified implicitly through the <code class="language-plaintext highlighter-rouge">condition</code>. The resulting validator module will contain the
following positive validation clause:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">valid?</span><span class="p">(%{</span><span class="ss">currency_pair:</span> <span class="o"><<</span><span class="s2">"EUR"</span><span class="p">,</span> <span class="n">_</span><span class="p">::</span><span class="n">binary</span><span class="o">>></span><span class="p">,</span> <span class="ss">valid:</span> <span class="n">valid</span><span class="p">}</span> <span class="o">=</span> <span class="n">m</span><span class="err">â</span><span class="n">p</span><span class="p">)</span>
<span class="ow">when</span> <span class="n">rate</span> <span class="o">>=</span> <span class="mf">1.0</span> <span class="ow">and</span> <span class="n">rate</span> <span class="o"><=</span> <span class="mf">2.0</span> <span class="ow">and</span> <span class="n">is_boolean</span><span class="p">(</span><span class="n">valid</span><span class="p">)</span> <span class="k">do</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span>
<span class="p">%{</span><span class="ss">currency_pair:</span> <span class="n">m</span><span class="err">â</span><span class="n">p</span><span class="p">[</span><span class="ss">:currency_pair</span><span class="p">],</span>
<span class="ss">valid:</span> <span class="n">m</span><span class="err">â</span><span class="n">p</span><span class="p">[</span><span class="ss">:valid</span><span class="p">],</span>
<span class="ss">rate:</span> <span class="n">m</span><span class="err">â</span><span class="n">p</span><span class="p">[</span><span class="ss">:rate</span><span class="p">]}}</span>
<span class="k">end</span>
</code></pre></div></div>
<p>As one can see, the quoted expression are to be used for both pattern match declaration <em>and</em> introducing
the variable to be used in custom guard. The reason is we cannot just put an arbitrary expression as a map value.
That said, <code class="language-plaintext highlighter-rouge">rules = [%{currency_pair: <<"EUR", _::binary>>}]</code> won’t pass the compilation stage.</p>
<p>To quote the expression we use the <a href="https://elixir-lang.org/getting-started/sigils.html#custom-sigils">custom sigils</a>.</p>
<hr />
<p>We ultimately want to support interpolation inside this sigil to allow dynamic expressions</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">rule_for_currency</span><span class="p">(</span><span class="o"><<</span><span class="n">currency</span><span class="p">::</span><span class="n">binary</span><span class="o">-</span><span class="n">size</span><span class="p">(</span><span class="mi">3</span><span class="p">),</span> <span class="n">_</span><span class="p">::</span><span class="n">binary</span><span class="o">>></span><span class="p">)</span> <span class="k">do</span>
<span class="p">[%{</span><span class="ss">currency_pair:</span> <span class="sx">~q[<<"#{currency}", _::binary>>]</span><span class="p">}]</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Luckily enough, there is the Elixir core where we might borrow the implementation from, slightly modified.
I would post here the most interesting clause, the rest might be easily found in the source code repository.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmacro</span> <span class="n">sigil_q</span><span class="p">({</span><span class="ss">:<</span><span class="o"><>></span><span class="p">,</span> <span class="n">meta</span><span class="p">,</span> <span class="n">pieces</span><span class="p">},</span> <span class="p">[])</span> <span class="k">do</span>
<span class="n">tokens</span> <span class="o">=</span>
<span class="k">case</span> <span class="ss">:elixir_interpolation</span><span class="o">.</span><span class="n">unescape_tokens</span><span class="p">(</span><span class="n">pieces</span><span class="p">)</span> <span class="k">do</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">unescaped_tokens</span><span class="p">}</span> <span class="o">-></span> <span class="n">unescaped_tokens</span>
<span class="p">{</span><span class="ss">:error</span><span class="p">,</span> <span class="n">reason</span><span class="p">}</span> <span class="o">-></span> <span class="k">raise</span> <span class="no">ArgumentError</span><span class="p">,</span> <span class="n">to_string</span><span class="p">(</span><span class="n">reason</span><span class="p">)</span>
<span class="k">end</span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="no">Code</span><span class="o">.</span><span class="n">string_to_quoted!</span><span class="p">(</span>
<span class="kn">unquote</span><span class="p">({</span><span class="ss">:<</span><span class="o"><>></span><span class="p">,</span> <span class="n">meta</span><span class="p">,</span> <span class="n">tokens</span><span class="p">}),</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">meta</span><span class="p">)</span>
<span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>We delegate the interpolation to Elixir core, and then construct an AST out of this string. Easy-peasy.</p>
<p>For plain variables, as in the first example, <code class="language-plaintext highlighter-rouge">~Q[var]</code> works exactly as <code class="language-plaintext highlighter-rouge">Macro.var(var, nil)</code>, constructing an AST tuple
like <code class="language-plaintext highlighter-rouge">{:var, [line: 1], nil}</code>.</p>
<h2 id="conclusion">Conclusion</h2>
<p>Custom sigils are nifty and I always wanted to find the application for one. Here I go.</p>
<p>Happy custom validating!</p>
Iteraptable → Swiss Knife For Structs2018-11-16T00:00:00+00:00https://rocket-science.ru/hacking/2018/11/16/iteraptable-extending-elixir-structs<p>Elixir <em>structs</em> are <a href="https://elixir-lang.org/getting-started/structs.html#structs-are-bare-maps-underneath"><em>are bare maps underneath</em></a>. While they provide the nifty way to <em>restrict keys to the predefined set</em>, <em>specify default values</em>, <em>require keys</em>, <em>pattern match selectively</em>, and <em>more</em>, they are still bare maps with some syntactic sugar on top of them. Structs exports <a href="https://github.com/elixir-lang/elixir/blob/v1.7.4/lib/elixir/lib/kernel.ex#L4162-L4168"><code class="language-plaintext highlighter-rouge">__struct__/1</code></a> method, allow <a href="https://github.com/elixir-lang/elixir/blob/v1.7.4/lib/elixir/lib/kernel.ex#L4182-L4185">protocol implementation inheritance</a> and allow pattern match <em>struct types</em> to distinguish different structs by their definition in different clauses.</p>
<p>Structs purposedly do not allow to iterate them over out of the box. Neither do they provide the default <code class="language-plaintext highlighter-rouge">Access</code> implementation. I would guess that’s because Elixir is very strict language in the sense of everything should work as expected no matter what, without exceptions. And neither <code class="language-plaintext highlighter-rouge">Enumerable</code> nor <code class="language-plaintext highlighter-rouge">Collectable</code> could not be ultimately defined for structs. Also, <code class="language-plaintext highlighter-rouge">Access</code> behavior requires <code class="language-plaintext highlighter-rouge">pop/3</code> to be implemented, which is impossible for structs be design.</p>
<p>But sometimes, you know, we are not as captious. We just want to make it working:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Enum</span><span class="o">.</span><span class="n">each</span><span class="p">(%</span><span class="no">MyStruct</span><span class="p">{},</span> <span class="o">...</span><span class="p">)</span>
<span class="c1"># or</span>
<span class="p">[</span><span class="ss">foo:</span> <span class="mi">42</span><span class="p">,</span> <span class="ss">bar:</span> <span class="ss">:baz</span><span class="p">]</span> <span class="o">|></span> <span class="no">Enum</span><span class="o">.</span><span class="n">into</span><span class="p">(%</span><span class="no">MyStruct</span><span class="p">{})</span>
<span class="c1"># or </span>
<span class="n">put_in</span><span class="p">(%</span><span class="no">MyStruct</span><span class="p">{},</span> <span class="p">[</span><span class="ss">:foo</span><span class="p">],</span> <span class="mi">42</span><span class="p">)</span>
</code></pre></div></div>
<p>Once we understand all the consequences, we indeed have an ability to implement all the above for structs ourselves. It rather quickly becomes boring. In 99% of cases implementations are literally equal. Keeping in mind, that I nevertheless have plans to support structs for deep iterations in <a href="https://hexdocs.pm/iteraptor"><code class="language-plaintext highlighter-rouge">Iteraptor</code></a>, I decided to provide a syntactic sugar for implementing the above in custom structs. Please note, it probably won’t work in <code class="language-plaintext highlighter-rouge">iex</code> for playing around, since protocol implementations should be consolidated, and they already are during <code class="language-plaintext highlighter-rouge">iex</code> startup. Also, this won’t work for dynamically created modules. <code class="language-plaintext highlighter-rouge">Access</code>, being a behaviour, would work though.</p>
<h3 id="syntax">Syntax</h3>
<p>To make the struct <code class="language-plaintext highlighter-rouge">Enumerable</code>, implement an <code class="language-plaintext highlighter-rouge">Access</code> and derive the implementation of the protocol <code class="language-plaintext highlighter-rouge">Foo</code>, one should do inside the struct module:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">use</span> <span class="no">Iteraptor</span><span class="o">.</span><span class="no">Iteraptable</span> <span class="ss">skip:</span> <span class="no">Collectable</span><span class="p">,</span> <span class="ss">derive:</span> <span class="no">Foo</span>
</code></pre></div></div>
<p>Arguments to keyword parameters might be both atoms or lists of atoms. Below I am going to share the approach I have taken. Another tiny tutorial on using macros in <em>Elixir</em>.</p>
<p>To enable <code class="language-plaintext highlighter-rouge">Access</code> for the struct created dynamically, one might use</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">use</span> <span class="no">Iteraptor</span><span class="o">.</span><span class="no">Iteraptable</span> <span class="ss">skip:</span> <span class="p">[</span><span class="no">Enumerable</span><span class="p">,</span> <span class="no">Collectable</span><span class="p">]</span>
</code></pre></div></div>
<h3 id="enumerable">Enumerable</h3>
<p>This would be the easiest one. Honestly, I am unsure why it’s not included by default. The implementation is <em>correct</em> by any mean and works for literally all structs without limitations. The only trick we need to exclude <code class="language-plaintext highlighter-rouge">__struct__</code> key that contains the metainformation and is put into maps by Elixir itself to help both compiler and runtime to distinguish structs. So, delegate everything to the <code class="language-plaintext highlighter-rouge">map</code> save for <code class="language-plaintext highlighter-rouge">count/0</code>:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">count</span><span class="p">(</span><span class="n">map</span><span class="p">)</span> <span class="k">do</span>
<span class="c1"># do not count :__struct__</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">map</span> <span class="o">|></span> <span class="no">Map</span><span class="o">.</span><span class="n">from_struct</span><span class="p">()</span> <span class="o">|></span> <span class="n">map_size</span><span class="p">()}</span>
<span class="k">end</span>
</code></pre></div></div>
<p>I do not use <code class="language-plaintext highlighter-rouge">{:ok, map_size(map) - 1}</code> since I want it to continue work properly when another private meta field will be added to structs by core team.</p>
<h3 id="collectable">Collectable</h3>
<p>It’s even shorter. I spent half of an hour thinking about how should I deal with an attempt to collect the key-value pair with a key not belonging to this struct, and found the simplest solution: <em>I do not do anything</em>. Struct itself will raise a proper error. <em>Fail Fast</em>.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defimpl</span> <span class="no">Collectable</span><span class="p">,</span> <span class="ss">for:</span> <span class="bp">__MODULE__</span> <span class="k">do</span>
<span class="k">def</span> <span class="n">into</span><span class="p">(</span><span class="n">original</span><span class="p">)</span> <span class="k">do</span>
<span class="p">{</span><span class="n">original</span><span class="p">,</span>
<span class="k">fn</span>
<span class="n">map</span><span class="p">,</span> <span class="p">{</span><span class="ss">:cont</span><span class="p">,</span> <span class="p">{</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="p">}}</span> <span class="o">-></span> <span class="ss">:maps</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="p">,</span> <span class="n">map</span><span class="p">)</span>
<span class="n">map</span><span class="p">,</span> <span class="ss">:done</span> <span class="o">-></span> <span class="n">map</span>
<span class="n">_</span><span class="p">,</span> <span class="ss">:halt</span> <span class="o">-></span> <span class="ss">:ok</span>
<span class="k">end</span><span class="p">}</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<h3 id="access">Access</h3>
<p>That was the hardest one. <code class="language-plaintext highlighter-rouge">pop/3</code> contract is to pop up the value for the key and return a tuple <code class="language-plaintext highlighter-rouge">{value, rest}</code>. The thing is I cannot remove keys from maps. I decided to nullify the value in the returned struct. Why not?</p>
<p>Besides the above, everything is quite straightforward.</p>
<h3 id="use-iteraptoriteraptable"><code class="language-plaintext highlighter-rouge">use Iteraptor.Iteraptable</code></h3>
<p>Here is the most exciting part. We need to embed the implementations into the caller’s context. Since we provide <code class="language-plaintext highlighter-rouge">skip</code> parameter, we cannot just build an AST and inject it. The goal is to build AST <em>selectively</em>.</p>
<p>Also we want to raise on attempt to <code class="language-plaintext highlighter-rouge">use</code> our helper in non-structs. Simple check for whether struct is declared on module won’t work, since <code class="language-plaintext highlighter-rouge">defstruct</code> will be likely called <em>after</em> our code injection. But hey, it’s Elixir. We have <a href="https://hexdocs.pm/elixir/Module.html#module-compile-callbacks">compile hooks</a>!</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">checker</span> <span class="o">=</span> <span class="kn">quote</span><span class="p">(</span><span class="ss">location:</span> <span class="ss">:keep</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="nv">@after_compile</span><span class="p">({</span><span class="no">Iteraptor</span><span class="o">.</span><span class="no">Utils</span><span class="p">,</span> <span class="ss">:struct_checker</span><span class="p">}))</span>
</code></pre></div></div>
<p>While we could simply inject <code class="language-plaintext highlighter-rouge">__after_compile__/2</code>, that might conflict with the callback declared by the module owner. That’s why we delegate to our own function (that in turn simply calls <code class="language-plaintext highlighter-rouge">env.module.__struct__</code> and allows Elixir to provide nifty error message is it is undefined.)</p>
<p>Then we prepare an AST for <code class="language-plaintext highlighter-rouge">@derive</code> <a href="https://github.com/am-kantox/elixir-iteraptor/blob/master/lib/iteraptor/iteraptable.ex#L138-L149">if needed</a>.</p>
<p>And the most exciting part would be the selective injection of implementations. For that we prepare a map of the following structure:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">@codepieces</span> <span class="p">%{</span>
<span class="no">Enumerable</span> <span class="o">=></span>
<span class="kn">quote</span> <span class="ss">location:</span> <span class="ss">:keep</span> <span class="k">do</span>
<span class="k">defimpl</span> <span class="no">Enumerable</span><span class="p">,</span> <span class="ss">for:</span> <span class="bp">__MODULE__</span> <span class="k">do</span>
<span class="o">...</span>
<span class="no">Collectable</span> <span class="o">=></span> <span class="o">...</span>
</code></pre></div></div>
<p>and iterating through all the possible implementations, we check whether this one should not be skipped, and if not, we inject it:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Enum</span><span class="o">.</span><span class="n">reduce</span><span class="p">(</span><span class="nv">@codepieces</span><span class="p">,</span> <span class="p">[</span><span class="n">checker</span> <span class="o">|</span> <span class="n">derive</span><span class="p">],</span> <span class="k">fn</span> <span class="p">{</span><span class="n">type</span><span class="p">,</span> <span class="n">ast</span><span class="p">},</span> <span class="n">acc</span> <span class="o">-></span>
<span class="k">if</span> <span class="no">Enum</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="n">excluded</span><span class="p">,</span> <span class="o">&</span><span class="p">(</span><span class="nv">&1</span> <span class="o">==</span> <span class="n">type</span><span class="p">)),</span> <span class="k">do</span><span class="p">:</span> <span class="n">acc</span><span class="p">,</span> <span class="k">else</span><span class="p">:</span> <span class="p">[</span><span class="n">ast</span> <span class="o">|</span> <span class="n">acc</span><span class="p">]</span>
<span class="k">end</span><span class="p">)</span>
</code></pre></div></div>
<p>Voilà.</p>
<h3 id="usage">Usage</h3>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">Iterapted</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">Iteraptor</span><span class="o">.</span><span class="no">Iteraptable</span>
<span class="k">defstruct</span> <span class="ss">foo:</span> <span class="mi">42</span><span class="p">,</span> <span class="ss">bar:</span> <span class="ss">:baz</span>
<span class="k">end</span>
<span class="no">Enum</span><span class="o">.</span><span class="n">each</span><span class="p">(%</span><span class="no">Iterapted</span><span class="p">{},</span> <span class="o">&</span><span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="o">/</span><span class="mi">1</span><span class="p">)</span>
<span class="c1">#⇒ {:bar, :baz}</span>
<span class="c1"># {:foo, 42}</span>
</code></pre></div></div>
<p>Happy iterating!</p>
¡AST FTW!2018-11-05T00:00:00+00:00https://rocket-science.ru/hacking/2018/11/05/AST-FTW<p><img src="/img/llibertat.jpg" alt="Llibertat" /></p>
<p>The most revolutionary and vivid idea (pattern, paradigm, design, younameit) in computer science was invented in 1958. It is still extremely underrated and very few developers fully understand it’s crucial importance to have a robust and easy to maintain code. It costs all the patterns invented after. If I were to be faced with a choice to pick up only one <em>key property of my language of choice</em>, that would be it.</p>
<p>Clean and exposable to the developer <a href="https://en.wikipedia.org/wiki/Abstract_syntax_tree">Abstract Syntax Tree</a>. An ability to have AST on hand. An ability to modify AST.</p>
<p>That ability alone makes the development process hundreds of times faster. The resulting code becomes concise and readable. DRY comes out for free out of the box. The only goodness AST does not bring into the development process is you cannot ask it to serve a whiskey for you.</p>
<p>Back in 1958 there was not much room for imagination in the language design, mostly due to a necessity to be as near to the machine codes as possible. <em>John McCarthy</em> found the best way to build a Turing-complete language for algorithms. He did not try to invent a fancy syntax, he just showed the simplest way to write the decision tree down to the paper. Yeah, Lots of Irritating Stupid Parentheses, or <a href="https://en.wikipedia.org/wiki/Lisp_(programming_language)">LISP</a>. Since then the <a href="https://en.wikipedia.org/wiki/Von_Neumann_architecture">von Neumann architecture</a> has not changed a lot, despite that we use silicon semiconductors performing GazziFlops in a cheap smartphone. Still, the stupid piece of iron (ok, ok, silicon) inside our supercomputers does <em>operations</em> on <em>two inputs</em> for it’s whole life. There were attempts to build computers build upon tri-state logic (the most successful was <a href="https://en.wikipedia.org/wiki/Setun">Setun</a> built in USSR in 1959,) but what we have now in front of us, does indeed call functions with two parameters only.</p>
<hr />
<p>For some weird reason, the profit of having access to AST is still not evident to the vast majority of computer engineers. Half of the century we were trying to make the computer language as readable for the human beings as possible (the first genius attempt was done by <a href="https://en.wikipedia.org/wiki/Grace_Hopper">Grace Hopper</a> with COBOL, and I would consider it being the most successful one towards this direction.) While trying to make it more and more readable we lost ties to the machine core. We found ourselves in a need to build the bloated complicated transpilers from not-yet-readable-enough code to what the semiconductors might understand. Even worse, we attempted to <em>fake</em> AST for the most popular languages.</p>
<p>Ruby has <a href="http://ruby-doc.org/stdlib/libdoc/ripper/rdoc/Ripper.html"><code class="language-plaintext highlighter-rouge">Ripper</code></a> in stdlib:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">pp</span> <span class="no">Ripper</span><span class="p">.</span><span class="nf">sexp</span><span class="p">(</span><span class="s1">'def hello(world) "Hello, #{world}!"; end'</span><span class="p">)</span> <span class="c1">#⇒</span>
<span class="p">[</span><span class="ss">:program</span><span class="p">,</span>
<span class="p">[[</span><span class="ss">:def</span><span class="p">,</span>
<span class="p">[</span><span class="ss">:@ident</span><span class="p">,</span> <span class="s2">"hello"</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">]],</span>
<span class="p">[</span><span class="ss">:paren</span><span class="p">,</span>
<span class="p">[</span><span class="ss">:params</span><span class="p">,</span> <span class="p">[[</span><span class="ss">:@ident</span><span class="p">,</span> <span class="s2">"world"</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">10</span><span class="p">]]],</span> <span class="kp">nil</span><span class="p">,</span> <span class="kp">nil</span><span class="p">,</span> <span class="kp">nil</span><span class="p">,</span> <span class="kp">nil</span><span class="p">,</span> <span class="kp">nil</span><span class="p">,</span> <span class="kp">nil</span><span class="p">]],</span>
<span class="p">[</span><span class="ss">:bodystmt</span><span class="p">,</span>
<span class="p">[[</span><span class="ss">:string_literal</span><span class="p">,</span>
<span class="p">[</span><span class="ss">:string_content</span><span class="p">,</span>
<span class="p">[</span><span class="ss">:@tstring_content</span><span class="p">,</span> <span class="s2">"Hello, "</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">18</span><span class="p">]],</span>
<span class="p">[</span><span class="ss">:string_embexpr</span><span class="p">,</span> <span class="p">[[</span><span class="ss">:var_ref</span><span class="p">,</span> <span class="p">[</span><span class="ss">:@ident</span><span class="p">,</span> <span class="s2">"world"</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">27</span><span class="p">]]]]],</span>
<span class="p">[</span><span class="ss">:@tstring_content</span><span class="p">,</span> <span class="s2">"!"</span><span class="p">,</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">33</span><span class="p">]]]]],</span>
<span class="kp">nil</span><span class="p">,</span>
<span class="kp">nil</span><span class="p">,</span>
<span class="kp">nil</span><span class="p">]]]]</span>
</code></pre></div></div>
<p>Javascript has some <a href="https://duckduckgo.com/?q=javascript+abstract+syntax+tree&t=canonical&ia=qa">3-rd party <em>leftpads</em></a> to spit out an AST.</p>
<p>Java has somewhat <a href="http://www.eclipse.org/jdt/ui/astview/index.php">implemented in Eclipse</a>.</p>
<p>All the above is fooling us, developers, because nobody needs read-only AST. The language syntax was explicitly designed to be <em>more readable</em> than AST. That’s its main purpose, for God’s sake. Why would I ever need to read something way more verbose and less readable?—There is no reason and that’s why many professionals even have no idea their languages of choice <em>have</em> kinda AST representation.</p>
<hr />
<p>Read-write AST is a game changer. <a href="https://doc.rust-lang.org/nightly/nightly-rustc/syntax/ast/index.html"><em>Rust</em> seems to start understanding that</a>. Maybe some other less common languages I am not aware of.</p>
<p>Elixir was born with AST being the first class citizen. Mostly because it runs over ErlangVM and the necessity to reuse as much as possible made AST handling a must. Both ways. No compromises.</p>
<p>We have a very well readable source, that flawlessly transpiles back and forth into AST on demand. That simple thing made it possible to have macros that are both ready to read and extremely powerful. In ruby (python, javascript, lua, java, c++) we cannot have a syntactically legit construct representing one branch in switch-case. In both LISP and Elixir we surely can. Natively in LISP and with <code class="language-plaintext highlighter-rouge">quote do</code> in Elixir.</p>
<p>AST is a tree in the first place. Meaning one might easily <em>plug out</em> branches and <em>plug in</em> other branches. That makes possible such things as not including the whole AST with calls to <code class="language-plaintext highlighter-rouge">Logger.debug</code> into release version—there are <em>exactly zero</em> processor ticks in prod, the code is simply not there, no conditionals needed.</p>
<p>Once we have AST, we might traverse it, modify and plug in back. Macros in Elixir receive the AST and return AST back. Since macros are expanded on the compilation stage, we might do literally whatever we want.</p>
<hr />
<p>Metaprogramming abilities are pathetic like provided by other languages ruby, python or java. I am positive, that sooner or later people will all of a sudden realize that AST solves most issues with the unreadable and unmaintainable code. The real development liberty is impossible without a direct access to AST.</p>
<p>Try it and you’ll never consider switching back.</p>
<hr />
<p><strong><em>Addendum 1.</em> Elixir AST.</strong></p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">|</span><span class="mi">1</span> <span class="err">▶</span> <span class="kn">quote</span> <span class="k">do</span><span class="p">:</span> <span class="p">({</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">result</span><span class="p">}</span> <span class="o">-></span> <span class="n">result</span><span class="p">)</span>
<span class="p">[{</span><span class="ss">:-</span><span class="o">></span><span class="p">,</span> <span class="p">[],</span> <span class="p">[[</span><span class="ss">ok:</span> <span class="p">{</span><span class="ss">:result</span><span class="p">,</span> <span class="p">[],</span> <span class="no">Elixir</span><span class="p">}],</span> <span class="p">{</span><span class="ss">:result</span><span class="p">,</span> <span class="p">[],</span> <span class="no">Elixir</span><span class="p">}]}]</span>
</code></pre></div></div>
<p>Happy treeing!</p>
Smart Validation In Elixir With Exvalibur2018-11-02T00:00:00+00:00https://rocket-science.ru/hacking/2018/11/02/smart-validation-in-elixir-with-exvalibur<p>As documentation states, <a href="https://hexdocs.pm/exvalibur/0.3.1/Exvalibur.html"><code class="language-plaintext highlighter-rouge">Exvalibur</code></a> is the generator for blazingly fast validators of maps based on sets of predefined rules.</p>
<p>Generally speaking, one provides a list of rules in a format of a map:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">rules</span> <span class="o">=</span> <span class="p">[</span>
<span class="p">%{</span><span class="ss">matches:</span> <span class="p">%{</span><span class="ss">currency_pair:</span> <span class="s2">"EURUSD"</span><span class="p">},</span>
<span class="ss">conditions:</span> <span class="p">%{</span><span class="ss">rate:</span> <span class="p">%{</span><span class="ss">min:</span> <span class="mf">1.0</span><span class="p">,</span> <span class="ss">max:</span> <span class="mf">2.0</span><span class="p">}}},</span>
<span class="p">%{</span><span class="ss">matches:</span> <span class="p">%{</span><span class="ss">currency_pair:</span> <span class="s2">"USDEUR"</span><span class="p">},</span>
<span class="ss">conditions:</span> <span class="p">%{</span><span class="ss">rate:</span> <span class="p">%{</span><span class="ss">min:</span> <span class="mf">1.2</span><span class="p">,</span> <span class="ss">max:</span> <span class="mf">1.3</span><span class="p">}}},</span>
<span class="p">]</span>
</code></pre></div></div>
<p>and calls <code class="language-plaintext highlighter-rouge">Exvalibur.validator!/2</code>. The latter produces a validator module with as many clauses of <code class="language-plaintext highlighter-rouge">valid?/1</code> function as we have rules above (plus one sink-everything clause.) Once generated, the <code class="language-plaintext highlighter-rouge">valid?/1</code> function might be called directly on the input data, providing blazingly fast validation based completely on pattern matching and guards.</p>
<p>This makes sense when the input coming from the third party / user requires validation of kind “if this field has a value <em>foo</em>, and that field has a value <em>bar</em>, and that failed might have a numeric value in a range from <em>this</em> to <em>that</em>, consider it’s valid.”</p>
<h2 id="workflow">Workflow</h2>
<p>As stated above, the first step would be to feed <code class="language-plaintext highlighter-rouge">Exvalibur.validator!/2</code> with a list of rules, each being a map having two keys: <code class="language-plaintext highlighter-rouge">matches</code> and <code class="language-plaintext highlighter-rouge">conditions</code>. Matches are used to generate different clauses of the validator and conditions are converted to guards within these clauses.</p>
<p>The module might be generated using <a href="https://hexdocs.pm/flow"><code class="language-plaintext highlighter-rouge">Flow</code></a> (by providing <code class="language-plaintext highlighter-rouge">flow: true</code> option in call to <code class="language-plaintext highlighter-rouge">validator!</code> to fasten the parsing of relatively huge rulesets.</p>
<p>The name of generated module is to be passed as <code class="language-plaintext highlighter-rouge">module_name: MyApp.MyValidator</code> option where the value is an atom for the name of the generated module.</p>
<p>By default, rules are <em>merged</em> into the existing ruleset. To replace the ruleset one might use <code class="language-plaintext highlighter-rouge">merge: false</code> option. The ruleset is hard-compiled into the module in the form of <code class="language-plaintext highlighter-rouge">term_to_binary(rule) => rule</code> map and is accessible via call to <code class="language-plaintext highlighter-rouge">MyApp.MyValidator.rules/0</code>.</p>
<h2 id="usage">Usage</h2>
<p>Assuming we already have the validator module compiled, the typical usage would be:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">case</span> <span class="no">MyApp</span><span class="o">.</span><span class="no">MyValidator</span><span class="o">.</span><span class="n">valid?</span><span class="p">(</span><span class="n">input</span><span class="p">)</span> <span class="k">do</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">_validated_fields</span><span class="p">}</span> <span class="o">-></span>
<span class="n">input</span>
<span class="ss">:error</span> <span class="o">-></span>
<span class="no">Logger</span><span class="o">.</span><span class="n">warn</span><span class="p">(</span><span class="s2">"Wrong input!"</span><span class="p">)</span>
<span class="no">nil</span>
<span class="k">end</span>
</code></pre></div></div>
<h2 id="matches-and-guards">Matches And Guards</h2>
<p>At the moment, rules do not support matching patterns (yet,) only static values are allowed. In a nutshell, <code class="language-plaintext highlighter-rouge">%{matches: %{foo: 42}}</code> rule would generate <code class="language-plaintext highlighter-rouge">def valid?(%{foo: 42})</code> clause and the condition <code class="language-plaintext highlighter-rouge">%{conditions: %{foo: {eq: 42}}}</code> would generate <code class="language-plaintext highlighter-rouge">def valid(%{foo: foo}) when foo == 42</code> clause.</p>
<h2 id="guards">Guards</h2>
<p>Out of the box <code class="language-plaintext highlighter-rouge">Exvalibur</code> provides <a href="https://hexdocs.pm/exvalibur/0.3.1/Exvalibur.Guards.Default.html#content"><code class="language-plaintext highlighter-rouge">Exvalibur.Guards.Default</code></a> module implementing the following set of guards:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">eq(var, val)</code> → guard for conditions like <code class="language-plaintext highlighter-rouge">%{eq: 1.0}</code>, exact equality</li>
<li><code class="language-plaintext highlighter-rouge">greater_than(var, val)</code> → guard for conditions like <code class="language-plaintext highlighter-rouge">%{greater_than: 1.0}</code>, like <code class="language-plaintext highlighter-rouge">min/2</code>, but the inequality is strict</li>
<li><code class="language-plaintext highlighter-rouge">less_than(var, val)</code> → guard for conditions like <code class="language-plaintext highlighter-rouge">%{less_than: 1.0}</code>, like <code class="language-plaintext highlighter-rouge">max/2</code>, but the inequality is strict</li>
<li><code class="language-plaintext highlighter-rouge">max(var, val)</code> → guard for conditions like <code class="language-plaintext highlighter-rouge">%{max: 2.0}</code>, implies actual value is less or equal than the parameter</li>
<li><code class="language-plaintext highlighter-rouge">min(var, val)</code> → guard for conditions like <code class="language-plaintext highlighter-rouge">%{min: 1.0}</code>, implies actual value is greater or equal than the parameter</li>
<li><code class="language-plaintext highlighter-rouge">max_length(var, val)</code> → guard for conditions like <code class="language-plaintext highlighter-rouge">%{max_length: 10}</code>, checks the byte length of the binary parameter</li>
<li><code class="language-plaintext highlighter-rouge">min_length(var, val)</code> → guard for conditions like <code class="language-plaintext highlighter-rouge">%{min_length: 10}</code>, checks the byte length of the binary parameter</li>
<li><code class="language-plaintext highlighter-rouge">one_of(var, val)</code> → guard for checking the includion in the list like <code class="language-plaintext highlighter-rouge">%{one_of: [42, 3.14]}</code></li>
<li><code class="language-plaintext highlighter-rouge">not_one_of(var, val)</code> → guard for checking the excludion from the list like <code class="language-plaintext highlighter-rouge">%{not_one_of: [42, 3.14]}</code></li>
</ul>
<p>If this set is not enough, one might implement their own set of guards. The guard implementation should be the function of arity 2, returning the AST which is valid as Elixir guard, e. g. suitable for use in guard expressions, exactly as <a href="https://hexdocs.pm/elixir/master/Kernel.html#defguard/1"><code class="language-plaintext highlighter-rouge">Kernel.defguard/1</code></a> does. Below is shown the typical implementation for such a module.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">MyApp</span><span class="o">.</span><span class="no">Guards</span> <span class="k">do</span>
<span class="kn">import</span> <span class="no">Exvalibur</span><span class="o">.</span><span class="no">Guards</span><span class="o">.</span><span class="no">Default</span><span class="p">,</span> <span class="ss">except:</span> <span class="p">[</span><span class="ss">min_length:</span> <span class="mi">2</span><span class="p">,</span> <span class="ss">max_length:</span> <span class="mi">2</span><span class="p">]</span>
<span class="k">def</span> <span class="n">min_length</span><span class="p">(</span><span class="n">var</span><span class="p">,</span> <span class="n">val</span><span class="p">)</span> <span class="ow">when</span> <span class="n">is_integer</span><span class="p">(</span><span class="n">val</span><span class="p">)</span> <span class="k">do</span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="n">is_bitstring</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">var</span><span class="p">))</span> <span class="ow">and</span> <span class="n">bytesize</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">var</span><span class="p">))</span> <span class="o">>=</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">val</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="n">bullshit</span><span class="p">(</span><span class="n">var</span><span class="p">,</span> <span class="n">val</span><span class="p">)</span> <span class="ow">when</span> <span class="n">is_integer</span><span class="p">(</span><span class="n">val</span><span class="p">)</span> <span class="k">do</span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="n">is_bitstring</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">var</span><span class="p">))</span> <span class="ow">and</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">val</span><span class="p">)</span> <span class="o">==</span> <span class="s2">"bullshit"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<h2 id="coming-soon">Coming Soon</h2>
<ul>
<li>allow generic patterns in matches, like <code class="language-plaintext highlighter-rouge">%{matches: %{currency_pair: <<"EUR", _ :: binary-size(3)>>}}</code></li>
<li>allow transformers in rules, like <code class="language-plaintext highlighter-rouge">%{transform: {MyMod, :to_changeset}}</code> applying to <em>validated</em> input, so that one might use the validator as a mapper</li>
<li>allow the ruleset given in CSV (or some kind of external format.)</li>
</ul>
<p>Several Imagine the application that receives some data from the external source. For the sake of an example let’s assume the data is currency rates stream.</p>
<p>The application has a set of rules to filter the incoming data stream. Let’s say we have a list of currencies we are interested in, and we want only the currencies from this list to pass through. Also, sometimes we receive invalid rates (nobody is perfect, our rates provider is not an exception.) So we maintain a long-lived validator that ensures that the rate in the stream looks fine for us and only then we allow the machinery to process it. Otherwise we just ignore it.</p>
<p>Happy validating!</p>
Generated Module As A Guard2018-10-30T00:00:00+00:00https://rocket-science.ru/hacking/2018/10/30/generated-module-as-a-guard<p>Imagine the application that receives some data from the external source. For the sake of an example let’s assume the data is currency rates stream.</p>
<p>The application has a set of rules to filter the incoming data stream. Let’s say we have a list of currencies we are interested in, and we want only the currencies from this list to pass through. Also, sometimes we receive invalid rates (nobody is perfect, our rates provider is not an exception.) So we maintain a long-lived validator that ensures that the rate in the stream looks fine for us and only then we allow the machinery to process it. Otherwise we just ignore it.</p>
<h2 id="naïve-approach">Naïve Approach</h2>
<p>The naïve approach to handle this use case would be to maintain a map, containing currency pairs as keys and rules as values, and apply rules to the incomind rates to check whether the rate is of interest or not. Rules would be simple maps specifying the acceptable interval for the rate as <code class="language-plaintext highlighter-rouge">min</code> and <code class="language-plaintext highlighter-rouge">max</code> values. Something like this (for the sake of an example let’s assume rates are coming as maps already, for instance from RabbitMQ or like):</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">Validator</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">Agent</span>
<span class="k">def</span> <span class="n">start_link</span><span class="p">,</span>
<span class="k">do</span><span class="p">:</span> <span class="no">Agent</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="k">fn</span> <span class="o">-></span> <span class="p">%{}</span> <span class="k">end</span><span class="p">,</span> <span class="ss">name:</span> <span class="bp">__MODULE__</span><span class="p">)</span>
<span class="k">def</span> <span class="n">update_rules</span><span class="p">(</span><span class="n">currency_pair</span><span class="p">,</span> <span class="n">rules</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="no">Agent</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="o">&</span><span class="p">(</span><span class="no">Map</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="nv">&1</span><span class="p">,</span> <span class="n">currency_pair</span><span class="p">,</span> <span class="n">rules</span><span class="p">))</span>
<span class="k">def</span> <span class="n">valid?</span><span class="p">(%{</span><span class="ss">currency_pair:</span> <span class="n">pair</span><span class="p">,</span> <span class="ss">rate:</span> <span class="n">rate</span><span class="p">})</span> <span class="k">do</span>
<span class="n">with</span> <span class="p">%{}</span> <span class="o">=</span> <span class="n">rules</span> <span class="o"><-</span> <span class="no">Agent</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="o">&</span> <span class="nv">&1</span><span class="p">),</span>
<span class="n">rule</span> <span class="ow">when</span> <span class="ow">not</span> <span class="n">is_nil</span><span class="p">(</span><span class="n">rule</span><span class="p">)</span> <span class="o"><-</span> <span class="n">rules</span><span class="p">[</span><span class="n">pair</span><span class="p">],</span>
<span class="n">r</span> <span class="ow">when</span> <span class="n">r</span> <span class="o">></span> <span class="n">rule</span><span class="o">.</span><span class="n">min</span> <span class="ow">and</span> <span class="n">r</span> <span class="o"><</span> <span class="n">rule</span><span class="o">.</span><span class="n">max</span> <span class="o"><-</span> <span class="n">rate</span> <span class="k">do</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">r</span><span class="p">},</span>
<span class="k">else</span>
<span class="n">_</span> <span class="o">-></span> <span class="ss">:error</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>This is good, and this works. But can we improve the performance?—Sure thing!</p>
<h2 id="pattern-matching">Pattern matching</h2>
<p>Instead of looking up the map with rules, we might simply generate the module, that will have one function <code class="language-plaintext highlighter-rouge">valid?</code> with as many clauses as we have rules. These clauses will directly pattern match the input and return <code class="language-plaintext highlighter-rouge">{:ok, rate}</code> tuple (the <em>existence</em> of this clause guarantees that the rate is good.) The last clause will accept all the non-matched garbage to return <code class="language-plaintext highlighter-rouge">:error</code>.</p>
<p>Sounds smart?—Indeed. Let’s implement it. Imagine we still have this map with rules.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">Validator</span> <span class="k">do</span>
<span class="k">def</span> <span class="n">instance!</span><span class="p">(</span><span class="n">rules</span><span class="p">)</span> <span class="k">do</span>
<span class="n">mod</span> <span class="o">=</span> <span class="no">Module</span><span class="o">.</span><span class="n">concat</span><span class="p">([</span><span class="s2">"Validator"</span><span class="p">,</span> <span class="s2">"Instance"</span><span class="p">])</span>
<span class="k">if</span> <span class="no">Code</span><span class="o">.</span><span class="n">ensure_compiled?</span><span class="p">(</span><span class="n">mod</span><span class="p">)</span> <span class="k">do</span>
<span class="ss">:code</span><span class="o">.</span><span class="n">purge</span><span class="p">(</span><span class="n">mod</span><span class="p">)</span>
<span class="ss">:code</span><span class="o">.</span><span class="n">delete</span><span class="p">(</span><span class="n">mod</span><span class="p">)</span>
<span class="k">end</span>
<span class="no">Module</span><span class="o">.</span><span class="n">create</span><span class="p">(</span><span class="n">mod</span><span class="p">,</span> <span class="n">ast</span><span class="p">(</span><span class="n">rules</span><span class="p">),</span> <span class="no">Macro</span><span class="o">.</span><span class="no">Env</span><span class="o">.</span><span class="n">location</span><span class="p">(</span><span class="n">__ENV__</span><span class="p">))</span>
<span class="k">end</span>
<span class="k">defp</span> <span class="n">ast</span><span class="p">(</span><span class="n">map</span><span class="p">)</span> <span class="k">do</span>
<span class="p">[</span>
<span class="kn">quote</span><span class="p">(</span><span class="k">do</span><span class="p">:</span> <span class="p">(</span><span class="k">def</span> <span class="n">valid?</span><span class="p">(</span><span class="n">_</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="ss">:error</span><span class="p">))</span> <span class="o">|</span>
<span class="no">Enum</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">map</span><span class="p">,</span> <span class="k">fn</span> <span class="p">{</span><span class="n">pair</span><span class="p">,</span> <span class="p">%{</span><span class="ss">min:</span> <span class="n">min</span><span class="p">,</span> <span class="ss">max:</span> <span class="n">max</span><span class="p">}}</span> <span class="o">-></span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="k">def</span> <span class="n">valid?</span><span class="p">(%{</span><span class="ss">currency_pair:</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">pair</span><span class="p">),</span> <span class="ss">rate:</span> <span class="n">rate</span><span class="p">})</span>
<span class="ow">when</span> <span class="n">rate</span> <span class="o">></span> <span class="kn">unquote</span><span class="p">(</span><span class="n">min</span><span class="p">)</span> <span class="ow">and</span> <span class="n">rate</span> <span class="o"><</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">max</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">rate</span><span class="p">}</span>
<span class="k">end</span>
<span class="k">end</span><span class="p">)</span>
<span class="p">]</span> <span class="o">|></span> <span class="ss">:lists</span><span class="o">.</span><span class="n">reverse</span><span class="p">()</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>When we need to update our rules, we just call <code class="language-plaintext highlighter-rouge">Validator.instance!/1</code> and receive back the module named <code class="language-plaintext highlighter-rouge">Validator.Instance</code>. It has several clauses for <code class="language-plaintext highlighter-rouge">valid/1</code> function. Let’s see this in action</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">|</span><span class="mi">1</span> <span class="err">▶</span> <span class="no">Validator</span><span class="o">.</span><span class="n">instance!</span><span class="p">(%{</span><span class="s2">"USDEUR"</span> <span class="o">=></span> <span class="p">%{</span><span class="ss">min:</span> <span class="mf">1.0</span><span class="p">,</span> <span class="ss">max:</span> <span class="mf">2.0</span><span class="p">}})</span>
<span class="p">{</span><span class="ss">:module</span><span class="p">,</span> <span class="no">Validator</span><span class="o">.</span><span class="no">Instance</span><span class="p">,</span>
<span class="o"><<</span><span class="mi">70</span><span class="p">,</span> <span class="mi">79</span><span class="p">,</span> <span class="mi">82</span><span class="p">,</span> <span class="mi">49</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">244</span><span class="p">,</span> <span class="mi">66</span><span class="p">,</span> <span class="mi">69</span><span class="p">,</span> <span class="mi">65</span><span class="p">,</span> <span class="mi">77</span><span class="p">,</span> <span class="mi">65</span><span class="p">,</span> <span class="mi">116</span><span class="p">,</span> <span class="mi">85</span><span class="p">,</span> <span class="mi">56</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">167</span><span class="p">,</span>
<span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">17</span><span class="p">,</span> <span class="mi">25</span><span class="p">,</span> <span class="mi">69</span><span class="p">,</span> <span class="mi">108</span><span class="p">,</span> <span class="mi">105</span><span class="p">,</span> <span class="mi">120</span><span class="p">,</span> <span class="mi">105</span><span class="p">,</span> <span class="mi">114</span><span class="p">,</span> <span class="mi">46</span><span class="p">,</span> <span class="mi">86</span><span class="p">,</span> <span class="mi">97</span><span class="p">,</span> <span class="mi">108</span><span class="p">,</span> <span class="mi">105</span><span class="p">,</span> <span class="mi">100</span><span class="p">,</span> <span class="mi">97</span><span class="p">,</span>
<span class="mi">116</span><span class="p">,</span> <span class="mi">111</span><span class="p">,</span> <span class="mi">114</span><span class="p">,</span> <span class="mi">46</span><span class="p">,</span> <span class="mi">73</span><span class="p">,</span> <span class="mi">110</span><span class="p">,</span> <span class="mi">115</span><span class="p">,</span> <span class="mi">116</span><span class="p">,</span> <span class="mi">97</span><span class="p">,</span> <span class="o">...>></span><span class="p">,</span> <span class="p">[</span><span class="ss">valid?:</span> <span class="mi">1</span><span class="p">,</span> <span class="ss">valid?:</span> <span class="mi">1</span><span class="p">]}</span>
<span class="n">iex</span><span class="o">|</span><span class="mi">2</span> <span class="err">▶</span> <span class="no">Validator</span><span class="o">.</span><span class="no">Instance</span><span class="o">.</span><span class="n">valid?</span><span class="p">(%{</span><span class="ss">currency_pair:</span> <span class="s2">"USDEUR"</span><span class="p">,</span> <span class="ss">rate:</span> <span class="mf">1.5</span><span class="p">})</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="mf">1.5</span><span class="p">}</span>
<span class="n">iex</span><span class="o">|</span><span class="mi">3</span> <span class="err">▶</span> <span class="no">Validator</span><span class="o">.</span><span class="no">Instance</span><span class="o">.</span><span class="n">valid?</span><span class="p">(%{</span><span class="ss">currency_pair:</span> <span class="s2">"USDEUR"</span><span class="p">,</span> <span class="ss">rate:</span> <span class="mf">0.5</span><span class="p">})</span>
<span class="ss">:error</span>
<span class="n">iex</span><span class="o">|</span><span class="mi">4</span> <span class="err">▶</span> <span class="no">Validator</span><span class="o">.</span><span class="no">Instance</span><span class="o">.</span><span class="n">valid?</span><span class="p">(%{</span><span class="ss">currency_pair:</span> <span class="s2">"USDGBP"</span><span class="p">,</span> <span class="ss">rate:</span> <span class="mf">1.5</span><span class="p">})</span>
<span class="ss">:error</span>
</code></pre></div></div>
<p>Exactly what we needed, and blazingly fast.</p>
<h2 id="further-improvement">Further Improvement</h2>
<p>If the amount of incoming rates is big enough, like thousands per a second, we might use <a href="https://hexdocs.pm/flow"><code class="language-plaintext highlighter-rouge">Flow</code></a> on <a href="https://hexdocs.pm/gen_stage"><code class="language-plaintext highlighter-rouge">GenStage</code></a> to validate them in bulks. That would be probably the topic of the next writing on the subject.</p>
<p>Happy generating!</p>
ActiveRecord Smell With Elixir/Ecto2018-10-30T00:00:00+00:00https://rocket-science.ru/hacking/2018/10/30/active-record-smell-in-elixir<h2 id="what-why">What? Why?</h2>
<p><a href="https://api.rubyonrails.org/classes/ActiveRecord.html"><code class="language-plaintext highlighter-rouge">ActiveRecord</code></a> is arguably the worst approach to deal with the database ever existed. While object-relational mapping definitely might bring some goodness to our lives, the rules <em>AR</em> is built upon can sometimes be shooting our legs.</p>
<p><a href="https://hexdocs.pm/ecto/3.0.0-rc.1"><code class="language-plaintext highlighter-rouge">Ecto</code></a> was really refreshing for everybody suffering from the necessity to conform wild guesses constantly made by <em>AR</em>. Unlike the latter, <code class="language-plaintext highlighter-rouge">Ecto</code> is deadly explicit. Nothing happens under the hood. It performs exactly what the developer wanted to be performed. No gazillions of unsolicited queries, no hidden DB updates when serialized fields change. It’s amazing.</p>
<p>After the year of a pure rapture, I found myself in a need to duplicate some boilerplate amongst different projects. So I decided to implement the most vivid (read: the best) parts of what <em>AR</em> provides in <em>Elixir</em> for <em>Ecto</em>.</p>
<p>Maybe some day I’ll build a package out of this code, now I want to share the way I had it implemented. This would be a tutorial on how to write macros in <em>Elixir</em> even more than reimplementation of <em>AR</em> helpers for the sake of decreasing the boilerplate.</p>
<p>Yes, some things from <em>AR</em> are good enough for me to miss them.</p>
<h2 id="goal">Goal</h2>
<p>What we want is to have something like this</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">Post</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">EctoAR</span><span class="o">.</span><span class="no">Query</span><span class="p">,</span>
<span class="ss">repo:</span> <span class="no">MyApp</span><span class="o">.</span><span class="no">Repo</span><span class="p">,</span>
<span class="ss">preload:</span> <span class="p">[</span><span class="ss">:user</span><span class="p">,</span> <span class="ss">:comments</span><span class="p">],</span>
<span class="ss">enums:</span> <span class="p">[</span><span class="ss">state:</span> <span class="sx">~w|draft published|</span><span class="p">]</span>
<span class="o">...</span>
</code></pre></div></div>
<p>to have some AR-like goodness for free. What I am after in this writing is:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">Post.find(id)</code> → <code class="language-plaintext highlighter-rouge">%Post{}</code></li>
<li><code class="language-plaintext highlighter-rouge">Post.scope(</code> <a href="https://github.com/elixir-ecto/ecto/blob/v3.0.0-rc.1/lib/ecto/queryable.ex"><code class="language-plaintext highlighter-rouge">Ecto.Queryable</code></a> <code class="language-plaintext highlighter-rouge">, params)</code> → <a href="https://hexdocs.pm/ecto/3.0.0-rc.1/Ecto.Query.html#content"><code class="language-plaintext highlighter-rouge">%Ecto.Query{}</code></a></li>
<li><code class="language-plaintext highlighter-rouge">Post.scope!(Ecto.Queryable, params)</code> → <code class="language-plaintext highlighter-rouge">[%Post{}]</code></li>
</ul>
<p>and some helpers for fun:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">Post.state_draft(params)</code> → <code class="language-plaintext highlighter-rouge">[%Post{}]</code></li>
<li><code class="language-plaintext highlighter-rouge">Post.state_published(params)</code> → <code class="language-plaintext highlighter-rouge">[%Post{}]</code></li>
</ul>
<h2 id="warning-note">Warning Note</h2>
<p>If you are not familiar with Elixir Macros, I’d strongly suggest reading the brilliant <a href="https://pragprog.com/book/cmelixir/metaprogramming-elixir">Metaprogramming Elixir</a> by Chris McCord. It is absolutely worth it even if you are not elixiring on daily basis.</p>
<h2 id="implementation">Implementation</h2>
<h3 id="scaffold">Scaffold</h3>
<p>So, we are to produce a macro that will inject the boilerplate into our bare <em>Ecto</em> schemas. Since I want to use it as <code class="language-plaintext highlighter-rouge">use EctoAR.Query</code>, that’ll be a module named <code class="language-plaintext highlighter-rouge">EctoAR.Query</code>, implementing <code class="language-plaintext highlighter-rouge">__using__/1</code> macro. Let’s start with a scaffold.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">EctoAR</span><span class="o">.</span><span class="no">Query</span> <span class="k">do</span>
<span class="nv">@moduledoc</span> <span class="no">false</span>
<span class="k">defmacrop</span> <span class="n">repo</span><span class="p">(</span><span class="n">opts</span><span class="p">)</span> <span class="k">do</span>
<span class="c1"># try `opts`, then try the topmost application’s `config`, then fallback `Repo`</span>
<span class="kn">quote</span> <span class="ss">bind_quoted:</span> <span class="p">[</span><span class="ss">opts:</span> <span class="n">opts</span><span class="p">],</span> <span class="ss">location:</span> <span class="ss">:keep</span> <span class="k">do</span>
<span class="n">with</span> <span class="no">nil</span> <span class="o"><-</span> <span class="no">Keyword</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">opts</span><span class="p">,</span> <span class="ss">:repo</span><span class="p">),</span>
<span class="p">[{</span><span class="n">me</span><span class="p">,</span> <span class="n">_</span><span class="p">,</span> <span class="n">_</span><span class="p">}</span> <span class="o">|</span> <span class="n">_</span><span class="p">]</span> <span class="o">=</span> <span class="no">Application</span><span class="o">.</span><span class="n">started_applications</span><span class="p">(),</span>
<span class="no">nil</span> <span class="o"><-</span> <span class="no">Application</span><span class="o">.</span><span class="n">get_env</span><span class="p">(</span><span class="n">me</span><span class="p">,</span> <span class="ss">:repo</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="no">Module</span><span class="o">.</span><span class="n">concat</span><span class="p">(</span><span class="no">Macro</span><span class="o">.</span><span class="n">camelize</span><span class="p">(</span><span class="n">to_string</span><span class="p">(</span><span class="n">me</span><span class="p">)),</span> <span class="s2">"Repo"</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">defmacrop</span> <span class="n">ast</span><span class="p">()</span> <span class="k">do</span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="sx">??</span><span class="err">?</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">defmacro</span> <span class="n">__using__</span><span class="p">(</span><span class="n">opts</span> <span class="p">\\</span> <span class="p">[])</span> <span class="k">do</span>
<span class="n">repo</span> <span class="o">=</span> <span class="n">repo</span><span class="p">(</span><span class="n">opts</span><span class="p">)</span>
<span class="n">preload</span> <span class="o">=</span>
<span class="n">opts</span>
<span class="o">|></span> <span class="no">Keyword</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="ss">:preload</span><span class="p">,</span> <span class="p">[])</span>
<span class="o">|></span> <span class="no">Macro</span><span class="o">.</span><span class="n">expand</span><span class="p">(</span><span class="n">__CALLER__</span><span class="p">)</span>
<span class="p">[</span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="kn">import</span><span class="p">(</span><span class="no">Ecto</span><span class="o">.</span><span class="no">Query</span><span class="p">)</span>
<span class="nv">@type</span> <span class="n">t</span> <span class="p">::</span> <span class="p">%</span><span class="bp">__MODULE__</span><span class="p">{}</span>
<span class="k">end</span>
<span class="o">|</span> <span class="n">ast</span><span class="p">()</span>
<span class="p">]</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The above is basically a scaffold we are to fill with the desired functionality. First comes the private macro to do our best in resolving the repository we are to use to connect to our data. It first tries to get the <code class="language-plaintext highlighter-rouge">:repo</code> keyword parameter from options passed to <code class="language-plaintext highlighter-rouge">use EctoAR.Query</code> call, then looks up the application config and finally falls back to default <code class="language-plaintext highlighter-rouge">MyApp.Repo</code>.</p>
<p>During the compilation stage, we cannot ensure the value provided is legit. It is possible to raise a meaningful exception with a solid description of what’s wrong (and I actually have it in the resulting code,) but for the sake of example let’s assume we are fine with reading docs and providing correct arguments in the call to our macro. For those curious how to implement graceful error in runtime when repo specified cannot be found, check <a href="https://hexdocs.pm/elixir/master/Code.html#ensure_loaded/1"><code class="language-plaintext highlighter-rouge">Code.ensure_loaded/1</code></a>.</p>
<p>Also, I’d explicitly note the wise use of <a href="https://hexdocs.pm/elixir/master/Kernel.SpecialForms.html#with/1"><code class="language-plaintext highlighter-rouge">Kernel.SpecialForms.with/1</code></a> <em>monadic</em> clause. It immediately returns a value as it was found and continues to the next clause otherwise.</p>
<h3 id="scope2"><code class="language-plaintext highlighter-rouge">scope/2</code></h3>
<p>The idea of <code class="language-plaintext highlighter-rouge">scope/2</code> is to behave nearly the same as AR’s <code class="language-plaintext highlighter-rouge">where</code> does, but with an explicit check for arguments passed. It should accept any queryable and return a queryable back to provide full support for embedding it in query chains. Here we go.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmacrop</span> <span class="n">scopes</span><span class="p">(</span><span class="n">preload</span><span class="p">)</span> <span class="k">do</span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="nv">@spec</span> <span class="n">scope</span><span class="p">(</span>
<span class="n">query</span> <span class="p">::</span> <span class="no">nil</span> <span class="o">|</span> <span class="no">Ecto</span><span class="o">.</span><span class="no">Query</span><span class="o">.</span><span class="n">t</span><span class="p">(),</span>
<span class="n">params</span> <span class="p">::</span> <span class="no">Keyword</span><span class="o">.</span><span class="n">t</span><span class="p">()</span>
<span class="p">)</span> <span class="p">::</span> <span class="no">Ecto</span><span class="o">.</span><span class="no">Query</span><span class="o">.</span><span class="n">t</span><span class="p">()</span>
<span class="nv">@doc</span> <span class="sd">"""
Returns an `Ecto.Query` for `where(params)`.
If the first parameter in call to this method
is omitted, defaults to `#{__MODULE__}`.
"""</span>
<span class="k">def</span> <span class="n">scope</span><span class="p">(</span><span class="n">query</span> <span class="p">\\</span> <span class="no">nil</span><span class="p">,</span> <span class="n">params</span><span class="p">)</span>
<span class="nv">@spec</span> <span class="n">do_scope</span><span class="p">(</span>
<span class="n">query</span> <span class="p">::</span> <span class="no">nil</span> <span class="o">|</span> <span class="no">Ecto</span><span class="o">.</span><span class="no">Query</span><span class="o">.</span><span class="n">t</span><span class="p">(),</span>
<span class="n">params</span> <span class="p">::</span> <span class="no">Keyword</span><span class="o">.</span><span class="n">t</span><span class="p">()</span>
<span class="p">)</span> <span class="p">::</span> <span class="no">Ecto</span><span class="o">.</span><span class="no">Query</span><span class="o">.</span><span class="n">t</span><span class="p">()</span>
<span class="k">defp</span> <span class="n">do_scope</span><span class="p">(</span><span class="n">query</span><span class="p">,</span> <span class="n">params</span><span class="p">)</span> <span class="k">do</span>
<span class="k">case</span> <span class="no">Keyword</span><span class="o">.</span><span class="n">keys</span><span class="p">(</span><span class="n">params</span><span class="p">)</span> <span class="o">--</span> <span class="n">__schema__</span><span class="p">(</span><span class="ss">:fields</span><span class="p">)</span> <span class="k">do</span>
<span class="p">[]</span> <span class="o">-></span>
<span class="n">from</span><span class="p">(</span><span class="n">data</span> <span class="ow">in</span> <span class="n">query</span><span class="p">,</span> <span class="ss">where:</span> <span class="o">^</span><span class="n">params</span><span class="p">)</span>
<span class="o">|></span> <span class="n">preload</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">preload</span><span class="p">))</span>
<span class="n">extra</span> <span class="o">-></span>
<span class="k">raise</span> <span class="sd">"""
Unsupported fields were passed in call to `#{__MODULE__}__.scope/2`.
Extra fields: #{inspect(extra)}
"""</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="n">scope</span><span class="p">(</span><span class="no">nil</span><span class="p">,</span> <span class="n">params</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="n">do_scope</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="n">params</span><span class="p">)</span>
<span class="k">def</span> <span class="n">scope</span><span class="p">(%</span><span class="no">Ecto</span><span class="o">.</span><span class="no">Query</span><span class="p">{}</span> <span class="o">=</span> <span class="n">query</span><span class="p">,</span> <span class="n">params</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="n">do_scope</span><span class="p">(</span><span class="n">query</span><span class="p">,</span> <span class="n">params</span><span class="p">)</span>
<span class="n">defoverridable</span><span class="p">(</span><span class="ss">scope:</span> <span class="mi">2</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The very last method should actually accept any implementation of <a href="https://github.com/elixir-ecto/ecto/blob/v3.0.0-rc.1/lib/ecto/queryable.ex#L1"><code class="language-plaintext highlighter-rouge">Ecto.Queryable</code></a>, but for the sake of the example, let it be just accepting queries.</p>
<p>OK, once the AST above is injected into <code class="language-plaintext highlighter-rouge">Ecto.Schema</code> instance, the latter receives the ability to call this function like:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Post</span><span class="o">.</span><span class="n">scope</span><span class="p">(</span><span class="ss">author_id:</span> <span class="mi">10</span><span class="p">,</span> <span class="ss">date:</span> <span class="sx">~D[2018-10-30]</span><span class="p">)</span>
<span class="c1">#⇒ #Ecto.Query<from data in Post,</span>
<span class="c1"># where: data.author_id == ^10 and data.date == ^~D[2018-10-30],</span>
<span class="c1"># preload: [:user, :comments]></span>
</code></pre></div></div>
<p>Note, that an attempt to pass the filter for inexisting field leads to the exception raised:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Post</span><span class="o">.</span><span class="n">scope</span><span class="p">(</span><span class="ss">author_id:</span> <span class="mi">10</span><span class="p">,</span> <span class="ss">answer:</span> <span class="mi">42</span><span class="p">)</span>
<span class="c1">#⇒ ** (RuntimeError) Unsupported fields were passed in call to `Elixir.Post.scope/2`.</span>
<span class="c1"># Extra fields:</span>
<span class="c1"># [answer: 42]</span>
</code></pre></div></div>
<p>Also note, that I made <code class="language-plaintext highlighter-rouge">scope/2</code> overridable FWIW.</p>
<h3 id="find1"><code class="language-plaintext highlighter-rouge">find/1</code></h3>
<p>That would be easy. In the real life, it is more complicated, since the primary key might be either inexistent or combined, and we would need to tackle with <code class="language-plaintext highlighter-rouge">__schema__(:primary_key)</code> and <a href="https://hexdocs.pm/elixir/master/Kernel.SpecialForms.html#unquote_splicing/1"><code class="language-plaintext highlighter-rouge">Kernel.SpecialForms.unquote_splicing/1</code></a>, but for the sake of this example let’s keep it simple.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmacrop</span> <span class="n">find</span><span class="p">(</span><span class="n">repo</span><span class="p">,</span> <span class="n">preload</span><span class="p">)</span> <span class="k">do</span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="k">def</span> <span class="n">find</span><span class="p">(</span><span class="n">id</span><span class="p">)</span> <span class="ow">when</span> <span class="n">is_integer</span><span class="p">(</span><span class="n">id</span><span class="p">)</span> <span class="k">do</span>
<span class="n">from</span><span class="p">(</span><span class="n">data</span> <span class="ow">in</span> <span class="bp">__MODULE__</span><span class="p">,</span> <span class="ss">where:</span> <span class="n">data</span><span class="o">.</span><span class="n">id</span> <span class="o">==</span> <span class="o">^</span><span class="n">id</span><span class="p">)</span>
<span class="o">|></span> <span class="n">preload</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">preload</span><span class="p">))</span>
<span class="o">|></span> <span class="kn">unquote</span><span class="p">(</span><span class="n">repo</span><span class="p">)</span><span class="o">.</span><span class="n">one</span><span class="p">()</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Use it as:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Post</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="mi">100</span><span class="p">)</span>
<span class="c1">#⇒ %Post{</span>
<span class="c1"># __meta__: #Ecto.Schema.Metadata<:loaded, "posts">,</span>
<span class="c1"># user: %User{</span>
<span class="c1"># ...</span>
<span class="c1"># },</span>
<span class="c1"># ...</span>
</code></pre></div></div>
<h3 id="anything-else">Anything Else?</h3>
<p>One might actually implement and inject as many helper methods as needed, but we are to stop here. I also have <code class="language-plaintext highlighter-rouge">scope!/2</code> that wraps the call to <code class="language-plaintext highlighter-rouge">scope/2</code> with actuall request against the database, returning <em>records</em> not a query.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">@spec</span> <span class="n">scope!</span><span class="p">(</span>
<span class="n">query</span> <span class="p">::</span> <span class="no">nil</span> <span class="o">|</span> <span class="no">Ecto</span><span class="o">.</span><span class="no">Query</span><span class="o">.</span><span class="n">t</span><span class="p">(),</span> <span class="n">params</span> <span class="p">::</span> <span class="no">Keyword</span><span class="o">.</span><span class="n">t</span><span class="p">()</span>
<span class="p">)</span> <span class="p">::</span> <span class="p">[</span><span class="no">Ecto</span><span class="o">.</span><span class="no">Schema</span><span class="o">.</span><span class="n">t</span><span class="p">()]</span>
<span class="nv">@doc</span> <span class="sx">~s|Returns a dataset for `where(params)`|</span>
<span class="k">def</span> <span class="n">scope!</span><span class="p">(</span><span class="n">query</span> <span class="p">\\</span> <span class="no">nil</span><span class="p">,</span> <span class="n">params</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">apply</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">repo</span><span class="p">),</span> <span class="ss">:all</span><span class="p">,</span> <span class="p">[</span><span class="n">scope</span><span class="p">(</span><span class="n">query</span><span class="p">,</span> <span class="n">params</span><span class="p">)])</span>
</code></pre></div></div>
<h3 id="helpers-for-enums">Helpers For Enums</h3>
<p>This is completely redundant, but let’s make them to show the power of <em>Elixir</em> macros again.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmacrop</span> <span class="n">enums</span><span class="p">(</span><span class="n">opts</span><span class="p">)</span> <span class="k">do</span>
<span class="n">opts</span>
<span class="o">|></span> <span class="no">Keyword</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="ss">:enums</span><span class="p">,</span> <span class="p">[])</span>
<span class="o">|></span> <span class="no">Enum</span><span class="o">.</span><span class="n">flat_map</span><span class="p">(</span><span class="k">fn</span> <span class="p">{</span><span class="n">prefix</span><span class="p">,</span> <span class="n">values</span><span class="p">}</span> <span class="o">-></span>
<span class="n">values</span>
<span class="o">|></span> <span class="no">Macro</span><span class="o">.</span><span class="n">expand</span><span class="p">(</span><span class="n">__CALLER__</span><span class="p">)</span> <span class="c1"># to allow sigils</span>
<span class="o">|></span> <span class="no">Enum</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="k">fn</span> <span class="n">value</span> <span class="o">-></span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="nv">@spec</span> <span class="kn">unquote</span><span class="p">(</span><span class="ss">:"</span><span class="si">#{</span><span class="n">prefix</span><span class="si">}</span><span class="ss">_</span><span class="si">#{</span><span class="n">value</span><span class="si">}</span><span class="ss">"</span><span class="p">)(</span><span class="n">opts</span> <span class="p">::</span> <span class="no">Keyword</span><span class="o">.</span><span class="n">t</span><span class="p">())</span> <span class="p">::</span> <span class="p">[</span><span class="no">Ecto</span><span class="o">.</span><span class="no">Schema</span><span class="o">.</span><span class="n">t</span><span class="p">()]</span>
<span class="nv">@doc</span> <span class="sx">~s|Returns a dataset for `where(#{unquote(prefix)}: "#{unquote(value)}")`|</span>
<span class="k">defp</span> <span class="kn">unquote</span><span class="p">(</span><span class="ss">:"</span><span class="si">#{</span><span class="n">prefix</span><span class="si">}</span><span class="ss">_</span><span class="si">#{</span><span class="n">value</span><span class="si">}</span><span class="ss">"</span><span class="p">)(</span><span class="n">opts</span><span class="p">)</span> <span class="k">do</span>
<span class="p">(</span><span class="n">data</span> <span class="ow">in</span> <span class="n">scope</span><span class="p">(</span><span class="n">opts</span><span class="p">))</span>
<span class="o">|></span> <span class="n">from</span><span class="p">(</span><span class="ss">where:</span> <span class="n">data</span><span class="o">.</span><span class="kn">unquote</span><span class="p">(</span><span class="n">prefix</span><span class="p">)</span> <span class="o">==</span> <span class="o">^</span><span class="kn">unquote</span><span class="p">(</span><span class="n">value</span><span class="p">))</span>
<span class="o">|></span> <span class="kn">unquote</span><span class="p">(</span><span class="n">repo</span><span class="p">)</span><span class="o">.</span><span class="n">all</span><span class="p">()</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span><span class="p">)</span>
<span class="k">end</span><span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Note, that specs and docs will be <em>caller specific</em>, including real names of functions for these particular enums.</p>
<h3 id="putting-it-all-together">Putting It All Together</h3>
<p>All we need to do to make it work, would be to build the AST in the scaffold mentioned above to be injected into our schemas. That would be as easy as</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">ast</span> <span class="o">=</span>
<span class="p">[</span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="kn">import</span><span class="p">(</span><span class="no">Ecto</span><span class="o">.</span><span class="no">Query</span><span class="p">)</span>
<span class="nv">@type</span> <span class="n">t</span> <span class="p">::</span> <span class="p">%</span><span class="bp">__MODULE__</span><span class="p">{}</span>
<span class="k">end</span>
<span class="o">|</span> <span class="p">[</span><span class="n">find</span><span class="p">(</span><span class="n">repo</span><span class="p">,</span> <span class="n">preload</span><span class="p">)</span> <span class="o">|</span> <span class="n">scopes</span><span class="p">(</span><span class="n">preload</span><span class="p">)</span> <span class="o">++</span> <span class="n">enums</span><span class="p">(</span><span class="n">opts</span><span class="p">)]</span>
<span class="p">]</span>
</code></pre></div></div>
<p>We are ready to <code class="language-plaintext highlighter-rouge">use</code> our helper in <em>Ecto</em> schemas. Happy macroing!</p>
Suggested New SPR Wording, or “We’re VMS and You’re Not.”2018-10-27T00:00:00+00:00https://rocket-science.ru/fun/2018/10/27/we-are-vms-and-you-are-not<h2 id="nb">NB!</h2>
<p>⚠️ <strong>Attention! This is a joke. It was written as a joke back in 1970s when the environment in CS was not as inclusive and we had no necessity to preface jokes with a disclaimer “Attention! This is a joke.”</strong></p>
<p>I am reposting it here because it is a marvelous piece of art and I am afraid it might be lost in ashes.</p>
<hr />
<h2 id="vms-version-3">VMS Version 3:</h2>
<p>Please stop submitting SPR’s. This is our system. We designed it, we built it, and we use it more than you do. If there are some features you think might be missing, if the system isn’t as effective as you think it could be, TOUGH! Give it back, we don’t need you. See figure.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> ---------------------------
! - !
! { } !
! | | !
! | | !
! .-.! !.-. !
! .-! ! ! !.-. !
! ! ! ! ; !
! \ ; !
! \ ; !
! ! : !
! ! | !
! | | !
! !
---------------------------
Figure 1.
</code></pre></div></div>
<p>Forget about your silly problem, let’s take a look at some of the features of the VMS operating system.</p>
<h4 id="1-options">1. Options.</h4>
<p>We’ve got lots of them. So many in fact, that you need two strong people to carry the documentation around. So many that it will be a cold day in hell before half of them are used. So many that you are probably not going to do your work right anyway. However, the number of options isn’t all that important, because we picked some interesting values for the options and called them…</p>
<h4 id="2-defaults">2. Defaults.</h4>
<p>We put a lot of thought into our defaults. We like them. If we didn’t, we would have made something else be the default. So keep your cotton-picking hands off our defaults. Don’t touch. Consider them mandatory. “Mandatory defaults” has a nice ring to it. Change them and your system crashes, tough. See figure 1.</p>
<h4 id="3-language-processors">3. Language processors.</h4>
<p>They work just fine. They take in source, and often produce object files as a reward for your efforts. You don’t like the code? Too bad! You can even try to call operating system services from them. For any that you can’t, use the assembler like we do. We spoke to the language processor developers about this, they think a lot like we do, they said “See figure 1”.</p>
<h4 id="4-debuggers">4. Debuggers.</h4>
<p>We’ve got debuggers, one we support and one we use. You shouldn’t make mistakes anyway, it is a waste of time. We don’t want to hear anything about debuggers, we’re not interested. See figure 1.</p>
<h4 id="5-error-logging">5. Error logging.</h4>
<p>Ignore it. Why give yourself an ulcer? You don’t want to give us the machine to get the problem fixed and we probably can’t do it anyway. Oh, and if something breaks between 17:00 and 18:00 or 9:30 and 10:30 or 11:30 and 13:30 or 14:30 and 15:30 don’t waste your time calling us, we’re out. See figure 1.</p>
<h4 id="6-command-language">6. Command language.</h4>
<p>We designed it ourselves, it’s perfect. We like it so much we put our name on it, DCL—Digital’s Command Language. In fact we’re so happy with it, we designed it once for each of our operating systems. We even try to keep it the same from release to release, sometimes we blow it though. See figure 1.</p>
<h4 id="7-real-time-performance">7. Real time performance.</h4>
<p>We got it. Who else could have done such a good job? So the system seems sluggish with all those priority 18 processes, no problem, just make them priority 1. Anyway, real time isn’t important anymore like it used to be. We changed our group’s name to get rid of the name realtime, we told all our realtime users to see figure 1 a long time ago.</p>
<p>In conclusion, stuff your SPR. Love VMS or leave it, but don’t complain.</p>
<p><a href="http://www.parc.xerox.com/csl/members/dourish/goodies/see-figure-1.html"><em>Original source (now revoked) at Xerox</em></a>.
<a href="https://web.archive.org/web/19980215205000/http://www.parc.xerox.com/csl/members/dourish/goodies/see-figure-1.html"><em>Source at WebMachine</em></a>.</p>
Better pry for Ruby REPL2018-10-27T00:00:00+00:00https://rocket-science.ru/hacking/2018/10/27/pry-with-whistles<p><img src="/img/pry_logo.png" alt="Pry Logo" /></p>
<h2 id="repl">REPL</h2>
<p><em>REPL</em> in CS stands for <a href="https://www.allacronyms.com/REPL">Real Eval Print Loop</a>.</p>
<p>Ruby, like many other languages, comes with it’s own <em>REPL</em> implementation called <code class="language-plaintext highlighter-rouge">irb</code>. While it likely does it’s primitive job, there is a way better alternative for it: <a href="http://pryrepl.org/"><code class="language-plaintext highlighter-rouge">pry</code></a>.</p>
<p>The excerpt from <code class="language-plaintext highlighter-rouge">pry</code>’s official site claims</p>
<blockquote>
<p>Pry is a <strong>powerful</strong> alternative to the standard IRB shell for Ruby. It features <strong>syntax highlighting</strong>, a <strong>flexible plugin architecture</strong>, <strong>runtime invocation</strong> and <strong>source and documentation browsing</strong>.</p>
</blockquote>
<p>And all the above is indeed true. I am going to show how to install and configure <code class="language-plaintext highlighter-rouge">pry</code> for user’s better experience. I would assume you already have Ruby up and running on the target machine.</p>
<h2 id="preparation-step">Preparation step</h2>
<p>Go the to console and execute:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ gem install pry pry-theme awesome_print coderay
</code></pre></div></div>
<p>Now you should be all set. Try to run <code class="language-plaintext highlighter-rouge">pry</code> and see no difference against <code class="language-plaintext highlighter-rouge">irb</code>. Frustrating enough.</p>
<p>Let’s teach <code class="language-plaintext highlighter-rouge">pry</code> to be smarter.</p>
<h2 id="configuration-matters">Configuration matters</h2>
<p>There is a config file <code class="language-plaintext highlighter-rouge">.pryrc</code> located in your home directory, that tells <code class="language-plaintext highlighter-rouge">pry</code> how it should look and behave like. If there is none, execute <code class="language-plaintext highlighter-rouge">cd && touch .pryrc</code> to create it.</p>
<h2 id="tweaking-pryrc">Tweaking <code class="language-plaintext highlighter-rouge">.pryrc</code></h2>
<p>I am going to reveal pry features step by step. For those impatient:</p>
<p><strong>TL;DR:</strong> <a href="https://gist.github.com/am-kantox/5ebb68916ccde06736e8d95026693742">gist with my <code class="language-plaintext highlighter-rouge">.pryrc</code></a></p>
<ul>
<li><a href="https://github.com/pry/pry/wiki"><code class="language-plaintext highlighter-rouge">pry</code> guides</a>: there is a ton of useful information for post-graduated pryests,</li>
<li><a href="https://github.com/kyrylo/pry-theme/wiki"><code class="language-plaintext highlighter-rouge">pry</code> themes</a>: use, create, howtos, wtfs.</li>
</ul>
<h3 id="editor">Editor</h3>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Pry</span><span class="p">.</span><span class="nf">editor</span> <span class="o">=</span> <span class="s1">'vi'</span> <span class="c1"># 'code', 'subl'</span>
</code></pre></div></div>
<h3 id="prompt">Prompt</h3>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Pry</span><span class="p">.</span><span class="nf">config</span><span class="p">.</span><span class="nf">prompt</span> <span class="o">=</span>
<span class="p">[</span>
<span class="o">-></span><span class="p">(</span><span class="n">_obj</span><span class="p">,</span> <span class="n">_nest_level</span><span class="p">,</span> <span class="n">_</span><span class="p">)</span> <span class="p">{</span> <span class="s2">"✎ "</span> <span class="p">},</span>
<span class="o">-></span><span class="p">(</span><span class="o">*</span><span class="p">)</span> <span class="p">{</span> <span class="s2">" "</span> <span class="p">}</span>
<span class="p">]</span>
</code></pre></div></div>
<p>One might dump the current object name, or maintain different propmts based on the level of nesting or whatever. Since <code class="language-plaintext highlighter-rouge">.pryrc</code> is a plain ruby file, this procs might execute any code you want or even mine bitcoins while you are waiting for the next prompt to appear.</p>
<p>I do use minimalistic setting to make copy-paste easy (not requiring any cleanup before pasting.)</p>
<h3 id="colors">Colors</h3>
<p>This is driven by <code class="language-plaintext highlighter-rouge">pry-theme</code> gem. I switch coloring on (unless started as <code class="language-plaintext highlighter-rouge">PRY_BW=true pry</code>) and set the predefined theme that does not hurt eyes much. For folders under <em>Rails</em> supervision (read: for <em>Rails</em> projects) that might show something even fancier. I hate <em>Rails</em>, that’s why I’d never seen it and I cannot tell what exactly happens there.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">unless</span> <span class="no">ENV</span><span class="p">[</span><span class="s1">'PRY_BW'</span><span class="p">]</span>
<span class="no">Pry</span><span class="p">.</span><span class="nf">color</span> <span class="o">=</span> <span class="kp">true</span>
<span class="no">Pry</span><span class="p">.</span><span class="nf">config</span><span class="p">.</span><span class="nf">theme</span> <span class="o">=</span> <span class="s2">"railscasts"</span>
<span class="no">Pry</span><span class="p">.</span><span class="nf">config</span><span class="p">.</span><span class="nf">prompt</span> <span class="o">=</span> <span class="no">PryRails</span><span class="o">::</span><span class="no">RAILS_PROMPT</span> <span class="k">if</span> <span class="k">defined?</span><span class="p">(</span><span class="no">PryRails</span><span class="o">::</span><span class="no">RAILS_PROMPT</span><span class="p">)</span>
<span class="no">Pry</span><span class="p">.</span><span class="nf">config</span><span class="p">.</span><span class="nf">prompt</span> <span class="o">||=</span> <span class="no">Pry</span><span class="p">.</span><span class="nf">prompt</span>
<span class="k">end</span>
</code></pre></div></div>
<h3 id="history">History</h3>
<p>This is a godsend. While in debugger mode, or whether the command (not an evaluation) was executed right before, press <kbd>⏎</kbd> and it’ll be repeated. Extremely helpful for stepping into code in debugger (unless you spot all the bugs with your glance and never ever enter debugging sessions, like me.)</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Pry</span><span class="p">.</span><span class="nf">config</span><span class="p">.</span><span class="nf">history</span><span class="p">.</span><span class="nf">should_save</span> <span class="o">=</span> <span class="kp">true</span>
<span class="no">Pry</span><span class="o">::</span><span class="no">Commands</span><span class="p">.</span><span class="nf">command</span> <span class="sr">/^$/</span><span class="p">,</span> <span class="s2">"repeat last command"</span> <span class="k">do</span>
<span class="n">_pry_</span><span class="p">.</span><span class="nf">run_command</span> <span class="no">Pry</span><span class="p">.</span><span class="nf">history</span><span class="p">.</span><span class="nf">to_a</span><span class="p">.</span><span class="nf">last</span>
<span class="k">end</span>
</code></pre></div></div>
<h3 id="commands">Commands</h3>
<p>Debugger. Don’t trust much this config, I just borrowed it from my teammate who is spending his whole life debugging stuff.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Pry</span><span class="p">.</span><span class="nf">commands</span><span class="p">.</span><span class="nf">alias_command</span> <span class="s1">'c'</span><span class="p">,</span> <span class="s1">'continue'</span> <span class="k">rescue</span> <span class="kp">nil</span>
<span class="no">Pry</span><span class="p">.</span><span class="nf">commands</span><span class="p">.</span><span class="nf">alias_command</span> <span class="s1">'s'</span><span class="p">,</span> <span class="s1">'step'</span> <span class="k">rescue</span> <span class="kp">nil</span>
<span class="no">Pry</span><span class="p">.</span><span class="nf">commands</span><span class="p">.</span><span class="nf">alias_command</span> <span class="s1">'n'</span><span class="p">,</span> <span class="s1">'next'</span> <span class="k">rescue</span> <span class="kp">nil</span>
<span class="no">Pry</span><span class="p">.</span><span class="nf">commands</span><span class="p">.</span><span class="nf">alias_command</span> <span class="s1">'f'</span><span class="p">,</span> <span class="s1">'finish'</span> <span class="k">rescue</span> <span class="kp">nil</span>
<span class="no">Pry</span><span class="p">.</span><span class="nf">commands</span><span class="p">.</span><span class="nf">alias_command</span> <span class="s1">'l'</span><span class="p">,</span> <span class="s1">'whereami'</span> <span class="k">rescue</span> <span class="kp">nil</span>
</code></pre></div></div>
<h3 id="listing-config">Listing config</h3>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Pry</span><span class="p">.</span><span class="nf">config</span><span class="p">.</span><span class="nf">ls</span><span class="p">.</span><span class="nf">separator</span> <span class="o">=</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span> <span class="c1"># new lines between methods</span>
<span class="no">Pry</span><span class="p">.</span><span class="nf">config</span><span class="p">.</span><span class="nf">ls</span><span class="p">.</span><span class="nf">heading_color</span> <span class="o">=</span> <span class="ss">:magenta</span>
<span class="no">Pry</span><span class="p">.</span><span class="nf">config</span><span class="p">.</span><span class="nf">ls</span><span class="p">.</span><span class="nf">public_method_color</span> <span class="o">=</span> <span class="ss">:green</span>
<span class="no">Pry</span><span class="p">.</span><span class="nf">config</span><span class="p">.</span><span class="nf">ls</span><span class="p">.</span><span class="nf">protected_method_color</span> <span class="o">=</span> <span class="ss">:yellow</span>
<span class="no">Pry</span><span class="p">.</span><span class="nf">config</span><span class="p">.</span><span class="nf">ls</span><span class="p">.</span><span class="nf">private_method_color</span> <span class="o">=</span> <span class="ss">:bright_black</span>
</code></pre></div></div>
<p>This should be self-explanatory.</p>
<h1 id="plugins">Plugins</h1>
<p>The below is just a sample of configuring <em>some</em> (read: <em>awesome</em>) plugin. I am positive you dislike bloated configs and tend to maintain them <del>line by line</del> pixel by pixel for decades. The snippet below is provided just for the sake of an example. Also comments in the code speak for themselves.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># `awesome_print` gem is a great syntax colorized printing</span>
<span class="c1"># look at `~/.aprc` for more settings for awesome_print</span>
<span class="k">begin</span>
<span class="nb">require</span> <span class="s1">'awesome_print'</span>
<span class="c1"># The following line enables awesome_print for all pry output,</span>
<span class="c1"># and it also enables paging</span>
<span class="no">Pry</span><span class="p">.</span><span class="nf">config</span><span class="p">.</span><span class="nf">print</span> <span class="o">=</span> <span class="nb">proc</span> <span class="p">{</span><span class="o">|</span><span class="n">output</span><span class="p">,</span> <span class="n">value</span><span class="o">|</span> <span class="no">Pry</span><span class="o">::</span><span class="no">Helpers</span><span class="o">::</span><span class="no">BaseHelpers</span><span class="p">.</span><span class="nf">stagger_output</span><span class="p">(</span><span class="s2">"=> </span><span class="si">#{</span><span class="n">value</span><span class="p">.</span><span class="nf">ai</span><span class="si">}</span><span class="s2">"</span><span class="p">,</span> <span class="n">output</span><span class="p">)}</span>
<span class="c1"># If you want awesome_print without automatic pagination, use the line below</span>
<span class="k">module</span> <span class="nn">AwesomePrint</span>
<span class="no">Formatter</span><span class="p">.</span><span class="nf">prepend</span><span class="p">(</span><span class="no">Module</span><span class="p">.</span><span class="nf">new</span> <span class="k">do</span>
<span class="k">def</span> <span class="nf">awesome_self</span><span class="p">(</span><span class="n">object</span><span class="p">,</span> <span class="n">type</span><span class="p">)</span>
<span class="k">return</span> <span class="k">super</span><span class="p">(</span><span class="n">object</span><span class="p">,</span> <span class="n">type</span><span class="p">)</span> <span class="k">unless</span> <span class="n">type</span> <span class="o">==</span> <span class="ss">:string</span>
<span class="k">return</span> <span class="k">super</span><span class="p">(</span><span class="n">object</span><span class="p">,</span> <span class="n">type</span><span class="p">)</span> <span class="k">unless</span> <span class="vi">@options</span><span class="p">[</span><span class="ss">:string_limit</span><span class="p">]</span>
<span class="k">return</span> <span class="k">super</span><span class="p">(</span><span class="n">object</span><span class="p">,</span> <span class="n">type</span><span class="p">)</span> <span class="k">unless</span> <span class="n">object</span><span class="p">.</span><span class="nf">inspect</span><span class="p">.</span><span class="nf">to_s</span><span class="p">.</span><span class="nf">length</span> <span class="o">></span> <span class="vi">@options</span><span class="p">[</span><span class="ss">:string_limit</span><span class="p">]</span>
<span class="n">colorize</span><span class="p">(</span><span class="n">object</span><span class="p">.</span><span class="nf">inspect</span><span class="p">.</span><span class="nf">to_s</span><span class="p">[</span><span class="mi">0</span><span class="o">..</span><span class="vi">@options</span><span class="p">[</span><span class="ss">:string_limit</span><span class="p">]]</span> <span class="o">+</span> <span class="s2">"..."</span><span class="p">,</span> <span class="n">type</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span><span class="p">)</span>
<span class="k">end</span>
<span class="no">AwesomePrint</span><span class="p">.</span><span class="nf">defaults</span> <span class="o">=</span> <span class="p">{</span>
<span class="ss">:string_limit</span> <span class="o">=></span> <span class="mi">80</span><span class="p">,</span>
<span class="ss">:indent</span> <span class="o">=></span> <span class="mi">2</span><span class="p">,</span>
<span class="ss">:multiline</span> <span class="o">=></span> <span class="kp">true</span>
<span class="p">}</span>
<span class="no">AwesomePrint</span><span class="p">.</span><span class="nf">pry!</span>
<span class="k">rescue</span> <span class="no">LoadError</span> <span class="o">=></span> <span class="n">err</span>
<span class="nb">puts</span> <span class="s2">"gem install awesome_print # <-- highly recommended"</span>
<span class="k">end</span>
</code></pre></div></div>
<h3 id="custom-commands">Custom commands</h3>
<p>Did I say that <code class="language-plaintext highlighter-rouge">pry</code> is very powerful? You can even define your own set of commands to be used within <code class="language-plaintext highlighter-rouge">pry</code>. The example below shows how to create <code class="language-plaintext highlighter-rouge">sql</code> command to execute raw SQL from the console (provided you have a working AR connection there) with a minimum of keystrokes.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">default_command_set</span> <span class="o">=</span> <span class="no">Pry</span><span class="o">::</span><span class="no">CommandSet</span><span class="p">.</span><span class="nf">new</span> <span class="k">do</span>
<span class="n">command</span> <span class="s2">"sql"</span><span class="p">,</span> <span class="s2">"Send sql over AR."</span> <span class="k">do</span> <span class="o">|</span><span class="n">query</span><span class="o">|</span>
<span class="k">if</span> <span class="no">ENV</span><span class="p">[</span><span class="s1">'RAILS_ENV'</span><span class="p">]</span> <span class="o">||</span> <span class="k">defined?</span><span class="p">(</span><span class="no">Rails</span><span class="p">)</span>
<span class="n">pp</span> <span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span><span class="p">.</span><span class="nf">connection</span><span class="p">.</span><span class="nf">select_all</span><span class="p">(</span><span class="n">query</span><span class="p">)</span>
<span class="k">else</span>
<span class="n">pp</span> <span class="s2">"No rails env defined"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="no">Pry</span><span class="p">.</span><span class="nf">config</span><span class="p">.</span><span class="nf">commands</span><span class="p">.</span><span class="nf">import</span> <span class="n">default_command_set</span>
</code></pre></div></div>
<h3 id="monkeypatches-and-globals">Monkeypatches and globals</h3>
<p>Yeah, you might tweak any Ruby code with your own monkeypatches, that are available in <code class="language-plaintext highlighter-rouge">pry</code> sessions only. The following is very handy for playing with arrays and hashes.</p>
<div class="language-pry highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="go">class Array
def self.sample(count = 10, &block)
Array.new(count, &(block || :succ))
end
end
</span><span class="err">
</span><span class="go">Hash.singleton_class.prepend(Module.new
def sample(count = 10)
(?a...count.times.reduce(?a) { |o| o.succ }).
map(&:to_sym).zip(0...count).to_h
end
end)
</span></code></pre></div></div>
<p><em>Sidenote:</em> if you think the snippet above is overcomplicated with <code class="language-plaintext highlighter-rouge">Integer#succ</code>, you surely never dealt with long hashes having more than 26 keys :)</p>
<h3 id="color-customization">Color customization</h3>
<p>Everything below is for customizing colors, using <code class="language-plaintext highlighter-rouge">coderay</code> gem. The result would be worth it. Symbols are red and numbers are blue, all that stuff.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">CodeRay</span><span class="p">.</span><span class="nf">scan</span><span class="p">(</span><span class="s2">"example"</span><span class="p">,</span> <span class="ss">:ruby</span><span class="p">).</span><span class="nf">term</span> <span class="c1"># just to load necessary files</span>
<span class="vg">$LOAD_PATH</span> <span class="o"><<</span> <span class="no">File</span><span class="p">.</span><span class="nf">dirname</span><span class="p">(</span><span class="no">File</span><span class="p">.</span><span class="nf">realpath</span><span class="p">(</span><span class="kp">__FILE__</span><span class="p">))</span>
<span class="nb">require</span> <span class="s2">"escaped_colors"</span>
<span class="k">module</span> <span class="nn">CodeRay</span>
<span class="k">module</span> <span class="nn">Encoders</span>
<span class="k">class</span> <span class="nc">Terminal</span> <span class="o"><</span> <span class="no">Encoder</span>
<span class="no">TERM_TOKEN_COLORS</span><span class="p">.</span><span class="nf">each_pair</span> <span class="k">do</span> <span class="o">|</span><span class="n">key</span><span class="p">,</span> <span class="n">value</span><span class="o">|</span>
<span class="no">TOKEN_COLORS</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="o">=</span> <span class="n">value</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<h2 id="thats-pretty-much-it">That’s pretty much it</h2>
<p>I hope I gave you the first impression on what <code class="language-plaintext highlighter-rouge">pry</code> is and how is it better than <code class="language-plaintext highlighter-rouge">irb</code>. I did not cover debugging experiences, mostly because I do not debug, I write the correct code from the scratch. But I hope I could interest you and—please—don’t hesitate to sink into and reconfigure everything to your own taste.</p>
<p>Happy prying!</p>
Developer Is The Next Ideological Beggar Job2018-10-24T00:00:00+00:00https://rocket-science.ru/hacking/2018/10/24/developer-is-the-next-ideological-beggar-job<p><img src="/img/willemsens-ideological-beggar.jpg" alt="professional beggars were often seen as people not deserving of aid" /></p>
<p><small>— Abraham Willemsens (Metropolitan Museum of Art)</small></p>
<p>This writing is a bit connected to <a href="https://dev.to/stereobooster/developer-is-the-next-blue-collar-job-269b">Developer is the next blue collar job</a>.</p>
<h2 id="lolwut">LOLWUT?</h2>
<p>Please don’t get me wrong. I am a professional developer with 30 years of experience. I can read and understand the code written in almost 20 languages. I am able to produce the professional code in up to ten of them. In my languages of choice I am listed in a stack overflow hall of fame (top-20 overall in <a href="https://stackoverflow.com/tags/ruby/topusers"><code class="language-plaintext highlighter-rouge">ruby</code></a> and <a href="https://stackoverflow.com/tags/elixir/topusers"><code class="language-plaintext highlighter-rouge">elixir</code></a>.) I love coding and I reject all the offers to “grow up” to management, CTO, whatever, despite it might heighten my income in times.</p>
<p>And I am still positive that <strong>computer science is suffering the worst desease during it’s whole history</strong>. There are many reasons for that and I am going to express my very biased and humble opinion on it.</p>
<h2 id="developers-are-extremely-overpaid">Developers Are Extremely Overpaid</h2>
<p>Yes, I said that. I am not talking about <em>real</em> developers, who calculate rocket’s trajectories in NASA or write a software for breathing support apparatuses. I understand why the guy who is responsible for whether we will get pictures from Mars in the next year, or past a decade after the next expedition, should not think about how to pay the next month bills. I am confident that heart muscle implant software should be paid generously.</p>
<p>But the mediocre developer writing pieces of banking software receiving at least twice as much as a bus driver is a nonsense. IQ required to code CRUD in any web framework is below average. I am pretty sure my smart washing machine can do that. Especially taking into account that my washing machine now has an access to the internets, and, hence, stack overflow and forums.</p>
<p>The thing is being overpaid has a drawback. <strong>Random people, who do not love to code, come to the profession after big salaries.</strong> This is terrible. Can you imagine a surgeon who’s afraid of blood but came to be cutting bodies with a scalpel for a bigger income? Would you sign up for his surgery? I doubt.</p>
<p>Can you imagine a lawyer who reads “Penal Code of Texas in 21 Day” stealthily in front of jury? Do you like them to handle your case? I doubt.</p>
<p>And even then to be allowed to practice as a surgeon, or as a lawyer, 5+ years of extensive education is required. To become a well-paid coder it takes 3 weeks in a bootcamp and a piece of luck with the first job. In three month one can claim middle level and some experience. This kills the industry.</p>
<h2 id="developers-are-extremely-exigent">Developers Are Extremely Exigent</h2>
<p>Let’s talk about bus drivers. They have a broken schedule, uncomfortable chair, a rolling wheel instead of the table and they have to hold an attention for the whole work shift. Otherwise, the tragedy may occur. When the developer loses their attention, reddit is being overflooded with dummy comments, that’s what occurs.</p>
<p>And despite that developers have unimaginably inflated demands. Standing work desk, macbook for travelling, a dozen of vertical monitors, hammock in the bathroom and mint mate in the toilet. <em>“Only that way we can be productive.”</em></p>
<p>John Carmack twitted a while ago:</p>
<blockquote class="twitter-tweet"><p lang="en" dir="ltr">I recently fact-checked an article that said I worked "18 hour days", which was just not true -- I have never been effective beyond 13 hours, so I have never worked a 100 hour week. On the other hand, I *like* working 60 hours a week. <a href="https://t.co/MMMr9VbeZ6">https://t.co/MMMr9VbeZ6</a></p>— John Carmack (@ID_AA_Carmack) <a href="https://twitter.com/ID_AA_Carmack/status/1051874929631789056?ref_src=twsrc%5Etfw">October 15, 2018</a></blockquote>
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<p><strong>I have great respect for John, and he is indeed the guy who does rocket science for his whole life.</strong> He enjoys his job. And he still cannot work beyond 13 hours. And you know what? Cashiers at the supermarket can. And somewhat do. Without hammocks and vertical monitors. For ten times less income. Is it fair? I doubt.</p>
<p>These guys do the real job, they have a complete right to claim fatigue, exhaustion and burnout.</p>
<p><img src="/img/coal-miners.jpg" alt="Coal miners after six hours in the bottomhole" /></p>
<p>When I hear about the burnout from people who were literally relocating pixels on the screen for eight hours, I have no emotions but laughter.</p>
<h2 id="developers-are-extremely-touchy">Developers Are Extremely Touchy</h2>
<p>Do you know of any other craft where laborers spend half of their working hours discussing how they could improve their feeling of the environment? Something like bartender unionists discussing whether “bloody Mary” should be now called “somewhat-red-colored bot” and denying serving it under the former misogynistic name? 80% of blog posts that are intended to be related to technology are about“unrelated skills that will help you to spend more time discussing these exciting topics instead of getting the damn job done.”</p>
<p>Ask a hundred of modern developers ‘what the most important skill in CS is’ and ninety of them would come up with kinda “empathy.” What about producing a robust solutions in time, huh? Would you like your car brakes were fixed by a gloomy mechanic with hands soiled with engine oil, or by an affable ignoramus in a white coat? Please, pick one, because both are rarely apply.</p>
<p>It happens quite often: brilliant professionals lack some social skills. They might lack joviality, even geniality. And still be great professionals. Basically what they are paid for. The tendency today would be to humiliate and ridicule scoundrels in every possible way. <strong>The proclaimed holy tolerance does not apply to them.</strong> Because fuck them, that’s why. They spoil our cheerful picture of the world.</p>
<p>Everyone likes when they are being stroked on the head and told “that’s fine, that’s ok, don’t worry.” But, you know, we are not in the kindergarten anymore. We make mistakes and we do pay for our mistakes. Everywhere. Save for IT industry. And this is terribly scary.</p>
<h2 id="developers-are-not-elite-anymore">Developers Are Not Elite Anymore</h2>
<p>Back in 1970 there were great developers who basically built a basement for everything we use for free now. They were incredibly talented and extremely humble. Both equally matter. They did their job as earliest explorers, in the wilderness of knowledge and under the windstorms of mistakes. I had never heard that a lack of the vertical monitor or an absense of the standing desk prevented them from doing their best.</p>
<p>We yield everything from them. <strong>They were elite and we are not.</strong></p>
<p>That roughly means we probably should stop asking for bonuses to our already bloated salaries, stop whining and stop demanding for better life. We already have one.</p>
<p>Let us please finally concentrate on doing our damn job well. That is important. And hopefully that way we will have something to be proud of again.</p>
Raise For The Rescue2018-10-21T00:00:00+00:00https://rocket-science.ru/hacking/2018/10/21/raise-for-the-rescue<h3 id="exceptional-behaviour">Exceptional Behaviour</h3>
<blockquote>
<p>There are only two hard things in Computer Science: cache invalidation and naming things.
<em>— Phil Karlton</em></p>
</blockquote>
<p>Actually, there are three. I would put making a decision on whether to raise or gracefully handle exceptions in that row.</p>
<p>There is the famous Erlang philosophy “let-it-crash,” which roughly means the process should not lie about it’s ability to deal with everything, rather it should simply refuse to handle unexpected data.</p>
<blockquote>
<p>[…] a clean separation of issues. We write code that solves problems and code that fixes problems, but the two are not intertwined.
<em>— Joe Armstrong, “<a href="https://pragprog.com/book/jaerlang2/programming-erlang">Programming Erlang</a>”</em></p>
</blockquote>
<p>That said, the code that <em>solves</em> problem should not ever try to <em>fix</em> problems. There is a rule of thumb.</p>
<p><strong>Code handling data must not make assumptions. If the code does not understand the data, it should raise an exception. Period.</strong></p>
<p>Imagine we have a function that processes the use input (say, currency conversion rate.) We usually tend to write something like this:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">params</span><span class="p">[</span><span class="ss">:inverse</span><span class="p">]</span> <span class="p">?</span> <span class="n">amount</span> <span class="o">/</span> <span class="n">params</span><span class="p">[</span><span class="ss">:rate</span><span class="p">]</span> <span class="p">:</span> <span class="n">amount</span> <span class="o">*</span> <span class="n">params</span><span class="p">[</span><span class="ss">:rate</span><span class="p">]</span>
</code></pre></div></div>
<p>But we are smart and we notice, that when the user passes zero, the code above will blow up, so we defend our application:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">if</span> <span class="n">params</span><span class="p">[</span><span class="ss">:inverse</span><span class="p">]</span>
<span class="k">unless</span> <span class="n">params</span><span class="p">[</span><span class="ss">:rate</span><span class="p">].</span><span class="nf">zero?</span>
<span class="n">amount</span> <span class="o">/</span> <span class="n">params</span><span class="p">[</span><span class="ss">:rate</span><span class="p">]</span>
<span class="k">else</span>
<span class="c1"># WHAT WOULD WE PUT HERE?</span>
<span class="k">end</span>
<span class="k">else</span>
<span class="n">amount</span> <span class="o">*</span> <span class="n">params</span><span class="p">[</span><span class="ss">:rate</span><span class="p">]</span>
<span class="k">end</span>
</code></pre></div></div>
<p>In the first place, the code already looks like a spaghetti monster (for two years already I reject any PRs having a nested <code class="language-plaintext highlighter-rouge">if</code> and I am going to start rejecting PRs having <code class="language-plaintext highlighter-rouge">if</code> at all as a sign of the architectural illness.) Besides that, there is no clean understanding of what should we do when the rate passed is zero. This code knows nothing about how to deal with the weird rate and it’s none of its business according to <a href="https://en.wikipedia.org/wiki/Single_responsibility_principle">SRP</a>. That means, that the explicit check for the <code class="language-plaintext highlighter-rouge">rate</code> value in the code above is a code smell. The former snippet is <em>correct</em>, the latter is a bloated piece of unrelated garbage.</p>
<p>The <strong>behaviour of this code when the rate is zero is <em>exceptional</em></strong> and unless we clearly know how to deal with it (e. g. if we are processing a view and might show <code class="language-plaintext highlighter-rouge">N/A</code> without any side effect), it <strong>should not interfere the control flow</strong>.</p>
<h3 id="let-it-crash">Let it crash</h3>
<blockquote>
<p>The real world actually has independent things communicating through messages. I’m an ex-physicist — we perceive the world by receiving messages. Packets of light and sound carry information. All that we know about the world is what we’ve learned by receiving messages. We don’t have shared memory. I have my memory, you have yours, and I don’t know what you think about anything. If I want to know what you think about something, then I have to ask you a question and wait for you to reply.
<em>— Joe Armstrong, “<a href="https://pragprog.com/book/jaerlang2/programming-erlang">Programming Erlang</a>”</em></p>
</blockquote>
<p>I have seen (and honestly produced) a lot of code that is as defensive as possible. The <code class="language-plaintext highlighter-rouge">case</code> conditional statement should have <code class="language-plaintext highlighter-rouge">else</code> clause, that is to process unexpected income, they say. But… wait. Let’s take a step back and look up the meaning of “unexpected” in Merriam-Webster. It says “unexpected” is a synonym of “unforeseen” and to my best knowledge the latter means one cannot predict it. I am also an ex-physicist and I am pretty sure we cannot cook the ready-to-use answer if we don’t know what the question was. They also call it <em>“causation”</em>.</p>
<p>That means, the code should not make silly attempts to cover all the cases. This leads to hidden, hard-to-hunt, arisen from nowhere bugs. Don’t screw future you up! Leave it not handled, or raise directly from there. That is one of rare cases when it’s better to receive <code class="language-plaintext highlighter-rouge">NullPointerException</code> right there than some incorrect value assigned to the unrelated variable ten thousands LOCs away.</p>
<p>And since you have the application-wide error handler (in erlang world we call it “ErlangVM”,) you are safe to propagate exception until some piece of code knows what to do with it. <strong>Don’t rescue</strong> if you are uncertain, what exactly should happen on error. Don’t rescue if many different things might happen, depending on the outside world state. In general, this advice (although it has some exceptions raised) should be read as “don’t rescue.”</p>
<p>Rescued excepions lead to hidden induced issues in the code that <em>was indeed exectuted afterwards, despite the data is known to be corrupted</em>.</p>
<h3 id="graceful-feedback">Graceful feedback</h3>
<p>Let me restate it again: do not return results of partially processed input, do not input “a meaningful value” when the input is incorrect, do not attempt to assume anything. Wrong means wrong. Let the calling process rethink and reenter the data even if the calling process is a beloved client. Do not fool your code users mimicking robust solid response when you don’t have one. They will definitely grumble in dissatisfaction, but they won’t stalk you to kill as in the case of “almost correct response.”</p>
<p>Instead of returning the half-proven response, put all the effort in the maintaining as descriptive and affable error message as possible. Tell them, whoever they are—function in the neighbour module, external API, or a human being living in Toledo, OH—what went wrong and how to fix an issue. Raise an exception with as many local information put in, as possible. And expose this data to the caller.</p>
<p>Do not hide skeletons in the closet, and nightmares will shun you away.</p>
<p>Happy raising!</p>
SeeAsVee Library For Handy CSV Processing2018-10-20T00:00:00+00:00https://rocket-science.ru/hacking/2018/10/20/see_as_vee-to-the-rescue<h2 id="intro">Intro</h2>
<p>In our daily work we deal a lot with CSV files uploaded by our clients. We love our customers and we turn a blind eye to their pranks. They have brilliant skills in their own business and unfortunately that business has nothing to do with conforming <code class="language-plaintext highlighter-rouge">RFC4180</code> in choosing correct CSV delimiters.</p>
<p>We receive CSVs with commas, semicolons, tabs and even mixed delimiters when they edit these files manually. Also, they tend to mess up with columns names, columns order, values formats. We receive <code class="language-plaintext highlighter-rouge">"3.358899E+20"</code> instead of <code class="language-plaintext highlighter-rouge">"12356890ABCDEF1234"</code> for their bank account numbers, because that’s how Excel has it stored. Our clients have a dozillion of more important things to do than checking the CSV file format before it’s being sent to us.</p>
<p>On the other hand, these CSV contain a valuable financial information that we are to process automatically. We have no resources to outsource CSV validation to AI. We should do it in plain Ruby.</p>
<p>If you are like us, you probably might be interested in our open source library called <a href="https://github.com/am-kantox/see_as_vee"><strong><code class="language-plaintext highlighter-rouge">SeeAsVee</code></strong></a>. It allows sophisticated checks, validations and transformations of the input. It is by no mean the fastest CSV processor, and it never intended to be. It perfectly suits the use case when you get crappy, but not very huge files and you want to handle them as gracefully as possible.</p>
<h2 id="in-general">In General</h2>
<p>This library allows input validation, input transformation, immediate callbacks on errors found in CSV lines and many more. It supports both <code class="language-plaintext highlighter-rouge">.csv</code> and <code class="language-plaintext highlighter-rouge">.xlsx</code> formats and allows transparent creation of CSV files from hashes. Also, it might return back the input CSV with all the found errors highlighted so that we can pass it back to the client as a ready-to-change example of what went wrong.</p>
<p>Besides custom rules, it also supports <a href="https://dry-rb.org/gems/dry-validation/"><code class="language-plaintext highlighter-rouge">dry-validation</code></a> as schemas for input validation. It also accepts <code class="language-plaintext highlighter-rouge">UTF-16 LE</code> with <code class="language-plaintext highlighter-rouge">BOM</code> and tabs as delimiters to please MS Excel users using default export settings.</p>
<p>There are also custom unique CSV output options: a value in the hash might be an array, or a string concatenated with commas, as returned by <code class="language-plaintext highlighter-rouge">GROUP_CONCAT</code> from any database.</p>
<h2 id="in-details">In Details</h2>
<h3 id="input-validation">Input Validation</h3>
<p>The most generic overcoplicated example of almost all the validation features would be:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># define schema</span>
<span class="n">schema</span> <span class="o">=</span> <span class="no">Dry</span><span class="o">::</span><span class="no">Validation</span><span class="o">.</span><span class="no">Params</span> <span class="k">do</span>
<span class="n">required</span><span class="p">(</span><span class="ss">:reference</span><span class="p">)</span> <span class="p">{</span> <span class="n">filled?</span> <span class="o">></span> <span class="n">str?</span> <span class="p">}</span>
<span class="n">required</span><span class="p">(</span><span class="ss">:trade_date</span><span class="p">).</span><span class="nf">filled</span><span class="p">(</span><span class="ss">:date?</span><span class="p">)</span>
<span class="n">required</span><span class="p">(</span><span class="ss">:amount</span><span class="p">).</span><span class="nf">filled</span><span class="p">(</span><span class="ss">:float?</span><span class="p">)</span>
<span class="k">end</span>
<span class="c1"># validate the input</span>
<span class="n">validation</span> <span class="o">=</span> <span class="no">SeeAsVee</span><span class="p">.</span><span class="nf">validate</span><span class="p">(</span><span class="s1">'input.csv'</span><span class="p">,</span> <span class="n">schema</span><span class="p">)</span>
<span class="k">raise</span> <span class="k">unless</span> <span class="n">validation</span><span class="p">.</span><span class="nf">all?</span> <span class="p">{</span> <span class="o">|</span><span class="n">vr</span><span class="o">|</span> <span class="n">vr</span><span class="p">.</span><span class="nf">errors</span><span class="p">.</span><span class="nf">empty?</span> <span class="p">}</span>
<span class="c1"># sophisticated validation and transformation</span>
<span class="n">sheet</span> <span class="o">=</span>
<span class="no">SeeAsVee</span><span class="p">.</span><span class="nf">harvest</span><span class="p">(</span>
<span class="s1">'input.xlsx'</span><span class="p">,</span> <span class="c1"># file exists ⇒ will be loaded</span>
<span class="ss">formatters: </span><span class="p">{</span> <span class="ss">trade_date: </span><span class="o">-></span><span class="p">(</span><span class="n">v</span><span class="p">)</span> <span class="p">{</span> <span class="no">DateTime</span><span class="p">.</span><span class="nf">parse</span><span class="p">(</span><span class="n">v</span><span class="p">)</span> <span class="p">}</span> <span class="p">},</span>
<span class="ss">checkers: </span><span class="p">{</span> <span class="ss">reference: </span><span class="o">-></span><span class="p">(</span><span class="n">v</span><span class="p">)</span> <span class="p">{</span> <span class="n">v</span><span class="p">.</span><span class="nf">nil?</span> <span class="p">}</span> <span class="p">},</span> <span class="c1"># must be present</span>
<span class="ss">skip_blank_rows: </span><span class="kp">true</span> <span class="c1"># optional, defaults to false</span>
<span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">idx</span><span class="p">,</span> <span class="n">errors</span><span class="p">,</span> <span class="nb">hash</span><span class="o">|</span> <span class="c1"># index, errors, as hash { header: value }</span>
<span class="o">...</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The above will run the input against validation schema and raise if the input is just malformed. On the second step, we process the input, yielding the row index, errors found in the row and the row itself as a hash to the block. The outcome of this operation would be an instance of <code class="language-plaintext highlighter-rouge">SeeAsVee::Sheet</code> class, basically a wrapper around the array of hashes.</p>
<h3 id="producing-the-csv">Producing the CSV</h3>
<p>We accept any arbitrary array of hashes, they will be merged into the ‘union’ or hashes, expanded for all the fields found in all the hashes. By default, the temporary file will be produced and the caller code might read it, copy to some extenral bucket or do whatever is needed. For the following input</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">SeeAsVee</span><span class="p">.</span><span class="nf">csv</span><span class="p">(</span>
<span class="p">[{</span><span class="ss">name: </span><span class="s1">'Aleksei'</span><span class="p">,</span> <span class="ss">age: </span><span class="mi">18</span><span class="p">},</span>
<span class="p">{</span><span class="ss">name: </span><span class="s1">'John'</span><span class="p">,</span> <span class="ss">value: </span><span class="mf">3.14</span><span class="p">}],</span>
<span class="ss">col_sep: </span><span class="s2">"</span><span class="se">\t</span><span class="s2">"</span>
<span class="p">)</span>
</code></pre></div></div>
<p>the CSV file with <em>three</em> columns (<code class="language-plaintext highlighter-rouge">name, age, value</code>) will be produced.</p>
<h3 id="why-now">Why now?</h3>
<p>We are extremely satisfied with how it works and it covers all our needs. There is a room for many improvements, though, including but not limited to:</p>
<ul>
<li>stream processing</li>
<li>inplace dry validation</li>
<li>more control on how the file with errors to be sent back to the client is being produced</li>
<li>more tied integration with the database and (possibly) <code class="language-plaintext highlighter-rouge">ActiveRecord</code>.</li>
</ul>
<p>Since we are fine with what it produces and I have luckily quitted Ruby development (it’s so boring!) I decided to ask the community, if there might be a need in some extended functionality. I can commit to this library a bit more, I like it and I don’t want it to just die.</p>
<p>So if anybody is curious about any other option to be added, feel free to <a href="https://github.com/am-kantox/see_as_vee/issues">fill an issue</a>.</p>
<p>Happy sheeting!</p>
Ruby metaprogramming for beginners → Elixir-like specs2018-10-17T00:00:00+00:00https://rocket-science.ru/hacking/2018/10/17/elixir-like-specs-in-ruby<h3 id="typespecs">Typespecs</h3>
<p><em>Erlang</em> and hence <em>Elixir</em> are dynamically typed languages. They both have an extended abilities to type check in compile time using <em>typespecs</em>. Here are <a href="https://elixir-lang.org/getting-started/typespecs-and-behaviours.html">an introduction</a> and <a href="https://hexdocs.pm/elixir/master/typespecs.html#content">more detailed documentation</a>.</p>
<p>Basically one might declare the specification of the function and if the actual function declaration violates this contract, the static code analysis tool called <a href="http://erlang.org/doc/man/dialyzer.html">dialyzer</a> will report an issue. The format of typespecs looks pretty nifty:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">@spec</span> <span class="n">concat</span><span class="p">(</span><span class="n">binary</span><span class="p">(),</span> <span class="n">any</span><span class="p">())</span> <span class="p">::</span> <span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">binary</span><span class="p">()}</span> <span class="o">|</span> <span class="p">{</span><span class="ss">:error</span><span class="p">,</span> <span class="n">any</span><span class="p">()}</span>
<span class="k">def</span> <span class="n">concat</span><span class="p">(</span><span class="n">origin</span><span class="p">,</span> <span class="n">any</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="n">origin</span> <span class="o"><></span> <span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="p">(</span><span class="n">any</span><span class="p">)</span>
</code></pre></div></div>
<p>During the work on explicit function typing in <a href="https://github.com/am-kantox/dry-behaviour"><code class="language-plaintext highlighter-rouge">Dry::Protocols</code></a> I experimented with the syntax of specs. The most exciting syntax I was able to get working was the almost copy of the Elixir’s one:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kp">include</span> <span class="no">Dry</span><span class="o">::</span><span class="no">Annotation</span>
<span class="vi">@spec</span><span class="p">[(</span><span class="n">this</span><span class="p">),</span> <span class="p">(</span><span class="n">string</span> <span class="o">|</span> <span class="n">any</span><span class="p">)</span> <span class="o">::</span> <span class="p">(</span><span class="n">string</span><span class="p">)]</span>
<span class="n">defmethod</span> <span class="ss">:append</span><span class="p">,</span> <span class="ss">:this</span><span class="p">,</span> <span class="ss">:any</span>
<span class="k">def</span> <span class="nf">append</span><span class="p">(</span><span class="n">this</span><span class="p">,</span> <span class="n">any</span><span class="p">)</span>
<span class="n">this</span> <span class="o"><<</span> <span class="n">any</span><span class="p">.</span><span class="nf">inspect</span>
<span class="k">end</span>
</code></pre></div></div>
<p>This <code class="language-plaintext highlighter-rouge">@spec</code> line is parsed and evaluated by standard ruby parser, if anybody’s curious. In this writing I am going to show how I had it implemented. For those brave who think they can do some ruby, I’d strongly suggest doing this golf yourselves, it was an exciting couple of hours while I get everything up and parsing properly (through crashes and weird error messages).</p>
<h3 id="getting-to-the-task">Getting to the task</h3>
<p>Well, for the sake of making the task exciting, I wanted to replicate Elixir’s syntax as close as possible. Of course I might go with boring Rails style declare-verbose-DSL-and-we-are-all-set. But this is so dull!</p>
<p>So, let’s see what we can do here. An orphan instance variable would be ignored by Ruby (well, it’s value will be returned and immediately discarded,) so I had two options here: to assign the “type” to theit, or to call it. I found <code class="language-plaintext highlighter-rouge">@spec[...]</code> (which is an alias for <code class="language-plaintext highlighter-rouge">@spec.call</code> btw) to be most sexy.</p>
<div class="language-diff highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="gd">- @spec = ...
- @spec.(...)
</span><span class="gi">+ @spec[...]
</span></code></pre></div></div>
<p>Now we need to handle parameters. The easiest way to gorge a list of words with Ruby interpretor is to create an instance of some specifically dedicated <em>accumulator</em> class and return <code class="language-plaintext highlighter-rouge">self</code> from each call to it’s <code class="language-plaintext highlighter-rouge">method_missing</code>. To avoid clashes, I’d derive this class from <a href="https://ruby-doc.org/core/BasicObject.html"><code class="language-plaintext highlighter-rouge">BasicObject</code></a>, not from the standard <code class="language-plaintext highlighter-rouge">Object</code>:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">AnnotationImpl</span> <span class="o"><</span> <span class="no">BasicObject</span>
<span class="k">def</span> <span class="nf">initialize</span>
<span class="vi">@types</span> <span class="o">=</span> <span class="p">{</span><span class="ss">args: </span><span class="p">[],</span> <span class="ss">result: </span><span class="p">[]}</span>
<span class="k">end</span>
<span class="k">def</span> <span class="err">🖊</span><span class="p">(</span><span class="nb">name</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">&</span><span class="err">λ</span><span class="p">)</span>
<span class="vi">@types</span><span class="p">[</span><span class="ss">:args</span><span class="p">]</span> <span class="o"><<</span> <span class="p">[</span><span class="n">args</span><span class="p">.</span><span class="nf">empty?</span> <span class="p">?</span> <span class="nb">name</span> <span class="p">:</span> <span class="p">[</span><span class="nb">name</span><span class="p">,</span> <span class="n">args</span><span class="p">,</span> <span class="err">λ</span><span class="p">]]</span>
<span class="nb">self</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">module</span> <span class="nn">Annotation</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">included</span><span class="p">(</span><span class="n">base</span><span class="p">)</span>
<span class="n">base</span><span class="p">.</span><span class="nf">instance_variable_set</span><span class="p">(</span><span class="ss">:@annotations</span><span class="p">,</span> <span class="no">AnnotationImpl</span><span class="p">.</span><span class="nf">new</span><span class="p">)</span>
<span class="n">base</span><span class="p">.</span><span class="nf">instance_variable_set</span><span class="p">(</span><span class="ss">:@spec</span><span class="p">,</span> <span class="o">-></span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">)</span> <span class="p">{</span> <span class="nb">puts</span> <span class="n">args</span><span class="p">.</span><span class="nf">inspect</span> <span class="p">})</span>
<span class="n">base</span><span class="p">.</span><span class="nf">instance_eval</span> <span class="k">do</span>
<span class="k">def</span> <span class="nf">method_missing</span><span class="p">(</span><span class="nb">name</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">&</span><span class="err">λ</span><span class="p">)</span>
<span class="vi">@annotations</span><span class="p">.</span><span class="nf">__send__</span><span class="p">(:</span><span class="err">🖊</span><span class="p">,</span> <span class="nb">name</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">&</span><span class="err">λ</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>I give such weird names to methods on purpose: I want to avoid name clashes as much as possible. Also, please note, that this approach is very dangerous because we are to overwrite <code class="language-plaintext highlighter-rouge">method_missing</code> on any class <code class="language-plaintext highlighter-rouge">Annotation</code> will be included into.</p>
<p>It’s fine for the demonstration purposes, though. And, apparently, for <code class="language-plaintext highlighter-rouge">Dry::Protocols</code>, because they are very isolated, are not supposed to be inherited and declare very few methods by design.</p>
<p>So far so good. We already have everything to support <code class="language-plaintext highlighter-rouge">@spec[foo, bar, baz]</code> syntax. Include <code class="language-plaintext highlighter-rouge">Annotation</code> into some fresh class and try it.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">C</span>
<span class="kp">include</span> <span class="no">Annotation</span>
<span class="vi">@spec</span><span class="p">[</span><span class="n">foo</span><span class="p">,</span> <span class="n">bar</span><span class="p">,</span> <span class="n">baz</span><span class="p">]</span>
<span class="k">end</span>
<span class="c1">#⇒ NoMethodError: undefined method `inspect' for #<AnnotationImpl:0x00564f9d7e0e80></span>
</code></pre></div></div>
<p>Sure thing, it’s an instance of <code class="language-plaintext highlighter-rouge">BasicObject</code>. Let’s define it:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">inspect</span>
<span class="vi">@types</span><span class="p">.</span><span class="nf">inspect</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Now the syntax somehow works. When I say “works” I just mean it does not blow up neither consuses both Ruby parser and interpreter.</p>
<h3 id="hardcore-boolean-or-for-types">Hardcore: boolean <em>or</em> for types</h3>
<p>But wait, we don’t want to stick to the only type permitted; we want to allow boolean <code class="language-plaintext highlighter-rouge">or</code> for them! In Elixir syntax that is done with a vertical bar <code class="language-plaintext highlighter-rouge">|</code>, so I decided to fully replicate this functionality. It might sound more complicated than just collecting words, but in fact it is not. Ruby classes <em>allow</em> redefinition of the <code class="language-plaintext highlighter-rouge">#|</code> method:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">|</span><span class="p">(</span><span class="n">_</span><span class="p">)</span>
<span class="vi">@types</span><span class="p">[</span><span class="ss">:args</span><span class="p">].</span><span class="nf">push</span><span class="p">(</span>
<span class="mi">2</span><span class="p">.</span><span class="nf">times</span><span class="p">.</span><span class="nf">map</span> <span class="p">{</span> <span class="vi">@types</span><span class="p">[</span><span class="ss">:args</span><span class="p">].</span><span class="nf">pop</span> <span class="p">}.</span><span class="nf">rotate</span><span class="p">.</span><span class="nf">reduce</span><span class="p">(</span><span class="o">&</span><span class="ss">:concat</span><span class="p">)</span>
<span class="p">)</span>
<span class="nb">self</span>
<span class="k">end</span>
</code></pre></div></div>
<p>What happens here? We pop two last elements from the array (the current one and the previous one,) join them together (preserving the order) and push them back to the array of arguments:</p>
<ul>
<li><strong><code class="language-plaintext highlighter-rouge">@types[:args]</code></strong> before: <code class="language-plaintext highlighter-rouge">[[:foo], [:bar], [:baz]]</code> where the <code class="language-plaintext highlighter-rouge">:baz</code> just came in</li>
<li>after 2 pops: <code class="language-plaintext highlighter-rouge">[[:foo]]</code> and <code class="language-plaintext highlighter-rouge">[[:baz], [:bar]].rotate.reduce(&:concat)</code> ≡ <code class="language-plaintext highlighter-rouge">[[:bar, :baz]]</code></li>
<li><strong><code class="language-plaintext highlighter-rouge">@types[:args]</code></strong> after: <code class="language-plaintext highlighter-rouge">[[:foo], [:bar, :baz]]</code></li>
</ul>
<p>That was simple. Also, to make it cleaner and to avoid the mess with precedence, I decided to explicitly require to surround the parameters with parentheses <code class="language-plaintext highlighter-rouge">@spec[(foo), (bar | baz)]</code>.</p>
<h3 id="nightmare-result-type">Nightmare: result type</h3>
<p>OK, that is where I expected issues. Of course, I could use hashrocket as lazy unambitious fellas do, but I am not like that. I didn’t want that trash, I wanted fancy Elixir-like colons:</p>
<div class="language-diff highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="gd">- @spec[(foo), (bar, baz) => (boo)]
</span><span class="gi">+ @spec[(foo), (bar, baz) :: (boo)]
</span></code></pre></div></div>
<p>But does this seem to be possible?—Yes, apparently it does. Remember one might call methods with double colon like <code class="language-plaintext highlighter-rouge">42::to_s #⇒ "42"</code>? Here we go.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">call</span><span class="p">(</span><span class="o">*</span><span class="p">)</span>
<span class="vi">@types</span><span class="p">[</span><span class="ss">:result</span><span class="p">]</span> <span class="o"><<</span> <span class="vi">@types</span><span class="p">[</span><span class="ss">:args</span><span class="p">].</span><span class="nf">pop</span>
<span class="nb">self</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Double colon invokes <code class="language-plaintext highlighter-rouge">call</code> method on the instance under the hood. When our implementation experiences <em>call</em>, it just pops the last argument and pushes it to the <code class="language-plaintext highlighter-rouge">result</code> array. Frankly, I though it’s gonna be harder.</p>
<h3 id="summing-it-up">Summing it up</h3>
<p>Technically, we are all done. The implementation is ready to go. Well, almost. There are some cosmetic changes to make it work with several different <code class="language-plaintext highlighter-rouge">@spec</code>s (the same way as <code class="language-plaintext highlighter-rouge">desc</code> works with rake tasks definitions, just collecting them and picking up the last one.)</p>
<p>Below is the whole code, for those still curious. I must repeat this: do not do that in real life. Not because the code is cumbersome and hard to read: it’s pretty vivid, clean and simple. But because polluting the space of the class including our module with heaps of nasty garbage as Rails do isshould be avoided. Whether we need a full load of unpredictable behaviours and tons of arisen from nowhere methods, we already have <code class="language-plaintext highlighter-rouge">ActiveSupport</code> for that. Enough is enough.</p>
<p>The code is provided mostly as an example of Ruby nearly infinite capabilities to habdle whatever crap comes to the developer’s head. In [<code class="language-plaintext highlighter-rouge">Dry::Protocol</code>] we’ll use way less sexy symbols for annotations.</p>
<hr />
<h3 id="appendix-i--complete-source-code">Appendix I :: complete source code</h3>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nn">Dry</span>
<span class="k">class</span> <span class="nc">AnnotationImpl</span> <span class="o"><</span> <span class="no">BasicObject</span>
<span class="k">def</span> <span class="nf">initialize</span>
<span class="vi">@spec</span> <span class="o">=</span> <span class="p">[]</span>
<span class="vi">@specs</span> <span class="o">=</span> <span class="p">[]</span>
<span class="vi">@types</span> <span class="o">=</span> <span class="p">{</span><span class="ss">args: </span><span class="p">[],</span> <span class="ss">result: </span><span class="p">[]}</span>
<span class="k">end</span>
<span class="k">def</span> <span class="err">📝</span> <span class="o">&</span><span class="err">λ</span>
<span class="k">return</span> <span class="vi">@spec</span> <span class="k">if</span> <span class="err">λ</span><span class="p">.</span><span class="nf">nil?</span>
<span class="p">(</span><span class="k">yield</span> <span class="vi">@spec</span><span class="p">).</span><span class="nf">tap</span> <span class="p">{</span> <span class="vi">@spec</span><span class="p">.</span><span class="nf">clear</span> <span class="p">}</span>
<span class="k">end</span>
<span class="k">def</span> <span class="err">📝📝</span>
<span class="vi">@specs</span>
<span class="k">end</span>
<span class="k">def</span> <span class="err">📇</span>
<span class="vi">@types</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">to_s</span>
<span class="vi">@specs</span><span class="p">.</span><span class="nf">reject</span> <span class="k">do</span> <span class="o">|</span><span class="n">type</span><span class="o">|</span>
<span class="sx">%i[args result]</span><span class="p">.</span><span class="nf">all?</span> <span class="p">{</span> <span class="o">|</span><span class="n">key</span><span class="o">|</span> <span class="n">type</span><span class="p">[</span><span class="n">key</span><span class="p">].</span><span class="nf">empty?</span> <span class="p">}</span>
<span class="k">end</span><span class="p">.</span><span class="nf">map</span> <span class="k">do</span> <span class="o">|</span><span class="n">type</span><span class="o">|</span>
<span class="s2">"@spec["</span> <span class="o"><<</span>
<span class="n">type</span><span class="p">.</span>
<span class="nf">values</span><span class="p">.</span>
<span class="nf">map</span> <span class="p">{</span> <span class="o">|</span><span class="n">args</span><span class="o">|</span> <span class="n">args</span><span class="p">.</span><span class="nf">map</span> <span class="p">{</span> <span class="o">|</span><span class="n">args</span><span class="o">|</span> <span class="s2">"(</span><span class="si">#{</span><span class="n">args</span><span class="p">.</span><span class="nf">join</span><span class="p">(</span><span class="s1">' | '</span><span class="p">)</span><span class="si">}</span><span class="s2">)"</span> <span class="p">}.</span><span class="nf">join</span><span class="p">(</span><span class="s1">', '</span><span class="p">)</span> <span class="p">}.</span>
<span class="nf">join</span><span class="p">(</span><span class="s1">' :: '</span><span class="p">)</span> <span class="o"><<</span> <span class="s2">"]"</span>
<span class="k">end</span><span class="p">.</span><span class="nf">join</span><span class="p">(</span><span class="s1">' || '</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">inspect</span>
<span class="vi">@specs</span><span class="p">.</span><span class="nf">reject</span> <span class="k">do</span> <span class="o">|</span><span class="n">type</span><span class="o">|</span>
<span class="sx">%i[args result]</span><span class="p">.</span><span class="nf">all?</span> <span class="p">{</span> <span class="o">|</span><span class="n">key</span><span class="o">|</span> <span class="n">type</span><span class="p">[</span><span class="n">key</span><span class="p">].</span><span class="nf">empty?</span> <span class="p">}</span>
<span class="k">end</span><span class="p">.</span><span class="nf">inspect</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">call</span><span class="p">(</span><span class="o">*</span><span class="p">)</span>
<span class="vi">@types</span><span class="p">[</span><span class="ss">:result</span><span class="p">]</span> <span class="o"><<</span> <span class="vi">@types</span><span class="p">[</span><span class="ss">:args</span><span class="p">].</span><span class="nf">pop</span>
<span class="nb">self</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">|</span><span class="p">(</span><span class="n">_</span><span class="p">)</span>
<span class="vi">@types</span><span class="p">[</span><span class="ss">:args</span><span class="p">].</span><span class="nf">push</span><span class="p">(</span>
<span class="mi">2</span><span class="p">.</span><span class="nf">times</span><span class="p">.</span><span class="nf">map</span> <span class="p">{</span> <span class="vi">@types</span><span class="p">[</span><span class="ss">:args</span><span class="p">].</span><span class="nf">pop</span> <span class="p">}.</span><span class="nf">rotate</span><span class="p">.</span><span class="nf">reduce</span><span class="p">(</span><span class="o">&</span><span class="ss">:concat</span><span class="p">)</span>
<span class="p">)</span>
<span class="nb">self</span>
<span class="k">end</span>
<span class="k">def</span> <span class="err">🖊️</span><span class="p">(</span><span class="nb">name</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">&</span><span class="err">λ</span><span class="p">)</span>
<span class="vi">@types</span><span class="p">[</span><span class="ss">:args</span><span class="p">]</span> <span class="o"><<</span> <span class="p">[</span><span class="n">args</span><span class="p">.</span><span class="nf">empty?</span> <span class="p">?</span> <span class="nb">name</span> <span class="p">:</span> <span class="p">[</span><span class="nb">name</span><span class="p">,</span> <span class="n">args</span><span class="p">,</span> <span class="err">λ</span><span class="p">]]</span>
<span class="nb">self</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">module</span> <span class="nn">Annotation</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">included</span><span class="p">(</span><span class="n">base</span><span class="p">)</span>
<span class="n">annotations</span> <span class="o">=</span> <span class="no">AnnotationImpl</span><span class="p">.</span><span class="nf">new</span>
<span class="n">base</span><span class="p">.</span><span class="nf">instance_variable_set</span><span class="p">(</span><span class="ss">:@annotations</span><span class="p">,</span> <span class="n">annotations</span><span class="p">)</span>
<span class="n">base</span><span class="p">.</span><span class="nf">instance_variable_set</span><span class="p">(</span><span class="ss">:@spec</span><span class="p">,</span> <span class="o">-></span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">)</span> <span class="p">{</span>
<span class="n">impl</span> <span class="o">=</span> <span class="n">args</span><span class="p">.</span><span class="nf">first</span>
<span class="n">last_spec</span> <span class="o">=</span> <span class="n">impl</span><span class="o">.</span><span class="err">📇</span><span class="p">.</span><span class="nf">map</span> <span class="p">{</span> <span class="o">|</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="o">|</span> <span class="p">[</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="p">.</span><span class="nf">dup</span><span class="p">]</span> <span class="p">}.</span><span class="nf">to_h</span>
<span class="c1"># TODO WARN IF SPEC IS EMPTY</span>
<span class="sx">%i[args result]</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">key</span><span class="o">|</span>
<span class="n">last_spec</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="o"><<</span> <span class="sx">%i[any]</span> <span class="k">if</span> <span class="n">last_spec</span><span class="p">[</span><span class="n">key</span><span class="p">].</span><span class="nf">empty?</span>
<span class="k">end</span>
<span class="n">base</span><span class="p">.</span><span class="nf">instance_variable_get</span><span class="p">(</span><span class="ss">:@annotations</span><span class="p">)</span><span class="o">.</span><span class="err">📝📝</span> <span class="o"><<</span> <span class="n">last_spec</span>
<span class="n">base</span><span class="p">.</span><span class="nf">instance_variable_get</span><span class="p">(</span><span class="ss">:@annotations</span><span class="p">)</span><span class="o">.</span><span class="err">📝</span><span class="p">.</span><span class="nf">replace</span><span class="p">([</span><span class="n">last_spec</span><span class="p">])</span>
<span class="n">impl</span><span class="o">.</span><span class="err">📝📝</span> <span class="o"><<</span> <span class="n">last_spec</span>
<span class="n">impl</span><span class="o">.</span><span class="err">📇</span><span class="p">.</span><span class="nf">each</span> <span class="p">{</span> <span class="o">|</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="o">|</span> <span class="n">v</span><span class="p">.</span><span class="nf">clear</span> <span class="p">}</span>
<span class="p">})</span>
<span class="n">base</span><span class="p">.</span><span class="nf">instance_eval</span> <span class="k">do</span>
<span class="k">def</span> <span class="nf">method_missing</span><span class="p">(</span><span class="nb">name</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">&</span><span class="err">λ</span><span class="p">)</span>
<span class="vi">@annotations</span><span class="p">.</span><span class="nf">__send__</span><span class="p">(:</span><span class="err">🖊️</span><span class="p">,</span> <span class="nb">name</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">&</span><span class="err">λ</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
RFC HTTP API Feedback Proposal2018-10-13T00:00:00+00:00https://rocket-science.ru/fun/2018/10/13/rfc-http-responses<blockquote>
<p>Internet Engineering Task Force (IETF) M. Skin, Ed.
Request for Comments: 13102018 AmotionCity
Obsoletes: — M. Skin, Ed.
Updates: 7231 Adobe
Category: Standards Track October 2018
ISSN: 1973-3009</p>
</blockquote>
<h2 id="http11-addendum-for-api-feedback-codes">HTTP/1.1: Addendum for API Feedback Codes</h2>
<h3 id="abstract">Abstract</h3>
<p>The Hypertext Transfer Protocol API Responses are a part of modern stateless Route3 communication between HTTP server and API consumers for distributed, collaborative, hypertext information systems.</p>
<p>This document defines the semantics of HTTP/1.1 API responses as expressed by response header fields, and response status codes, along with the payload of messages (metadata and body content) and mechanisms for content negotiation.</p>
<h3 id="status-of-this-memo">Status of This Memo</h3>
<p>This is an Internet Standards Track document.</p>
<p>This document is a product of the Internet Engineering Task Force
(IETF). It represents the consensus of the IETF community. It has
received public review and has been approved for publication by the
Internet Engineering Steering Group (IESG). Further information on
Internet Standards is available in Section 2 of RFC 5741.</p>
<p>Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
http://www.rfc-editor.org/info.</p>
<h3 id="proposal-for-new-api-response-codes">Proposal for New API Response Codes</h3>
<p>[…]</p>
<hr />
<h3 id="6xx-consumer-feedback">6xx Consumer feedback</h3>
<p>The consumer succeeded to process a request and wants to share their opinion.</p>
<p>Response feedback codes beginning with the digit <strong>“6”</strong> indicate cases in which the consumer is aware that it has successfully received and processed the API response and feels a necessity to provide the server with the feedback. Except when processing a 5×× response, the consumer should include an entity containing an explanation of the feedback conditions, and indicate whether it is a temporary or permanent impression. Likewise, servers should display any included entity to the development and/or marketing team. These feedback codes are applicable to any API server response.</p>
<h4 id="600-ok"><strong><code class="language-plaintext highlighter-rouge">600 OK</code></strong></h4>
<p>Standard feedback for successful API response. The actual response will depend on the mood of the API at the moment. In a 2×× response, the feedback will contain an impression created during response processing. In a 4××/5×× response, the feedback will contain an entity describing or containing the result of the action, the words of dissatisfaction and optionally the of level of API disappointment.</p>
<h4 id="601-considered"><strong><code class="language-plaintext highlighter-rouge">601 Considered</code></strong></h4>
<p>The response has been accepted for processing, but the processing has not been completed. The response might or might not be eventually acted upon, and may be completely forgotten in a matter of minutes.</p>
<h4 id="602-confused"><strong><code class="language-plaintext highlighter-rouge">602 Confused</code></strong></h4>
<p>The response contained an unexpected content. The response might or might not be eventually liked/disliked, and may be completely unrelated to the actual request API sent.</p>
<h4 id="603-none-of-your-business"><strong><code class="language-plaintext highlighter-rouge">603 None of your business</code></strong></h4>
<p>The response was valid, but the API is refusing to share the impression back to the server. The server might not have the necessary reputation for an API, or may be banned/blocked of some sort.</p>
<h4 id="604-not-sure"><strong><code class="language-plaintext highlighter-rouge">604 Not sure</code></strong></h4>
<p>The response was valid, but the API is hesitating to tell whether is was good or not. The server might re-send the same response later when API will be in the mood.</p>
<h4 id="605-delighted"><strong><code class="language-plaintext highlighter-rouge">605 Delighted</code></strong></h4>
<p>The response was lovely. The API heartfully appreciates the response.</p>
<h4 id="606-not-acceptable"><strong><code class="language-plaintext highlighter-rouge">606 Not acceptable</code></strong></h4>
<p>The response was harmful/abusive. The incident will be reported.</p>
<h4 id="608-retweeted"><strong><code class="language-plaintext highlighter-rouge">608 Retweeted</code></strong></h4>
<p>The response contained a valuable and/or interesting content that was shared by API with all the API subscribers. The response might or might not be eventually considered, but API just felt a necessity to broadcast it to public API channel.</p>
<hr />
<p>Please consider commenting / voting this RFC for the above to happen sooner.</p>
Protocols in Ruby → Allow Implicit Inheritance2018-10-11T00:00:00+00:00https://rocket-science.ru/hacking/2018/10/11/dry-behaviour-better-error-handling-and-implicit-inheritance<p><a href="https://github.com/am-kantox/dry-behaviour"><strong><code class="language-plaintext highlighter-rouge">dry-behaviour</code></strong></a> is a ruby implementation of <a href="http://elixir-lang.org/getting-started/protocols.html"><em>Elixir protocols</em></a>.</p>
<p>For those who sees this concept for the first time, here is a quick example.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nn">ToString</span>
<span class="kp">include</span> <span class="no">Dry</span><span class="o">::</span><span class="no">Protocol</span>
<span class="n">defprotocol</span> <span class="k">do</span>
<span class="n">defmethod</span> <span class="ss">:dump</span><span class="p">,</span> <span class="ss">:this</span>
<span class="n">defimpl</span> <span class="ss">target: </span><span class="no">User</span> <span class="k">do</span>
<span class="k">def</span> <span class="nf">dump</span><span class="p">(</span><span class="n">this</span><span class="p">)</span>
<span class="s2">"</span><span class="si">#{</span><span class="n">this</span><span class="p">.</span><span class="nf">name</span><span class="si">}</span><span class="s2"> <</span><span class="si">#{</span><span class="n">this</span><span class="p">.</span><span class="nf">email</span><span class="si">}</span><span class="s2">>"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="n">defimpl</span> <span class="ss">target: </span><span class="no">Post</span> <span class="k">do</span>
<span class="k">def</span> <span class="nf">dump</span><span class="p">(</span><span class="n">this</span><span class="p">)</span>
<span class="s2">"</span><span class="si">#{</span><span class="n">this</span><span class="p">.</span><span class="nf">title</span><span class="si">}</span><span class="s2"> <</span><span class="si">#{</span><span class="n">this</span><span class="p">.</span><span class="nf">date</span><span class="si">}</span><span class="s2">>"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="no">ToString</span><span class="p">.</span><span class="nf">dump</span><span class="p">(</span><span class="no">User</span><span class="p">.</span><span class="nf">last</span><span class="p">)</span>
<span class="c1">#⇒ Aleksei <am@mudasobwa.ru></span>
<span class="no">ToString</span><span class="p">.</span><span class="nf">dump</span><span class="p">(</span><span class="no">Post</span><span class="p">.</span><span class="nf">last</span><span class="p">)</span>
<span class="c1">#⇒ Allow Implicit Inheritance <2018-10-11></span>
</code></pre></div></div>
<p>Basically, this is another way to implement a polymorhism in ruby, isolated and allowing avoid monkeypatching at all.</p>
<hr />
<p>In the last release we have significantly improved error handling, including, but not limited to:</p>
<ul>
<li>very descriptive, elixirish error messages that include possible causes, suggestions on how to resolve the issue, etc</li>
<li>the whole stacktrace is carefully saved with <code class="language-plaintext highlighter-rouge">cause</code></li>
<li>the error object does now store all the environment where the exception occured</li>
<li>internal exceptions related to wrong implementation do now point to the proper lines in the client code (internal trace lines are removed).</li>
</ul>
<p>Here is the example of the error message:</p>
<p><img src="/img/protocols/error-message.png" alt="New error" /></p>
<hr />
<p><code class="language-plaintext highlighter-rouge">Protocols</code> implementation is very flexible. Besides direct implementation, it allows delegation of methods to the target object). Before this version we allowed the implicit delegation when the method was not implemented:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nn">Namer</span>
<span class="kp">include</span> <span class="no">Dry</span><span class="o">::</span><span class="no">Protocol</span>
<span class="n">defprotocol</span> <span class="k">do</span>
<span class="n">defmethod</span> <span class="ss">:email</span><span class="p">,</span> <span class="ss">:this</span>
<span class="n">defimpl</span> <span class="ss">target: </span><span class="no">User</span> <span class="k">do</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="no">Namer</span><span class="p">.</span><span class="nf">email</span><span class="p">(</span><span class="no">User</span><span class="p">.</span><span class="nf">last</span><span class="p">)</span>
<span class="c1">#⇒ am@mudasobwa.ru</span>
</code></pre></div></div>
<p>There are problems with implicitness. Always. Sooner or later you’ll get into very hard to understand and debug issue, induced by some indirect initialization. Yeah, the warning message about implicit delegation was printed during class loading, but who does read these messages.</p>
<p>In the latest version we’ve deprecated implicit delegation with the intent to remove it completely in <code class="language-plaintext highlighter-rouge">1.0</code> and raise if the implementation is incomplete.</p>
<p><img src="/img/protocols/deprecation-warning.png" alt="Deprecation warning" /></p>
<p>Instead we now allow implicit delegation to <code class="language-plaintext highlighter-rouge">super</code> (parent protocol implementation) provided the protocol definition is allowing this.</p>
<div class="language-diff highlighter-rouge"><div class="highlight"><pre class="highlight"><code> module Namer
include Dry::Protocol
- defprotocol do
<span class="gi">+ defprotocol implicit_inheritance: true do
</span> defmethod :email
def email(this)
this.email
end
defimpl target: User do
<span class="gd">- def email(this)
- super(this)
- end
</span> end
end
end
</code></pre></div></div>
<hr />
<p>Also, the most complete environment is stored within the exception to simplify debug if there are some issues:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Foo</span> <span class="o">=</span> <span class="no">Class</span><span class="p">.</span><span class="nf">new</span>
<span class="n">ex</span> <span class="o">=</span>
<span class="k">begin</span>
<span class="no">Namer</span><span class="p">.</span><span class="nf">email</span><span class="p">(</span><span class="no">Foo</span><span class="p">.</span><span class="nf">new</span><span class="p">,</span> <span class="kp">true</span><span class="p">)</span>
<span class="k">rescue</span> <span class="o">=></span> <span class="n">e</span>
<span class="n">e</span>
<span class="k">end</span>
<span class="c1">#⇒ Protocol “Namer” is not implemented for “Foo”.></span>
<span class="n">ex</span><span class="p">.</span><span class="nf">details</span>
<span class="c1">#⇒ {:method=>:email,</span>
<span class="c1"># :receiver=>#<Foo:0x00560922447308>,</span>
<span class="c1"># :args=>[true],</span>
<span class="c1"># :self=>Namer,</span>
<span class="c1"># :message=>"Protocol “Namer” is not implemented for “Foo”."}</span>
</code></pre></div></div>
<hr />
<p>As one might see, this minor release is mostly focused on polishing error handling, documentation, “explicit-over-implicit” and cosmetics. It brings only one new feature (implicit inheritance inside protocol ancestors tree,) and even that was induced by the necessity to removedeprecate the implicit delegation.</p>
<p>I perfectly understand now why José is constantly repeating “documentaion is the first class citizen.” The code is now way more pleasantly to use. Enjoy!</p>
<hr />
<p>For those who is curious about how the library is implemented under the hood, there is more <a href="http://rocket-science.ru/hacking/2016/12/19/dry-behaviour">technical introduction</a> <em>and</em>, of course, the <a href="https://github.com/am-kantox/dry-behaviour">source is open</a>.</p>
<p>Happy behaving!</p>
Adopting Property Testing in Elixir2018-10-10T00:00:00+00:00https://rocket-science.ru/hacking/2018/10/10/adopting-property-testing-in-elixir<p>When I released the first robust version of <a href="https://hexdocs.pm/iteraptor"><strong><code class="language-plaintext highlighter-rouge">Iteraptor</code></strong></a>, the <em>Elixir</em> library for enumerations on steroids, I announced it on <a href="https://elixirforum.com/t/iteraptor-iterating-nested-terms-like-i-m-five/13536"><em>Elixir Forum</em></a>, asking for how better can I test it. In the very first comment I received the reply that literally changed my understanding of how testing should be performed:</p>
<blockquote>
<p>This kind of functionality seems like the perfect fit for Property-based testing 😃</p>
</blockquote>
<p>I never trusted tests much. It’s time to throw stones at me. I was always able to find better excuses for writing as few tests as possible. I rejected to test <em>third-party functionality</em>, <em>implementation details</em>, <em>internals of core library</em>, etc. Nah, seriously, either one trusts the framework in use, or it’d better by any mean to avoid using it at all.</p>
<p>All these countless questions on SO like <em>“how would I test that my rails controller receives <code class="language-plaintext highlighter-rouge">"foo"</code> when the fronend sends <code class="language-plaintext highlighter-rouge">"foo"</code>”</em> drive me bonkers.—<em>“You don’t damn do it.”</em> DHH already did. Trust him or switch to Wordpress.</p>
<p>Also, this is a human nature to test only what indeed passes. Mostly because if we knew it does not pass, we’d have it covered in the code so that it passes. Don’t tell me about TDD as well, it is truly better than covering the existing code with greenies, but still it is all about what we could imagine out of our head. The typical testsuite to cover the HTTP request would be:</p>
<ul>
<li>correct data → ✓ 200</li>
<li>wrong data → ✗ 404</li>
</ul>
<p>I am mature enough developer to produce a correct <code class="language-plaintext highlighter-rouge">if</code> statement, check the input validness and respond with either 200 or 404. I pretty fine understand also that this example is contrived, I am exaggerating and the real life is more complicated. True that. But we still fail to fortunetell the case where and by what our code is going to be screwed up. In the example above that would be something like � symbol coming from the user input. Malformed UTF8. Connection lost. Rats had the cable eaten. All that crap.</p>
<hr />
<p><strong>We have to test what we cannot predict. Unfortunately, we are not able to predict what exactly we cannot predict.</strong></p>
<p>I am to repeat this again: testing good input, and bad input, and malformed input, and whatever else is an undoubted goodness. Everybody does it, and that’s great. That makes future code changes safer, simplifies the life and all that. That is not enough, though. Because see above.</p>
<hr />
<p>Turning back to my library, it allows to <code class="language-plaintext highlighter-rouge">iterate</code>/<code class="language-plaintext highlighter-rouge">map</code>/<code class="language-plaintext highlighter-rouge">reduce</code>/<code class="language-plaintext highlighter-rouge">filter</code> deeply nested structures in <em>Elixir</em>, like maps, lists, keywords. I could come up with 50 examples or cumbersome deeply nested terms, no issue. I had exactly zero confidence that I have all the cases covered. And here property testing comes to the scene.</p>
<p>I have received this aforementioned advice to try property testing in the beginning of April. Two weeks later I attended <a href="http://www.elixirconf.eu/">ElixirConfEU in Warsaw</a> and listened to the <a href="https://www.youtube.com/watch?v=p84DMv8TQuo">brilliant talk by Andrea Leopardi</a> who is basically the core team member who <a href="https://andrealeopardi.com/posts/the-guts-of-a-property-testing-library/">had it implemented for <em>Elixir</em></a>. It was a life changer.</p>
<p>Back in Barcelona I spent several hours implementing the <a href="https://github.com/am-kantox/elixir-iteraptor/blob/v1.2.1/test/property/iteraptor_test.exs">property testing for <code class="language-plaintext highlighter-rouge">Iteraptor</code></a>. The whole file contains 75 LOCs and assures me my library works for many randomly generated nested structures of the max depth 25 (I performed the over-night run once I have it written—1K times with the depth 200.) Now I am 99% positive the declared functionality is robust.</p>
<hr />
<p>It is indeed very easy to use. I will just drop a couple of examples here to show the general approach, for details please refer to Andrea’s post linked above.</p>
<p>First of all, you need to declare your possible data variants. I am iterating nested enumerables, hence I’ve had there</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="no">StreamData</span>
<span class="k">defmacrop</span> <span class="n">aib</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="kn">quote</span> <span class="k">do</span><span class="p">:</span>
<span class="n">one_of</span><span class="p">([</span><span class="n">atom</span><span class="p">(</span><span class="ss">:alphanumeric</span><span class="p">),</span> <span class="n">integer</span><span class="p">(),</span> <span class="n">binary</span><span class="p">()])</span>
<span class="k">defmacrop</span> <span class="n">leaf_list</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="kn">quote</span> <span class="k">do</span><span class="p">:</span> <span class="n">list_of</span><span class="p">(</span><span class="n">aib</span><span class="p">())</span>
<span class="c1"># ...other enumerables...</span>
<span class="k">defmacrop</span> <span class="n">leaf</span><span class="p">,</span>
<span class="k">do</span><span class="p">:</span> <span class="kn">quote</span> <span class="k">do</span><span class="p">:</span> <span class="n">one_of</span><span class="p">([</span><span class="n">leaf_list</span><span class="p">(),</span> <span class="o">...</span><span class="p">])</span>
<span class="k">defmacrop</span> <span class="n">non_leaf_list</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="kn">quote</span> <span class="k">do</span><span class="p">:</span> <span class="n">list_of</span><span class="p">(</span><span class="n">leaf</span><span class="p">())</span>
<span class="c1"># ...other enumerables...</span>
<span class="k">defmacrop</span> <span class="n">non_leaf</span><span class="p">,</span>
<span class="k">do</span><span class="p">:</span> <span class="kn">quote</span> <span class="k">do</span><span class="p">:</span> <span class="n">one_of</span><span class="p">([</span><span class="n">non_leaf_list</span><span class="p">(),</span> <span class="o">...</span><span class="p">])</span>
<span class="k">defmacrop</span> <span class="n">maybe_leaf_list</span><span class="p">,</span>
<span class="k">do</span><span class="p">:</span> <span class="kn">quote</span> <span class="k">do</span><span class="p">:</span> <span class="n">list_of</span><span class="p">(</span><span class="n">one_of</span><span class="p">([</span><span class="n">leaf</span><span class="p">(),</span> <span class="n">non_leaf</span><span class="p">()]))</span>
</code></pre></div></div>
<p>That’s it. Now one simply needs to declare the expectations for any technically random data</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">check</span> <span class="n">all</span> <span class="n">term</span> <span class="o"><-</span> <span class="n">maybe_leaf</span><span class="p">(),</span> <span class="ss">max_runs:</span> <span class="mi">25</span> <span class="k">do</span>
<span class="n">reduced</span> <span class="o">=</span>
<span class="no">Iteraptor</span><span class="o">.</span><span class="n">reduce</span><span class="p">(</span>
<span class="n">term</span><span class="p">,</span> <span class="p">[],</span> <span class="k">fn</span> <span class="n">_</span><span class="p">,</span> <span class="n">acc</span> <span class="o">-></span> <span class="p">[</span><span class="s2">"."</span> <span class="o">|</span> <span class="n">acc</span><span class="p">]</span> <span class="k">end</span>
<span class="p">)</span>
<span class="n">mapped</span> <span class="o">=</span>
<span class="n">term</span>
<span class="o">|></span> <span class="no">Iteraptor</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="k">fn</span> <span class="n">_</span> <span class="o">-></span> <span class="s2">"."</span> <span class="k">end</span><span class="p">)</span>
<span class="o">|></span> <span class="no">Iteraptor</span><span class="o">.</span><span class="n">to_flatmap</span><span class="p">()</span>
<span class="o">|></span> <span class="no">Map</span><span class="o">.</span><span class="n">values</span><span class="p">()</span>
<span class="n">assert</span> <span class="n">reduced</span> <span class="o">==</span> <span class="n">mapped</span>
<span class="k">end</span>
</code></pre></div></div>
<p>That’s it.</p>
Banned for not being flattering2018-09-13T00:00:00+00:00https://rocket-science.ru/culture/2018/09/13/banned-from-dev-to<h2 id="today-my-post-was-banned-from-devto">Today my post was banned from <a href="https://dev.to">dev.to</a></h2>
<p><img src="/img/dev.to/banned_post.png" alt="How To Pass Any Tech Interview" /></p>
<p>This one. During 8 hours of it’s availability it received 32 positive responses. It was tweeted several times by strangers. And then all of a sudden I have received this email:</p>
<p><img src="/img/dev.to/notification.png" alt="“we received a number of (valid) complaints that the article was clickbait-y and didn't establish a productive launching point for a true conversation”" /></p>
<p>I have exactly zero comments. This is a perfect example of how people tend to treat diversity (unless there is a difference in points of view,) how snitching is more powerful than producing a content, and how the crowd would deal with anybody who does not babble the words of admiration, delight, and rapture to the community members.</p>
<p>This is what people call “healthy environment” today. Because fuck me.</p>
<p>I am fine with that. Are you?</p>
Hype Demythified or Pizza With Pineapple Topping2018-09-10T00:00:00+00:00https://rocket-science.ru/hacking/2018/09/10/pizza-with-ananas-topping<h2 id="the-problem">The Problem</h2>
<p>This post is made mostly as a response to <a href="https://elixirforum.com/t/learning-elixir-frst-impressions-plz-dont-kill-me">Learning Elixir, Frst Impressions</a> posted yesterday at <a href="https://elixirforum.com">Elixir Forum</a>. The real matter I have finally come to writing this would be I feel a necessity to sum up what I constantly tell to my colleagues, mates and wife. Here it is:</p>
<blockquote>
<h4 id="thou-shalt-not-make-into-thee-any-graven-image-do-not-worship-anything-or-anybody-blindly">thou shalt not make into thee any graven image; do not worship anything or anybody blindly</h4>
</blockquote>
<p>This writing would consist of a number of modern hype topics demythified (spelling intended.)</p>
<h2 id="functional-programming-is-the-path">Functional Programming Is The Path</h2>
<p>If I were to publish this 10 years ago, the “OOP Is A Way To Go” would come first. Even that makes me think that taken as a doctrine it’s a bullshit. Functional programming is not better than procedural, object oriented, or even the one using assembly language and machine codes. The correct answer is “it depends.” Whether one is constantly producing new CRUD endpoints for the long-lived, stored in the database, tightly bound and twisted, objects—sooner or later they have Rails invented. Which is—I insist—way better for CRUDing basecamp entities than any fancy modern approach. That is the perfect tool to solve this particular task.</p>
<p>Not to solve <em>all the tasks</em>. Not to solve <em>your current task</em>, unless it’s similar to what DHH is solving for decades.</p>
<p>The same way being pure functional is brilliant when you are doing an academical talk entitled <em>Adjusting Models With Category Theory In New Generation Language Paradigm</em>, but when you are being paid for less sexy stuff, like doing what’s called Data Science in a startup, you’d better pick up <code class="language-plaintext highlighter-rouge">R</code> and/or <code class="language-plaintext highlighter-rouge">Python</code>. No matter how <em>pure</em> and <em>elegant</em> they are (they are not at all.)</p>
<p>Being functional might help, it might injure, it might even hurt. The developer might like it, might hate it, might use it, might be like “wut the heck is functional.” There is no goodness in function purity. There is no goodness in currying as is. I hear your objections, and here goes my answer: you are wrong. Data-first or data-last, currying or piping, purity or side effects—all that crap does not matter much. There are tons of different problems when strong typing is obstructing and impeding. There are great developers who never need currying (thank God.) Nobody should ever care whether data-is-last or data-is-first. Even data in the middle is fine, if it works.</p>
<p>And here we smoothly pass to the second myth.</p>
<h2 id="oop-is-a-way-to-go">OOP Is A Way To Go</h2>
<p>Nah. The father of OOP, <a href="https://en.wikipedia.org/wiki/Alan_Kay">Alan Kay</a>, the inventor of <em>classes</em> and <em>objects</em> said in the mid of 1990s:</p>
<blockquote>
<p>I’m sorry that I long ago coined the term “objects” for this topic because it gets many people to focus on the lesser idea. The big idea is “messaging.”</p>
</blockquote>
<p>That does not mean <em>objects</em> are not a good concept. Mutable objects. I said that. You heard me. Mutable objects are great, they are a better, cleaner and easier-to-get-to concept, than pure immutable data. After all, when we break the glass of our office fire siren alarm (trust me, don’t check it yourself, please)—we end up with a harrowing siren alarm <em>and</em> a broken glass, not with a brand new broken glass <em>and</em> an original one, untouched and intact. Objects are fine.</p>
<p>Everybody who had a chance to develop with <a href="https://en.wikipedia.org/wiki/Apache_Hadoop">Apache Haddop</a> would tell you: this is a perfect example of extremely well put paradigm. Mappers and even reducers there are great because they are using unceasingly mutated objects. That’s more straightforward and less memory-consuming, and all that stuff.</p>
<p>Being object-oriented might help, but it might injure, it might even hurt. One should check what the real consequences of using an OOP paradigm are before going fully OOP. And even inside OOP paradigm, there are still many places where functional approaches work very well.</p>
<h2 id="immutability-is-a-silver-bullet">Immutability Is A Silver Bullet</h2>
<p>Bullshit. There are tons of applications where mutability is a pure virtue, besides aforementioned basecamps and hadoops. For instance, any task that involves a conveyor and some state-dependent local storage would make you life a nightmare if you cannot mutate the state.</p>
<h2 id="strong-typing-is-a-panacea">Strong Typing Is A Panacea</h2>
<p>There are fashionable rumors all around pushing that strong typing drastically decreases the number of mistakes, allows to catch any error on compilation stage and even obsoletes testing. In academic test stands maybe. In fancy slides. Not in the real life.</p>
<p>Because real life is full of side effects. One gets data from third parties, from the internet, from the user, from the universe. And the universe (I am not talking about the users) is not strongly typed.</p>
<p>That does not mean I deny the profit from applying the category theory to computer science. Not at all. There are areas where it works pretty well. Unfortunately, there are areas where it does not. And being 100% strong types hinders there.</p>
<h2 id="monads-are-a-godsend">Monads Are A Godsend</h2>
<p>They are not. Sometimes they work. Sometimes they just bring a boilerplate of the inconceivable size for nothing. If the control flow is best described as a chain, each step might succeed or fail and we are to stop in the case of fail and immediately return the result—monads are great. But hey, how many processes in our lives are as straightforward? Even bringing <code class="language-plaintext highlighter-rouge">tee</code> monad makes the code piece an unreadable crap.</p>
<h2 id="the-summing-up">The Summing Up</h2>
<p>There is no pill. The salad is almost always more tasty than a cucumber, a tomato, or an olive oil. Pizzas are usually made from many ingredients. Try to order a pizza with salami. Literally: a piece of dough covered with cropped salami; roommates would consider you a nerd.</p>
<p>Don’t hesitate to pick the most applicable patterns from different paradigms and mix them to achieve the best result for this particular task. There is nor a prophet neither a fortuneteller who knows what’s best for <em>you</em> and <em>your current problem</em>. And always ask for a weird topping, like pineapple or something. Here is the key to both most satisfactory and most profitable development process:</p>
<h2 id="the-gold-ratio">The Gold Ratio</h2>
<p>There were many languages invented because of the real need, with business goals in mind. Those are (including but not limited to):</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">c</code> to make an assembly language a bit more humane (and <code class="language-plaintext highlighter-rouge">rust</code> now as a great disciple)</li>
<li><code class="language-plaintext highlighter-rouge">erlang</code> to make a telecom work with millions of connections simultaneously</li>
<li><code class="language-plaintext highlighter-rouge">php</code> to make static webpages dynamic for no cost</li>
<li><code class="language-plaintext highlighter-rouge">javascript</code> to make drop-down menus and poison the <code class="language-plaintext highlighter-rouge">marquee</code> tag</li>
<li><code class="language-plaintext highlighter-rouge">haskell</code> to prove some mathematical concepts and make a perfect ideal academic language in a vacuum to teach students</li>
<li><code class="language-plaintext highlighter-rouge">java</code> to support enterprise-level scalability</li>
<li><code class="language-plaintext highlighter-rouge">ruby</code> to make programmers happy (according to Matz, and I agree)</li>
<li><code class="language-plaintext highlighter-rouge">go</code> to drastically decrease an entry level and learning curve to make it possible to hire teenagers</li>
</ul>
<p>All of them are great in their niches. One might even write pure immutable fully-functional code with ruby, or spawn a million of parallel tasks with javascript. The thing is one should not abuse languages for what they are not good enough. Don’t be a slave of technology, hype, fashion, public opinion, authorities.</p>
<p>Use the best from each world. Use strong typing when it applies and forget about it where it does not. Use currying if the language treats it as a first class citizen and avoid building a weird surrogates to support currying in the languages that are built with another control flow in mind. Benefit from <code class="language-plaintext highlighter-rouge">while(*dest++ = *src++);</code> implementation of <code class="language-plaintext highlighter-rouge">strcpy</code> where it makes sense.</p>
<p>I could have been providing examples for another hundred of pages, but enough is enough. There is no one single paradigm as by now that works well under any circumstances. Let me repeat: <strong>don’t be a slave of technology, hype, fashion, public opinion, authorities</strong>. Pick up the set of patterns accordingly to the problem, not to what you are most familiar with, or love more, or were told is the best thing ever.</p>
Parent Of My Child Is Not Exactly Me2018-08-27T00:00:00+00:00https://rocket-science.ru/hacking/2018/08/27/parent-of-my-child-is-not-exactly-me<h2 id="the-problem">The Problem</h2>
<blockquote>
<p>A man and his son are driving in a car one day, when they get into a fatal accident. The man is killed instantly. The boy is knocked unconscious, but he is still alive. He is rushed to hospital, and will need immediate surgery. The doctor enters the emergency room, looks at the boy, and says <em>“I can’t operate on this boy, he is my son.”</em>
How is this possible? The answer is simple: the doctor is the boy’s mother.</p>
</blockquote>
<p>Sooner or later in each complex application involving data manipulation backed up by the database arises a necessity to have a parent-child relationship inside the same table/model. The easiest example that comes into my mind would be several related blog posts, maintaining the long story chapters. The sequence of these posts would be a whole topic but since people rare like to read more than 280 symbols at once nowadays we tend to split long writings into parts.</p>
<h2 id="the-wrong-approach">The wrong approach</h2>
<p>These posts basically maintain a linked list. The naïve database approach to store the relation between neighbour chapters would be to add a reference field to the table/model.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Post</span> <span class="o"><</span> <span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span>
<span class="n">has_one</span> <span class="ss">:next</span><span class="p">,</span> <span class="ss">class: </span><span class="s1">'Post'</span><span class="p">,</span> <span class="ss">foreign_key: :next_post_id</span>
<span class="n">belongs_to</span> <span class="ss">:previous</span><span class="p">,</span> <span class="ss">class: </span><span class="s1">'Post'</span><span class="p">,</span> <span class="ss">inverse_of: :next</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Here is my strong advice: <strong>do not do that!</strong> Never ever.</p>
<h2 id="the-better-approach">The better approach</h2>
<p>Create a new join table <code class="language-plaintext highlighter-rouge">posts_relationships</code> and use it to link the sibling posts chains. It sounds like overcomplicating things, but in practice it is not. It will pay back soon, helping to avoid a nighmare debugging session full of <em>whys</em> and <em>wtfs</em> while the deleted objects will keep resurrecting out of the ashes.</p>
<p>Let’s assume we use the straight approach and we are adding new options for the <em>children</em> objects. Like a check for whether the parent was updated, or whatever. We do:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Post</span> <span class="o"><</span> <span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span>
<span class="o">...</span>
<span class="k">def</span> <span class="nf">check_parent</span>
<span class="n">flash</span><span class="p">(</span><span class="s2">"Parent updated!"</span><span class="p">)</span> <span class="k">if</span> <span class="n">previous</span><span class="p">.</span><span class="nf">updated_at</span> <span class="o">></span> <span class="n">updated_at</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The example is contrived, I do omit all the guards and checks. So far so good. Now we want to implement a functionality to unlink parent if needed. We do:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Post</span> <span class="o"><</span> <span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span>
<span class="k">def</span> <span class="nf">unlink_parent</span>
<span class="n">update_attributes!</span> <span class="ss">previous: </span><span class="kp">nil</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>And here is when monsters come. If any other process had the instance of <em>parent</em> loaded at the moment, any call to <code class="language-plaintext highlighter-rouge">save</code> it would <em>restore</em> the link between objects.</p>
<p>One cannot rely on optimistic locking here, because the <em>parent</em> object is not stale by any mean.</p>
<p>One cannot rely on <code class="language-plaintext highlighter-rouge">reload</code> inside the parent code, because <code class="language-plaintext highlighter-rouge">reload</code> will trash all the <em>wanted</em> scheduled changes to the <em>parent</em> out.</p>
<p>One cannot rely on anything, save for an ugly explicit check of <code class="language-plaintext highlighter-rouge">next.reload.previous.nil?</code>.</p>
<p>With an intermediate relationships table, the issue goes away for free. The reference between objects is <em>deleted</em> from this table when <code class="language-plaintext highlighter-rouge">parent.save</code> is called, and the latter will fail immediately, violating referential integrity.</p>
<p>That simple. Do not be lazy to implement the relationships table. Or, better, do not use mutable variables to store objects in general and <code class="language-plaintext highlighter-rouge">ActiveRecord</code> in particular.</p>
Tarearbol now allows subscriptions to task results2018-08-03T00:00:00+00:00https://rocket-science.ru/hacking/2018/08/03/tarearbol-now-allows-subscriptions<h3 id="intro">Intro</h3>
<p><a href="https://hexdocs.pm/tarearbol/intro.html"><strong><code class="language-plaintext highlighter-rouge">Tarearbol</code></strong></a> is a lightweight task manager, allowing retries, callbacks, assurance that the task succeeded, and more. It starts and manages the task supervision tree for granted. With <code class="language-plaintext highlighter-rouge">Tarearbol</code> one might ensure the task, that might occasionally fail due to reasons beyond caller’s control, will be retried until succeeded. It’s highly configurable and allows callbacks on:</p>
<ul>
<li>task successful completion,</li>
<li>task retries,</li>
<li>task fail (when the max limit of retries is reached.)</li>
</ul>
<p>It also provides a functionality to manage many tasks at once with <a href="https://hexdocs.pm/tarearbol/Tarearbol.html#ensure_all/2"><code class="language-plaintext highlighter-rouge">Tarearbol.ensure_all/2</code></a> and <a href="https://hexdocs.pm/tarearbol/Tarearbol.html#ensure_all_streamed/2"><code class="language-plaintext highlighter-rouge">Tarearbol.ensure_all_streamed/2</code></a>, run tasks with a delay using
<a href="https://hexdocs.pm/tarearbol/Tarearbol.html#run_at/3"><code class="language-plaintext highlighter-rouge">Tarearbol.run_at/3</code></a> and <a href="https://hexdocs.pm/tarearbol/Tarearbol.html#run_in/3"><code class="language-plaintext highlighter-rouge">Tarearbol.run_in/3</code></a>, and drain the scheduled tasks queue with <a href="https://hexdocs.pm/tarearbol/Tarearbol.html#drain/1"><code class="language-plaintext highlighter-rouge">Tarearbol.drain/1</code></a>.</p>
<p>Starting with <a href="https://github.com/am-kantox/tarearbol/releases/tag/v0.8.1">version 0.8.1</a> it married <a href="https://hexdocs.pm/envio"><code class="language-plaintext highlighter-rouge">Envío</code></a> to make it drastically easy to subscribe to task processing callbacks.</p>
<h3 id="tarearbol-common-syntax"><code class="language-plaintext highlighter-rouge">Tarearbol</code> common syntax</h3>
<p>The most widely used function of <code class="language-plaintext highlighter-rouge">Tarearbol</code> would be probably [<code class="language-plaintext highlighter-rouge">ensure/2</code>], accepting a job either in a form of anonymous function or as a MFA-tuple, and a set of options. Options are:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">attempts</code> <em>[default: :infinity]</em> number of attemtps before fail, the integer value</li>
<li><code class="language-plaintext highlighter-rouge">delay</code> <em>[default: 1 msec]</em> number of milliseconds between attempts, <code class="language-plaintext highlighter-rouge">1_000</code> or <code class="language-plaintext highlighter-rouge">1.0</code> for one second, <code class="language-plaintext highlighter-rouge">:timeout</code> for five seconds, etc</li>
<li><code class="language-plaintext highlighter-rouge">timeout</code> <em>[default: 5_000]</em> the timeout for the underlying task</li>
<li><code class="language-plaintext highlighter-rouge">raise</code> <em>[default: false]</em> whether <code class="language-plaintext highlighter-rouge">ensure/2</code> should <code class="language-plaintext highlighter-rouge">raise</code> on fail</li>
<li><code class="language-plaintext highlighter-rouge">accept_not_ok</code> <em>[default: true]</em> when <code class="language-plaintext highlighter-rouge">false</code>, only the <code class="language-plaintext highlighter-rouge">:ok</code> atom or <code class="language-plaintext highlighter-rouge">{:ok, _}</code>
tuple considered to be valid returned values from the task</li>
<li><code class="language-plaintext highlighter-rouge">on_success</code> <em>[default: nil]</em> the function to be called on successful execution (<code class="language-plaintext highlighter-rouge">arity</code> ∈ <code class="language-plaintext highlighter-rouge">[0, 1]</code> or tuple <code class="language-plaintext highlighter-rouge">{Mod, fun}</code> where fun is of arity zero or one,) for the 1-arity, the result of task execution is passed</li>
<li><code class="language-plaintext highlighter-rouge">on_retry</code> <em>[default: nil]</em> same as above, called <em>on retries</em> after unsuccessful attempts or one of <code class="language-plaintext highlighter-rouge">[:debug, :info, :warn, :error]</code> atoms to log a retry with default logger</li>
<li><code class="language-plaintext highlighter-rouge">on_fail</code> <em>[default: nil]</em> same as above, called when the task finally failed after attempts amount of unsuccessful attempts</li>
</ul>
<p>The usual invocation might look like:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">case</span> <span class="no">Tarearbol</span><span class="o">.</span><span class="n">ensure</span><span class="p">(</span><span class="k">fn</span> <span class="o">-></span> <span class="k">raise</span> <span class="s2">"¡?"</span> <span class="k">end</span><span class="p">,</span> <span class="ss">attempts:</span> <span class="mi">1</span><span class="p">,</span> <span class="k">raise</span><span class="p">:</span> <span class="no">false</span><span class="p">)</span> <span class="k">do</span>
<span class="p">{</span><span class="ss">:error</span><span class="p">,</span> <span class="p">%{</span><span class="ss">job:</span> <span class="n">_job</span><span class="p">,</span> <span class="ss">outcome:</span> <span class="n">outcome</span><span class="p">}}</span> <span class="o">-></span>
<span class="no">Logger</span><span class="o">.</span><span class="n">warn</span><span class="p">(</span><span class="s2">"Error. Returned outcome: </span><span class="si">#{</span><span class="n">inspect</span> <span class="n">outcome</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">result</span><span class="p">}</span> <span class="o">-></span>
<span class="n">result</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The above always returns an error (because the wrapped function does always raise)
and logs the exception.</p>
<h3 id="pub_sub-instead-of-logging">Pub_Sub instead of logging</h3>
<p>Logging is not always handy when it comes to the real life. One might want to handle the task results from some other process, sending emails on fails, or retrying in a smarter way, or whatever. For that purpose all three callbacks are now publishing a message to subscribers using <code class="language-plaintext highlighter-rouge">Registry</code> through it’s handy wrapper <code class="language-plaintext highlighter-rouge">Envío</code> (effective <code class="language-plaintext highlighter-rouge">@since 0.8.1</code>.)</p>
<p>Channels are:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">{Tarearbol.Publisher, :error}</code> to subscribe to task failures only,</li>
<li><code class="language-plaintext highlighter-rouge">{Tarearbol.Publisher, :warn}</code> to subscribe to task retries only,</li>
<li><code class="language-plaintext highlighter-rouge">{Tarearbol.Publisher, :info}</code> to subscribe to task successful executions only,</li>
<li><code class="language-plaintext highlighter-rouge">{Tarearbol.Publisher, :all}</code> to subscribe to everything above.</li>
</ul>
<p>Now the application using <code class="language-plaintext highlighter-rouge">Tarearbol</code> might simply add a subscriber into their supervision tree:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">MyApp</span><span class="o">.</span><span class="no">Subscriber</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">Envio</span><span class="o">.</span><span class="no">Subscriber</span><span class="p">,</span> <span class="ss">channels:</span> <span class="p">[{</span><span class="no">Tarearbol</span><span class="o">.</span><span class="no">Publisher</span><span class="p">,</span> <span class="ss">:error</span><span class="p">}]</span>
<span class="k">def</span> <span class="n">handle_envio</span><span class="p">(</span><span class="n">message</span><span class="p">,</span> <span class="n">state</span><span class="p">)</span> <span class="k">do</span>
<span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="p">(</span><span class="n">message</span><span class="p">,</span> <span class="ss">label:</span> <span class="s2">"Task failed"</span><span class="p">)</span>
<span class="p">{</span><span class="ss">:noreply</span><span class="p">,</span> <span class="n">state</span><span class="p">}</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>From now on <em>all</em> task failures will be handled by the module above.</p>
<p>Happy subscribing!</p>
Envío as a reincarnation of GenEvent²2018-07-30T00:00:00+00:00https://rocket-science.ru/hacking/2018/07/30/envio-as-genevent-2-0<p><a href="https://hexdocs.pm/envio"><strong><code class="language-plaintext highlighter-rouge">Envío</code></strong></a> is kinda <code class="language-plaintext highlighter-rouge">GenEvent²</code>, the modern
idiomatic pub-sub implementation of event passing.</p>
<p>In version <code class="language-plaintext highlighter-rouge">1.4</code>, Elixir core team deprecated
<a href="https://hexdocs.pm/elixir/master/GenEvent.html"><code class="language-plaintext highlighter-rouge">GenEvent</code></a> and introduced
<a href="https://hexdocs.pm/elixir/master/Registry.html"><code class="language-plaintext highlighter-rouge">Registry</code></a>. The reason behind
that is <code class="language-plaintext highlighter-rouge">GenEvent</code> is unable to exploit concurrency out of the box and
processes multiple handlers serially. This was done on purpose because
<code class="language-plaintext highlighter-rouge">GenEvent</code> was mainly used for logging stuff and one probably wants the
log entries to appear in order as they were emitted. This is not the case for
most event applications, though.</p>
<p>While <a href="http://blog.plataformatec.com.br/2016/11/replacing-genevent-by-a-supervisor-genserver/">José Valim suggests</a>
using a simple approach involving a <code class="language-plaintext highlighter-rouge">Supervisor</code> with many instances of
<code class="language-plaintext highlighter-rouge">GenServer</code> acting as subscribers (this is how it was done in <a href="https://github.com/elixir-lang/elixir/commit/e7f2d14036121916fffafda4be9a2137ee39b5bd"><code class="language-plaintext highlighter-rouge">ExUnit</code></a>,)
this way requires a load of redundant boilerplate and it is not as transparent
for the developer as it could be.</p>
<p>That all forced me to introduce <a href="https://hexdocs.pm/envio"><strong><code class="language-plaintext highlighter-rouge">Envío</code></strong></a> package.
It is a drop-in replacement for <code class="language-plaintext highlighter-rouge">GenEvent</code>‘like functionality, built on top
of <a href="https://hexdocs.pm/elixir/master/Registry.html"><code class="language-plaintext highlighter-rouge">Registry</code></a> and supporting
both synchronous and asychronous subscriptions (called <code class="language-plaintext highlighter-rouge">:dispatch</code> and <code class="language-plaintext highlighter-rouge">pub_sub</code>
respectively, with names gracefully stolen from <code class="language-plaintext highlighter-rouge">Registry</code> vocabulary.)</p>
<p>With this in mind, the main goal was to drastically simplify creation of both
publishers and subscribers, hide the boilerplate and make things work out of the
box with a minimal amount of additional code required.</p>
<p><strong><code class="language-plaintext highlighter-rouge">Envío</code></strong> also manages it’s own registry, <code class="language-plaintext highlighter-rouge">Envio.Registry</code> to deliberately
relieve the amount of code you need to create as a boilerplate.</p>
<h3 id="application-example">Application example</h3>
<p>One of most typical examples of how <strong><code class="language-plaintext highlighter-rouge">Envío</code></strong> is supposed to be used, would be
asynchronous publishing notifications to <code class="language-plaintext highlighter-rouge">Slack</code>. Here is the <em>whole</em> code
needed to provide this functionality in your application:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># lib/slack_publisher.ex</span>
<span class="k">defmodule</span> <span class="no">SlackPublisher</span> <span class="k">do</span>
<span class="c1"># use SlackPublisher.broadcast(term()) anywhere to publish</span>
<span class="kn">use</span> <span class="no">Envio</span><span class="o">.</span><span class="no">Publisher</span><span class="p">,</span> <span class="ss">channel:</span> <span class="ss">:main</span>
<span class="k">end</span>
<span class="c1"># config/prod.exs</span>
<span class="n">config</span> <span class="ss">:envio</span><span class="p">,</span> <span class="ss">:backends</span><span class="p">,</span> <span class="p">%{</span>
<span class="no">Envio</span><span class="o">.</span><span class="no">Slack</span> <span class="o">=></span> <span class="p">%{</span>
<span class="p">{</span><span class="no">SlackPublisher</span><span class="p">,</span> <span class="ss">:main</span><span class="p">}</span> <span class="o">=></span> <span class="p">[</span>
<span class="ss">hook_url:</span> <span class="p">{</span><span class="ss">:system</span><span class="p">,</span> <span class="s2">"SLACK_ENVIO_HOOK_URL"</span><span class="p">}</span>
<span class="p">]</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>That’s it. Once environment variable <code class="language-plaintext highlighter-rouge">SLACK_ENVIO_HOOK_URL</code> is setup to point
to any <a href="https://api.slack.com/slack-apps">Slack application</a> enabled for your
team, you are all set. Do anywhere in your code:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">SlackPublisher</span><span class="o">.</span><span class="n">broadcast</span><span class="p">(%{</span>
<span class="ss">title:</span> <span class="s2">"Foo changed"</span><span class="p">,</span>
<span class="ss">level:</span> <span class="ss">:info</span><span class="p">,</span>
<span class="ss">foo:</span> <span class="p">%{</span><span class="ss">bar:</span> <span class="p">[</span><span class="ss">baz:</span> <span class="p">[</span><span class="mi">42</span><span class="p">,</span> <span class="mf">3.1415</span><span class="p">]]}})</span>
</code></pre></div></div>
<p>And receive a notification in your slack channel.</p>
<h3 id="complicated-workflows">Complicated workflows</h3>
<p><strong><code class="language-plaintext highlighter-rouge">Envío</code></strong> simplifies the creation of both publishers and subscribers
hiding all the boilerplate behind <code class="language-plaintext highlighter-rouge">use Envio.Publisher</code> and
<code class="language-plaintext highlighter-rouge">use Envio.Subscriber</code>. The syntax of <a href="https://hexdocs.pm/envio/Envio.Publisher.html#content"><code class="language-plaintext highlighter-rouge">Envio.Publisher</code></a> is shown above in the
<code class="language-plaintext highlighter-rouge">Slack</code> example; for subscriber it’s nearly the same:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">MySubscriber</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">Envio</span><span class="o">.</span><span class="no">Subscriber</span><span class="p">,</span> <span class="ss">channels:</span> <span class="p">[{</span><span class="no">MyPublisher</span><span class="p">,</span> <span class="ss">:featured</span><span class="p">}]</span>
<span class="k">def</span> <span class="n">handle_envio</span><span class="p">(</span><span class="n">message</span><span class="p">,</span> <span class="n">state</span><span class="p">)</span> <span class="k">do</span>
<span class="p">{</span><span class="ss">:noreply</span><span class="p">,</span> <span class="n">state</span><span class="p">}</span> <span class="o">=</span> <span class="kn">super</span><span class="p">(</span><span class="n">message</span><span class="p">,</span> <span class="n">state</span><span class="p">)</span>
<span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="p">({</span><span class="n">message</span><span class="p">,</span> <span class="n">state</span><span class="p">},</span> <span class="ss">label:</span> <span class="s2">"Received"</span><span class="p">)</span>
<span class="p">{</span><span class="ss">:noreply</span><span class="p">,</span> <span class="n">state</span><span class="p">}</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p><a href="https://hexdocs.pm/envio/Envio.Subscriber.html#c:handle_envio/2"><code class="language-plaintext highlighter-rouge">handle_envio/2</code></a>
is the only callback defined for this behaviour. Implement it to whatever
you need and all the messages sent to <code class="language-plaintext highlighter-rouge">:featured</code> channel by <code class="language-plaintext highlighter-rouge">MyPublisher</code>
will now be handled.</p>
<p>For <code class="language-plaintext highlighter-rouge">:dispatch</code> type of the subscription, it’s even easier: just put</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Envio</span><span class="o">.</span><span class="n">register</span><span class="p">(</span>
<span class="p">{</span><span class="no">MySubscriber</span><span class="p">,</span> <span class="ss">:on_envio</span><span class="p">},</span> <span class="c1"># the function of arity 2 must exist</span>
<span class="ss">dispatch:</span> <span class="p">%</span><span class="no">Envio</span><span class="o">.</span><span class="no">Channel</span><span class="p">{</span><span class="ss">source:</span> <span class="no">Publisher</span><span class="p">,</span> <span class="ss">name:</span> <span class="ss">:featured</span><span class="p">}</span>
<span class="p">)</span>
</code></pre></div></div>
<p>anywhere in your code and start receiving messages synchronously.</p>
<h3 id="backends">Backends</h3>
<p><strong><code class="language-plaintext highlighter-rouge">Envío</code></strong> introduces a concept of backends. <code class="language-plaintext highlighter-rouge">Slack</code> mentioned above would be
one of them, shipped with a package itself. Backends are supposed to implement
<a href="https://hexdocs.pm/envio/Envio.Backend.html#content"><code class="language-plaintext highlighter-rouge">Envio.Backend</code></a> behaviour
currently consisting of the only function <code class="language-plaintext highlighter-rouge">on_envio/2</code>. The backend, once
specified in the configuration file, will be automatically plugged into the
internally managed <code class="language-plaintext highlighter-rouge">Registry</code> and subscribed to the channel set in config.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">config</span> <span class="ss">:envio</span><span class="p">,</span> <span class="ss">:backends</span><span class="p">,</span> <span class="p">%{</span>
<span class="no">Envio</span><span class="o">.</span><span class="no">MyBackend</span> <span class="o">=></span> <span class="p">%{</span>
<span class="p">{</span><span class="no">MyPublisher</span><span class="p">,</span> <span class="ss">:featured</span><span class="p">}</span> <span class="o">=></span> <span class="p">[</span>
<span class="ss">callback_url:</span> <span class="p">{</span><span class="ss">:system</span><span class="p">,</span> <span class="s2">"SLACK_ENVIO_HOOK_URL"</span><span class="p">},</span>
<span class="ss">callback_headers:</span> <span class="p">%{</span><span class="s2">"X-Envio-From"</span> <span class="o">=></span> <span class="s2">"MyPublisher"</span><span class="p">}</span>
<span class="p">]</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">Envio.MyBackend</code> is expected to exist and impelement <code class="language-plaintext highlighter-rouge">Envio.Backend</code>.
The options from the config file will be passed alongside message/payload
as a second parameter. For instance, the <code class="language-plaintext highlighter-rouge">Slack</code> implementation does literally
the following:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nv">@impl</span> <span class="no">true</span>
<span class="k">def</span> <span class="n">on_envio</span><span class="p">(</span><span class="n">message</span><span class="p">,</span> <span class="n">meta</span><span class="p">)</span> <span class="k">do</span>
<span class="k">case</span> <span class="n">meta</span> <span class="k">do</span>
<span class="p">%{</span><span class="ss">hook_url:</span> <span class="n">hook_url</span><span class="p">}</span> <span class="o">-></span>
<span class="no">HTTPoison</span><span class="o">.</span><span class="n">post</span><span class="p">(</span>
<span class="n">hook_url</span><span class="p">,</span>
<span class="n">format</span><span class="p">(</span><span class="n">message</span><span class="p">),</span>
<span class="p">[{</span><span class="s2">"Content-Type"</span><span class="p">,</span> <span class="s2">"application/json"</span><span class="p">}]</span>
<span class="p">)</span>
<span class="n">_</span> <span class="o">-></span> <span class="p">{</span><span class="ss">:error</span><span class="p">,</span> <span class="ss">:no_hook_url_in_envio</span><span class="p">}</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>That’s it. ¡Envíos feliz!</p>
Ruby Challenges as QotD2018-05-31T00:00:00+00:00https://rocket-science.ru/hacking/2018/05/31/qotd-ruby-challenges<h2 id="intro">Intro</h2>
<p>In our company we have a custom to have weekly challenges. Once a week,
sometimes more frequent, sometimes more rare, we publish to slack channel
the small ruby challenge. The reason of doing this is mostly to stretch the
cerebral muscle and/or learn some new fancy ruby tricks.</p>
<p>Below are the top 10 challenges (on my own taste). Feel free to pick some
to interview ruby developers.</p>
<h2 id="quizzz">Quizzz</h2>
<p>What will be printed?</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">print</span> <span class="p">?</span><span class="no">A</span><span class="o"><<</span><span class="sc">?A</span><span class="p">.</span><span class="nf">to_i</span><span class="p">(</span><span class="mi">16</span><span class="p">)</span>
</code></pre></div></div>
<hr />
<p>What would be the output executing this?</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">enum</span> <span class="o">=</span> <span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="mi">3</span><span class="p">).</span><span class="nf">each</span>
<span class="nb">print</span> <span class="n">enum</span><span class="p">.</span><span class="nf">next</span>
<span class="n">enum</span><span class="p">.</span><span class="nf">each</span> <span class="p">{</span> <span class="o">|</span><span class="n">i</span><span class="o">|</span> <span class="nb">print</span> <span class="n">i</span> <span class="p">}</span>
<span class="n">enum</span><span class="p">.</span><span class="nf">next</span>
<span class="kp">loop</span> <span class="p">{</span> <span class="nb">print</span> <span class="n">enum</span><span class="p">.</span><span class="nf">next</span> <span class="p">}</span>
</code></pre></div></div>
<hr />
<p>What would be the result of the following code?</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">m</span><span class="p">(</span><span class="n">o</span><span class="p">)</span>
<span class="n">o</span> <span class="o"><<</span> <span class="s1">'_O'</span>
<span class="k">end</span>
<span class="n">o</span> <span class="o">=</span> <span class="s1">'O'</span><span class="p">;</span>
<span class="s2">"</span><span class="si">#{</span><span class="n">o</span><span class="si">}</span><span class="s2"> | </span><span class="si">#{</span><span class="n">m</span> <span class="n">o</span><span class="si">}</span><span class="s2"> | </span><span class="si">#{</span><span class="n">m</span> <span class="n">o</span><span class="si">}</span><span class="s2"> | </span><span class="si">#{</span><span class="n">o</span><span class="si">}</span><span class="s2">"</span>
</code></pre></div></div>
<hr />
<p>What would be printed in both cases?</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">foo1</span><span class="p">(</span><span class="o">**</span><span class="n">kw</span><span class="p">);</span> <span class="nb">puts</span> <span class="n">kw</span><span class="p">;</span> <span class="k">end</span>
<span class="n">foo1</span><span class="p">(</span><span class="ss">arg?: </span><span class="mi">42</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">foo2</span><span class="p">(</span><span class="n">arg?</span><span class="p">:,</span> <span class="o">**</span><span class="n">kw</span><span class="p">);</span> <span class="nb">puts</span> <span class="n">kw</span><span class="p">;</span> <span class="k">end</span>
<span class="n">foo2</span><span class="p">(</span><span class="ss">arg?: </span><span class="mi">42</span><span class="p">)</span>
</code></pre></div></div>
<hr />
<p>What would be the difference between <code class="language-plaintext highlighter-rouge">instance.is_a?(MyClass)</code>
and <code class="language-plaintext highlighter-rouge">instance.class <= MyClass</code>?</p>
<hr />
<p>What would be the result?</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">a1</span>
<span class="ss">:normal</span>
<span class="k">ensure</span>
<span class="ss">:ensure</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">a2</span>
<span class="k">return</span> <span class="ss">:normal</span>
<span class="k">ensure</span>
<span class="k">return</span> <span class="ss">:ensure</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">a3</span>
<span class="ss">:normal</span>
<span class="n">finally</span>
<span class="ss">:finally</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">a4</span>
<span class="k">return</span> <span class="ss">:normal</span>
<span class="n">finally</span>
<span class="k">return</span> <span class="ss">:finally</span>
<span class="k">end</span>
<span class="p">[</span>
<span class="p">(</span><span class="n">a1</span> <span class="k">rescue</span> <span class="ss">:raised</span><span class="p">),</span>
<span class="p">(</span><span class="n">a2</span> <span class="k">rescue</span> <span class="ss">:raised</span><span class="p">),</span>
<span class="p">(</span><span class="n">a3</span> <span class="k">rescue</span> <span class="ss">:raised</span><span class="p">),</span>
<span class="p">(</span><span class="n">a4</span> <span class="k">rescue</span> <span class="ss">:raised</span><span class="p">)</span>
<span class="p">]</span>
</code></pre></div></div>
<hr />
<p>Correct the function below so that it completes successfully.</p>
<p><em>NB:</em> relatively hard.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">print_down_to</span><span class="p">(</span><span class="n">input</span><span class="p">,</span> <span class="n">down_to</span><span class="p">)</span>
<span class="nb">p</span> <span class="n">input</span>
<span class="n">print_down_to</span><span class="p">(</span><span class="n">input</span> <span class="o">-</span> <span class="mi">1</span><span class="p">,</span> <span class="n">down_to</span><span class="p">)</span> <span class="k">if</span> <span class="n">input</span> <span class="o">></span> <span class="n">down_to</span>
<span class="k">end</span>
<span class="n">print_down_to</span><span class="p">(</span><span class="mi">1_000_000</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
<span class="c1">#⇒ .... lots of integers</span>
<span class="c1">#⇒ ** SystemStackError</span>
</code></pre></div></div>
<hr />
<p>What would be the output of:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kp">true</span> <span class="o">==</span> <span class="kp">true</span> <span class="o">==</span> <span class="kp">true</span>
</code></pre></div></div>
<hr />
<p>What would this code produce?</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span><span class="o">*</span><span class="sc">?a</span><span class="o">..</span><span class="sc">?z</span><span class="p">].</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">e</span><span class="o">|</span>
<span class="k">next</span> <span class="k">unless</span> <span class="p">(</span><span class="n">e</span> <span class="o">=~</span> <span class="sr">/[aeiou]/</span><span class="p">)</span><span class="o">..</span><span class="p">(</span><span class="n">e</span> <span class="o">=~</span> <span class="sr">/[aeiou]/</span><span class="p">)</span>
<span class="nb">print</span> <span class="n">e</span>
<span class="k">end</span>
</code></pre></div></div>
<hr />
<p>Given a huge array of integers, calculate an amount of <code class="language-plaintext highlighter-rouge">1</code>, followed by <code class="language-plaintext highlighter-rouge">2</code>
(not necessarily immediately). The solution must be not worse than <code class="language-plaintext highlighter-rouge">O(N)</code>
(and <code class="language-plaintext highlighter-rouge">O(1)</code> on memory.) E.g. if the list is <code class="language-plaintext highlighter-rouge">[3,1,2,2,1,1,1,3,2]</code>,
the answer would be <code class="language-plaintext highlighter-rouge">2</code> (indices <code class="language-plaintext highlighter-rouge">1</code> and <code class="language-plaintext highlighter-rouge">6</code>.) Ask me for hints and tips
(mine is 36 symbols.)</p>
<p><em>NB:</em> relatively hard task (50+ symbols solution is relatively easy.)</p>
<hr />
<h2 id="happy-challenging">Happy Challenging!</h2>
These Weird Accents2018-05-29T00:00:00+00:00https://rocket-science.ru/hacking/2018/05/29/these-weird-accents<h2 id="history-ruby-hard-way">History. Ruby Hard Way.</h2>
<p>I was born in the country where we don’t use the Latin alphabet. Unlike the
computer science, that was indeed born with the 26-letters-tied mother tongue.</p>
<p>That basically forced me to study another alphabet, besides my primordial one.
Unlike the computer science, that mostly used the pristine ASCII7 till near past.</p>
<p>As the computer science was spreading throughout the world, people from different
countries were looking for opportunities to use their own native languages. Asian
peoples, using the hieroglyphic writing, were forced to invent a brand new way
to represent their characters; Europeans tried to be content with half measures,
still saving a space on hard drives and, more significant, a traffic over the
coaxial cables the network was built with back then.</p>
<p>Umlauts in German, <code class="language-plaintext highlighter-rouge">Æ</code>, <code class="language-plaintext highlighter-rouge">Ø</code> and <code class="language-plaintext highlighter-rouge">Å</code> in Dano-Norwegian, <code class="language-plaintext highlighter-rouge">Ñ</code> in Spanish and <code class="language-plaintext highlighter-rouge">Ç</code> in
Catalan, all these symbols were squeezed and crammed into one byte that was
supposed to represent a character (<code class="language-plaintext highlighter-rouge">char</code> as we call it with regard to computers.)
Norwegians gave a little shit to the existence of cedillas, while Germans were
indifferent to breves and carons. That’s how many Latin1-derived encodings were
created. The word “encoding” means that the same byte is represented
in the different ways. Here is a tiny example for positions<code class="language-plaintext highlighter-rouge">210–216</code> (<code class="language-plaintext highlighter-rouge">ruby</code>):</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">(</span><span class="mi">210</span><span class="o">..</span><span class="mi">216</span><span class="p">).</span><span class="nf">map</span> <span class="k">do</span> <span class="o">|</span><span class="n">i</span><span class="o">|</span>
<span class="p">[</span><span class="n">i</span><span class="p">,</span> <span class="o">*</span><span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="mi">16</span><span class="p">).</span><span class="nf">map</span> <span class="k">do</span> <span class="o">|</span><span class="n">j</span><span class="o">|</span>
<span class="n">i</span><span class="p">.</span>
<span class="nf">chr</span><span class="p">.</span>
<span class="nf">force_encoding</span><span class="p">(</span><span class="no">Encoding</span><span class="p">.</span><span class="nf">const_get</span><span class="p">(</span><span class="s2">"ISO8859_</span><span class="si">#{</span><span class="n">j</span><span class="si">}</span><span class="s2">"</span><span class="p">)).</span>
<span class="nf">encode</span><span class="p">(</span><span class="no">Encoding</span><span class="o">::</span><span class="no">UTF_8</span><span class="p">)</span> <span class="k">rescue</span> <span class="kp">nil</span>
<span class="k">end</span><span class="p">.</span><span class="nf">compact</span><span class="p">.</span><span class="nf">join</span><span class="p">]</span>
<span class="k">end</span><span class="p">.</span><span class="nf">to_h</span>
<span class="c1">#⇒ {</span>
<span class="c1"># 210=>"ÒŇÒŌвزÒŌาŅÒÒÒ",</span>
<span class="c1"># 211=>"ÓÓÓĶгسΣÓÓำÓÓÓÓ",</span>
<span class="c1"># 212=>"ÔÔÔÔдشΤÔÔิŌÔÔÔ",</span>
<span class="c1"># 213=>"ÕŐĠÕеصΥÕÕีÕÕÕŐ",</span>
<span class="c1"># 214=>"ÖÖÖÖжضΦÖÖึÖÖÖÖ",</span>
<span class="c1"># 215=>"××××зطΧ×Ũื×Ṫ׌",</span>
<span class="c1"># 216=>"ØŘĜØиظΨØØุŲØØŰ"</span>
<span class="c1"># }</span>
</code></pre></div></div>
<p>Meanwhile not all the mail/proxy servers were able to understand <em>the encoding</em>,
and those using Cyrillic alphabet invented a genious way to outwit them: so-called
<code class="language-plaintext highlighter-rouge">KOI-8</code> encoding put the Cyrillic letters in the places where the <em>similar
pronouncing latin letters were placed</em>. That way even if the gateway damaged
the original encoding, the text was still somehow readable (“ПРИВЕТ” became “PRIVET”.)</p>
<p>Anyway, this was a mess and Unicode Consortium (after a couple of fails with
stillborn <code class="language-plaintext highlighter-rouge">UCS</code> encodings,) finally invented <code class="language-plaintext highlighter-rouge">UTF-8</code>. Which allows to encode
everything needed (plus a dozillion of emojis.) Accents were re-thought and
they were given a legal green card to exist.
<a href="https://en.wikipedia.org/wiki/Combining_Diacritical_Marks">Combined diacritics</a>
came to the scene. Instead of looking through all the alphabets, collecting
letters that walks like a duck and quacks like a duck, but having three legs,
letters and accents were finally distinguished. To type a graved <em>a</em> in voilà,
I don’t need to remap my keyboard to have this symbol under my pinkie, I can
simply use a combined diacritics, typing <code class="language-plaintext highlighter-rouge">a</code> and the accent ` ̀ ` subsequently.
And this is great.</p>
<p>Unfortunately, there is a legacy. Spanish keyboard has a single key to type
“ñ” and I doubt it would be welcome to deprecate this key in favor of typing
a combined diacritics. Hence the decision to keep old good symbols like “Å” and
“Ø” was taken. To type any of those, one might either press a key on their
Norwegian keyboard, or type “A” followed by the combining ring above or “O”
followed by combining solidus overlay. The disaster is: those produce <em>two
different strings</em>. Already frightened?—Wait a second. Now: <em>these strings
have different length</em>.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="sx">%w|mañana mañana|</span><span class="p">.</span><span class="nf">map</span><span class="p">(</span><span class="o">&</span><span class="ss">:length</span><span class="p">)</span>
<span class="c1">#⇒ [7, 6]</span>
</code></pre></div></div>
<p>I am not kidding. Try it yourself. This distinction is known as <em>composed</em> vs.
<em>decomposed</em> form.</p>
<p>FWIW, Ruby 2.5 introduced (and it was backported to Ruby 2.3+)
<a href="https://ruby-doc.org/core/String.html#method-i-unicode_normalize"><code class="language-plaintext highlighter-rouge">String#unicode_normalize</code></a>
method, accepting euther <code class="language-plaintext highlighter-rouge">:nfc</code> or <code class="language-plaintext highlighter-rouge">:nfd</code> parameter to compose or decompose the
receiver respectively. To catch “ñ” in the string no matter how it was typed,
one <strong>must</strong> use:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># use composed one as a matcher ⇓</span>
<span class="sx">%w|mañana mañana|</span><span class="p">.</span><span class="nf">map</span> <span class="p">{</span> <span class="o">|</span><span class="n">s</span><span class="o">|</span> <span class="n">s</span><span class="p">.</span><span class="nf">unicode_normalize</span><span class="p">(</span><span class="ss">:nfc</span><span class="p">)[</span><span class="sr">/ñ/</span><span class="p">]</span> <span class="p">}</span>
<span class="c1">#⇒ ["ñ", "ñ"]</span>
</code></pre></div></div>
<hr />
<h2 id="elixir-thanks-josé">Elixir. Thanks, José!</h2>
<p>Fortunately enough, José Valim, the creator or Elixir, has an acute in his name.
And—Elixir has had a proper support for Unicode from the scratch. Elixir does it’s
best to allow us not to bury into encoding issues. We have
<a href="https://hexdocs.pm/elixir/String.html#graphemes/1"><code class="language-plaintext highlighter-rouge">String.graphemes/1</code></a> that
lists graphemes:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="sx">~w|mañana mañana|</span>
<span class="o">|></span> <span class="no">Enum</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="o">&</span><span class="no">String</span><span class="o">.</span><span class="n">graphemes</span><span class="o">/</span><span class="mi">1</span><span class="p">)</span>
<span class="o">|></span> <span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="p">()</span>
<span class="c1">#⇒ [["m", "a", "ñ", "a", "n", "a"], ["m", "a", "ñ", "a", "n", "a"]]</span>
<span class="o">|></span> <span class="no">Enum</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="o">&</span><span class="no">Enum</span><span class="o">.</span><span class="n">join</span><span class="o">/</span><span class="mi">1</span><span class="p">)</span>
<span class="c1">#⇒ ["mañana", "mañana"]</span>
</code></pre></div></div>
<p><a href="https://hexdocs.pm/elixir/String.html#length/1"><code class="language-plaintext highlighter-rouge">String.length/1</code></a> works
as expected, without surprises:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="sx">~w|mañana mañana|</span> <span class="o">|></span> <span class="no">Enum</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="o">&</span><span class="no">String</span><span class="o">.</span><span class="n">length</span><span class="o">/</span><span class="mi">1</span><span class="p">)</span>
<span class="c1">#⇒ [6, 6]</span>
</code></pre></div></div>
<p>That is all transparently available because Elixir is smart, even though
the input still differs:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="sx">~w|mañana mañana|</span> <span class="o">|></span> <span class="no">Enum</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="o">&</span><span class="no">String</span><span class="o">.</span><span class="n">codepoints</span><span class="o">/</span><span class="mi">1</span><span class="p">)</span>
<span class="c1">#⇒ [</span>
<span class="c1"># ["m", "a", "n", "̃", "a", "n", "a"],</span>
<span class="c1"># ["m", "a", "ñ", "a", "n", "a"]</span>
<span class="c1"># ]</span>
</code></pre></div></div>
<p>And Elixir still provides
<a href="https://hexdocs.pm/elixir/String.html#normalize/2"><code class="language-plaintext highlighter-rouge">String.normalize/2</code></a> to
manually fix the discrepancy issue, producing a known form, either composed,
or decomposed, depending on what the goal is. And this is the way to go,
no matter how smart Elixir is, sometimes it’s better to not let the
steering wheel out of hands:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">String</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"mañana mañana"</span><span class="p">,</span> <span class="s2">"ñ"</span><span class="p">,</span> <span class="s2">"-"</span><span class="p">)</span>
<span class="c1">#⇒ "mañana ma-ana"</span>
<span class="no">Regex</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="sr">~r/ñ/</span><span class="p">,</span> <span class="s2">"mañana mañana"</span><span class="p">,</span> <span class="s2">"-"</span><span class="p">)</span>
<span class="c1">#⇒ "mañana ma-ana"</span>
</code></pre></div></div>
<p>But:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="s2">"mañana mañana"</span>
<span class="o">|></span> <span class="no">String</span><span class="o">.</span><span class="n">normalize</span><span class="p">(</span><span class="ss">:nfc</span><span class="p">)</span>
<span class="o">|></span> <span class="no">String</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s2">"ñ"</span><span class="p">,</span> <span class="s2">"-"</span><span class="p">)</span>
<span class="c1">#⇒ "ma-ana ma-ana"</span>
</code></pre></div></div>
<hr />
<p>I foresee the times when all the above became a legacy as well since everybody
will just use emojis instead of words. Emojis have luckily no decomposed form.</p>
To IF Or Not To IF—That’s The Conditional Statement2018-05-21T00:00:00+00:00https://rocket-science.ru/hacking/2018/05/21/to-if-or-not-to-if<p>This post is written mostly as a response to <a href="https://dev.to/genta/theres-no-else-if-in-js--24f9">that discussion thread</a>. I was advocating
to never use <code class="language-plaintext highlighter-rouge">if</code> conditional statement and many people were arguing.</p>
<p><strong>TL;DR:</strong> The text below is very biased and opionated. That does not mean
it’s plain wrong. I consider <code class="language-plaintext highlighter-rouge">if</code> statements a code smell. Period.</p>
<p><em>I am not a blinkered moron, though, and I understand there are circumstances
under which <code class="language-plaintext highlighter-rouge">if</code> might be of perfect use. The rule having a dozen of
exceptions is hard to memorize, that’s why I state the rule as
“don’t use <code class="language-plaintext highlighter-rouge">if</code> statements.” “Unless you pretty fine understand why do
you use <code class="language-plaintext highlighter-rouge">if</code>,” adds my internal rubber duck whom I have proofread the
draft of this writing to.</em></p>
<hr />
<h2 id="stone-age">Stone Age</h2>
<p>In the stone age there were no programming languages and to kindly
ask the machine to perform any operation, humans were using
<a href="https://en.wikipedia.org/wiki/Machine_code">codes</a> that were understood
by the machine as is. One of the first books I was able to get to reveal
the the mystical veil of development was <a href="https://www.goodreads.com/author/list/163189.Peter_Norton">one of these by Peter Norton</a>. Sorry, Peter,
I do not remember which one. It started with a chapter explaining the
source of <em>Norton Disk Editor</em> utility, written completely in machine
codes. Line by line. It looked like that (don’t try to run it, I just
randomly generated a garbage for the sake of an example):</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>61 01 67 30 30 8D 65 A4
E1 4C 9F 3F 96 B1 EB 85
F6 54 92 B8 51 E1 E5 7D
05 93 EC 2D BF 10 34 FB
98 AD E0 96 B7 52 21 09
50 FA 45 59 05 DA 17 CB
6B 95 0C 4F E7 24 60 1F
DE 5D BA F9 72 B7 F8 09
03 10 13 1A 8D 75 79 BF
5E 8D 05 B3 F0 2C F6 D8
</code></pre></div></div>
<p>I spent a week trying to understand the code and finally realized, I was
worse than any machine in understanding codes. (FWIW, I still am.)</p>
<p>I believe I was not the only person frustrated by the necessity to talk
to machines using <em>their</em> language, and
<a href="https://en.wikipedia.org/wiki/Assembly_language">Assembly</a> came to the scene.</p>
<p>That was indeed a huge step towards preserving sanity of human minds. Now the
program listing became …uuuhm… readable:</p>
<pre><code class="language-asm">mov al, 42
mov ah, .answer
cmp al, ah
jne .answer_err
</code></pre>
<p><em>Move</em> <code class="language-plaintext highlighter-rouge">42</code> to the register named <code class="language-plaintext highlighter-rouge">al</code>, <em>move</em> the value of <code class="language-plaintext highlighter-rouge">.answer</code> variable
into the register <code class="language-plaintext highlighter-rouge">ah</code>, <em>compare</em> values and <em>jump</em> to <code class="language-plaintext highlighter-rouge">.answer_err</code>
instruction if values are not equal. If this is not a poetry, I don’t know what is.</p>
<p>The reason behind this structure of the code is that the CPU cannot
permorm some sophisticated operations, like “make a cappuccino.” It can
add integer numbers, and it can transfer a control. That’s it.</p>
<p>Note for those bored and falling asleep: I am not going to describe
the whole evolution process of programming languages. Actually, I just
wanted to show this piece of assembly because I am going to refer to it later.</p>
<hr />
<h2 id="we-are-human-beings">We Are Human Beings</h2>
<p>Unlike computers, we don’t need to decompose compound tasks into the
sequence of additions and conditional jumps. We just know whether we
want to eat an apple right now or not. There is no <code class="language-plaintext highlighter-rouge">if</code> implied there.
We don’t think like “if I want an apple, I’ll eat it, otherwise I won’t.”</p>
<p>This is important.</p>
<p>I don’t know how exactly it’s done inside my head (I have heard, even
neuroscientists are somehow in doubt,) but from my point of view it
looks like “Oh, an apple, I’ll take it.” Or I just ignore it. I don’t
think about what I’d do if I wanted it, I just damn don’t give a shit.
Pardon my appleish.</p>
<p>Every single morning, save for weekends, I am leaving my house,
heading towards the office. I use a metro to reach the office building.
The route has one transfer. And, for the sake of this post,
I decided to write a code, that could deliver me to the office.</p>
<p>I am going to use ruby, but the syntax should be considered a pseudocode.</p>
<hr />
<h2 id="making-decisions">Making Decisions</h2>
<p>The first naïve attempt would be to use <code class="language-plaintext highlighter-rouge">if</code> to check if today
is a workday in the first place. Because, you know, I am not writing
my PhD anymore and I don’t work on Saturdays. Not in the office, at least.</p>
<p>That said, the early adopted intent would be to start with something like:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">if</span> <span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="mi">5</span><span class="p">).</span><span class="nf">cover?</span> <span class="no">Date</span><span class="p">.</span><span class="nf">today</span><span class="p">.</span><span class="nf">wday</span>
<span class="n">go_to_the_office</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Well, so far so good. The code is clean and easy to read. But wait,
on Tuesdays and Thursdays I go to gym:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">today</span> <span class="o">=</span> <span class="no">Date</span><span class="p">.</span><span class="nf">today</span>
<span class="k">if</span> <span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="mi">5</span><span class="p">).</span><span class="nf">cover?</span> <span class="n">today</span><span class="p">.</span><span class="nf">wday</span>
<span class="k">if</span> <span class="n">today</span><span class="p">.</span><span class="nf">tuesday?</span> <span class="o">||</span> <span class="n">today</span><span class="p">.</span><span class="nf">thursday?</span>
<span class="n">go_to_the_gym</span>
<span class="k">end</span>
<span class="n">go_to_the_office</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Oh, jeez, I have forgotten to mention that on Wednesdays I am working from home:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="mi">1</span> <span class="n">today</span> <span class="o">=</span> <span class="no">Date</span><span class="p">.</span><span class="nf">today</span>
<span class="mi">2</span> <span class="k">if</span> <span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="mi">5</span><span class="p">).</span><span class="nf">cover?</span> <span class="n">today</span><span class="p">.</span><span class="nf">wday</span>
<span class="mi">3</span> <span class="k">if</span> <span class="n">today</span><span class="p">.</span><span class="nf">tuesday?</span> <span class="o">||</span> <span class="n">today</span><span class="p">.</span><span class="nf">thursday?</span>
<span class="mi">4</span> <span class="n">go_to_the_gym</span>
<span class="mi">5</span> <span class="k">end</span>
<span class="mi">6</span> <span class="k">if</span> <span class="o">!</span><span class="n">today</span><span class="p">.</span><span class="nf">wednesday?</span>
<span class="mi">7</span> <span class="n">go_to_the_office</span>
<span class="mi">8</span> <span class="k">end</span>
<span class="mi">9</span> <span class="k">end</span>
</code></pre></div></div>
<p>At this point the code is already very hard to maintain. Also, there are 9 LOCs
and we all expect it should be easy to tell what the code is doing, but unfortunately
that’s not the case. The <em>real code</em> there is actually contained in lines 4 and 7,
all the other stuff is about checking conditions.</p>
<p><em>Sidenote:</em> yes, I pretty fine understand this example is highly exaggerated,
there are postfix conditionals in ruby etc. But trust me: what had been started
as a single nifty <code class="language-plaintext highlighter-rouge">if</code> sooner or later will become this kind of spaghetti monster.
The road to hell is paved with good intentions.</p>
<h2 id="stop-making-unnecessary-decisions">Stop Making Unnecessary Decisions</h2>
<p>The pill would be to use so-called <a href="https://en.wikipedia.org/wiki/Fail-fast">“Fail-fast</a>”
ideology.</p>
<h3 id="oop-approach">OOP Approach</h3>
<p>Once I decided to provide a ruby code snippets, and ruby is somehow
an object-oriented language, let’s start with just OOP to implement the desired
functionality. Prepare to see an unexpectedly huge boilerplate, but it’ll pay back
very soon (if one is writing a one-time-run shell script, I’d suggest to stop
reading this and go with an <code class="language-plaintext highlighter-rouge">if</code>.)</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nn">Schedule</span>
<span class="k">class</span> <span class="nc">Day</span>
<span class="k">def</span> <span class="nf">trip!</span>
<span class="n">go_to_the_gym</span>
<span class="n">go_to_the_office</span>
<span class="n">go_home</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">go_to_the_gym</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">go_to_the_office</span>
<span class="no">Metro</span><span class="p">.</span><span class="nf">office!</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">go_home</span>
<span class="no">Metro</span><span class="p">.</span><span class="nf">home!</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="sx">%w[mon fri]</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">wd</span><span class="o">|</span>
<span class="no">Schedule</span><span class="p">.</span><span class="nf">const_set</span><span class="p">(</span><span class="s2">"</span><span class="si">#{</span><span class="n">wd</span><span class="p">.</span><span class="nf">capitalize</span><span class="si">}</span><span class="s2">Schedule"</span><span class="p">,</span> <span class="no">Class</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="no">Day</span><span class="p">)</span> <span class="k">do</span>
<span class="k">end</span><span class="p">)</span>
<span class="k">end</span>
<span class="sx">%w[tue thu]</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">wd</span><span class="o">|</span>
<span class="no">Schedule</span><span class="p">.</span><span class="nf">const_set</span><span class="p">(</span><span class="s2">"</span><span class="si">#{</span><span class="n">wd</span><span class="p">.</span><span class="nf">capitalize</span><span class="si">}</span><span class="s2">Schedule"</span><span class="p">,</span> <span class="no">Class</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="no">Day</span><span class="p">)</span> <span class="k">do</span>
<span class="k">def</span> <span class="nf">go_to_the_gym</span>
<span class="no">Metro</span><span class="p">.</span><span class="nf">gym!</span>
<span class="k">end</span>
<span class="k">end</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">class</span> <span class="nc">Schedule</span><span class="o">.</span><span class="no">WedSchedule</span> <span class="o"><</span> <span class="no">Day</span>
<span class="k">def</span> <span class="nf">go_to_the_office</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Now we can <em>instantiate</em> the respective class, basing on current week day
and call it’s instance <code class="language-plaintext highlighter-rouge">trip!</code> method. For both Saturday and Sunday this
instantiation would throw an exception, <em>which is just fine</em> (assuming we
handle exceptions on the top level and don’t leak them as is to the user.)</p>
<h3 id="fsm-approach">FSM Approach</h3>
<p><a href="https://en.wikipedia.org/wiki/Finite-state_machine">Finite-state automata</a>
could be used instead of OOP. The code would be looking pretty much as in
the previous (OOP) approach, save for instead of different classes we’d
use different transitions from the state <code class="language-plaintext highlighter-rouge">home</code> depending on the day
of the week. One might argue that there will be <code class="language-plaintext highlighter-rouge">if</code>s, and I’d say yes,
but a) they might be completely avoided if desired and b) they are hidden
for the user of this FSM behind the proper abstraction. Also, SRP won’t be
violated.</p>
<p>Since Ruby does not have an FSM implementation in the core lib, I am
to omit the code example here. It’s more or less trivial.</p>
<h3 id="pattern-matching">Pattern matching</h3>
<p>Ruby does not support pattern matching out of the box, therefore we’d
use Elixir to demonstrate the control flow.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">Schedule</span> <span class="k">do</span>
<span class="k">def</span> <span class="n">trip!</span><span class="p">(</span><span class="n">day</span><span class="p">)</span> <span class="k">do</span>
<span class="n">go_to_the_gym</span><span class="p">(</span><span class="n">day</span><span class="p">)</span>
<span class="n">go_to_the_office</span><span class="p">(</span><span class="n">day</span><span class="p">)</span>
<span class="n">go_home</span><span class="p">(</span><span class="n">day</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="n">go_to_the_gym</span><span class="p">(</span><span class="n">day</span><span class="p">)</span> <span class="ow">when</span> <span class="n">day</span> <span class="ow">in</span> <span class="sx">~w|tue thu|</span> <span class="k">do</span>
<span class="no">Metro</span><span class="o">.</span><span class="n">gym!</span>
<span class="k">end</span>
<span class="k">def</span> <span class="n">go_to_the_gym</span><span class="p">(</span><span class="n">day</span><span class="p">)</span> <span class="ow">when</span> <span class="n">day</span> <span class="ow">in</span> <span class="sx">~w|mon wed fri|</span> <span class="k">do</span>
<span class="k">end</span>
<span class="k">def</span> <span class="n">go_to_the_office</span><span class="p">(</span><span class="s2">"wed"</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="ss">:ok</span>
<span class="k">def</span> <span class="n">go_to_the_office</span><span class="p">(</span><span class="n">_</span><span class="p">)</span>
<span class="no">Metro</span><span class="o">.</span><span class="n">office!</span>
<span class="k">end</span>
<span class="k">def</span> <span class="n">go_home</span><span class="p">(</span><span class="n">_</span><span class="p">)</span>
<span class="no">Metro</span><span class="o">.</span><span class="n">home!</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>As in the OOP example, the code is readable, maintainable, and—most importantly—extendable.</p>
<h3 id="this-is-not-true">This Is Not True</h3>
<p>Well, it indeed is. Whether in doubt, try to add the weekend activity to any
of the aforementioned approaches, then try to do the same with the very first
nested <code class="language-plaintext highlighter-rouge">if</code>s structure.</p>
<h2 id="conclusion">Conclusion</h2>
<p>Any time I found myself blindly typing <code class="language-plaintext highlighter-rouge">if</code>, I pause for a while and
talk to my internal rubber duck (her name is Jess, btw.) “Jess, is there any
way to avoid <code class="language-plaintext highlighter-rouge">if</code> clause here?” I ask inevitably. And you know what?—In most
cases she responds with “yes” and we invent a robust well-designed solution
for a problem together.</p>
<p>Sometimes she responds “no,” it’s a perfect use-case for an <code class="language-plaintext highlighter-rouge">if</code>, like
yesterday when I was to implement a function producing a string representation
of current time for an American audience. I ended up with</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">if</span> <span class="n">hours</span> <span class="o"><</span> <span class="mi">12</span> <span class="k">then</span> <span class="s1">'AM'</span> <span class="k">else</span> <span class="s1">'PM'</span>
</code></pre></div></div>
<p>I understand why I used <code class="language-plaintext highlighter-rouge">if</code> there (thanks, Jessica,) and it was a last resort.
It was a wisely made decision. There is no room for any abstraction. The calendar
and the string time representation are not going to change until I am retired.</p>
<p>In all other cases I ended up with a better abstraction, that saved me tons
of hours of refactoring, just because I had asked myself (and Jess) whether we could
do better than just <code class="language-plaintext highlighter-rouge">if</code>. And we usually did.</p>
<p>That is why I insist on memorizing the rule <em>“<code class="language-plaintext highlighter-rouge">if</code> statements are a code smell, period.”</em>
Even despite it sounds too cruel and arrogant. Vague and blurry rules always
lose. Solid rules do win. And they surely do have exceptions.</p>
Interview on Senior Poet Position2018-05-17T00:00:00+00:00https://rocket-science.ru/jokes/2018/05/17/interview-on-senior-poet-position<p>“Good morning. Welcome. How did you find us, if I may?”<br />
“There was a vacancy at PoetryFlow.”<br />
“Do you have an account there?”<br />
“Yes, and at PoemHub as well. My best poem was starred three hundred times.”<br />
“Fine, fine. We’ve surely read the résumé, but could you please briefly introduce yourself?”<br />
“Sure. I started to write poems in high school. ’Twas Hexameter 1.0, on used punchcards, you know. Then Haiku++ (that one having 19 syllables.) Also I did some pure rhymes.”<br />
“Good,” the interviewer makes some notes.
“The only noticeable one was a poem to my classmate Sara in the college. Plus some small exercises.”<br />
“What was your first production-level poem about?”<br />
“It was ‘Mr. Smith is full of shit,’ written on the wall of our college. It’s still not wiped clean.”<br />
“What areas do you have writing experience at?”<br />
“I prefer amphibrach, but, unfortunately, it’s rarely in demand nowadays. Also anapest and iambic. The project I mentioned is done mostly on choree.”<br />
“Good,” the interviewer makes some notes.
“Also I sometimes comment on blog posts. For free. But this happens only in my free time. For that, I use mostly dactyl.”<br />
“Are you a hipster?”<br />
“No, no, no way. That’s for fun. I understand that dactyl is too demanding to resources, plus neither AWS nor Azure support guitar accompaniment.”<br />
“Have you ever been working with the free verse?”<br />
“Not in production, only a couple of pet projects.”<br />
“You see, we have a project in the slightly modified version of the free verse, say, in-house edition,” the interviewer conspiratorially smiles.
“Some fork, like Auden’s?”<br />
“Not really. We have added to the free verse what it lacked: isotonic, rhyme, isosyllabism and some other fancy whistles.”<br />
“What was wrong with a plain old good iambic?”<br />
“Honestly, this is a legacy. The project has been started by an ancient Hindu poet, who could do nothing save for the free verse. In addition, 80% of commercial poetry in the world is written in the free verse.”<br />
“Got it.”<br />
“We have more or less standard backend: love and war. R&D department is trying to use extras like jokes, oxymorons, ruptures of discourse… But in the production, all these nifty features are still off.”<br />
“Got it.”<br />
“Well, whether you’re fine with that, let’s do a tiny whiteboard testing. The tasks would be very simple, that’s mostly to test your way of rhyming rather than deep poetry knowledge.”<br />
“OK.”<br />
“Please finish the stanza ‘What killed Voldemort’…”<br />
“Uhm. I need to think. OK. Here we go. Uhm.” <em>(coughs)</em>
“No rush, take your time.”<br />
“What can I use to implement the ending?”<br />
“Whatever you want.”<br />
“Iambic then. For instance ‘Accidental gunshot.’”<br />
“Almost, but let’s try to improve it a bit. Look. Voldemort is immortal, hence the gunshot, while it definitely could cause some damage…”<br />
“Yes, yes, sure, sec. Here we go. Choree: ‘Bubble sort, bubble sort!’ Or, even better, dactyl. Oh, there is no native interface from your stanza to dactyl. Just a sec…”<br />
“OK, everything is fine, there is another one. How would you implement the anniversary ceremony for some honorable person?”<br />
“Here it’s dactyl for sure. Plus I’d use plugins ‘tears of joy’ and ‘elation’”<br />
“Dactyl? Who do you think would support it in the future?”<br />
“Support? How could one reuse the ceremony greeting?”<br />
“That said, you expect no anniversary in the next year?”<br />
“Uh-oh. Sure, of course.”<br />
“Well, let’s have one question more, and that’s all. What do you think would be the ideal poetic size for everything?”<br />
“The free verse with rhymes and isosyllabism.”<br />
“You are hired.”</p>
Elixir Iteraptor :: Iterating Nested Terms Like I’m Five2018-04-06T00:00:00+00:00https://rocket-science.ru/hacking/2018/04/06/iteraptor-for-elixir<p>A week ago I <a href="http://rocket-science.ru/hacking/2018/03/29/iteraptor-for-the-rescue">have released</a>
the Ruby’s library called <a href="https://github.com/am-kantox/iteraptor"><code class="language-plaintext highlighter-rouge">Iteraptor</code></a>
that allows easy handy iterating over cumbersome nested structures, consisting
of any enumerables. This blog post announces it’s sibling for Elixir.
I love the name, that’s why it’s also called <code class="language-plaintext highlighter-rouge">Iteraptor</code>.</p>
<p><strong>TL;DR:</strong></p>
<ul>
<li><a href="https://github.com/am-kantox/elixir-iteraptor">source code</a>;</li>
<li><a href="https://hexdocs.pm/iteraptor/Iteraptor.html">documentation</a>.</li>
</ul>
<hr />
<p>Iterating both maps and lists in Elixir is charming. One might chain iterators,
map, reduce, filter, select, reject, zip… Everybody having at least eight
hours of experience with Elixir has definitely seen (and even maybe written)
something like this:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">~</span><span class="n">w</span><span class="o">|</span><span class="n">aleksei</span> <span class="n">saverio</span><span class="o">|</span>
<span class="o">|></span> <span class="no">Enum</span><span class="p">.</span><span class="nf">map</span><span class="p">(</span><span class="o">&</span> <span class="no">String</span><span class="p">.</span><span class="nf">capitalize</span><span class="o">/</span><span class="mi">1</span><span class="p">)</span>
<span class="o">|></span> <span class="no">Enum</span><span class="p">.</span><span class="nf">each</span><span class="p">(</span><span class="n">fn</span> <span class="n">capitalized_name</span> <span class="o">-></span>
<span class="no">IO</span><span class="p">.</span><span class="nf">puts</span> <span class="s2">"Hello, </span><span class="si">#{</span><span class="n">capitalized_name</span><span class="si">}</span><span class="s2">!"</span>
<span class="k">end</span><span class="p">)</span>
</code></pre></div></div>
<p>That is really handy. The things gets cumbersome when it comes to deeply nested
structures, like a map having nested keywords, lists etc. The good example of
that would be any configuration file, having nested subsections.</p>
<p>While Elixir provides helpers to update elements deeply inside such a term:</p>
<ul>
<li><a href="https://hexdocs.pm/elixir/Kernel.html#get_in/2"><code class="language-plaintext highlighter-rouge">Kernel.get_in/2</code></a></li>
<li><a href="https://hexdocs.pm/elixir/Kernel.html#put_in/2"><code class="language-plaintext highlighter-rouge">Kernel.put_in/{2,3}</code></a></li>
<li><a href="https://hexdocs.pm/elixir/Kernel.html#update_in/2"><code class="language-plaintext highlighter-rouge">Kernel.update_in/{2,3}</code></a></li>
<li><a href="https://hexdocs.pm/elixir/Kernel.html#get_and_update_in/2"><code class="language-plaintext highlighter-rouge">Kernel.get_and_update_in/{2,3}</code></a></li>
</ul>
<p>all the above would work if and only all the parent levels in the structure exist.
The exception would be <code class="language-plaintext highlighter-rouge">get_in/2</code> which is happily returning <code class="language-plaintext highlighter-rouge">nil</code> being asked
for whatever inexisting.</p>
<p>The amount of questions on Stack Overflow asking “how would I modify a nested
structure” forced me to finally create this library. The implementation in Elixir
looks a bit more convoluted since everything is immutable and one cannot just
traverse a structure down to leaves, modifying whatever needed in-place.
The iteration-wide accumulator is required.</p>
<p>That is probably the only example I met in my life where mutability makes things
easier. As a bonus the implementation of <code class="language-plaintext highlighter-rouge">bury/4</code> to store the value deeply inside
a structure, creating the intermediate keys as necessary, was introduced.
It behaves as a <a href="https://bugs.ruby-lang.org/issues/11747">proposed but rejected in ruby core</a>
<code class="language-plaintext highlighter-rouge">Hash#bury</code>.</p>
<hr />
<p>So, welcome the library that makes the iteration of any nested map/keyword/list
combination almost as easy as the natural Elixir <code class="language-plaintext highlighter-rouge">map</code> and <code class="language-plaintext highlighter-rouge">each</code>.</p>
<p>• <a href="https://github.com/am-kantox/elixir-iteraptor"><strong>Iteraptor</strong></a></p>
<h3 id="features">Features</h3>
<ul>
<li><a href="https://hexdocs.pm/iteraptor/Iteraptor.html#each/3"><code class="language-plaintext highlighter-rouge">Iteraptor.each/3</code></a>
to iterate a deeply nested map/list/keyword;</li>
<li><a href="https://hexdocs.pm/iteraptor/Iteraptor.html#map/3"><code class="language-plaintext highlighter-rouge">Iteraptor.map/3</code></a>
to map a deeply nested map/list/keyword;</li>
<li><a href="https://hexdocs.pm/iteraptor/Iteraptor.html#reduce/4"><code class="language-plaintext highlighter-rouge">Iteraptor.reduce/4</code></a>
to reduce a deeply nested map/list/keyword;</li>
<li><a href="https://hexdocs.pm/iteraptor/Iteraptor.html#map_reduce/4"><code class="language-plaintext highlighter-rouge">Iteraptor.map_reduce/4</code></a>
to map and reduce a deeply nested map/list/keyword;</li>
<li><a href="https://hexdocs.pm/iteraptor/Iteraptor.html#filter/3"><code class="language-plaintext highlighter-rouge">Iteraptor.filter/3</code></a>
to filter a deeply nested map/list/keyword;</li>
<li><a href="https://hexdocs.pm/iteraptor/Iteraptor.html#to_flatmap/2"><code class="language-plaintext highlighter-rouge">Iteraptor.to_flatmap/2</code></a>
to flatten a deeply nested map/list/keyword into
flatten map with concatenated keys;</li>
<li><a href="https://hexdocs.pm/iteraptor/Iteraptor.html#from_flatmap/3"><code class="language-plaintext highlighter-rouge">Iteraptor.from_flatmap/3</code></a>
to “unveil”/“unflatten” the previously flattened map into nested structure;</li>
<li><a href="https://hexdocs.pm/iteraptor/Iteraptor.Iteraptable.html"><code class="language-plaintext highlighter-rouge">use Iteraptor.Iteraptable</code></a>
to automagically implement <code class="language-plaintext highlighter-rouge">Enumerable</code> and <code class="language-plaintext highlighter-rouge">Collectable</code> protocols, as well as
<code class="language-plaintext highlighter-rouge">Access</code> behaviour on the structure.</li>
</ul>
<h3 id="words-are-cheap-show-me-the-code">Words are cheap, show me the code</h3>
<h4 id="iterating-mapping-reducing">Iterating, Mapping, Reducing</h4>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># each</span>
<span class="n">iex</span><span class="o">></span> <span class="p">%{</span><span class="ss">a:</span> <span class="p">%{</span><span class="ss">b:</span> <span class="p">%{</span><span class="ss">c:</span> <span class="mi">42</span><span class="p">}}}</span> <span class="o">|></span> <span class="no">Iteraptor</span><span class="o">.</span><span class="n">each</span><span class="p">(</span><span class="o">&</span><span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="o">/</span><span class="mi">1</span><span class="p">,</span> <span class="ss">yield:</span> <span class="ss">:all</span><span class="p">)</span>
<span class="c1"># {[:a], %{b: %{c: 42}}}</span>
<span class="c1"># {[:a, :b], %{c: 42}}</span>
<span class="c1"># {[:a, :b, :c], 42}</span>
<span class="p">%{</span><span class="ss">a:</span> <span class="p">%{</span><span class="ss">b:</span> <span class="p">%{</span><span class="ss">c:</span> <span class="mi">42</span><span class="p">}}}</span>
<span class="c1"># map</span>
<span class="n">iex</span><span class="o">></span> <span class="p">%{</span><span class="ss">a:</span> <span class="p">%{</span><span class="ss">b:</span> <span class="p">%{</span><span class="ss">c:</span> <span class="mi">42</span><span class="p">}}}</span> <span class="o">|></span> <span class="no">Iteraptor</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="k">fn</span> <span class="p">{</span><span class="n">k</span><span class="p">,</span> <span class="n">_</span><span class="p">}</span> <span class="o">-></span> <span class="no">Enum</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">k</span><span class="p">)</span> <span class="k">end</span><span class="p">)</span>
<span class="p">%{</span><span class="ss">a:</span> <span class="p">%{</span><span class="ss">b:</span> <span class="p">%{</span><span class="ss">c:</span> <span class="s2">"abc"</span><span class="p">}}}</span>
<span class="n">iex</span><span class="o">></span> <span class="p">%{</span><span class="ss">a:</span> <span class="p">%{</span><span class="ss">b:</span> <span class="p">%{</span><span class="ss">c:</span> <span class="mi">42</span><span class="p">}}}</span>
<span class="o">...></span> <span class="o">|></span> <span class="no">Iteraptor</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="k">fn</span>
<span class="o">...></span> <span class="p">{[</span><span class="n">_</span><span class="p">],</span> <span class="n">_</span><span class="p">}</span> <span class="o">=</span> <span class="n">self</span> <span class="o">-></span> <span class="n">self</span>
<span class="o">...></span> <span class="p">{[</span><span class="n">_</span><span class="p">,</span> <span class="n">_</span><span class="p">],</span> <span class="n">_</span><span class="p">}</span> <span class="o">-></span> <span class="s2">"YAY"</span>
<span class="o">...></span> <span class="k">end</span><span class="p">,</span> <span class="ss">yield:</span> <span class="ss">:all</span><span class="p">)</span>
<span class="p">%{</span><span class="ss">a:</span> <span class="p">%{</span><span class="ss">b:</span> <span class="s2">"YAY"</span><span class="p">}}</span>
<span class="c1"># reduce</span>
<span class="n">iex</span><span class="o">></span> <span class="p">%{</span><span class="ss">a:</span> <span class="p">%{</span><span class="ss">b:</span> <span class="p">%{</span><span class="ss">c:</span> <span class="mi">42</span><span class="p">}}}</span>
<span class="o">...></span> <span class="o">|></span> <span class="no">Iteraptor</span><span class="o">.</span><span class="n">reduce</span><span class="p">([],</span> <span class="k">fn</span> <span class="p">{</span><span class="n">k</span><span class="p">,</span> <span class="n">_</span><span class="p">},</span> <span class="n">acc</span> <span class="o">-></span>
<span class="o">...></span> <span class="p">[</span><span class="no">Enum</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">k</span><span class="p">,</span> <span class="s2">"_"</span><span class="p">)</span> <span class="o">|</span> <span class="n">acc</span><span class="p">]</span>
<span class="o">...></span> <span class="k">end</span><span class="p">,</span> <span class="ss">yield:</span> <span class="ss">:all</span><span class="p">)</span>
<span class="o">...></span> <span class="o">|></span> <span class="ss">:lists</span><span class="o">.</span><span class="n">reverse</span><span class="p">()</span>
<span class="p">[</span><span class="s2">"a"</span><span class="p">,</span> <span class="s2">"a_b"</span><span class="p">,</span> <span class="s2">"a_b_c"</span><span class="p">]</span>
<span class="c1"># map-reduce</span>
<span class="n">iex</span><span class="o">></span> <span class="p">%{</span><span class="ss">a:</span> <span class="p">%{</span><span class="ss">b:</span> <span class="p">%{</span><span class="ss">c:</span> <span class="mi">42</span><span class="p">}}}</span>
<span class="o">...></span> <span class="o">|></span> <span class="no">Iteraptor</span><span class="o">.</span><span class="n">map_reduce</span><span class="p">([],</span> <span class="k">fn</span>
<span class="o">...></span> <span class="p">{</span><span class="n">k</span><span class="p">,</span> <span class="p">%{}</span> <span class="o">=</span> <span class="n">v</span><span class="p">},</span> <span class="n">acc</span> <span class="o">-></span> <span class="p">{</span><span class="err"></span><span class="p">{</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="p">},</span> <span class="p">[</span><span class="no">Enum</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">k</span><span class="p">,</span> <span class="s2">"."</span><span class="p">)</span> <span class="o">|</span> <span class="n">acc</span><span class="p">]}</span>
<span class="o">...></span> <span class="p">{</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="p">},</span> <span class="n">acc</span> <span class="o">-></span> <span class="p">{</span><span class="err"></span><span class="p">{</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="o">*</span> <span class="mi">2</span><span class="p">},</span> <span class="p">[</span><span class="no">Enum</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">k</span><span class="p">,</span> <span class="s2">"."</span><span class="p">)</span> <span class="o"><></span> <span class="s2">"="</span> <span class="o">|</span> <span class="n">acc</span><span class="p">]}</span>
<span class="o">...></span> <span class="k">end</span><span class="p">,</span> <span class="ss">yield:</span> <span class="ss">:all</span><span class="p">)</span>
<span class="p">{</span><span class="err"></span><span class="p">%{</span><span class="ss">a:</span> <span class="p">%{</span><span class="ss">b:</span> <span class="p">%{</span><span class="ss">c:</span> <span class="mi">42</span><span class="p">}}},</span> <span class="p">[</span><span class="s2">"a.b.c="</span><span class="p">,</span> <span class="s2">"a.b"</span><span class="p">,</span> <span class="s2">"a"</span><span class="p">]}</span>
<span class="c1"># filter</span>
<span class="n">iex</span><span class="o">></span> <span class="p">%{</span><span class="ss">a:</span> <span class="p">%{</span><span class="ss">b:</span> <span class="mi">42</span><span class="p">,</span> <span class="ss">e:</span> <span class="p">%{</span><span class="ss">f:</span> <span class="mf">3.14</span><span class="p">,</span> <span class="ss">c:</span> <span class="mi">42</span><span class="p">},</span> <span class="ss">d:</span> <span class="p">%{</span><span class="ss">c:</span> <span class="mi">42</span><span class="p">}},</span> <span class="ss">c:</span> <span class="mi">42</span><span class="p">,</span> <span class="ss">d:</span> <span class="mf">3.14</span><span class="p">}</span>
<span class="o">...></span> <span class="o">|></span> <span class="no">Iteraptor</span><span class="o">.</span><span class="n">filter</span><span class="p">(</span><span class="k">fn</span> <span class="p">{</span><span class="n">key</span><span class="p">,</span> <span class="n">_</span><span class="p">}</span> <span class="o">-></span> <span class="ss">:c</span> <span class="ow">in</span> <span class="n">key</span> <span class="k">end</span><span class="p">,</span> <span class="ss">yield:</span> <span class="ss">:none</span><span class="p">)</span>
<span class="p">%{</span><span class="ss">a:</span> <span class="p">%{</span><span class="ss">e:</span> <span class="p">%{</span><span class="ss">c:</span> <span class="mi">42</span><span class="p">},</span> <span class="ss">d:</span> <span class="p">%{</span><span class="ss">c:</span> <span class="mi">42</span><span class="p">}},</span> <span class="ss">c:</span> <span class="mi">42</span><span class="p">}</span>
</code></pre></div></div>
<h4 id="flattening">Flattening</h4>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">></span> <span class="p">%{</span><span class="ss">a:</span> <span class="p">%{</span><span class="ss">b:</span> <span class="p">%{</span><span class="ss">c:</span> <span class="mi">42</span><span class="p">,</span> <span class="ss">d:</span> <span class="p">[</span><span class="no">nil</span><span class="p">,</span> <span class="mi">42</span><span class="p">]},</span> <span class="ss">e:</span> <span class="p">[</span><span class="ss">:f</span><span class="p">,</span> <span class="mi">42</span><span class="p">]}}</span>
<span class="o">...></span> <span class="o">|></span> <span class="no">Iteraptor</span><span class="o">.</span><span class="n">to_flatmap</span><span class="p">(</span><span class="ss">delimiter:</span> <span class="s2">"_"</span><span class="p">)</span>
<span class="c1">#⇒ %{"a_b_c" => 42, "a_b_d_0" => nil, "a_b_d_1" => 42, "a_e_0" => :f, "a_e_1" => 42}</span>
<span class="n">iex</span><span class="o">></span> <span class="p">%{</span><span class="s2">"a.b.c"</span><span class="p">:</span> <span class="mi">42</span><span class="p">,</span> <span class="s2">"a.b.d.0"</span><span class="p">:</span> <span class="no">nil</span><span class="p">,</span> <span class="s2">"a.b.d.1"</span><span class="p">:</span> <span class="mi">42</span><span class="p">,</span> <span class="s2">"a.e.0"</span><span class="p">:</span> <span class="ss">:f</span><span class="p">,</span> <span class="s2">"a.e.1"</span><span class="p">:</span> <span class="mi">42</span><span class="p">}</span>
<span class="o">...></span> <span class="o">|></span> <span class="no">Iteraptor</span><span class="o">.</span><span class="n">from_flatmap</span>
<span class="c1">#⇒ %{a: %{b: %{c: 42, d: [nil, 42]}, e: [:f, 42]}}</span>
</code></pre></div></div>
<h4 id="extras">Extras</h4>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">></span> <span class="no">Iteraptor</span><span class="o">.</span><span class="no">Extras</span><span class="o">.</span><span class="n">bury</span><span class="p">([</span><span class="ss">foo:</span> <span class="ss">:bar</span><span class="p">],</span> <span class="sx">~w|a b c d|a</span><span class="p">,</span> <span class="mi">42</span><span class="p">)</span>
<span class="p">[</span><span class="ss">a:</span> <span class="p">[</span><span class="ss">b:</span> <span class="p">[</span><span class="ss">c:</span> <span class="p">[</span><span class="ss">d:</span> <span class="mi">42</span><span class="p">]]],</span> <span class="ss">foo:</span> <span class="ss">:bar</span><span class="p">]</span>
</code></pre></div></div>
<h3 id="in-details">In Details</h3>
<h4 id="iterating">Iterating</h4>
<p><strong><code class="language-plaintext highlighter-rouge">Iteraptor.each(term, fun/1, opts)</code></strong> — iterates the nested structure, yielding
the key and value. The returned from the function value is discarded.</p>
<ul>
<li><em>function argument:</em> <strong><code class="language-plaintext highlighter-rouge">{key, value}</code></strong> tuple</li>
<li><em>options</em>: <strong><code class="language-plaintext highlighter-rouge">yield: [:all, :maps, :lists, :none]</code></strong>, <code class="language-plaintext highlighter-rouge">:none</code> is the default</li>
<li><em>return value</em>: <strong><code class="language-plaintext highlighter-rouge">self</code></strong></li>
</ul>
<h4 id="mapping-and-reducing">Mapping and Reducing</h4>
<p><strong><code class="language-plaintext highlighter-rouge">Iteraptor.map(term, fun/1, opts)</code></strong> — iterates the nested structure,
yielding the key and value. The value, returned from the block
should be either a single value or a <code class="language-plaintext highlighter-rouge">{key, value}</code> tuple.</p>
<ul>
<li><em>function argument:</em> <strong><code class="language-plaintext highlighter-rouge">{key, value}</code></strong> tuple</li>
<li><em>options</em>: <strong><code class="language-plaintext highlighter-rouge">yield: [:all, :maps, :lists, :none]</code></strong>, <code class="language-plaintext highlighter-rouge">:none</code> is the default</li>
<li><em>return value</em>: <strong><code class="language-plaintext highlighter-rouge">mapped</code></strong></li>
</ul>
<p><strong><code class="language-plaintext highlighter-rouge">Iteraptor.reduce(term, fun/2, opts)</code></strong> — iterates the nested structure,
yielding the key and value. The value, returned from the block
should be an accumulator value.</p>
<ul>
<li><em>function arguments:</em> <strong><code class="language-plaintext highlighter-rouge">{key, value}, acc</code></strong> pair</li>
<li><em>options</em>: <strong><code class="language-plaintext highlighter-rouge">yield: [:all, :maps, :lists, :none]</code></strong>, <code class="language-plaintext highlighter-rouge">:none</code> is the default</li>
<li><em>return value</em>: <strong><code class="language-plaintext highlighter-rouge">accumulator</code></strong></li>
</ul>
<p><strong><code class="language-plaintext highlighter-rouge">Iteraptor.map_reduce(term, fun/2, opts)</code></strong> — iterates the nested structure,
yielding the key and value. The value, returned from the block
should be a <code class="language-plaintext highlighter-rouge">{{key, value}, acc}</code> value. The first element of this tuple is
used for mapping, the last—accumulating the result.</p>
<ul>
<li><em>function arguments:</em> <strong><code class="language-plaintext highlighter-rouge">{key, value}, acc</code></strong> pair</li>
<li><em>options</em>: <strong><code class="language-plaintext highlighter-rouge">yield: [:all, :maps, :lists, :none]</code></strong>, <code class="language-plaintext highlighter-rouge">:none</code> is the default</li>
<li><em>return value</em>: <strong><code class="language-plaintext highlighter-rouge">{mapped, accumulator}</code></strong> tuple</li>
</ul>
<h4 id="filtering">Filtering</h4>
<p><strong><code class="language-plaintext highlighter-rouge">Iteraptor.filter(term, filter/1, opts)</code></strong> — filters the structure
according to the value returned from each iteration (<code class="language-plaintext highlighter-rouge">true</code> to leave
the element, <code class="language-plaintext highlighter-rouge">false</code> to discard.)</p>
<ul>
<li><em>function argument:</em> <strong><code class="language-plaintext highlighter-rouge">{key, value}</code></strong> tuple</li>
<li><em>options</em>: <strong><code class="language-plaintext highlighter-rouge">yield: [:all, :maps, :lists, :none]</code></strong>, <code class="language-plaintext highlighter-rouge">:none</code> is the default</li>
<li><em>return value</em>: <strong><code class="language-plaintext highlighter-rouge">filtered</code></strong></li>
</ul>
<h4 id="flattening-1">Flattening</h4>
<p><strong><code class="language-plaintext highlighter-rouge">Iteraptor.to_flatmap(term, opts)</code></strong> — flattens the structure into
the flatten map/keyword, concatenating keys with a delimiter.</p>
<ul>
<li><em>options</em>: <strong><code class="language-plaintext highlighter-rouge">delimiter: binary(), into: term()</code></strong>,
defaults: <code class="language-plaintext highlighter-rouge">delimiter: ".", into: %{}</code></li>
<li><em>return value</em>: <strong><code class="language-plaintext highlighter-rouge">flattened</code></strong></li>
</ul>
<p><strong><code class="language-plaintext highlighter-rouge">Iteraptor.from_flatmap(term, fun/1, opts)</code></strong> — de-flattens the structure from
the flattened map/keyword, splitting keys by a delimiter. An optional transformer
function might be called after the value is deflattened.</p>
<ul>
<li><em>function argument:</em> <strong><code class="language-plaintext highlighter-rouge">{key, value}</code></strong> tuple</li>
<li><em>options</em>: <strong><code class="language-plaintext highlighter-rouge">delimiter: binary(), into: term()</code></strong>,
defaults: <code class="language-plaintext highlighter-rouge">delimiter: ".", into: %{}</code></li>
<li><em>return value</em>: <strong><code class="language-plaintext highlighter-rouge">Map.t | Keyword.t | List.t</code></strong></li>
</ul>
<hr />
<p>The source code is linked above, the package is available through
<a href="https://hex.pm/packages/iteraptor"><code class="language-plaintext highlighter-rouge">hex</code></a>.</p>
Iteraptor :: Iterating Nested Terms Like I’m Five2018-03-29T00:00:00+00:00https://rocket-science.ru/hacking/2018/03/29/iteraptor-for-the-rescue<p>Iterating both hashes and arrays in ruby is charming. One might chain iterators,
map, reduce, filter, select, reject, zip… Everybody having at least eight
hours of experience with ruby has definitely seen (and even maybe written)
something like this:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="sx">%w[aleksei saverio]</span><span class="p">.</span><span class="nf">map</span> <span class="k">do</span> <span class="o">|</span><span class="nb">name</span><span class="o">|</span>
<span class="nb">name</span><span class="p">.</span><span class="nf">capitalize</span>
<span class="k">end</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">capitalized_name</span><span class="o">|</span>
<span class="nb">puts</span> <span class="s2">"Hello, </span><span class="si">#{</span><span class="n">capitalized_name</span><span class="si">}</span><span class="s2">!"</span>
<span class="k">end</span>
</code></pre></div></div>
<p>That is really handy. The things gets cumbersome when it comes to deeply nested
structures, like a hash having nested hashes, arrays etc. The good example of
that would be any configuration file, loaded from YAML.</p>
<p>So, welcome the library that makes the iteration of any nested hash/array
combination almost as easy as the natural ruby <code class="language-plaintext highlighter-rouge">map</code> and <code class="language-plaintext highlighter-rouge">each</code>.</p>
<p>• <a href="https://github.com/am-kantox/iteraptor"><strong>Iteraptor</strong></a></p>
<h3 id="intro">Intro</h3>
<p>Since the library monkeypatches core classes, it uses spanish names for
iteration methods. There is a plan to make it better memorizable, see more
about it in the end of this post.</p>
<h3 id="features">Features</h3>
<ul>
<li><code class="language-plaintext highlighter-rouge">cada</code> (<em>sp.</em> <code class="language-plaintext highlighter-rouge">each</code>) iterates through all the levels of the nested <code class="language-plaintext highlighter-rouge">Enumerable</code>,
yielding <code class="language-plaintext highlighter-rouge">parent, element</code> tuple; parent is returned as a delimiter-joined string</li>
<li><code class="language-plaintext highlighter-rouge">mapa</code> (<em>sp.</em> <code class="language-plaintext highlighter-rouge">map</code>) iterates all the elements, yielding <code class="language-plaintext highlighter-rouge">parent, (key, value)</code>;
the mapper should return either <code class="language-plaintext highlighter-rouge">[key, value]</code> array or <code class="language-plaintext highlighter-rouge">nil</code> to remove this
element;
<ul>
<li><em>NB</em> this method always maps to <code class="language-plaintext highlighter-rouge">Hash</code>, to map to <code class="language-plaintext highlighter-rouge">Array</code> use <code class="language-plaintext highlighter-rouge">plana_mapa</code></li>
<li><em>NB</em> this method will raise if the returned value is neither <code class="language-plaintext highlighter-rouge">[key, value]</code> tuple nor <code class="language-plaintext highlighter-rouge">nil</code></li>
</ul>
</li>
<li><code class="language-plaintext highlighter-rouge">plana_mapa</code> iterates yielding <code class="language-plaintext highlighter-rouge">key, value</code>, maps to the yielded value,
whatever it is; <code class="language-plaintext highlighter-rouge">nil</code>s are not treated in some special way</li>
<li><code class="language-plaintext highlighter-rouge">aplanar</code> (<em>sp.</em> <code class="language-plaintext highlighter-rouge">flatten</code>) the analogue of <code class="language-plaintext highlighter-rouge">Array#flatten</code>, but flattens
the deep enumerable into <code class="language-plaintext highlighter-rouge">Hash</code> instance</li>
<li><code class="language-plaintext highlighter-rouge">recoger</code> (<em>sp.</em> <code class="language-plaintext highlighter-rouge">harvest</code>, <code class="language-plaintext highlighter-rouge">collect</code>) the opposite to <code class="language-plaintext highlighter-rouge">aplanar</code>, it builds
the nested structure out of flattened hash</li>
<li><code class="language-plaintext highlighter-rouge">segar</code> (<em>sp.</em> <code class="language-plaintext highlighter-rouge">yield</code>), alias <code class="language-plaintext highlighter-rouge">escoger</code> (<em>sp.</em> <code class="language-plaintext highlighter-rouge">select</code>) allows to filter
and collect elelements</li>
<li><code class="language-plaintext highlighter-rouge">rechazar</code> (<em>sp.</em> <code class="language-plaintext highlighter-rouge">reject</code>) allows to filter out and collect elelements</li>
<li><code class="language-plaintext highlighter-rouge">compactar</code> (<em>sp.</em> <code class="language-plaintext highlighter-rouge">compact</code>), allows to filter out all <code class="language-plaintext highlighter-rouge">nil</code>s.</li>
</ul>
<h3 id="words-are-cheap-show-me-the-code">Words are cheap, show me the code</h3>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">▶</span> <span class="nb">require</span> <span class="s1">'iteraptor'</span>
<span class="c1">#⇒ true</span>
<span class="err">▶</span> <span class="nb">hash</span> <span class="o">=</span> <span class="p">{</span><span class="ss">company: </span><span class="p">{</span><span class="ss">name: </span><span class="s2">"Me"</span><span class="p">,</span> <span class="ss">currencies: </span><span class="p">[</span><span class="s2">"A"</span><span class="p">,</span> <span class="s2">"B"</span><span class="p">,</span> <span class="s2">"C"</span><span class="p">],</span>
<span class="err">▷</span> <span class="ss">password: </span><span class="s2">"12345678"</span><span class="p">,</span>
<span class="err">▷</span> <span class="ss">details: </span><span class="p">{</span><span class="ss">another_password: </span><span class="s2">"QWERTYUI"</span><span class="p">}}}</span>
<span class="c1">#⇒ {:company=>{:name=>"Me", :currencies=>["A", "B", "C"],</span>
<span class="c1"># :password=>"12345678",</span>
<span class="c1"># :details=>{:another_password=>"QWERTYUI"}}}</span>
<span class="err">▶</span> <span class="nb">hash</span><span class="p">.</span><span class="nf">segar</span><span class="p">(</span><span class="sr">/password/i</span><span class="p">)</span> <span class="p">{</span> <span class="s2">"*"</span> <span class="o">*</span> <span class="mi">8</span> <span class="p">}</span>
<span class="c1">#⇒ {"company"=>{"password"=>"********",</span>
<span class="c1"># "details"=>{"another_password"=>"********"}}}</span>
<span class="err">▶</span> <span class="nb">hash</span><span class="p">.</span><span class="nf">segar</span><span class="p">(</span><span class="sr">/password/i</span><span class="p">)</span> <span class="p">{</span> <span class="o">|*</span><span class="n">args</span><span class="o">|</span> <span class="nb">puts</span> <span class="n">args</span><span class="p">.</span><span class="nf">inspect</span> <span class="p">}</span>
<span class="p">[</span><span class="s2">"company.password"</span><span class="p">,</span> <span class="s2">"12345678"</span><span class="p">]</span>
<span class="p">[</span><span class="s2">"company.details.another_password"</span><span class="p">,</span> <span class="s2">"QWERTYUI"</span><span class="p">]</span>
<span class="c1">#⇒ {"company"=>{"password"=>nil, "details"=>{"another_password"=>nil}}}</span>
<span class="err">▶</span> <span class="nb">hash</span><span class="p">.</span><span class="nf">rechazar</span><span class="p">(</span><span class="sr">/password/</span><span class="p">)</span>
<span class="c1">#⇒ {"company"=>{"name"=>"Me", "currencies"=>["A", "B", "C"]}}</span>
<span class="err">▶</span> <span class="nb">hash</span><span class="p">.</span><span class="nf">aplanar</span>
<span class="c1">#⇒ {"company.name"=>"Me",</span>
<span class="c1"># "company.currencies.0"=>"A",</span>
<span class="c1"># "company.currencies.1"=>"B",</span>
<span class="c1"># "company.currencies.2"=>"C",</span>
<span class="c1"># "company.password"=>"12345678",</span>
<span class="c1"># "company.details.another_password"=>"QWERTYUI"}</span>
<span class="err">▶</span> <span class="nb">hash</span><span class="p">.</span><span class="nf">aplanar</span><span class="p">.</span><span class="nf">recoger</span>
<span class="c1">#⇒ {"company"=>{"name"=>"Me", "currencies"=>["A", "B", "C"],</span>
<span class="c1"># "password"=>"12345678",</span>
<span class="c1"># "details"=>{"another_password"=>"QWERTYUI"}}}</span>
<span class="err">▶</span> <span class="nb">hash</span><span class="p">.</span><span class="nf">aplanar</span><span class="p">.</span><span class="nf">recoger</span><span class="p">(</span><span class="ss">symbolize_keys: </span><span class="kp">true</span><span class="p">)</span>
<span class="c1">#⇒ {:company=>{:name=>"Me", :currencies=>["A", "B", "C"],</span>
<span class="c1"># :password=>"12345678",</span>
<span class="c1"># :details=>{:another_password=>"QWERTYUI"}}}</span>
</code></pre></div></div>
<h3 id="in-details">In Details</h3>
<h4 id="simple-iterating">Simple Iterating</h4>
<p><strong><code class="language-plaintext highlighter-rouge">Iteraptor#cada(**params, &λ)</code></strong> — iterates the nested structure, yielding
the keys (concatenated with <code class="language-plaintext highlighter-rouge">Iteraptor::DELIMITER</code> or whatever is passed
as <code class="language-plaintext highlighter-rouge">delimiter</code> keyword argument.) The returned from the block value is discarded.</p>
<p><em>block arguments:</em> <strong><code class="language-plaintext highlighter-rouge">key, value</code></strong></p>
<p><em>Example:</em></p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">▶</span> <span class="p">{</span><span class="ss">foo1: </span><span class="mi">42</span><span class="p">,</span> <span class="ss">foo2: </span><span class="sx">%i[bar1 bar2]</span><span class="p">,</span> <span class="ss">foo3: </span><span class="p">{</span><span class="ss">foo4: </span><span class="p">{</span><span class="ss">foo5: </span><span class="mf">3.14</span><span class="p">,</span> <span class="ss">foo6: :baz</span><span class="p">}}}</span><span class="o">.</span>
<span class="err">▷</span> <span class="p">.</span><span class="nf">cada</span> <span class="p">{</span> <span class="o">|</span><span class="n">key</span><span class="p">,</span> <span class="n">value</span><span class="o">|</span> <span class="nb">puts</span> <span class="p">[</span><span class="n">key</span><span class="p">,</span> <span class="n">value</span><span class="p">].</span><span class="nf">inspect</span> <span class="p">}</span>
<span class="c1"># ["foo1", 42]</span>
<span class="c1"># ["foo2", [:bar1, :bar2]]</span>
<span class="c1"># ["foo2.0", :bar1]</span>
<span class="c1"># ["foo2.1", :bar2]</span>
<span class="c1"># ["foo3", {:foo4=>{:foo5=>3.14, :foo6=>:baz}}]</span>
<span class="c1"># ["foo3.foo4", {:foo5=>3.14, :foo6=>:baz}]</span>
<span class="c1"># ["foo3.foo4.foo5", 3.14]</span>
<span class="c1"># ["foo3.foo4.foo6", :baz]</span>
<span class="c1">#⇒ {:foo1=>42, :foo2=>[:bar1, :bar2], :foo3=>{:foo4=>{:foo5=>3.14, :foo6=>:baz}}}</span>
</code></pre></div></div>
<h4 id="simple-mapping">Simple Mapping</h4>
<p><strong><code class="language-plaintext highlighter-rouge">Iteraptor#mapa(**params, &λ)</code></strong> — iterates the nested structure,
yielding the parent key, key and value. The value, returned from the block
should be a single value (while iterating through arrays, <code class="language-plaintext highlighter-rouge">value</code> block
argument is <code class="language-plaintext highlighter-rouge">nil</code>,) of either <code class="language-plaintext highlighter-rouge">[key, value]</code> tuple or <code class="language-plaintext highlighter-rouge">nil</code> while
iterating over hashes. In the latter case if <code class="language-plaintext highlighter-rouge">nil</code> is returned, the resulting
value is removed from the result (<em>NB:</em> this behaviour might change.)</p>
<p><em>block arguments:</em> <strong><code class="language-plaintext highlighter-rouge">parent, (key, value)</code></strong></p>
<p><em>Example:</em></p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">▶</span> <span class="p">{</span><span class="ss">foo1: </span><span class="mi">42</span><span class="p">,</span> <span class="ss">foo2: </span><span class="sx">%i[bar1 bar2]</span><span class="p">,</span> <span class="ss">foo3: </span><span class="p">{</span><span class="ss">foo4: </span><span class="p">{</span><span class="ss">foo5: </span><span class="mf">3.14</span><span class="p">,</span> <span class="ss">foo6: :baz</span><span class="p">}}}</span><span class="o">.</span>
<span class="err">▷</span> <span class="n">mapa</span> <span class="p">{</span> <span class="o">|</span><span class="n">parent</span><span class="p">,</span> <span class="p">(</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="p">)</span><span class="o">|</span> <span class="nb">puts</span> <span class="n">parent</span><span class="p">;</span> <span class="n">v</span> <span class="p">?</span> <span class="p">[</span><span class="n">k</span><span class="p">,</span> <span class="s2">"==</span><span class="si">#{</span><span class="n">v</span><span class="si">}</span><span class="s2">=="</span><span class="p">]</span> <span class="p">:</span> <span class="n">k</span> <span class="p">}</span>
<span class="c1"># foo1</span>
<span class="c1"># foo2.0</span>
<span class="c1"># foo2.1</span>
<span class="c1"># foo3.foo4.foo5</span>
<span class="c1"># foo3.foo4.foo6</span>
<span class="c1">#⇒ {:foo1=>"==42==", :foo2=>[:bar1, :bar2],</span>
<span class="c1"># :foo3=>{:foo4=>{:foo5=>"==3.14==", :foo6=>"==baz=="}}}</span>
</code></pre></div></div>
<h4 id="filtering">Filtering</h4>
<p><strong><code class="language-plaintext highlighter-rouge">Iteraptor#escoger(*filters, **params, &λ)</code></strong> — filters the receiver
according to the set of filters given (filters use case-equality) and,
optionally, iterates the resulting structure if the block was given. Might
be treated an extended analogue of <code class="language-plaintext highlighter-rouge">Enumerable#select</code>.</p>
<p><em>alias:</em> <strong><code class="language-plaintext highlighter-rouge">segar</code></strong>, <em>block arguments:</em> <strong><code class="language-plaintext highlighter-rouge">key, value</code></strong></p>
<p><em>Example:</em></p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">▶</span> <span class="p">{</span><span class="ss">foo1: </span><span class="mi">42</span><span class="p">,</span> <span class="ss">foo2: </span><span class="sx">%i[bar1 bar2]</span><span class="p">,</span> <span class="ss">foo3: </span><span class="p">{</span><span class="ss">foo4: </span><span class="p">{</span><span class="ss">foo5: </span><span class="mf">3.14</span><span class="p">,</span> <span class="ss">foo6: :baz</span><span class="p">}}}</span><span class="o">.</span>
<span class="err">▷</span> <span class="n">escoger</span><span class="p">(</span><span class="sr">/foo4/</span><span class="p">)</span>
<span class="c1">#⇒ {"foo3"=>{"foo4"=>{"foo5"=>3.14, "foo6"=>:baz}}}</span>
<span class="err">▷</span> <span class="n">escoger</span><span class="p">(</span><span class="o">-></span><span class="p">(</span><span class="n">k</span><span class="p">)</span> <span class="p">{</span> <span class="n">k</span> <span class="o">==</span> <span class="s2">"foo3"</span> <span class="p">})</span>
<span class="c1">#⇒ {"foo3"=>{"foo4"=>{"foo5"=>3.14, "foo6"=>:baz}}}</span>
<span class="err">▶</span> <span class="p">{</span><span class="ss">foo1: </span><span class="mi">42</span><span class="p">,</span> <span class="ss">foo2: </span><span class="sx">%i[bar1 bar2]</span><span class="p">,</span> <span class="ss">foo3: </span><span class="p">{</span><span class="ss">foo4: </span><span class="p">{</span><span class="ss">foo5: </span><span class="mf">3.14</span><span class="p">,</span> <span class="ss">foo6: :baz</span><span class="p">}}}</span><span class="o">.</span>
<span class="err">▷</span> <span class="n">segar</span><span class="p">(</span><span class="sr">/foo[16]/</span><span class="p">)</span> <span class="p">{</span> <span class="o">|</span><span class="n">key</span><span class="p">,</span> <span class="n">value</span><span class="o">|</span> <span class="mf">3.14</span> <span class="p">}</span>
<span class="c1">#⇒ {"foo1"=>3.14, "foo3"=>{"foo4"=>{"foo6"=>3.14}}}</span>
</code></pre></div></div>
<p><strong><code class="language-plaintext highlighter-rouge">Iteraptor#rechazar(*filters, **params, &λ)</code></strong> — the exactly opposite to
<code class="language-plaintext highlighter-rouge">Iteraptor#escoger</code>. Might be treated an extended analogue of <code class="language-plaintext highlighter-rouge">Enumerable#select</code>.</p>
<p><em>Example:</em></p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">▶</span> <span class="p">{</span><span class="ss">foo1: </span><span class="mi">42</span><span class="p">,</span> <span class="ss">foo2: </span><span class="sx">%i[bar1 bar2]</span><span class="p">,</span> <span class="ss">foo3: </span><span class="p">{</span><span class="ss">foo4: </span><span class="p">{</span><span class="ss">foo5: </span><span class="mf">3.14</span><span class="p">,</span> <span class="ss">foo6: :baz</span><span class="p">}}}</span><span class="o">.</span>
<span class="err">▷</span> <span class="n">rechazar</span><span class="p">(</span><span class="sr">/[15]/</span><span class="p">)</span>
<span class="c1">#⇒ {"foo2"=>[:bar1], "foo3"=>{"foo4"=>{"foo6"=>:baz}}}</span>
</code></pre></div></div>
<p><strong><code class="language-plaintext highlighter-rouge">Iteraptor#compactar(**params)</code></strong> — the analogue of <code class="language-plaintext highlighter-rouge">Array#compact</code> to some
extent. <code class="language-plaintext highlighter-rouge">Iteraptor#compactar</code> removes all the deeply nested keys having <code class="language-plaintext highlighter-rouge">nil</code> value.</p>
<p><em>Example:</em></p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">▶</span> <span class="p">{</span><span class="ss">foo1: </span><span class="kp">nil</span><span class="p">,</span> <span class="ss">foo2: </span><span class="sx">%i[nil bar2]</span><span class="p">,</span> <span class="ss">foo3: </span><span class="p">{</span><span class="ss">foo4: </span><span class="p">{</span><span class="ss">foo5: </span><span class="kp">nil</span><span class="p">,</span> <span class="ss">foo6: :baz</span><span class="p">}}}</span><span class="o">.</span>
<span class="err">▷</span> <span class="n">compactar</span><span class="p">()</span>
<span class="c1">#⇒ {"foo2"=>[:bar2], "foo3"=>{"foo4"=>{"foo6"=>:baz}}}</span>
</code></pre></div></div>
<h4 id="flattening">Flattening</h4>
<p><strong><code class="language-plaintext highlighter-rouge">Iteraptor#aplanar(**params, &λ)</code></strong> — flattens the receiver, concatenating
keys with <code class="language-plaintext highlighter-rouge">Iteraptor::DELIMITER</code> or whatever is passed as <code class="language-plaintext highlighter-rouge">delimiter</code> keyword
argument. Might be treated an extended analogue of <code class="language-plaintext highlighter-rouge">Enumerable#flatten</code>.
If block passed, <code class="language-plaintext highlighter-rouge">key, value</code> pair is yielded to it. The returned value
is discarded.</p>
<p><em>block arguments:</em> <strong><code class="language-plaintext highlighter-rouge">key, value</code></strong></p>
<p><em>Example:</em></p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">▶</span> <span class="p">{</span><span class="ss">foo1: </span><span class="mi">42</span><span class="p">,</span> <span class="ss">foo2: </span><span class="sx">%i[bar1 bar2]</span><span class="p">,</span> <span class="ss">foo3: </span><span class="p">{</span><span class="ss">foo4: </span><span class="p">{</span><span class="ss">foo5: </span><span class="mf">3.14</span><span class="p">,</span> <span class="ss">foo6: :baz</span><span class="p">}}}</span><span class="o">.</span>
<span class="err">▷</span> <span class="n">aplanar</span><span class="p">(</span><span class="ss">delimiter: </span><span class="s2">"_"</span><span class="p">)</span>
<span class="c1">#⇒ {"foo1"=>42, "foo2_0"=>:bar1, "foo2_1"=>:bar2,</span>
<span class="c1"># "foo3_foo4_foo5"=>3.14, "foo3_foo4_foo6"=>:baz}</span>
</code></pre></div></div>
<p><strong><code class="language-plaintext highlighter-rouge">Iteraptor#recoger(**params)</code></strong> — de-flattens the receiver, building
the nested structure back. Knows now to deal with arrays.</p>
<p><em>Example:</em></p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">▶</span> <span class="p">{</span><span class="s2">"foo1"</span><span class="o">=></span><span class="mi">42</span><span class="p">,</span> <span class="s2">"foo2_0"</span><span class="o">=></span><span class="ss">:bar1</span><span class="p">,</span> <span class="s2">"foo2_1"</span><span class="o">=></span><span class="ss">:bar2</span><span class="p">,</span> <span class="s2">"foo3_foo4_foo5"</span><span class="o">=></span><span class="mf">3.14</span><span class="p">,</span>
<span class="err">▷</span> <span class="s2">"foo3_foo4_foo6"</span><span class="o">=></span><span class="ss">:baz</span><span class="p">}.</span><span class="nf">recoger</span><span class="p">(</span><span class="ss">delimiter: </span><span class="s2">"_"</span><span class="p">,</span> <span class="ss">symbolize_keys: </span><span class="kp">true</span><span class="p">)</span>
<span class="c1">#⇒ {:foo1=>42, :foo2=>[:bar1, :bar2], :foo3=>{:foo4=>{:foo5=>3.14, :foo6=>:baz}}}</span>
</code></pre></div></div>
<hr />
<p>The source code is linked above, the <code class="language-plaintext highlighter-rouge">gem</code> is available through rubygems.</p>
Elixir Pipeline Operators2018-03-16T00:00:00+00:00https://rocket-science.ru/hacking/2018/03/16/elixir-pipeline-operators<p>Yesterday I was answering <a href="https://stackoverflow.com/a/49314637/2035262">this SO question</a>,
roughly asking for how to use an uncommon syntax in pipe operator.</p>
<p>“Man, that’s Elixir,” was my very first response. “We have macros for that.”
The pitfall here is: while we can restrict exporting the standard
<a href="https://hexdocs.pm/elixir/Kernel.html#%7C%3E/2"><code class="language-plaintext highlighter-rouge">Kernel.|>/2</code></a> and overload it,
this approach does not sound as a good proposal since we’ll lose all the nifty
default functionality. Of course, one might check the right operand and if it’s
one of those expected by standard pipe, fallback to default <code class="language-plaintext highlighter-rouge">Kernel.|>(right)</code>,
but this is not nifty at all.</p>
<p>So, I grepped Elixir source code for <em>“pipeline”</em> keyword and <a href="https://github.com/elixir-lang/elixir/blob/master/lib/elixir/lib/code/formatter.ex#L22">found this</a>:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">@pipeline_operators</span> <span class="p">[</span><span class="ss">:|</span><span class="o">></span><span class="p">,</span> <span class="ss">:~</span><span class="o">>></span><span class="p">,</span> <span class="ss">:<</span><span class="o"><</span><span class="err">~</span><span class="p">,</span> <span class="ss">:~</span><span class="o">></span><span class="p">,</span> <span class="ss">:<</span><span class="err">~</span><span class="p">,</span> <span class="ss">:<</span><span class="err">~</span><span class="o">></span><span class="p">,</span> <span class="ss">:<</span><span class="o">|></span><span class="p">]</span>
</code></pre></div></div>
<p>I was unable to <em>duckduckgo</em> anything related to what are permitted pipe
operators in Elixir, and I even am not sure this behaviour being an
implementation detail or is it guaranteed to remain the same, but I wrote
some checks and all those are working pretty fine as pipe operators in all
the modern versions of Elixir I have on hand.</p>
<p>I am going to show how to use the custom pipe operator in your codebase
to make the code looking sexier, in a case you have a colleague to impress.
Let’s stay with the question as it was stated on SO:</p>
<blockquote>
<p>There is a need to update the struct without breaking the pipeline up.
Something like this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>my_struct
|> %{ | my_field_in_struct: a_new_value}
|> my_funct1
|> %{ | my_field_in_struct: a_new_value}
|> my_funct2
|> %{ | my_field_in_struct: a_new_value}
|> my_funct3
</code></pre></div> </div>
</blockquote>
<p>Let’s start with introducing our own pipe operator implementation:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">StructPiper</span> <span class="k">do</span>
<span class="k">defmacro</span> <span class="n">__using__</span><span class="p">(</span><span class="n">_</span><span class="p">)</span> <span class="k">do</span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="kn">require</span> <span class="kn">unquote</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">)</span>
<span class="kn">import</span> <span class="kn">unquote</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="c1"># TODO: implement this</span>
<span class="k">defmacro</span> <span class="n">left</span> <span class="err">~</span><span class="o">>></span> <span class="n">right</span> <span class="k">do</span>
<span class="no">IO</span><span class="o">.</span><span class="n">inspect</span> <span class="p">{</span><span class="n">left</span><span class="p">,</span> <span class="n">right</span><span class="p">},</span> <span class="ss">label:</span> <span class="s2">"DEBUG"</span>
<span class="ss">:ok</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The code above might be used like this:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">MyStruct</span> <span class="k">do</span>
<span class="k">defstruct</span> <span class="sx">~w|foo bar baz|a</span>
<span class="k">end</span>
<span class="k">defmodule</span> <span class="no">StructPiper</span><span class="o">.</span><span class="no">Test</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">StructPiper</span>
<span class="k">def</span> <span class="n">test</span> <span class="k">do</span>
<span class="p">%</span><span class="no">MyStruct</span><span class="p">{</span><span class="ss">foo:</span> <span class="mi">42</span><span class="p">}</span>
<span class="o">|></span> <span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="p">(</span><span class="ss">label:</span> <span class="s2">"1"</span><span class="p">)</span>
<span class="err">~</span><span class="o">>></span> <span class="p">[</span><span class="ss">bar:</span> <span class="mf">3.14</span><span class="p">]</span>
<span class="o">|></span> <span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="p">(</span><span class="ss">label:</span> <span class="s2">"2"</span><span class="p">)</span>
<span class="err">~</span><span class="o">>></span> <span class="p">[</span><span class="ss">baz:</span> <span class="s2">"FOOBAR"</span><span class="p">]</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="p">(</span><span class="no">StructPiper</span><span class="o">.</span><span class="no">Test</span><span class="o">.</span><span class="n">test</span> <span class="o">==</span>
<span class="p">%</span><span class="no">MyStruct</span><span class="p">{</span><span class="ss">bar:</span> <span class="mf">3.14</span><span class="p">,</span> <span class="ss">baz:</span> <span class="s2">"FOOBAR"</span><span class="p">,</span> <span class="ss">foo:</span> <span class="mi">42</span><span class="p">},</span> <span class="ss">label:</span> <span class="s2">"Test"</span><span class="p">)</span>
</code></pre></div></div>
<p>Let’s run the test to check what do we have there:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">|</span><span class="mi">1</span> <span class="err">▶</span> <span class="n">m</span> <span class="o">=</span> <span class="p">%</span><span class="no">MyStruct</span><span class="p">{</span><span class="ss">foo:</span> <span class="mi">42</span><span class="p">}</span>
<span class="n">iex</span><span class="o">|</span><span class="mi">2</span> <span class="err">▶</span> <span class="n">m</span> <span class="err">~</span><span class="o">>></span> <span class="p">[</span><span class="ss">bar:</span> <span class="mf">3.14</span><span class="p">]</span>
<span class="c1">#⇒ DEBUG: {{:m, [line: 24], nil}, [bar: 3.14]}</span>
</code></pre></div></div>
<p>OK, we receive <code class="language-plaintext highlighter-rouge">left</code> and <code class="language-plaintext highlighter-rouge">right</code> operands as expected. The only thing we need
would be to transform them to the AST for <code class="language-plaintext highlighter-rouge">%{left | right}</code>. Let’s check
how the latter looks like:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">|</span><span class="mi">3</span> <span class="err">▶</span> <span class="kn">quote</span> <span class="k">do</span><span class="p">:</span> <span class="p">%{</span><span class="n">m</span> <span class="o">|</span> <span class="ss">bar:</span> <span class="mf">3.14</span><span class="p">}</span>
<span class="p">{</span><span class="ss">:%{}</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[{</span><span class="ss">:|</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[{</span><span class="ss">:m</span><span class="p">,</span> <span class="p">[],</span> <span class="no">Elixir</span><span class="p">},</span> <span class="p">[</span><span class="ss">bar:</span> <span class="mf">3.14</span><span class="p">]]}]}</span>
</code></pre></div></div>
<p>Check the last part of the AST related to <code class="language-plaintext highlighter-rouge">:|</code> operator in this map/struct
updating syntax: arguments are <em>exactly</em> the same as we got in the call to
our pipe macro. That said, in this particular case it would be easier not to
tackle with quoting/unquoting and return the AST out of the box:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmacro</span> <span class="n">left</span> <span class="err">~</span><span class="o">>></span> <span class="n">right</span> <span class="k">do</span>
<span class="p">{</span><span class="ss">:%{}</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[{</span><span class="ss">:|</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[</span><span class="n">left</span><span class="p">,</span> <span class="n">right</span><span class="p">]}]}</span>
<span class="k">end</span>
</code></pre></div></div>
<p>We’re all set! Below is the full working implementation of the pipe operator
macro <code class="language-plaintext highlighter-rouge">:~>></code>:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">StructPiper</span> <span class="k">do</span>
<span class="k">defmacro</span> <span class="n">__using__</span><span class="p">(</span><span class="n">_</span><span class="p">)</span> <span class="k">do</span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="kn">require</span> <span class="kn">unquote</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">)</span>
<span class="kn">import</span> <span class="kn">unquote</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">defmacro</span> <span class="n">left</span> <span class="err">~</span><span class="o">>></span> <span class="n">right</span> <span class="k">do</span>
<span class="p">{</span><span class="ss">:%{}</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[{</span><span class="ss">:|</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[</span><span class="n">left</span><span class="p">,</span> <span class="n">right</span><span class="p">]}]}</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>If you don’t like this particular pipe operator, feel free to pick another
from the list this post starts with.</p>
Beware of YAGNI2018-03-06T00:00:00+00:00https://rocket-science.ru/tips/2018/03/06/beware-of-yagni<blockquote>
<p>“You aren’t gonna need it” (<em>acronym:</em> YAGNI) is a principle of extreme
programming (XP) that states a programmer should not add functionality
until deemed necessary.</p>
<p>XP co-founder Ron Jeffries has written: “Always implement things when you
actually need them, never when you just foresee that you need them.”</p>
<p><small> <a href="https://en.wikipedia.org/wiki/You_aren't_gonna_need_it">You ain’t gonna need it @Wiki</a></small></p>
</blockquote>
<p>This is the most misinterpreted and hence awful, hazardous, and detrimental
principle ever stated in the Book of the Wrong Advises for Developers.</p>
<p>As far as I can tell, originally it meant “Don’t install a fifth wheel on your
car unless you are participating the 5-wheeled-cars-rally.” Nowadays it’s widely
used as an excuse to reject doing their job from the lazy and arrogant
developers. I rarely agree with Martin Fowler, but this is a brilliant wording
explaining the issue:</p>
<blockquote>
<p>Yagni only applies to capabilities built into the software to support
a presumptive feature, it does not apply to effort to make the
software easier to modify.</p>
<p><small> <a href="https://www.martinfowler.com/bliki/Yagni.html">Yagni @MartinFowler</a></small></p>
</blockquote>
<p>This principle works well when a Smart Senior Developer does a code review for
an ambitious young Junior, who wants to conquer the world with their first PR.
Being given a task to implement a flag in the user profile, denoting whether
the user is active or not, Juniors often tend to invent a whole General Purpose
Flag Subsystem. With an embedded LISP implementation for easy DSL, running
in the cloud as an independent microservice, written in Lua to make it possible
to launch it as Nginx plugin.</p>
<p>In that particular case YAGNI multiplied by the authority of this Smart Senior
does the trick: the flag remains the boolean column in the database, the task
is delivered in time (meaning, in this century, as an opposite to what was
proposed by the proactive Junior) and everybody (including the business) is happy.</p>
<blockquote>
<p>Truth is mighty and will prevail. There is nothing wrong with this, except that
it ain’t so.</p>
<p><small> Mark Twain</small></p>
</blockquote>
<p>That’s the thing. Even a kitchen knife might be used to kill people, and this
YAGNI concept is a perfect example of such a menace. It’s a great excuse to
reject nearly any CR suggestions: “this already works, <em>yagn</em> anything else,
period.” Unfortunately, in my experience 9 out of 10 times, I accepted YAGNI
argument, it resulted in <em>me</em> implementing the stuff we were not going to need.
Usually it happens in a month, sometimes—tomorrow. In some cases, it was not only
implementing what I suggested, but <em>completely rewriting</em> the whole code piece,
because <em>that</em> YAGNI implies unmaintainable, not ready for extensions code.</p>
<p>In 1930s Pepsi-Cola launched 12oz bottles fighting Coca-Cola’s domination.
Coca-Cola responded with YAGNI and lost 30% of the market.</p>
<p>In 1970s Toyota, Nissan, Mitsubishi and smaller Japanese companies made
a bid on low fuel consumption. ‘YAGNI,’ responded American autoconcern and
lost nearly half of the market in the next decade.</p>
<p>Nowadays every single YAGNI used as an excuse for unwillingness to accomplish
the task considering all the consequences, results in 10× times to recover from
the YAGNI-code. Which is nearly always unmaintainable, error-prone and
not ready for any subsequent changes. The rule of thumb would be:</p>
<blockquote>
<p><em>While in code review, the reviewer might use YAGNI to prevent a waste of time</em>
<em>on unwanted and/or unexpected features. The code owner has no right to use</em>
<em>YAGNI as an argument to proof their delusion and/or reluctance to make the</em>
<em>code friendly to future changes.</em></p>
</blockquote>
<p>That simple, thus it works.</p>
<p>And, the last but not the least: never, never excuse yourself with YAGNI.
There are usually many people all around: team leader, project manager, product
manager, CTO, CEO, your spouse whining about you spend too little time with them,
to prevent you from doing a redundant stuff. There are many, many colleagues,
who will tell you ‘YAGNI’ when needed. But as soon as you have caught yourself
choosing the easiest path because of YAGNI, find the strength in yourself to
resist. Unless you are going to qualify as Development Evangelist and Coach in
Silicon Valley next year, of course. There works any garbage.</p>
Pattern matching on dynamic struct types2018-02-23T00:00:00+00:00https://rocket-science.ru/hacking/2018/02/23/pattern-matching-on-dynamic-struct-types<p>Pattern matching is great.</p>
<p>Strictly speaking, I could end this post right here, but occasionally
I have an interesting Elixir feature on hand. That is related to pattern
matching. That is, I bet, not widely known at all.</p>
<p>One can pattern match on dynamic struct type with pin operator
<a href="https://hexdocs.pm/elixir/Kernel.SpecialForms.html#%5E/1"><code class="language-plaintext highlighter-rouge">Kernel.SpecialForms.^/1</code></a>.
It’s documentation says:</p>
<blockquote>
<p>Accesses an already bound variable in match clauses. Also known as the pin operator.</p>
</blockquote>
<p>Not quite expressive, neither informative. But while a documentation walks, the
code talks. Check this:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">|</span><span class="mi">1</span> <span class="err">▶</span> <span class="k">defmodule</span> <span class="no">MyMod</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="k">defstruct</span> <span class="sx">~w|foo bar|a</span>
<span class="n">iex</span><span class="o">|</span><span class="mi">2</span> <span class="err">▶</span> <span class="n">mod_ok</span> <span class="o">=</span> <span class="no">MyMod</span>
<span class="n">iex</span><span class="o">|</span><span class="mi">3</span> <span class="err">▶</span> <span class="n">mod_ko</span> <span class="o">=</span> <span class="no">Integer</span>
<span class="n">iex</span><span class="o">|</span><span class="mi">4</span> <span class="err">▶</span> <span class="p">%</span><span class="o">^</span><span class="n">mod_ok</span><span class="p">{}</span> <span class="o">=</span> <span class="p">%</span><span class="no">MyMod</span><span class="p">{</span><span class="ss">foo:</span> <span class="mi">42</span><span class="p">,</span> <span class="ss">bar:</span> <span class="mf">3.14</span><span class="p">}</span>
<span class="c1">#⇒ %MyMod{bar: 3.14, foo: 42}</span>
<span class="n">iex</span><span class="o">|</span><span class="mi">5</span> <span class="err">▶</span> <span class="p">%</span><span class="o">^</span><span class="n">mod_ko</span><span class="p">{}</span> <span class="o">=</span> <span class="p">%</span><span class="no">MyMod</span><span class="p">{</span><span class="ss">foo:</span> <span class="mi">42</span><span class="p">,</span> <span class="ss">bar:</span> <span class="mf">3.14</span><span class="p">}</span>
<span class="c1">#⇒ ** (MatchError) no match of right hand side value:</span>
<span class="c1"># %MyMod{bar: 3.14, foo: 42}</span>
</code></pre></div></div>
<p>Wow. We can explicitly pattern match on dynamic struct types! It also works
in <code class="language-plaintext highlighter-rouge">case</code> clauses:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">|</span><span class="mi">6</span> <span class="err">▶</span> <span class="k">case</span> <span class="p">%</span><span class="no">MyMod</span><span class="p">{</span><span class="ss">foo:</span> <span class="mi">42</span><span class="p">,</span> <span class="ss">bar:</span> <span class="mf">3.14</span><span class="p">}</span> <span class="k">do</span>
<span class="o">...|</span><span class="mi">6</span> <span class="err">▶</span> <span class="p">%</span><span class="o">^</span><span class="n">mod_ok</span><span class="p">{}</span> <span class="o">=</span> <span class="p">%{</span><span class="ss">foo:</span> <span class="n">_foo</span><span class="p">}</span> <span class="o">-></span>
<span class="o">...|</span><span class="mi">6</span> <span class="err">▶</span> <span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="p">(</span><span class="n">mod_ok</span><span class="p">,</span> <span class="ss">label:</span> <span class="s2">"Pinned module"</span><span class="p">)</span>
<span class="o">...|</span><span class="mi">6</span> <span class="err">▶</span> <span class="k">end</span>
<span class="c1">#⇒ Pinned module: MyMod</span>
</code></pre></div></div>
<p>FWIW, the latter might be used without a <em>pin oerator</em> to get the struct type</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">|</span><span class="mi">7</span> <span class="err">▶</span> <span class="k">case</span> <span class="p">%</span><span class="no">MyMod</span><span class="p">{</span><span class="ss">foo:</span> <span class="mi">42</span><span class="p">,</span> <span class="ss">bar:</span> <span class="mf">3.14</span><span class="p">}</span> <span class="k">do</span>
<span class="o">...|</span><span class="mi">7</span> <span class="err">▶</span> <span class="p">%</span><span class="n">mod</span><span class="p">{}</span> <span class="o">-></span> <span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="p">(</span><span class="n">mod</span><span class="p">,</span> <span class="ss">label:</span> <span class="s2">"Matched module"</span><span class="p">)</span>
<span class="o">...|</span><span class="mi">7</span> <span class="err">▶</span> <span class="k">end</span>
<span class="c1">#⇒ Matched module: MyMod</span>
</code></pre></div></div>
<p>This match is the cumbersome spelling of <code class="language-plaintext highlighter-rouge">%MyMod{foo: 42, bar: 3.14}.__struct__</code>,
though.</p>
<hr />
<p>Permalink to the Elixir codebase:
<a href="https://github.com/elixir-lang/elixir/blob/cbde356d104996e082b1752b559a64e5c6576f51/lib/elixir/test/elixir/map_test.exs#L205">MapTest.exs</a>.</p>
Real applications of flip-flop in ruby2018-02-22T00:00:00+00:00https://rocket-science.ru/hacking/2018/02/22/flip-flop-real-application<blockquote>
<p>ruby inherited the Perl philosophy of having more than one way to do the same
thing. I inherited that philosophy from Larry Wall, who is my hero actually.
I want to make ruby users free. I want to give them the freedom to choose.
<small>Yukihiro Matsumoto</small></p>
</blockquote>
<p>Ruby surely inherited a flip-flop operator from Perl, amongst others. Why?
Because programming is sexy, ruby is sexy and—you get it—flip-flop is very sexy.</p>
<p>Uh. I probably should start with a brief explanation of what flip-flop is
in general. (There is another
<a href="http://nithinbekal.com/posts/ruby-flip-flop/">great writing</a> on the subject by
Nithin Bekal.)</p>
<p>The term <a href="https://en.wikipedia.org/wiki/Flip-flop_(electronics)">came from the electronics</a>
where it roughly means two-state machine with memory. Basically, its meaning
can be expressed as a boolean variable that somehow depends on two conditions
<em>and</em> the history. It’s initiated with <code class="language-plaintext highlighter-rouge">false</code>.
Once set to <code class="language-plaintext highlighter-rouge">true</code> because of the first condition, it remains
<code class="language-plaintext highlighter-rouge">true</code> until the second condition is evaluated to <code class="language-plaintext highlighter-rouge">true</code>. Now it turns back
to <code class="language-plaintext highlighter-rouge">false</code> until the first condition… I bet you got the point. Since it holds
the state and depends on it, flip-flop makes sense <em>only</em> inside loops.</p>
<p>The simplest implementation of flip-flop in ruby (if it had no dedicated syntax,)
would be:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">flip_flop</span><span class="p">(</span><span class="n">enum</span><span class="p">,</span> <span class="n">cond1</span><span class="p">,</span> <span class="n">cond2</span><span class="p">,</span> <span class="n">fun</span><span class="p">)</span>
<span class="n">enum</span><span class="p">.</span><span class="nf">inject</span><span class="p">(</span><span class="kp">false</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">acc</span><span class="p">,</span> <span class="n">value</span><span class="o">|</span>
<span class="n">acc</span> <span class="o">||=</span> <span class="n">cond1</span><span class="o">.</span><span class="p">(</span><span class="n">value</span><span class="p">)</span>
<span class="n">fun</span><span class="o">.</span><span class="p">(</span><span class="n">value</span><span class="p">)</span> <span class="k">if</span> <span class="n">acc</span>
<span class="n">acc</span> <span class="o">&&</span> <span class="o">!</span><span class="n">cond2</span><span class="o">.</span><span class="p">(</span><span class="n">value</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>and it might be used as:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">enum</span> <span class="o">=</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span><span class="mi">1</span><span class="p">,</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">2</span><span class="p">]</span>
<span class="n">cond1</span> <span class="o">=</span> <span class="o">-></span><span class="p">(</span><span class="n">i</span><span class="p">)</span> <span class="p">{</span> <span class="n">i</span><span class="p">.</span><span class="nf">odd?</span> <span class="p">}</span>
<span class="n">cond2</span> <span class="o">=</span> <span class="o">-></span><span class="p">(</span><span class="n">i</span><span class="p">)</span> <span class="p">{</span> <span class="n">i</span><span class="p">.</span><span class="nf">even?</span> <span class="p">}</span>
<span class="n">fun</span> <span class="o">=</span> <span class="o">-></span><span class="p">(</span><span class="n">value</span><span class="p">)</span> <span class="p">{</span> <span class="nb">print</span> <span class="n">value</span> <span class="p">}</span>
<span class="n">flip_flop</span><span class="p">(</span><span class="n">enum</span><span class="p">,</span> <span class="n">cond1</span><span class="p">,</span> <span class="n">cond2</span><span class="p">,</span> <span class="n">fun</span><span class="p">)</span>
<span class="c1">#⇒ 112</span>
</code></pre></div></div>
<p>In the code above, it turned to have <code class="language-plaintext highlighter-rouge">truthy</code> state on the first <em>odd</em> number
and turned back to <code class="language-plaintext highlighter-rouge">falsey</code> on the first even number. That simple.</p>
<h3 id="flip-flip-out-of-the-box">Flip-flip out of the box</h3>
<p>The thing is ruby comes with a default syntax for flip-flop operator,
that looks a bit discouraging at first glance. It’s a range operator with
left range boundary being a first condition, and the right one—being the second.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">cond1</span><span class="o">..</span><span class="n">cond2</span>
</code></pre></div></div>
<p>Also, it is treated as flip-flop <em>only</em> inside conditionals and ternary operator.
The example above might be rewritten using standard ruby syntax as:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span><span class="mi">2</span><span class="p">,</span><span class="mi">1</span><span class="p">,</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">2</span><span class="p">].</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">i</span><span class="o">|</span>
<span class="nb">print</span> <span class="n">i</span> <span class="k">if</span> <span class="n">i</span><span class="p">.</span><span class="nf">odd?</span><span class="o">..</span><span class="n">i</span><span class="p">.</span><span class="nf">even?</span>
<span class="k">end</span>
<span class="c1">#⇒ 112</span>
</code></pre></div></div>
<p>Cute, isn’t it?</p>
<h3 id="real-applications">Real applications</h3>
<p>Well, this is all sexy, but hey what’s the real purpose of this? This operator
is definitely not as common as most of others, but still.</p>
<p>Yesterday I needed to process a huge list of strings, finding matching pairs
where one string comes after another (not necessarily immediately) and counting
them. For those curious, it’s a sanity check for the FSM; basically I
stress-tested the FSM, printed out the states and after all I wanted to make
sure there were no state order violations (in this case the count should be zero
if everything is fine.)</p>
<p>For the sake of brevity I will use integers in the example below. So, let’s say
we have a huge list of integers and we want to count how many times <code class="language-plaintext highlighter-rouge">2</code> comes
after <code class="language-plaintext highlighter-rouge">1</code>.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">input</span> <span class="o">=</span> <span class="p">[</span><span class="mi">3</span><span class="p">,</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">1</span><span class="p">,</span><span class="mi">1</span><span class="p">,</span><span class="mi">1</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span><span class="mi">2</span><span class="p">]</span>
<span class="n">input</span><span class="p">.</span><span class="nf">count</span> <span class="p">{</span> <span class="o">|</span><span class="n">i</span><span class="o">|</span> <span class="n">i</span> <span class="o">==</span> <span class="mi">2</span> <span class="k">if</span> <span class="p">(</span><span class="n">i</span><span class="o">==</span><span class="mi">1</span><span class="p">)</span><span class="o">..</span><span class="p">(</span><span class="n">i</span><span class="o">==</span><span class="mi">2</span><span class="p">)</span> <span class="p">}</span>
</code></pre></div></div>
<p>That’s basically it. Imagine, how clunky would look any other implementation,
when you have seen this one.</p>
<h3 id="real-applications-more">Real applications …more</h3>
<ul>
<li>split ini-file into sections (yeah, I hear your screaming “regexp,” but still)</li>
<li>split log file <em>on</em> request IP / time change</li>
<li>nearly everything that could be expressed as <em>blah-blah-blah-boom</em>.</li>
</ul>
<p>I wish I could invent more of nifty examples, but as I said, this operator
is not of very common usage. Still, it’s a must-know of every Ruby developer
who pretends to have the Junior position outgrown.</p>
Why immutability rules2018-02-20T00:00:00+00:00https://rocket-science.ru/hacking/2018/02/20/why-immutability-rules<p>The arguing on pros and cons of mutability is aging it’s Diamond jubilee.
As far as I can tell, nowadays people tend to blame mutability for hardly
manageable code and “floating” errors that are nearly impossible to catch.
Also, I’ve heard that immutability is great because it leaves no room
to make a silly mistake for the developer.</p>
<p>While that is true, that is <em>very</em> perfunctory view on the difference.
I have an experience in writing code in both assembly language and C and I
am positive that programming differs from visiting spa facilities: it’s
still up to the developer to take care about, you know, code quality. Any coder,
having the IQ that slightly exceeds the IQ of my vacuum cleaner, must be able
to deal with the mutability. That’s not a rocket science.</p>
<p>What really matters is a performance. Code efficiency and viability. The human
being should be responsible to understand any cumbersome code, while it brings
a gain in the user experience. One of the most valuable metrics of UX is
a response time. Not the developer comfort. I believe there is an epitaph
graved in stone in the programmer’s hell, saying <strong>“Thou shalt have no
boon before UX.”</strong></p>
<p>I got distracted, though.</p>
<p>Immutability grants <em>a compiler</em> with a great opportunity for free: there is
no need to copy data. Everything might be passed by reference. For free.
Because the data is guaranteed to remain unchanged forever.</p>
<p>I accidentally realized that there are some people all around who didn’t think
about that particular aspect of immutability, so please let me repeat it in bold.</p>
<p><strong>Data is nearly never copied, no matter what. It is passed by reference. Always.</strong></p>
<p>That said, if one wants to return, for instance, a complicated structure from
a function, there is no need to declare a constant upfront. In the code below,
the only instance of this map will be allocated. All subsequent calls will
reference the previously allocated memory.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">my_data</span> <span class="k">do</span>
<span class="p">%{</span>
<span class="ss">address:</span> <span class="s2">"Carrer de la Marina, 16, 08005 Barcelona"</span><span class="p">,</span>
<span class="ss">poi:</span> <span class="s2">"Torre Mapfre"</span><span class="p">,</span>
<span class="ss">weird_numbers:</span> <span class="p">[</span><span class="mi">42</span><span class="p">,</span> <span class="mf">3.14159265</span><span class="p">],</span>
<span class="ss">inner_map:</span> <span class="p">%{</span><span class="ss">i:</span> <span class="ss">:was</span><span class="p">,</span> <span class="ow">not</span><span class="p">:</span> <span class="ss">:able</span><span class="p">,</span> <span class="ss">to:</span> <span class="ss">:invent</span><span class="p">,</span> <span class="ss">better:</span> <span class="ss">:example</span><span class="p">}</span>
<span class="p">}</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Please note, that the same function called in e. g. <em>ruby</em> 50 times would
allocate the memory <em>50 times</em>.</p>
<p>That is very significant. If you still think it is not, let me restate:
<em>immutability grants caching of everything for free.</em> This is an immortal
cache that does not require invalidation for the application lifetime.</p>
<hr />
<p>Plus, of course, it saves our asses from passing a list/array to a function
in a loop and wondering why the heck it gets truncated when the loop exits.
I am also able to value the amenity for developers, even despite I care for
compiler much more.</p>
Pattern matching on binaries takes over Regex2018-01-12T00:00:00+00:00https://rocket-science.ru/hacking/2018/01/12/parse-cumbersome-data<p>Last week I spent evenings tweaking
<a href="https://github.com/amotion-city/lib_lat_lon"><code class="language-plaintext highlighter-rouge">{:ok, 📍} LibLatLon</code></a>,
the helper library for geocoding. Currently it supports both <code class="language-plaintext highlighter-rouge">OpenStreetMap</code> and
<code class="language-plaintext highlighter-rouge">GoogleMaps</code> providers (other might be easily added,) and provides fancy lookup
given <em>an image with gps coordinates included</em> as an argument</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">|</span><span class="mi">1</span> <span class="err">▶</span> <span class="no">LibLatLon</span><span class="o">.</span><span class="n">lookup</span> <span class="s2">"images/IMG_1.jpg"</span>
<span class="c1">#⇒ %LibLatLon.Info{</span>
<span class="c1"># address: "Avinguda del Litoral, [...] España",</span>
<span class="c1"># bounds: %LibLatLon.Bounds{</span>
<span class="c1"># ...</span>
</code></pre></div></div>
<p>OK, leading shameless ad is over; let’s turn back to the theme of today’s
talk. This library claims it can read a latitude and longitude information from
almost anything that is somehow looking like a thing, that includes coordinates.</p>
<p>It was easy for:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">{lat, lon} when is_number()</code> tuples;</li>
<li><code class="language-plaintext highlighter-rouge">{[degree, minute, second], reference}</code> values;</li>
<li>some weird combinations of the above, used here and there;</li>
<li>images with <em>GPS</em> geo information included (that was easy!).</li>
</ul>
<p>One might check
<a href="https://hexdocs.pm/lib_lat_lon/LibLatLon.Coords.html#borrow/1">the diversity of types accepted</a>
and the examples below. And everything was fun unless I stepped into accepting
strings as an input. If you wonder, Google supports the nifty properly
typographed format:</p>
<ul>
<li><a href="https://maps.google.com?search=41°22´33.612˝N,2°8´55.242˝E">https://maps.google.com?search=41°22´33.612˝N,2°8´55.242˝E</a></li>
</ul>
<p>Cool? Yes. I decided I need to support this format as well. In other words,
I was to parse the input like <code class="language-plaintext highlighter-rouge">"41°22´33.612˝N"</code> and produce a float out of
this. I love regular expressions. If I ever will launch my own programming
language, it’ll support none syntax but regular expressions.</p>
<p>The only drawback is productivity and inability to damn test all the corner
cases with a regular expression. One might go with something like:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>\d{1,2}°\d{1,2}´\d{1,2}(\.\d{1,})?˝[NWES]
</code></pre></div></div>
<p>or, even, be more precise and disallow degrees, greater than <code class="language-plaintext highlighter-rouge">89</code> and minutes
greater than <code class="language-plaintext highlighter-rouge">59</code> and all that. My goal was different: I wanted to use
Elixir binary pattern matching to accomplish the task. Because I love regular
expressions, but binary pattern matching is still way sexier.</p>
<p>The issue is one cannot pattern match binaries of undeternmined length
in the middle of the match. My first idea was to strictly disallow malformed
input like <code class="language-plaintext highlighter-rouge">"42°0´6.57252˝N,3°8´28.13388˝E"</code>, but a friend of mine having
an address <em>“17257 Fontanilles, Girona, Spain”</em> would complain and grudge that
<code class="language-plaintext highlighter-rouge">{42, 3.14159265}</code> is accepted fine, while <code class="language-plaintext highlighter-rouge">"42°0´0˝N,3°14´15.9˝E"</code> is not.</p>
<p>Well, Elixir provides great opportunities for macro programming, would probably
yell here the astute reader, and yes, here we go. We are about to <em>generate</em>
all possible variants of the string above in a compile time.</p>
<p>Let’s do it for the single blahtitude:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="n">for</span> <span class="n">id</span> <span class="o"><-</span> <span class="mi">1</span><span class="o">..</span><span class="mi">2</span><span class="p">,</span>
<span class="n">im</span> <span class="o"><-</span> <span class="mi">1</span><span class="o">..</span><span class="mi">2</span><span class="p">,</span>
<span class="n">is</span> <span class="o"><-</span> <span class="mi">1</span><span class="o">..</span><span class="nv">@decimal_precision</span> <span class="k">do</span>
<span class="k">def</span> <span class="n">parse</span><span class="p">(</span><span class="o"><<</span>
<span class="n">d</span><span class="p">::</span><span class="n">binary</span><span class="o">-</span><span class="n">size</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">id</span><span class="p">)),</span> <span class="s2">"°"</span><span class="p">,</span>
<span class="n">m</span><span class="p">::</span><span class="n">binary</span><span class="o">-</span><span class="n">size</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">im</span><span class="p">)),</span> <span class="s2">"´"</span><span class="p">,</span>
<span class="n">s</span><span class="p">::</span><span class="n">binary</span><span class="o">-</span><span class="n">size</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">is</span><span class="p">)),</span> <span class="s2">"˝"</span><span class="p">,</span>
<span class="n">ss</span><span class="p">::</span><span class="n">binary</span><span class="o">-</span><span class="n">size</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="o">>></span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="no">Enum</span><span class="o">.</span><span class="n">map</span><span class="p">([</span><span class="n">d</span><span class="p">,</span> <span class="n">m</span><span class="p">,</span> <span class="n">s</span><span class="p">],</span> <span class="k">fn</span> <span class="n">v</span> <span class="o">-></span>
<span class="n">with</span> <span class="p">{</span><span class="n">v</span><span class="p">,</span> <span class="s2">""</span><span class="p">}</span> <span class="o"><-</span> <span class="no">Float</span><span class="o">.</span><span class="n">parse</span><span class="p">(</span><span class="n">v</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="n">v</span>
<span class="k">end</span><span class="p">)</span>
</code></pre></div></div>
<p>Hey, it was simple! <code class="language-plaintext highlighter-rouge">@decimal_precision</code> is a parameter that is small in
developement environment and set to <code class="language-plaintext highlighter-rouge">12</code> in production. 12 gives 48 different
implementations only for the single blahtitude and it takes some noticable time
to compile, while the runtime execution is blazingly fast.</p>
<p>The result would be nearly the same as we had copy-pasted
<code class="language-plaintext highlighter-rouge">def parse(<<.........>>), do: {}</code> 48 times and changed the details here
and there.</p>
<p>The implementation of the whole match does not differ much: another three
comprehension loops are added and the total number of generated functions
is raised drastically. That’s mostly it.</p>
<p>The same technique might be applied to parsing dates, times, floating point numbers
with a limited mantissa (with an unlimited one, it’s still possible with
a fallback to regular expression when the amount of digits is greater than <code class="language-plaintext highlighter-rouge">42</code>,)
you name it.</p>
<hr />
<p><em>Author note:</em> sometimes plain old good regular expression looks at least way more sane.</p>
Automate pattern matching for structs2018-01-08T00:00:00+00:00https://rocket-science.ru/hacking/2018/01/08/automate-struct-pattern-matching<p>In the previous post we’ve been dealing with
<a href="/hacking/2018/01/03/pattern-matcher-for-protocols">pattern matching <code class="language-plaintext highlighter-rouge">Protocol</code>s</a>.</p>
<p>Now let’s dive into Elixir macro world even deeper to see if we can provide
a handy way to pattern match arbitrary structs. I am not sure this code might
be of any value, neither I can invent any meaningful application of it; that’s
why I decided to write this post. I hate throwing the working code away.</p>
<p>The goal would be to produce a simple syntax to tell Elixir “Hey, I want these
functions’ arguments to be pattern matched against these structs respectively.”</p>
<p>The possible application would be to use this technique in the domestic logger
implementation with an ability to tune <em>types</em> of objects to be logged. That
could help in real-time debug (for non-matched structs we are to yield zero AST.)</p>
<p>The code is more of an example of macro dealing, rather that a ready-to-use
drop-in for anything. Also, it might be used as a drop-in, for those braves
<a href="https://gist.github.com/am-kantox/17540ab90343c87e76071ed2b7f428a2">there is a gist</a>.</p>
<p>For all others I have a step-by-step explanation. What we want is to have
something like this in our target class:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">MyLogger</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">DryStructMatch</span><span class="p">,</span>
<span class="ss">log:</span> <span class="p">[</span>
<span class="p">{</span><span class="no">LoginChecker</span><span class="p">,</span> <span class="p">{</span><span class="no">Logger</span><span class="p">,</span> <span class="ss">:warn</span><span class="p">}},</span>
<span class="p">{</span><span class="no">User</span><span class="p">,</span> <span class="o">&</span><span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="o">/</span><span class="mi">1</span><span class="p">},</span>
<span class="p">{[</span><span class="no">Post</span><span class="p">,</span> <span class="no">Comment</span><span class="p">],</span> <span class="p">{</span><span class="no">IO</span><span class="p">,</span> <span class="ss">:inspect</span><span class="p">,</span> <span class="p">[[</span><span class="ss">label:</span> <span class="s2">"inplace"</span><span class="p">]]}}]</span>
<span class="k">def</span> <span class="n">log</span><span class="p">(</span><span class="n">_</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="ss">:ok</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Now if we call <code class="language-plaintext highlighter-rouge">MyLogger.log(object)</code> somewhere in the code,
the call will be properly routed to whatever handler we have provided.
If no handler is provided, the fallback, declared in the core module,
will be used.</p>
<p>While there is not much sense of using this notation instead of four
normal <code class="language-plaintext highlighter-rouge">def</code> clauses with different parameters, the implementation of
this senseless module is intriguing. Let’s start with the basics.</p>
<p>First of all, let’s define callbacks for our handlers. As it might be seen
in the example above, we are to support all kinds of notations:</p>
<ul>
<li>anonymous functions <code class="language-plaintext highlighter-rouge">fn arg -> arg end</code>;</li>
<li>references to functions <code class="language-plaintext highlighter-rouge">&IO.inspect/1</code>;</li>
<li>ready-to-apply functions <code class="language-plaintext highlighter-rouge">{IO, :inspect, [label: "★"]}</code>.</li>
</ul>
<p>The quoted code for that would be:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># assuming we have `name` on hand</span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="k">defp</span> <span class="kn">unquote</span><span class="p">(</span><span class="ss">:"</span><span class="si">#{</span><span class="n">name</span><span class="si">}</span><span class="ss">_callback"</span><span class="p">)({</span><span class="n">mod</span><span class="p">,</span> <span class="n">fun</span><span class="p">},</span> <span class="n">result</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="n">apply</span><span class="p">(</span><span class="n">mod</span><span class="p">,</span> <span class="n">fun</span><span class="p">,</span> <span class="p">[</span><span class="n">result</span><span class="p">])</span>
<span class="k">defp</span> <span class="kn">unquote</span><span class="p">(</span><span class="ss">:"</span><span class="si">#{</span><span class="n">name</span><span class="si">}</span><span class="ss">_callback"</span><span class="p">)({</span><span class="n">mod</span><span class="p">,</span> <span class="n">fun</span><span class="p">,</span> <span class="n">args</span><span class="p">},</span> <span class="n">result</span><span class="p">)</span> <span class="ow">when</span> <span class="n">is_list</span><span class="p">(</span><span class="n">args</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">apply</span><span class="p">(</span><span class="n">mod</span><span class="p">,</span> <span class="n">fun</span><span class="p">,</span> <span class="p">[</span><span class="n">result</span> <span class="o">|</span> <span class="n">args</span><span class="p">])</span>
<span class="k">defp</span> <span class="kn">unquote</span><span class="p">(</span><span class="ss">:"</span><span class="si">#{</span><span class="n">name</span><span class="si">}</span><span class="ss">_callback"</span><span class="p">)(</span><span class="n">fun</span><span class="p">,</span> <span class="n">result</span><span class="p">)</span> <span class="ow">when</span> <span class="n">is_function</span><span class="p">(</span><span class="n">fun</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="n">fun</span><span class="o">.</span><span class="p">(</span><span class="n">result</span><span class="p">)</span>
<span class="k">defp</span> <span class="kn">unquote</span><span class="p">(</span><span class="ss">:"</span><span class="si">#{</span><span class="n">name</span><span class="si">}</span><span class="ss">_callback"</span><span class="p">)(</span><span class="n">_</span><span class="p">,</span> <span class="n">result</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="n">result</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Here we declare three callback handlers and <em>match-it-all</em> handler to pass
the result through when the handler is not allowed. Now the issue is to
generate all the clauses; let’s start with declaring a helper for that:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmacrop</span> <span class="n">clause!</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">mod</span><span class="p">,</span> <span class="n">fun</span><span class="p">)</span> <span class="k">do</span>
<span class="kn">quote</span> <span class="ss">bind_quoted:</span> <span class="p">[</span><span class="ss">name:</span> <span class="n">name</span><span class="p">,</span> <span class="ss">mod:</span> <span class="n">mod</span><span class="p">,</span> <span class="ss">fun:</span> <span class="n">fun</span><span class="p">]</span> <span class="k">do</span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="k">def</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">name</span><span class="p">)(</span>
<span class="kn">unquote</span><span class="p">({</span><span class="ss">:%</span><span class="p">,</span> <span class="p">[],</span>
<span class="p">[{</span><span class="ss">:__aliases__</span><span class="p">,</span> <span class="p">[</span><span class="ss">alias:</span> <span class="no">false</span><span class="p">],</span> <span class="p">[</span><span class="n">mod</span><span class="p">]},</span>
<span class="p">{</span><span class="ss">:%{}</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[]}]})</span> <span class="o">=</span> <span class="n">struct</span>
<span class="p">)</span> <span class="k">do</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">struct</span> <span class="c1"># NB here one might tweak input</span>
<span class="kn">unquote</span><span class="p">(</span><span class="ss">:"</span><span class="si">#{</span><span class="n">name</span><span class="si">}</span><span class="ss">_callback"</span><span class="p">)(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">fun</span><span class="p">),</span> <span class="n">result</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>That helper will produce a function, specified by <code class="language-plaintext highlighter-rouge">name</code> argument, that in turn
will call the <code class="language-plaintext highlighter-rouge">name_callback</code> helper from the previous snippet. One might
probably write a matcher in more readable way, but I ❤ AST.</p>
<p>OK, we are almost done. Let’s cheat with <code class="language-plaintext highlighter-rouge">__using/1</code> macro to embed this to
our target modules:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># assuming we have `mods` as the list of handlers</span>
<span class="c1"># in one of the allowed forms</span>
<span class="no">Enum</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">mods</span><span class="p">,</span> <span class="k">fn</span>
<span class="p">{</span><span class="n">mods</span><span class="p">,</span> <span class="n">fun</span><span class="p">}</span> <span class="ow">when</span> <span class="n">is_list</span><span class="p">(</span><span class="n">mods</span><span class="p">)</span> <span class="o">-></span>
<span class="no">Enum</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">mods</span><span class="p">,</span> <span class="o">&</span><span class="n">clause!</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="nv">&1</span><span class="p">,</span> <span class="n">fun</span><span class="p">))</span>
<span class="p">{</span><span class="n">mod</span><span class="p">,</span> <span class="n">fun</span><span class="p">}</span> <span class="o">-></span>
<span class="n">clause!</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">mod</span><span class="p">,</span> <span class="n">fun</span><span class="p">)</span>
<span class="n">mod</span> <span class="o">-></span>
<span class="n">clause!</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">mod</span><span class="p">,</span> <span class="no">nil</span><span class="p">)</span>
<span class="k">end</span><span class="p">)</span> <span class="o">++</span>
<span class="p">[{</span><span class="ss">:defoverridable</span><span class="p">,</span>
<span class="p">[</span><span class="ss">context:</span> <span class="no">Elixir</span><span class="p">,</span> <span class="kn">import</span><span class="p">:</span> <span class="no">Kernel</span><span class="p">],</span>
<span class="p">[[{</span><span class="n">name</span><span class="p">,</span> <span class="mi">1</span><span class="p">}]]}]</span>
</code></pre></div></div>
<p>We just construct the quoted expressions for everything passed as parameters
with a help of <code class="language-plaintext highlighter-rouge">clause!</code> private macro described above. In the very end, we
don’t forget to allow all our functions to be overridable.</p>
<p>Let’s see how it works.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span><span class="p">(</span><span class="no">Foo</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="k">defstruct</span><span class="p">(</span><span class="ss">foo:</span> <span class="mi">42</span><span class="p">))</span>
<span class="k">defmodule</span><span class="p">(</span><span class="no">Foo1</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="k">defstruct</span><span class="p">(</span><span class="ss">foo:</span> <span class="mi">42</span><span class="p">))</span>
<span class="k">defmodule</span><span class="p">(</span><span class="no">Foo2</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="k">defstruct</span><span class="p">(</span><span class="ss">foo:</span> <span class="mi">42</span><span class="p">))</span>
<span class="k">defmodule</span><span class="p">(</span><span class="no">Bar</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="k">defstruct</span><span class="p">(</span><span class="ss">bar:</span> <span class="mi">42</span><span class="p">))</span>
<span class="k">defmodule</span><span class="p">(</span><span class="no">Baz</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="k">defstruct</span><span class="p">(</span><span class="ss">baz:</span> <span class="mi">42</span><span class="p">))</span>
<span class="k">defmodule</span> <span class="no">A</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">DryStructMatch</span><span class="p">,</span>
<span class="ss">update:</span> <span class="p">[</span>
<span class="no">Foo</span><span class="p">,</span>
<span class="p">{</span><span class="no">Bar</span><span class="p">,</span> <span class="o">&</span><span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="o">/</span><span class="mi">1</span><span class="p">},</span>
<span class="p">{[</span><span class="no">Foo1</span><span class="p">,</span> <span class="no">Foo2</span><span class="p">],</span>
<span class="p">{</span><span class="no">IO</span><span class="p">,</span> <span class="ss">:inspect</span><span class="p">,</span> <span class="p">[[</span><span class="ss">label:</span> <span class="s2">"inplace"</span><span class="p">]]}}],</span>
<span class="ss">empty:</span> <span class="no">Foo1</span>
<span class="k">def</span> <span class="n">empty</span><span class="p">(</span><span class="n">input</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="kn">super</span><span class="p">(</span><span class="n">input</span><span class="p">)</span> <span class="o">&&</span> <span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="p">(</span><span class="n">input</span><span class="p">,</span> <span class="ss">label:</span> <span class="s2">"overloaded"</span><span class="p">)</span>
<span class="k">end</span>
<span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="p">(</span><span class="no">A</span><span class="o">.</span><span class="n">update</span><span class="p">(%</span><span class="no">Foo</span><span class="p">{}),</span> <span class="ss">label:</span> <span class="s2">"explicit"</span><span class="p">)</span>
<span class="c1">#⇒ explicit: %Foo{foo: 42}</span>
<span class="no">A</span><span class="o">.</span><span class="n">update</span><span class="p">(%</span><span class="no">Bar</span><span class="p">{})</span>
<span class="c1">#⇒ %Bar{bar: 42} # via callback</span>
<span class="no">A</span><span class="o">.</span><span class="n">update</span><span class="p">(%</span><span class="no">Foo2</span><span class="p">{})</span> <span class="c1"># via callback</span>
<span class="c1">#⇒ inplace: %Foo2{foo: 42}</span>
<span class="no">A</span><span class="o">.</span><span class="n">empty</span><span class="p">(%</span><span class="no">Foo1</span><span class="p">{})</span>
<span class="c1">#⇒ overloaded: %Foo1{foo: 42}</span>
<span class="c1"># raises `FunctionClauseError`</span>
<span class="no">A</span><span class="o">.</span><span class="n">update</span><span class="p">(%</span><span class="no">Baz</span><span class="p">{})</span>
<span class="c1"># ** (FunctionClauseError) no function clause matching in A.update/1</span>
<span class="c1"># The following arguments were given to A.update/1:</span>
<span class="c1"># # 1</span>
<span class="c1"># %Baz{baz: 42}</span>
<span class="c1"># A.update/1</span>
<span class="c1"># (elixir) lib/code.ex:678: Code.require_file/2</span>
</code></pre></div></div>
<p>The summing up, we have the following code:</p>
<script src="https://gist.github.com/am-kantox/17540ab90343c87e76071ed2b7f428a2.js"></script>
<p>Enjoy!</p>
Pattern matcher for Protocols2018-01-03T00:00:00+00:00https://rocket-science.ru/hacking/2018/01/03/pattern-matcher-for-protocols<p>One of the best features of Erlang (and hence Elixir) is the ability to
pattern match and use guards directly in function clauses:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">M</span> <span class="k">do</span>
<span class="k">def</span> <span class="n">m</span><span class="p">(</span><span class="no">nil</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="s2">"nil given"</span>
<span class="k">def</span> <span class="n">m</span><span class="p">([]),</span> <span class="k">do</span><span class="p">:</span> <span class="s2">"empty list given"</span>
<span class="k">def</span> <span class="n">m</span><span class="p">([</span><span class="n">h</span><span class="o">|</span><span class="n">_</span><span class="p">]),</span> <span class="k">do</span><span class="p">:</span> <span class="s2">"non-empty list given (head: </span><span class="si">#{</span><span class="n">inspect</span><span class="p">(</span><span class="n">head</span><span class="p">)</span><span class="si">}</span><span class="s2">.)"</span>
<span class="k">def</span> <span class="n">m</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span> <span class="ow">when</span> <span class="n">is_binary</span><span class="p">(</span><span class="n">message</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="s2">"message given (msg: </span><span class="si">#{</span><span class="n">msg</span><span class="si">}</span><span class="s2">.)"</span>
<span class="k">end</span>
</code></pre></div></div>
<p>That works perfectly, routing the calls to respective handlers. But what if
we want to pattern match <em>the implementation of the protocol</em>?</p>
<p>That is not possible out of the box and might seem to be tricky. The naïve
approach would be to <em>emulate</em> pattern matching:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defprotocol</span> <span class="no">P</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="k">def</span> <span class="n">pm</span><span class="p">(</span><span class="n">this</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="n">inspect</span><span class="p">(</span><span class="n">this</span><span class="p">)</span>
<span class="k">defmodule</span> <span class="no">M</span> <span class="k">do</span>
<span class="k">def</span> <span class="n">m</span><span class="p">(</span><span class="n">p</span><span class="p">)</span> <span class="k">do</span>
<span class="k">unless</span> <span class="no">P</span><span class="o">.</span><span class="n">impl_for</span><span class="p">(</span><span class="n">p</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="k">raise</span> <span class="no">MatchError</span><span class="p">,</span> <span class="ss">term:</span> <span class="n">p</span>
<span class="c1"># ...</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>That kinda works, but hey! We have all the ingredients: we have macros in Elixir,
we have
<a href="https://hexdocs.pm/elixir/Kernel.html#defprotocol/2-reflection"><code class="language-plaintext highlighter-rouge">Kernel#reflections</code></a>
for protocols, we also have an ardour. Let’s cook the matchers ourselves.</p>
<p>We are going to generate all the clauses for all the consolidated protocols,
known to the system. Let’s do it:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">ProtoMatcher</span> <span class="k">do</span>
<span class="k">defmacro</span> <span class="n">defprotomatchers</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">mod</span><span class="p">,</span> <span class="n">fun</span><span class="p">)</span> <span class="k">do</span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="n">with</span> <span class="p">{</span><span class="ss">:consolidated</span><span class="p">,</span> <span class="n">mods</span><span class="p">}</span> <span class="o"><-</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">mod</span><span class="p">)</span><span class="o">.</span><span class="n">__protocol__</span><span class="p">(</span><span class="ss">:impls</span><span class="p">)</span> <span class="k">do</span>
<span class="n">for</span> <span class="n">impl</span> <span class="o"><-</span> <span class="n">mods</span> <span class="k">do</span>
<span class="k">def</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">name</span><span class="p">)(%{</span><span class="ss">__struct__:</span> <span class="n">impl</span><span class="p">}</span> <span class="o">=</span> <span class="n">this</span><span class="p">)</span> <span class="k">do</span>
<span class="kn">unquote</span><span class="p">(</span><span class="n">fun</span><span class="p">)</span><span class="o">.</span><span class="p">(</span><span class="n">this</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">else</span>
<span class="ss">:not_consolidated</span> <span class="o">-></span>
<span class="k">raise</span> <span class="p">[</span>
<span class="s2">"Protocols are not consolidated."</span><span class="p">,</span>
<span class="s2">"That usually happens in `iex`."</span><span class="p">,</span>
<span class="s2">"Please start iex as `iex -S mix`"</span><span class="p">,</span>
<span class="s2">" in the project directory to use this feature."</span><span class="p">]</span>
<span class="o">|></span> <span class="no">Enum</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Now this might be used as:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">M</span> <span class="k">do</span>
<span class="kn">require</span> <span class="no">ProtoMatcher</span>
<span class="no">ProtoMatcher</span><span class="o">.</span><span class="n">defprotomatchers</span><span class="p">(</span><span class="ss">:checker</span><span class="p">,</span> <span class="no">Enumerable</span><span class="p">,</span> <span class="k">fn</span> <span class="n">this</span> <span class="o">-></span> <span class="n">inspect</span><span class="p">(</span><span class="n">this</span><span class="p">)</span> <span class="k">end</span><span class="p">)</span>
<span class="k">end</span>
<span class="no">M</span><span class="o">.</span><span class="n">checker</span><span class="p">(%{})</span>
<span class="c1">#⇒ ** (FunctionClauseError) no function clause matching in M.checker/1</span>
<span class="no">M</span><span class="o">.</span><span class="n">checker</span><span class="p">(%</span><span class="no">File</span><span class="o">.</span><span class="no">Stream</span><span class="p">{})</span>
<span class="c1">#⇒ "%File.Stream{line_or_bytes: :line, modes: [], path: nil, raw: true}"</span>
</code></pre></div></div>
<p>That said, the <code class="language-plaintext highlighter-rouge">defmodule M</code> above produced:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="ss">:module</span><span class="p">,</span> <span class="no">M</span><span class="p">,</span>
<span class="o"><<</span><span class="mi">70</span><span class="p">,</span> <span class="mi">79</span><span class="p">,</span> <span class="mi">82</span><span class="p">,</span> <span class="mi">49</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">56</span><span class="p">,</span> <span class="mi">66</span><span class="p">,</span> <span class="mi">69</span><span class="p">,</span> <span class="mi">65</span><span class="p">,</span> <span class="mi">77</span><span class="p">,</span> <span class="mi">65</span><span class="p">,</span> <span class="mi">116</span><span class="p">,</span> <span class="mi">85</span><span class="p">,</span> <span class="mi">56</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">174</span><span class="p">,</span>
<span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">17</span><span class="p">,</span> <span class="mi">8</span><span class="p">,</span> <span class="mi">69</span><span class="p">,</span> <span class="mi">108</span><span class="p">,</span> <span class="mi">105</span><span class="p">,</span> <span class="mi">120</span><span class="p">,</span> <span class="mi">105</span><span class="p">,</span> <span class="mi">114</span><span class="p">,</span> <span class="mi">46</span><span class="p">,</span> <span class="mi">77</span><span class="p">,</span> <span class="mi">8</span><span class="p">,</span> <span class="mi">95</span><span class="p">,</span> <span class="mi">95</span><span class="p">,</span> <span class="mi">105</span><span class="p">,</span> <span class="mi">110</span><span class="p">,</span>
<span class="mi">102</span><span class="p">,</span> <span class="mi">111</span><span class="p">,</span> <span class="mi">95</span><span class="p">,</span> <span class="mi">95</span><span class="p">,</span> <span class="mi">9</span><span class="p">,</span> <span class="mi">102</span><span class="p">,</span> <span class="mi">117</span><span class="p">,</span> <span class="mi">110</span><span class="p">,</span> <span class="mi">99</span><span class="p">,</span> <span class="o">...>></span><span class="p">,</span>
<span class="p">[</span>
<span class="ss">checker:</span> <span class="mi">1</span><span class="p">,</span>
<span class="ss">checker:</span> <span class="mi">1</span><span class="p">,</span>
<span class="ss">checker:</span> <span class="mi">1</span><span class="p">,</span>
<span class="ss">checker:</span> <span class="mi">1</span><span class="p">,</span>
<span class="ss">checker:</span> <span class="mi">1</span><span class="p">,</span>
<span class="ss">checker:</span> <span class="mi">1</span><span class="p">,</span>
<span class="ss">checker:</span> <span class="mi">1</span><span class="p">,</span>
<span class="ss">checker:</span> <span class="mi">1</span><span class="p">,</span>
<span class="ss">checker:</span> <span class="mi">1</span><span class="p">,</span>
<span class="ss">checker:</span> <span class="mi">1</span><span class="p">,</span>
<span class="ss">checker:</span> <span class="mi">1</span><span class="p">,</span>
<span class="ss">checker:</span> <span class="mi">1</span>
<span class="p">]}</span>
</code></pre></div></div>
<p>with 11 implementations of <code class="language-plaintext highlighter-rouge">M.checker/1</code>:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">M</span><span class="o">.</span><span class="n">__info__</span><span class="p">(</span><span class="ss">:functions</span><span class="p">)</span>
<span class="c1">#⇒ [checker: 1]</span>
</code></pre></div></div>
<p>Now you have a function (a set of 11 functions, to be precise,) that effectively
matches <em>only</em> implementations of the <code class="language-plaintext highlighter-rouge">Enumerable</code> protocol.</p>
<hr />
<p>There is definitely a room for improvements, like packing everything into a <code class="language-plaintext highlighter-rouge">__using__/1</code>
macro, allowing blocks and delegates besides the anonymous function as a handler,
better customization for how to handle different implementations, callbacks, cold beer…</p>
<p>Maybe once I’ll publish this as a package to hex, but for now this blog is fine enough.</p>
.iex.exs to the rescue2017-12-29T00:00:00+00:00https://rocket-science.ru/hacking/2017/12/29/iex.exs-to-the-rescue<p>My development cycle looks like:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>invent feature ⇒ write test ⇒ implement everything
⇓ ⇑
hack in iex write more tests
⇓ ⇑
make tests pass
</code></pre></div></div>
<p>Right now I’m going to talk about “hack in <code class="language-plaintext highlighter-rouge">iex</code>” part. If you are like me,
you probably have a good <code class="language-plaintext highlighter-rouge">seeds.exs</code> file that feeds the <code class="language-plaintext highlighter-rouge">dev</code> database with
everything it needs, but… occasionally you tweak the data, and you don’t
need a fresh start, and every new single console session starts with annoying
<kbd>Ctrl</kbd>+<kbd>R</kbd> history searches for the data manipulations.</p>
<p>BTW, it you still have no history in your <code class="language-plaintext highlighter-rouge">iex</code> console, you are going to
upgrade Erlang to version 20 and issue</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>export ERL_AFLAGS="-kernel shell_history enabled"
</code></pre></div></div>
<p>somewhere inside your <code class="language-plaintext highlighter-rouge">.zsh.after</code>, or <code class="language-plaintext highlighter-rouge">.bashrc</code>, or even some fishy script.
If you are somehow stuck to erlang version less than 20, check
<a href="https://github.com/ferd/erlang-history">this link</a>. History is a must,
no doubt. There is <a href="https://hexdocs.pm/iex/IEx.html#module-shell-history">more on enabling history in <code class="language-plaintext highlighter-rouge">IEx</code> documentation</a>.</p>
<p>Turning back to <code class="language-plaintext highlighter-rouge">iex</code>, one might want to produce a temporary <em>seed</em>, that after
the hacking session might be moved to <code class="language-plaintext highlighter-rouge">seeds.exs</code> file. To do so, just
create a <code class="language-plaintext highlighter-rouge">.iex.exs</code> file in the project root directory and put any elixir
code you want to get executed when the console starts.</p>
<p>Before we continue with the main example, let me introduce my own <code class="language-plaintext highlighter-rouge">.iex.exs</code>
file, because it actually has both project-related and system-wide
configurations.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">global_settings</span> <span class="o">=</span> <span class="s2">"~/.iex.exs"</span>
<span class="k">if</span> <span class="no">File</span><span class="o">.</span><span class="n">exists?</span><span class="p">(</span><span class="n">global_settings</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="no">Code</span><span class="o">.</span><span class="n">require_file</span><span class="p">(</span><span class="n">global_settings</span><span class="p">)</span>
<span class="no">Application</span><span class="o">.</span><span class="n">put_env</span><span class="p">(</span><span class="ss">:elixir</span><span class="p">,</span> <span class="ss">:ansi_enabled</span><span class="p">,</span> <span class="no">true</span><span class="p">)</span>
<span class="no">IEx</span><span class="o">.</span><span class="n">configure</span><span class="p">(</span>
<span class="ss">colors:</span> <span class="p">[</span>
<span class="ss">eval_result:</span> <span class="p">[</span><span class="ss">:cyan</span><span class="p">,</span> <span class="ss">:bright</span><span class="p">]</span> <span class="p">,</span>
<span class="ss">eval_error:</span> <span class="p">[[</span><span class="ss">:red</span><span class="p">,</span> <span class="ss">:bright</span><span class="p">,</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">▶▶▶</span><span class="se">\n</span><span class="s2">"</span><span class="p">]],</span>
<span class="ss">eval_info:</span> <span class="p">[</span><span class="ss">:yellow</span><span class="p">,</span> <span class="ss">:bright</span> <span class="p">],</span>
<span class="p">],</span>
<span class="ss">default_prompt:</span> <span class="p">[</span>
<span class="s2">"</span><span class="se">\e</span><span class="s2">[G"</span><span class="p">,</span> <span class="c1"># cursor ⇒ column 1</span>
<span class="ss">:blue</span><span class="p">,</span> <span class="s2">"%prefix"</span><span class="p">,</span> <span class="ss">:yellow</span><span class="p">,</span> <span class="s2">"|"</span><span class="p">,</span> <span class="ss">:blue</span><span class="p">,</span> <span class="s2">"%counter"</span><span class="p">,</span> <span class="s2">" "</span><span class="p">,</span> <span class="ss">:yellow</span><span class="p">,</span> <span class="s2">"▶"</span><span class="p">,</span> <span class="ss">:reset</span>
<span class="p">]</span> <span class="o">|></span> <span class="no">IO</span><span class="o">.</span><span class="no">ANSI</span><span class="o">.</span><span class="n">format</span> <span class="o">|></span> <span class="no">IO</span><span class="o">.</span><span class="n">chardata_to_string</span>
<span class="p">)</span>
<span class="n">alias</span> <span class="no">MyApp</span><span class="o">.</span><span class="p">{</span><span class="no">Repo</span><span class="p">,</span><span class="no">User</span><span class="p">,</span><span class="no">Photo</span><span class="p">,</span><span class="no">Album</span><span class="p">}</span>
<span class="p">[</span><span class="n">am</span><span class="p">]</span> <span class="o">=</span> <span class="no">Repo</span><span class="o">.</span><span class="n">all</span> <span class="no">User</span>
</code></pre></div></div>
<p>First two lines enforce the system-wide configuration to be used as
“overridable defaults” here. For instance, the whole <code class="language-plaintext highlighter-rouge">IEx.configure</code> part
might be moved there if you are OK with having same settings for all
the projects.</p>
<p>My advise would be: invent the prompt shape you like and put
<em>the <code class="language-plaintext highlighter-rouge">default_prompt</code></em> part to system-wide <code class="language-plaintext highlighter-rouge">~/.iex.exs</code>. Don’t be lazy to
tune up <em>different colors</em> for each project. That way you’ll never lose
the point on what project this console is opened for. If you are working
on one project at any moment, you might want to put the whole <code class="language-plaintext highlighter-rouge">IEx</code>
configuration into system-wide file.</p>
<p>So far so good. This is how my prompt looks like:</p>
<p><img src="/img/20171229-1.jpg" alt="iex 1>" /></p>
<p>One might see the bright red error message. That is configured by
<code class="language-plaintext highlighter-rouge">[colors: [eval_error: ...]]</code> setting. Everything else is pretty
self-explanatory. OK, when we have things tuned up, let’s tweak our project.</p>
<p>When I start to work on some feature, I might need some very filtered set
of the data in <code class="language-plaintext highlighter-rouge">dev</code> environment, not the wholy load of everything from
<code class="language-plaintext highlighter-rouge">seeds.exs</code>. I usually start with creating some new data by typing in a console
and as soon as I find myself re-evaluating the same line of code from
the history more and more, I put the respective line into my project’s <code class="language-plaintext highlighter-rouge">.iex.exs</code>
file. Or, I might do some clean up after last console session there.</p>
<p>In the example I provided above, last two lines are aliasing <em>everything</em>
for my current project <em>and</em> assign the first available user instance from the
database to the variable named <code class="language-plaintext highlighter-rouge">am</code>. In some projects I have dozens of lines
there. For some projects I do not bother to create this file at all, relying
on the system-wide one.</p>
<p><a href="https://hexdocs.pm/iex/IEx.html#module-the-iex-exs-file">Official documentation for <code class="language-plaintext highlighter-rouge">IEx</code></a>.</p>
Idiomatic function memoization in Elixir2017-11-23T00:00:00+00:00https://rocket-science.ru/hacking/2017/11/23/idiomatic-function-memoization-in-elixir<p>Today there was a <a href="https://stackoverflow.com/questions/47452163/writing-the-function-once-in-elixir">question raised on SO</a>.</p>
<blockquote>
<p>“it’s possible to write a higher order function ‘once’ which returns a function
that will invoke the passed in function only once, and returns the previous
result on subsequent calls?”</p>
</blockquote>
<blockquote>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>var once = (func) => {
var wasCalled = false, prevResult;
return (...args) => {
if (wasCalled) return prevResult;
wasCalled = true;
return prevResult = func(...args);
}
}
</code></pre></div> </div>
</blockquote>
<p>The above is an example of this behaviour in JS, provided by OP. There were
many different approaches given.</p>
<p>The first would be to use an
<a href="https://hexdocs.pm/elixir/Agent.html#content"><code class="language-plaintext highlighter-rouge">Agent</code></a> to store the value
and lookup it before any subsequent execution.</p>
<p>Another one would be to use the
<a href="https://hexdocs.pm/elixir/Process.html#content"><code class="language-plaintext highlighter-rouge">Process</code></a> dictionary when all
the calls are done within the same process, or <code class="language-plaintext highlighter-rouge">ETS</code>/<code class="language-plaintext highlighter-rouge">DETS</code> for inter-process
memoization.</p>
<p>I don’t think any of this would be an idiomatic approach. I would introduce the
dedicated <a href="https://hexdocs.pm/elixir/GenServer.html#content"><code class="language-plaintext highlighter-rouge">GenServer</code></a> for
this, that will run the heavy function inside it’s <code class="language-plaintext highlighter-rouge">init</code> callback:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">M</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">GenServer</span>
<span class="k">def</span> <span class="n">start_link</span><span class="p">(</span><span class="n">_opts</span> <span class="p">\\</span> <span class="p">[])</span> <span class="k">do</span>
<span class="no">GenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="no">nil</span><span class="p">,</span> <span class="ss">name:</span> <span class="bp">__MODULE__</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="n">init</span><span class="p">(</span><span class="n">_args</span><span class="p">)</span> <span class="k">do</span>
<span class="no">Process</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mi">1_000</span><span class="p">)</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="s2">"42"</span><span class="p">}</span>
<span class="k">end</span>
<span class="k">def</span> <span class="n">value</span><span class="p">()</span> <span class="k">do</span>
<span class="n">label</span> <span class="o">=</span>
<span class="k">case</span> <span class="n">start_link</span><span class="p">()</span> <span class="k">do</span>
<span class="p">{</span><span class="ss">:error</span><span class="p">,</span> <span class="p">{</span><span class="ss">:already_started</span><span class="p">,</span> <span class="n">_</span><span class="p">}}</span> <span class="o">-></span> <span class="s2">"using memoized value: "</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">_</span><span class="p">}</span> <span class="o">-></span> <span class="s2">"calculated the value: "</span>
<span class="k">end</span>
<span class="n">label</span> <span class="o"><></span> <span class="no">GenServer</span><span class="o">.</span><span class="n">call</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="ss">:value</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="n">handle_call</span><span class="p">(</span><span class="ss">:value</span><span class="p">,</span> <span class="n">_from</span><span class="p">,</span> <span class="n">state</span><span class="p">)</span> <span class="k">do</span>
<span class="p">{</span><span class="ss">:reply</span><span class="p">,</span> <span class="n">state</span><span class="p">,</span> <span class="n">state</span><span class="p">}</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>to check it’s working as expected, one might use:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">|</span><span class="mi">1</span><span class="o">></span> <span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="mi">5</span><span class="p">)</span> <span class="o">|></span> <span class="no">Enum</span><span class="o">.</span><span class="n">each</span><span class="p">(</span><span class="o">&</span><span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="p">(</span><span class="no">M</span><span class="o">.</span><span class="n">value</span><span class="p">(),</span> <span class="ss">label:</span> <span class="n">to_string</span><span class="p">(</span><span class="nv">&1</span><span class="p">)))</span>
<span class="c1">#⇒ one second delay happens here</span>
<span class="c1"># 1: "calculated the value: 42"</span>
<span class="c1"># 2: "using memoized value: 42"</span>
<span class="c1"># 3: "using memoized value: 42"</span>
<span class="c1"># 4: "using memoized value: 42"</span>
<span class="c1"># 5: "using memoized value: 42"</span>
</code></pre></div></div>
<p>One might notice, that the first value is printed with a delay,
while all subsequent values are printed immediately.</p>
<p>This is an exact analog of the memoized function from JS, built using
<code class="language-plaintext highlighter-rouge">GenServer</code>. <code class="language-plaintext highlighter-rouge">GenServer.start_link/3</code> returns one of the following:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="c1">#PID<0.80.0>}</span>
<span class="p">{</span><span class="ss">:error</span><span class="p">,</span> <span class="p">{</span><span class="ss">:already_started</span><span class="p">,</span> <span class="c1">#PID<0.80.0>}}</span>
</code></pre></div></div>
<p>and this value is used in the code above to print the leading label.
In the real life it might be just omitted:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="k">def</span> <span class="n">value</span><span class="p">()</span> <span class="k">do</span>
<span class="n">start_link</span><span class="p">()</span>
<span class="no">GenServer</span><span class="o">.</span><span class="n">call</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="ss">:value</span><span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">GenServer</code> will not be resstarted if it’s already started.
We can not bother to check the returned value since we are all set in any case:
if it’s the initial start, we call the heavy function. If the server was already
started, the value is already at fingers in the state.</p>
<p>That might be turned into a helper module, like:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">Memoized</span> <span class="k">do</span>
<span class="k">defmacro</span> <span class="n">__using__</span><span class="p">(</span><span class="n">opts</span><span class="p">)</span> <span class="k">do</span>
<span class="n">with</span> <span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">fun</span><span class="p">}</span> <span class="o"><-</span> <span class="no">Keyword</span><span class="o">.</span><span class="n">fetch</span><span class="p">(</span><span class="n">opts</span><span class="p">,</span> <span class="ss">:fun</span><span class="p">),</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">name</span><span class="p">}</span> <span class="o"><-</span> <span class="no">Keyword</span><span class="o">.</span><span class="n">fetch</span><span class="p">(</span><span class="n">opts</span><span class="p">,</span> <span class="ss">:name</span><span class="p">)</span> <span class="k">do</span>
<span class="n">block</span> <span class="o">=</span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">GenServer</span>
<span class="k">def</span> <span class="n">start_link</span><span class="p">(</span><span class="n">_opts</span> <span class="p">\\</span> <span class="p">[]),</span>
<span class="k">do</span><span class="p">:</span> <span class="no">GenServer</span><span class="o">.</span><span class="n">start_link</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="no">nil</span><span class="p">,</span> <span class="ss">name:</span> <span class="bp">__MODULE__</span><span class="p">)</span>
<span class="k">def</span> <span class="n">init</span><span class="p">(</span><span class="n">_args</span><span class="p">)</span> <span class="k">do</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">fun</span><span class="p">)</span><span class="o">.</span><span class="p">()}</span>
<span class="k">end</span>
<span class="k">def</span> <span class="n">value</span><span class="p">()</span> <span class="k">do</span>
<span class="n">start_link</span><span class="p">()</span>
<span class="no">GenServer</span><span class="o">.</span><span class="n">call</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="ss">:value</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="n">handle_call</span><span class="p">(</span><span class="ss">:value</span><span class="p">,</span> <span class="n">_from</span><span class="p">,</span> <span class="n">state</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="ss">:reply</span><span class="p">,</span> <span class="n">state</span><span class="p">,</span> <span class="n">state</span><span class="p">}</span>
<span class="k">end</span>
<span class="kn">quote</span> <span class="k">do</span><span class="p">:</span> <span class="no">Kernel</span><span class="o">.</span><span class="k">defmodule</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">name</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">block</span><span class="p">))</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">defmodule</span> <span class="no">M</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">Memoized</span><span class="p">,</span> <span class="ss">name:</span> <span class="no">Fun</span><span class="p">,</span> <span class="ss">fun:</span> <span class="k">fn</span> <span class="o">-></span> <span class="no">Process</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mi">1_000</span><span class="p">);</span> <span class="mi">42</span> <span class="k">end</span>
<span class="k">def</span> <span class="n">check</span><span class="p">(),</span> <span class="k">do</span><span class="p">:</span> <span class="no">Enum</span><span class="o">.</span><span class="n">each</span><span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="mi">5</span><span class="p">,</span> <span class="o">&</span><span class="no">IO</span><span class="o">.</span><span class="n">inspect</span><span class="p">(</span><span class="no">Fun</span><span class="o">.</span><span class="n">value</span><span class="p">(),</span> <span class="ss">label:</span> <span class="n">to_string</span><span class="p">(</span><span class="nv">&1</span><span class="p">)))</span>
<span class="k">end</span>
<span class="no">M</span><span class="o">.</span><span class="n">check</span><span class="p">()</span>
<span class="c1">#⇒ 1 second delay</span>
<span class="c1"># 1: 42</span>
<span class="c1"># 2: 42</span>
<span class="c1"># 3: 42</span>
<span class="c1"># 4: 42</span>
<span class="c1"># 5: 42</span>
</code></pre></div></div>
<p>That’s it.</p>
Clarity Over Verbosity Everywhere2017-11-15T00:00:00+00:00https://rocket-science.ru/culture/2017/11/15/clarity-over-verbosity-everywhere<p>Last day my colleague has shared <a href="https://signalvnoise.com/posts/3250-clarity-over-brevity-in-variable-and-method-names">the post by DHH</a>,
written back in 2012 and called “Clarity over brevity in variable and method names.”</p>
<p>David advocates long variable and method names there. The example from the
<em>Basecamp</em> code base illustrates the idea. I beg your pardon if you do read
this from the smartphone and/or tablet:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">make_person_an_outside_subscriber_if_all_accesses_revoked</span>
<span class="n">person</span><span class="p">.</span><span class="nf">update_attribute</span><span class="p">(</span><span class="ss">:outside_subscriber</span><span class="p">,</span> <span class="kp">true</span><span class="p">)</span> <span class="k">if</span> <span class="n">person</span><span class="p">.</span><span class="nf">reload</span><span class="p">.</span><span class="nf">accesses</span><span class="p">.</span><span class="nf">blank?</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The point is (if I got it properly): <em>the method name is clear because of it’s
verbosity</em>. This is bullshit.</p>
<p>When two persons are having a conversation in Spanish, the phrase in Russian,
even being very clear, verbose and syllabically flawless, won’t help to
describe the topic better. Development is being done with so-called
programming languages, that differ from human languages. And there is a
good reason for that.</p>
<p>Abstraction.</p>
<p>While coding, I don’t give a piece of cake whether this entity is a person,
a cat or even a forest troll. The literal meaning of “subscriber” is of no value
at all. It is just an association. It could be <em>the hair color</em> as well.
Or, say, <em>monthly income</em>. The good developer should not care about
implementation details. Good abstraction level is 80% of the success in
producing maintainable, flexible and reliable codebase. Not long names.</p>
<p><code class="language-plaintext highlighter-rouge">if_all_accesses_revoked</code>. What? Who damn cares? Make it generic and benefit
from having an abstraction, that serves access rights. <em>Any possible</em> access
rights, both existing and those to be created in the next year.</p>
<p>I am positive, that while one’s reading the method name shown above, good
developer will produce the whole implementation of the <em>access rights</em> abstraction.
Programming languages differ from human languages. They extract
the essense to decrease a possibility of the ambiguity and make the code less
error-prone. All that is impossible when we start to talk (and therefore think)
on such a microlevel as “outside subscribers in a case when all accesses are
revoked.” This granularity breaks the whole picture, makes the codebase bloated
and in general introduces unmaintainable code, since every single update
in <em>accesses</em> would typically introduce hundreds of new methods.</p>
<p>We are developers and we are supposed to know the syntax of the language
we work with. It should not be a quest to understand what the code does.</p>
<p>Long names say nothing about meaningful things, like: control flow, side effects,
current state, etc. All that is valuable. And “outside subscriber” is not.
Call it “chapuza,” or “stuff,” or “хня,” the programming language speaks for
itself. BTW, COBOL is the most readable language that was ever invented,
that’s why I proposed to call Rails COBOL². The whole attempt of Rails’
so-called readability was invented by
<a href="https://en.wikipedia.org/wiki/Grace_Hopper">Grace Hopper</a> 60 yrs ago.</p>
<p>And my advice for those having problems understanding oneliners would be:
train yourself in the programming language. Solve quizes. Help on Stack Overflow.
Write more code.</p>
<p>Introducing long method names would just bury you disability to write and read
clean code deeper in the ground. I <a href="http://rocket-science.ru/hacking/2016/04/22/stack-overflow-achievements">have been writing</a> about
that already. Nothing has changed. Good code is granular, dense and strong.
Long sentenses are good in medieval literature, not in the development.</p>
Timeo Juniors et ideas ferentes2017-11-13T00:00:00+00:00https://rocket-science.ru/culture/2017/11/13/juniors-and-ideas<p>At work, we have a slack channel named “morethings” where all team members are
to throw links to any resources found in the internet, that might be interesting
for others. Yesterday one of my colleagues threw a link to the blogpost cited below:</p>
<blockquote>
<p>One of the areas I often see senior engineers struggle with,
is raising junior engineers to the next level.
Often this is because we don’t give them the space to explore,
learn and understand how to approach problems for themselves.
<small><a href="http://silverwraith.com/blog/2017/10/the-senior-engineers-guide-to-helping-others-make-decisions/">The Senior Engineer’s Guide to Helping Others Make Decisions</a></small></p>
</blockquote>
<p>I read the article thoroughly. While it contains many good and valuable advises,
I disagree with the main point. Senior engineers are seldom blocking ideas.
Actual Senior engineer is always ready to discuss any crap that is spontaneously
raised. Juniors, raising ideas are the dream of every single real Senior.</p>
<p>In my 19 years of experience being Senior dev, overruling Juniors’ ideas is not
an issue. As soon as Junior raises ideas, <em>we are all set</em>.</p>
<p>The real problem is: Juniors rare if never come up with ideas. In C/C++/Java world
they tend to ask Seniors on every subject (which is good,) or grab-n-paste answers
from SO, which is bad, because Seniors usually explain <em>whys</em> and SO only provides <em>hows</em>
(if we were lucky enough to find a correct answer.) In Rails world the situation
is even worse, because Rails provides an ability to build some likely working stuff
without even understanding how it all works under the hood. Juniors don’t need
to ask Seniors anymore, nor even grep SO: <em>for every problem there is a solution
which is simple, clean and wrong</em>. There is no room for ideas: why would I think
about architecture, if I already have all that callback zoo, helpers and
even validations out of the box?</p>
<p>When I started my career, there was no SO, no documentation, no open source to
learn from. We all were forced to come up with ideas, because there was no source
to steal any from. When we had a database, we started to code with handling
the connection on socket level. Here you are either come up with ideas, or die.</p>
<p>And during this time there was no issue to ask anyone about anything and
to come up with any weird proposal.</p>
<p>Nowadays we have tons of information, we are literally sinking in.
Besides tutorials, video lessons, crash courses of any flavor, we have all these
so called technical blogs, mostly written by not so experienced people.
And the worst thing is: since they have no sufficient knowledge about the technology,
they do write motivational posts. Like that one linked above.
“Make love not war” and all that crap.</p>
<p>It does not work that way in software development. When I entered the field,
I was thankful for any advise, even whether I was beaten by a baseball bat
for getting it. We all raised ideas and 99% of them were ridiculed by Seniors,
and each subsequent episode was another lesson, that, as we unconditionally believed,
headed us towards gaining skills and harvesting expertise.</p>
<p>So, the summing up, the rumors about Seniors harassing Juniors for raising ideas
are heavily exaggerated. Seniors love Juniors bringing ideas on table and most
of them are ready to make their own job during the weekend to free a time
to talk to Juniors to discuss their ideas. The thing is: Juniors are [in my experience]
often pretty fine with just getting paid for what they can do right now.
No ideas at all. And that’s the real issue.</p>
<p>If any Senior is indeed concerned to help Juniors to grow up, instead of being
overpolite and timid—they should find ways to intrigue Juniors to be greedy to
learn new stuff and raise new ideas.</p>
<p>Seniors are not babysitters for Juniors after all, but mentors.</p>
Unveil Erlang Code of Your Elixir Project2017-09-01T00:00:00+00:00https://rocket-science.ru/hacking/2017/09/01/unveil_erlang_code_of_your_elixir_project<h2 id="elixir-is-erlang-in-a-nutshell">Elixir is Erlang in a nutshell</h2>
<blockquote>
<p><code class="language-plaintext highlighter-rouge">Elixir</code> leverages the Erlang VM […]<br />
<small><a href="https://elixir-lang.org/">elixir-lang.org</a></small></p>
</blockquote>
<p>What does mean “leverages”? It means Elixir is compiled into BEAMs, and the latter are
executed in Erlang VM. Elixir brings a lot of goodness, like arguable better syntax and
extensive macro support, but after all it’s the language on top of Erlang.</p>
<p>BEAMs are known to support decompilation back into Erlang code (providing they were
compiled with <code class="language-plaintext highlighter-rouge">debug_info</code> compiler option on.)</p>
<p>So, let’s configure our project to support decompilation back into Erlang code
in <code class="language-plaintext highlighter-rouge">MIX_ENV=dev</code> environment. That is relatively easy and might be helpful in some cases:
both educational and “wtf–investigations.” To do so we’ll need three things:</p>
<h2 id="update-your-configdevexs-to-enable-debug_info">Update your <code class="language-plaintext highlighter-rouge">config/dev.exs</code> to enable <code class="language-plaintext highlighter-rouge">debug_info</code></h2>
<p>Put this line in the very top of your <code class="language-plaintext highlighter-rouge">config/dev.exs</code> file:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># https://hexdocs.pm/elixir/Code.html#compiler_options/1</span>
<span class="no">Code</span><span class="o">.</span><span class="n">compiler_options</span><span class="p">(</span><span class="ss">debug_info:</span> <span class="no">true</span><span class="p">)</span>
</code></pre></div></div>
<p>According to the <a href="https://hexdocs.pm/elixir/Code.html#compiler_options/1">official documentation</a>,</p>
<blockquote>
<p><code class="language-plaintext highlighter-rouge">:debug_true</code> option is set to <code class="language-plaintext highlighter-rouge">true</code> to retain debug information in the compiled
module; this allows a developer to reconstruct the original source code, <code class="language-plaintext highlighter-rouge">false</code> by default.<br />
[…]<br />
These options are global since they are stored by Elixir’s Code Server.</p>
</blockquote>
<h2 id="prepare-the-elixir-beams-to-be-available">Prepare the Elixir BEAMs to be available</h2>
<p>Before we are to create the script itself, let’s discover what do we need to support
Elixir code decompilation. Right: Elixir’s core compiled BEAMs themselves,
otherwise we would hardly decompile Elixir’s own goodness, like comprehensions etc.</p>
<p>I have no idea whether it’s possible with a standard Elixir distribution, but lickily
I nevertheless use trunk version (managed with <a href="https://github.com/mururu/exenv"><code class="language-plaintext highlighter-rouge">exenv</code></a>
version manager.) So, install <code class="language-plaintext highlighter-rouge">exenv</code> unless you have already done it, download
the source code of Elixir somewhere and compile it:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">cd</span> /usr/local/src <span class="o">&&</span> <span class="se">\</span>
git clone https://github.com/elixir-lang/elixir.git <span class="o">&&</span> <span class="se">\</span>
<span class="nb">cd </span>elixir <span class="o">&&</span> <span class="se">\</span>
make clean <span class="nb">test</span>
</code></pre></div></div>
<p>If everything went good, teach your <code class="language-plaintext highlighter-rouge">exenv</code> to use trunk version:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">cd</span> ~/.exenv/versions <span class="o">&&</span> <span class="se">\</span>
<span class="nb">ln</span> <span class="nt">-s</span> /usr/local/src/elixir trunk <span class="o">&&</span> <span class="se">\</span>
<span class="nb">cd </span>PATH_TO_MY_ELIXIR_PROJECT <span class="o">&&</span> <span class="se">\</span>
exenv <span class="nb">local </span>trunk
</code></pre></div></div>
<p>Check we are all set:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>iex
Erlang/OTP 20 <span class="o">[</span>erts-9.0] <span class="o">[</span><span class="nb">source</span><span class="o">]</span> <span class="o">[</span>64-bit] <span class="o">[</span>...]
Interactive Elixir <span class="o">(</span>1.6.0-dev<span class="o">)</span> - press Ctrl+C to <span class="nb">exit</span> <span class="o">(</span><span class="nb">type </span>h<span class="o">()</span> ENTER <span class="k">for </span><span class="nb">help</span><span class="o">)</span>
iex|1 ▶
</code></pre></div></div>
<h2 id="create-a-script-to-unveil-the-code">Create a script to unveil the code</h2>
<p>Copy the following lines into <code class="language-plaintext highlighter-rouge">/usr/local/bin/delixir</code>:</p>
<div class="language-erlang highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">#</span><span class="o">!/</span><span class="n">usr</span><span class="o">/</span><span class="n">bin</span><span class="o">/</span><span class="n">env</span> <span class="n">escript</span>
<span class="c">% -*- mode: erlang -*-
</span>
<span class="nf">main</span><span class="p">([</span><span class="nv">BeamFile</span><span class="p">])</span> <span class="o">-></span>
<span class="nv">Dir</span> <span class="o">=</span> <span class="s">"/usr/local/src/elixir/lib/elixir/ebin"</span><span class="p">,</span>
<span class="nn">code</span><span class="p">:</span><span class="nf">add_patha</span><span class="p">(</span><span class="nv">Dir</span><span class="p">),</span>
<span class="p">{</span><span class="n">ok</span><span class="p">,{_,[{</span><span class="n">abstract_code</span><span class="p">,{_,</span><span class="nv">AC</span><span class="p">}}]}}</span> <span class="o">=</span> <span class="nn">beam_lib</span><span class="p">:</span><span class="nf">chunks</span><span class="p">(</span><span class="nv">BeamFile</span><span class="p">,[</span><span class="n">abstract_code</span><span class="p">]),</span>
<span class="nn">io</span><span class="p">:</span><span class="nf">fwrite</span><span class="p">(</span><span class="s">"</span><span class="si">~s~n</span><span class="s">"</span><span class="p">,</span> <span class="p">[</span><span class="nn">erl_prettypr</span><span class="p">:</span><span class="nf">format</span><span class="p">(</span><span class="nn">erl_syntax</span><span class="p">:</span><span class="nf">form_list</span><span class="p">(</span><span class="nv">AC</span><span class="p">))]).</span>
</code></pre></div></div>
<p>Make the file executable with <code class="language-plaintext highlighter-rouge">chmod +x /usr/local/bin/delixir</code> and you are all set.
Let’s check how it works:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>delixir _build/dev/lib/my_app/ebin/Elixir.MyApp.beam
</code></pre></div></div>
<p>You should see something like:</p>
<div class="language-erlang highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">-</span><span class="ni">file</span><span class="p">(</span><span class="s">"/home/user/proj/my_app/lib/"</span>
<span class="s">"my_app.ex"</span><span class="p">,</span>
<span class="mi">1</span><span class="p">).</span>
<span class="p">-</span><span class="ni">module</span><span class="p">(</span><span class="n">'Elixir.MyApp'</span><span class="p">).</span>
<span class="p">-</span><span class="ni">compile</span><span class="p">(</span><span class="n">no_auto_import</span><span class="p">).</span>
<span class="p">-</span><span class="ni">behaviour</span><span class="p">(</span><span class="n">'Elixir.GenServer'</span><span class="p">).</span>
<span class="p">-</span><span class="ni">export</span><span class="p">([</span><span class="n">'__info__'</span><span class="o">/</span><span class="mi">1</span><span class="p">,</span> <span class="p">...]).</span>
<span class="p">-</span><span class="ni">spec</span> <span class="nf">'__info__'</span><span class="p">(</span><span class="n">attributes</span> <span class="p">|</span> <span class="n">compile</span> <span class="p">|</span> <span class="n">exports</span> <span class="p">|</span>
<span class="n">functions</span> <span class="p">|</span> <span class="n">macros</span> <span class="p">|</span> <span class="nb">md5</span> <span class="p">|</span> <span class="n">module</span><span class="p">)</span> <span class="o">-></span> <span class="nf">atom</span><span class="p">()</span> <span class="p">|</span>
<span class="p">[{</span><span class="nf">atom</span><span class="p">(),</span> <span class="nf">any</span><span class="p">()}</span> <span class="p">|</span>
<span class="p">{</span><span class="nf">atom</span><span class="p">(),</span> <span class="nf">byte</span><span class="p">(),</span>
<span class="nf">integer</span><span class="p">()}].</span>
<span class="nf">'__info__'</span><span class="p">(</span><span class="n">functions</span><span class="p">)</span> <span class="o">-></span>
</code></pre></div></div>
<p>etc. This is an Erlang code that is basically exactly what Elixir was
“transpiled” into before compilation. Or, better to say, this is how
Erlang virtual machine sees your Elixir code.</p>
Workflow as FSM: lost transitions2017-07-27T00:00:00+00:00https://rocket-science.ru/hacking/2017/07/27/workflow-as-fsm<h2 id="workflow-as-fsm">Workflow as FSM</h2>
<blockquote>
<p>A <strong>finite-state machine</strong> (<strong>FSM</strong>) or <strong>finite-state automaton</strong> (<strong>FSA</strong>,
plural: <em>automata</em>), <strong>finite automaton</strong>, or simply <strong>a state machine</strong>,
is a mathematical model of computation. It is an abstract machine that can be
in exactly one of a finite number of states at any given time.<br />
The FSM can change from one state to another in response to some external inputs;
the change from one state to another is called a transition.<br />
An FSM is defined by a list of its states, its initial state,
and the conditions for each transition.<br />
<small><a href="https://en.wikipedia.org/wiki/Finite-state_machine">Wikipedia</a></small></p>
</blockquote>
<p><em>Wikipedia</em> provides a coin-operated turnstile as an example of state machine:</p>
<p><img src="https://upload.wikimedia.org/wikipedia/commons/9/9e/Turnstile_state_machine_colored.svg" alt="State diagram for a turnstile" /></p>
<p>Even from this example it’s clear that the definition above is not accurate.
To be precise, the statement “the change from one state to another is called
a transition” is actually incomplete. Check the <code class="language-plaintext highlighter-rouge">push</code> transition when the state
is <code class="language-plaintext highlighter-rouge">locked</code> and <code class="language-plaintext highlighter-rouge">coin</code> transition when the state is <code class="language-plaintext highlighter-rouge">unlocked</code>. Both <em>do not change
the state</em>.</p>
<p>This terminology inaccuracy leads to misunderstanding of how FSM should be built
to become a friend of a developer, not an enemy to fight against.</p>
<h2 id="bonus-program-tracking-naïve-approach">Bonus program tracking (naïve approach)</h2>
<p>Let’s imagine we have a bonus program to be developed for our clients. Each
client receives a fixed number of bonus points for each subsequent action,
and, as soon as the total amount reaches some value, the customer is awarded
with something fruitful. We want to implement bonus as a model, evidently
having three states: <code class="language-plaintext highlighter-rouge">:initial</code>, <code class="language-plaintext highlighter-rouge">:collecting</code> and <code class="language-plaintext highlighter-rouge">:bonus</code>. The naïve FSM
would look like:</p>
<p><img src="/img/fsm_bonus_naive.png" alt="Naïve FSM" /></p>
<p>Each subsequent bonus point moves us towards <code class="language-plaintext highlighter-rouge">:bonus</code> state. The code, supporting
this workflow could look somewhat like below (I am using
<a href="https://github.com/geekq/workflow"><code class="language-plaintext highlighter-rouge">workflow</code></a> gem here, but it really does not
matter.)</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">workflow</span> <span class="k">do</span>
<span class="n">state</span> <span class="ss">:initial</span> <span class="k">do</span>
<span class="n">event</span> <span class="ss">:gain</span><span class="p">,</span> <span class="ss">transitions_to: :collecting</span>
<span class="k">end</span>
<span class="n">state</span> <span class="ss">:collecting</span> <span class="k">do</span>
<span class="n">event</span> <span class="ss">:bonus_reached</span><span class="p">,</span> <span class="ss">transitions_to: :bonus</span>
<span class="k">end</span>
<span class="n">state</span> <span class="ss">:bonus</span> <span class="k">do</span>
<span class="n">event</span> <span class="ss">:restart</span><span class="p">,</span> <span class="ss">transitions_to: :initial</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">coins_gained!</span><span class="p">(</span><span class="n">amount</span><span class="p">)</span>
<span class="vi">@coins</span> <span class="o">=</span> <span class="vi">@coins</span><span class="p">.</span><span class="nf">to_i</span> <span class="o">+</span> <span class="n">amount</span>
<span class="k">if</span> <span class="vi">@coins</span> <span class="o">>=</span> <span class="mi">100</span>
<span class="n">bonus_reached!</span>
<span class="vi">@coins</span> <span class="o">-=</span> <span class="mi">100</span>
<span class="n">restart!</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Right?—Unfortunately, no. This is not a robust workflow. It shifts too much of
it’s duties elsewhere. It does not control intermediate state (<code class="language-plaintext highlighter-rouge">:collecting</code>)
properly. Imagine, two days after launch we were asked by our marketing department
to email customers when their points amount reaches 20 and 50 points. What would
we do?—Yes, we would blow our <code class="language-plaintext highlighter-rouge">coins_gained!</code> method up:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">coins_gained!</span><span class="p">(</span><span class="n">amount</span><span class="p">)</span>
<span class="n">old_amount</span> <span class="o">=</span> <span class="vi">@coins</span><span class="p">.</span><span class="nf">to_i</span>
<span class="n">new_amount</span> <span class="o">=</span> <span class="n">old_amount</span> <span class="o">+</span> <span class="n">amount</span>
<span class="k">if</span> <span class="n">old_amount</span> <span class="o"><</span> <span class="mi">20</span> <span class="o">&&</span> <span class="n">new_amount</span> <span class="o">>=</span> <span class="mi">20</span> <span class="o">||</span>
<span class="n">old_amount</span> <span class="o"><</span> <span class="mi">50</span> <span class="o">&&</span> <span class="n">new_amount</span> <span class="o">>=</span> <span class="mi">50</span>
<span class="n">send_email_to_customer</span>
<span class="k">end</span>
<span class="vi">@coins</span> <span class="o">=</span> <span class="n">new_amount</span>
<span class="k">if</span> <span class="vi">@coins</span> <span class="o">>=</span> <span class="mi">100</span>
<span class="n">bonus_reached!</span>
<span class="vi">@coins</span> <span class="o">-=</span> <span class="mi">100</span>
<span class="n">restart!</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Yeah, yeah, I know that the whole checking and sending email might be extracted
to it’s own method, as we always do in Rails making our models finally
to contain a dozillion of private helpers calling other private helpers
to satisfy rubocop with her 10-LOC-per-method-body limit.</p>
<h2 id="better-approach">Better approach</h2>
<p>The real issue is: our workflow is not robust. There is a transition we
missed: noop from <code class="language-plaintext highlighter-rouge">:collecting</code> to <code class="language-plaintext highlighter-rouge">:collecting</code> state:</p>
<p><img src="/img/fsm_bonus_correct.png" alt="Proper FSM" /></p>
<p>The workflow transitions are now named properly in the first place.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">workflow</span> <span class="k">do</span>
<span class="n">state</span> <span class="ss">:initial</span> <span class="k">do</span>
<span class="n">event</span> <span class="ss">:gain</span><span class="p">,</span> <span class="ss">transitions_to: :collecting</span>
<span class="k">end</span>
<span class="n">state</span> <span class="ss">:collecting</span> <span class="k">do</span>
<span class="n">event</span> <span class="ss">:gain</span><span class="p">,</span> <span class="ss">transitions_to: :collecting</span>
<span class="n">event</span> <span class="ss">:bonus_reached</span><span class="p">,</span> <span class="ss">transitions_to: :bonus</span>
<span class="k">end</span>
<span class="n">state</span> <span class="ss">:bonus</span> <span class="k">do</span>
<span class="n">event</span> <span class="ss">:restart</span><span class="p">,</span> <span class="ss">transitions_to: :initial</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Better?—Yes, but still not perfect:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">gain</span><span class="p">(</span><span class="n">amount</span><span class="p">)</span>
<span class="vi">@coins</span> <span class="o">=</span> <span class="vi">@coins</span><span class="p">.</span><span class="nf">to_i</span> <span class="o">+</span> <span class="n">amount</span>
<span class="n">bonus_reached!</span> <span class="k">if</span> <span class="vi">@coins</span> <span class="o">>=</span> <span class="mi">100</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">bonus_reached</span>
<span class="vi">@coins</span> <span class="o">-=</span> <span class="mi">100</span>
<span class="n">restart!</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">on_transition</span><span class="p">(</span><span class="n">amount</span><span class="p">)</span>
<span class="n">send_email_to_customer</span> <span class="k">if</span> <span class="n">need_email?</span> <span class="c1"># encapsulated check</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The code is now clean, no explicit <code class="language-plaintext highlighter-rouge">if</code> on checking what state to be next exists,
the email will be sent despite the target state. What could be improved?—Well,
sending email is <em>also a state</em>.</p>
<h2 id="overdesigned-approach">Overdesigned approach</h2>
<p>Should not we also introduce a particular state for “sending an email”?</p>
<p><img src="/img/fsm_bonus_overdesigned.png" alt="Overdesigned FSM" /></p>
<p>The answer is: no, we probably shouldn’t. It will just make things cumbersome,
it will implicitly require more unnecessary checks, it will make the transitions
less explicit:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">gain</span><span class="p">(</span><span class="n">amount</span><span class="p">)</span>
<span class="o">...</span>
<span class="n">emailing!</span> <span class="k">if</span> <span class="n">need_email?</span>
<span class="c1"># what if bonus is reached?</span>
<span class="o">...</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">emailing</span>
<span class="n">send_email_to_customer</span>
<span class="n">email_sent!</span>
<span class="c1"># back to collecting ⇒ void transition implies</span>
<span class="c1"># potential glitches when</span>
<span class="c1"># handlers are implemented</span>
<span class="k">end</span>
</code></pre></div></div>
<h2 id="conslusion">Conslusion</h2>
<p>When some action is taken place from outside, it should be covered with
transition. Even if it’s same-state transition.</p>
<p>When the action is completely internal, it does not deserve it’s state.</p>
StringNaming to call UTF8 by name2017-06-20T00:00:00+00:00https://rocket-science.ru/hacking/2017/06/20/string-naming-module<h2 id="wtf-utf8">WTF UTF8?</h2>
<p>There are many good tutorials on how Elixir deals with UTF8 and why it is the only
language so far that properly converts “José” string to capitals out of the box.
If you are unfamiliar with the subject, I would recommend the brilliant writing
<a href="https://www.bignerdranch.com/blog/unicode-and-utf-8-explained/">Unicode and UTF-8 Explained</a>
by Nathan Long.</p>
<p>In a nutshell, Elixir’s <a href="https://hexdocs.pm/elixir/String.html#upcase/1"><code class="language-plaintext highlighter-rouge">String.upcase/1</code></a>
walks through the string, performs a normalization (for accents and other combined
symbols), and finally converts graphemes to their capital representation. This
process is fully automated and—which is more important—is always up-to-date,
since it reads the conversion rules directly from Consortium’s
<a href="http://www.unicode.org/Public/UCD/latest/ucd/">definition files</a>. Two used in
case conversion are <code class="language-plaintext highlighter-rouge">UnicodeData.txt</code> and <code class="language-plaintext highlighter-rouge">SpecialCasing.txt</code>, if you one’s curious.</p>
<p>In this post we’ll build the relatively same stuff to address fancy UTF8 symbols
by their names. The whole codebase contains 119 LOCs. In the end we’ll yield
a bundle of modules, providing functions returning the UTF8 symbol given it’s name
as a function name. The practical use of this package is beyond the scope of this
post, but I am positive there could be many.</p>
<h2 id="bruteforce-approach">Bruteforce approach</h2>
<p>So far, so good. Our goal is to end up with something like:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">|</span><span class="mi">1</span> <span class="err">▶</span> <span class="no">StringNaming</span><span class="o">.</span><span class="no">AnimalSymbols</span><span class="o">.</span><span class="n">monkey</span>
<span class="s2">"🐒"</span>
</code></pre></div></div>
<p>The gracefully stolen from native Elixir <a href="https://github.com/elixir-lang/elixir/blob/master/lib/elixir/unicode/properties.ex"><code class="language-plaintext highlighter-rouge">unicode/properties.ex</code></a> approach would be to:</p>
<p>— parse text file provided by Consortium;<br />
— prepare the data structure to walk through;<br />
— build the functions in compile time using meta programming features.</p>
<p>This is all mighty and will prevail, except it ain’t so.</p>
<p>There is no problem reading the file, as well as the data structure needed
is to be built without a glitch. The problem is that we need nested modules
to be produced on the fly. Look:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">|</span><span class="mi">2</span> <span class="err">▶</span> <span class="no">StringNaming</span><span class="o">.</span><span class="no">AnimalSymbols</span><span class="o">.<</span><span class="no">TAB</span><span class="o">></span>
<span class="no">Baby</span> <span class="no">Bactrian</span> <span class="no">Dromedary</span> <span class="no">Fox</span>
<span class="no">Front</span> <span class="no">Hatching</span> <span class="no">Lady</span> <span class="no">Lion</span>
<span class="no">Paw</span> <span class="no">Spiral</span> <span class="no">Tropical</span> <span class="no">Unicorn</span>
<span class="no">Water</span> <span class="n">ant</span><span class="o">/</span><span class="mi">0</span> <span class="n">bat</span><span class="o">/</span><span class="mi">0</span> <span class="n">bird</span><span class="o">/</span><span class="mi">0</span>
<span class="n">blowfish</span><span class="o">/</span><span class="mi">0</span> <span class="n">boar</span><span class="o">/</span><span class="mi">0</span> <span class="n">bug</span><span class="o">/</span><span class="mi">0</span> <span class="n">butterfly</span><span class="o">/</span><span class="mi">0</span>
<span class="n">cat</span><span class="o">/</span><span class="mi">0</span> <span class="n">chicken</span><span class="o">/</span><span class="mi">0</span> <span class="n">chipmunk</span><span class="o">/</span><span class="mi">0</span> <span class="n">cow</span><span class="o">/</span><span class="mi">0</span>
<span class="n">crab</span><span class="o">/</span><span class="mi">0</span> <span class="n">crocodile</span><span class="o">/</span><span class="mi">0</span> <span class="n">deer</span><span class="o">/</span><span class="mi">0</span> <span class="n">dog</span><span class="o">/</span><span class="mi">0</span>
<span class="n">dolphin</span><span class="o">/</span><span class="mi">0</span> <span class="n">dragon</span><span class="o">/</span><span class="mi">0</span> <span class="n">duck</span><span class="o">/</span><span class="mi">0</span> <span class="n">eagle</span><span class="o">/</span><span class="mi">0</span>
<span class="n">elephant</span><span class="o">/</span><span class="mi">0</span> <span class="n">fish</span><span class="o">/</span><span class="mi">0</span> <span class="n">goat</span><span class="o">/</span><span class="mi">0</span> <span class="n">gorilla</span><span class="o">/</span><span class="mi">0</span>
<span class="n">honeybee</span><span class="o">/</span><span class="mi">0</span> <span class="n">horse</span><span class="o">/</span><span class="mi">0</span> <span class="n">koala</span><span class="o">/</span><span class="mi">0</span> <span class="n">leopard</span><span class="o">/</span><span class="mi">0</span>
<span class="n">lizard</span><span class="o">/</span><span class="mi">0</span> <span class="n">monkey</span><span class="o">/</span><span class="mi">0</span> <span class="n">mouse</span><span class="o">/</span><span class="mi">0</span> <span class="n">octopus</span><span class="o">/</span><span class="mi">0</span>
<span class="n">owl</span><span class="o">/</span><span class="mi">0</span> <span class="n">ox</span><span class="o">/</span><span class="mi">0</span> <span class="n">penguin</span><span class="o">/</span><span class="mi">0</span> <span class="n">pig</span><span class="o">/</span><span class="mi">0</span>
<span class="n">poodle</span><span class="o">/</span><span class="mi">0</span> <span class="n">rabbit</span><span class="o">/</span><span class="mi">0</span> <span class="n">ram</span><span class="o">/</span><span class="mi">0</span> <span class="n">rat</span><span class="o">/</span><span class="mi">0</span>
<span class="n">rhinoceros</span><span class="o">/</span><span class="mi">0</span> <span class="n">rooster</span><span class="o">/</span><span class="mi">0</span> <span class="n">scorpion</span><span class="o">/</span><span class="mi">0</span> <span class="n">shark</span><span class="o">/</span><span class="mi">0</span>
<span class="n">sheep</span><span class="o">/</span><span class="mi">0</span> <span class="n">shrimp</span><span class="o">/</span><span class="mi">0</span> <span class="n">snail</span><span class="o">/</span><span class="mi">0</span> <span class="n">snake</span><span class="o">/</span><span class="mi">0</span>
<span class="n">spider</span><span class="o">/</span><span class="mi">0</span> <span class="n">squid</span><span class="o">/</span><span class="mi">0</span> <span class="n">tiger</span><span class="o">/</span><span class="mi">0</span> <span class="n">turkey</span><span class="o">/</span><span class="mi">0</span>
<span class="n">turtle</span><span class="o">/</span><span class="mi">0</span> <span class="n">whale</span><span class="o">/</span><span class="mi">0</span>
<span class="n">iex</span><span class="o">|</span><span class="mi">2</span> <span class="err">▶</span> <span class="no">StringNaming</span><span class="o">.</span><span class="no">AnimalSymbols</span><span class="o">.</span><span class="no">Baby</span><span class="o">.</span><span class="n">chick</span>
<span class="s2">"🐤"</span>
</code></pre></div></div>
<p>From the above we see that there are nested modules along with plain functions
inside nearly each module. And, unluckily, Elixir does not allow to re-open
modules as Ruby does. So, we are to use recursion in our metaprogramming
adventure. Exciting?</p>
<h3 id="reading-the-file">Reading the file</h3>
<p>There is nothing special with reading the file: it’s plain text in very
simple format. Category names are prepended with <code class="language-plaintext highlighter-rouge">"@\t"</code> making it easy
to pattern match, codepoints are followed by their names: take and parse.</p>
<p>On that stage we simply collect everything in the list of tuples
<code class="language-plaintext highlighter-rouge">{code, name, category}</code>. E.g. for the monkey face above, this tuple would
be <code class="language-plaintext highlighter-rouge">{"1F412", "MONKEY", "AnimalSymbols"}</code>.</p>
<p>The boring code, in case anybody is curious, <a href="https://github.com/am-kantox/string_naming/blob/master/lib/string_naming.ex#L81-L97">may be found here</a>.</p>
<h3 id="converting-the-list-to-nested-map">Converting the list to nested map</h3>
<p>Second stage is to convert everything to the nested map, so that later on
we could recursively iterate it. This is
<a href="https://github.com/am-kantox/string_naming/blob/master/lib/string_naming.ex#L99-L112">plain old good Elixir</a>
as well: no tricks, no exciting shining ideas.</p>
<p>Codepoint is the leaf in each nested map.</p>
<h3 id="building-modules">Building modules</h3>
<p>That is the part the whole post was written for. Impatient readers might
just read <a href="https://github.com/am-kantox/string_naming/blob/master/lib/string_naming.ex#L1-L36">these 36 LOCs</a>,
for others we’ll walk through step by step.</p>
<p>The first pitfall is that we need to call to produce the nested modules from inside
the definition of the currently operated one. That said, we need to declare
the dedicated module to deal with that, otherwise scopes won’t allow us to do that.
BTW, we’ll <code class="language-plaintext highlighter-rouge">:code.delete</code> and <code class="language-plaintext highlighter-rouge">:code.purge</code> this helper module afterward.</p>
<p>The first step is to write a flat level iteration. That is relatively easy:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="n">nesteds</span><span class="p">(</span><span class="n">nested</span><span class="p">,</span> <span class="p">%{}</span> <span class="o">=</span> <span class="n">map</span><span class="p">)</span> <span class="k">do</span>
<span class="no">Enum</span><span class="o">.</span><span class="n">each</span><span class="p">(</span><span class="n">map</span><span class="p">,</span> <span class="k">fn</span>
<span class="p">{</span><span class="n">_key</span><span class="p">,</span> <span class="n">code</span><span class="p">}</span> <span class="ow">when</span> <span class="n">is_binary</span><span class="p">(</span><span class="n">code</span><span class="p">)</span> <span class="o">-></span> <span class="ss">:ok</span> <span class="c1"># leaf, skip it</span>
<span class="p">{</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="p">}</span> <span class="o">-></span>
<span class="n">mod</span> <span class="o">=</span> <span class="ss">:lists</span><span class="o">.</span><span class="n">reverse</span><span class="p">([</span><span class="n">k</span> <span class="o">|</span> <span class="ss">:lists</span><span class="o">.</span><span class="n">reverse</span><span class="p">(</span><span class="n">nested</span><span class="p">)])</span>
<span class="no">StringNaming</span><span class="o">.</span><span class="no">H</span><span class="o">.</span><span class="n">nested_module</span><span class="p">(</span><span class="n">mod</span><span class="p">,</span> <span class="n">v</span><span class="p">)</span>
<span class="k">end</span><span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The above just iterates the map and calls a producer for all the nested
modules. That simple. <code class="language-plaintext highlighter-rouge">StringNaming.H.nested_module/2</code> is where the deal
happens. First of all, we are to split functions (leaves) and modules (branches).
We could not prepare this in advance, since we had no clue at parsing stage
whether this would be a leaf or not.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span><span class="n">funs</span><span class="p">,</span> <span class="n">mods</span><span class="p">]</span> <span class="o">=</span> <span class="no">Enum</span><span class="o">.</span><span class="n">reduce</span><span class="p">(</span><span class="n">children</span><span class="p">,</span> <span class="p">[%{},</span> <span class="p">%{}],</span> <span class="k">fn</span>
<span class="p">{</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="p">},</span> <span class="p">[</span><span class="n">funs</span><span class="p">,</span> <span class="n">mods</span><span class="p">]</span> <span class="ow">when</span> <span class="n">is_binary</span><span class="p">(</span><span class="n">v</span><span class="p">)</span> <span class="o">-></span>
<span class="p">[</span><span class="no">Map</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="n">funs</span><span class="p">,</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="p">),</span> <span class="n">mods</span><span class="p">]</span>
<span class="p">{</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="p">},</span> <span class="p">[</span><span class="n">funs</span><span class="p">,</span> <span class="n">mods</span><span class="p">]</span> <span class="o">-></span>
<span class="p">[</span><span class="n">funs</span><span class="p">,</span> <span class="no">Map</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="n">mods</span><span class="p">,</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="p">)]</span>
<span class="k">end</span><span class="p">)</span>
</code></pre></div></div>
<p>We consider binaries to be a codepoint value, and, hence, a leaf. Yes, I am aware
of <a href="https://hexdocs.pm/elixir/Enum.html#split_with/2"><code class="language-plaintext highlighter-rouge">Enum.split_with/2</code></a>, but
here it’s simpler (and faster) to produce maps explicitly. Now we have two maps.
It’s time to rock!</p>
<p>The first—hacky and basically wrong—approach was to use
<a href="https://hexdocs.pm/elixir/Code.html#eval_quoted/3"><code class="language-plaintext highlighter-rouge">Code.eval_quoted/3</code></a>,
since I could not figure out how to dynamically create a module inside other module:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">Module</span><span class="o">.</span><span class="n">concat</span><span class="p">(</span><span class="n">mod</span><span class="p">)</span> <span class="k">do</span>
<span class="no">Enum</span><span class="o">.</span><span class="n">each</span><span class="p">(</span><span class="n">funs</span><span class="p">,</span> <span class="k">fn</span> <span class="p">{</span><span class="n">name</span><span class="p">,</span> <span class="n">value</span><span class="p">}</span> <span class="o">-></span>
<span class="c1"># name might be numeric, e.g. 1 ⇒ make it a proper atom here</span>
<span class="n">name</span> <span class="o">=</span> <span class="n">name</span>
<span class="o">|></span> <span class="no">String</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="sr">~r/\A(\d)/</span><span class="p">,</span> <span class="s2">"N_</span><span class="se">\\</span><span class="s2">1"</span><span class="p">)</span>
<span class="o">|></span> <span class="no">Macro</span><span class="o">.</span><span class="n">underscore</span>
<span class="o">|></span> <span class="no">String</span><span class="o">.</span><span class="n">to_atom</span>
<span class="c1"># abstract syntax tree of</span>
<span class="c1"># ★ def monkey, do: "🐒"</span>
<span class="c1"># value is a codepoint</span>
<span class="n">ast</span> <span class="o">=</span> <span class="kn">quote</span> <span class="k">do</span>
<span class="k">def</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">name</span><span class="p">)()</span> <span class="k">do</span>
<span class="o"><<</span><span class="no">String</span><span class="o">.</span><span class="n">to_integer</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">value</span><span class="p">),</span> <span class="mi">16</span><span class="p">)::</span><span class="n">utf8</span><span class="o">>></span>
<span class="k">end</span>
<span class="k">end</span>
<span class="no">Code</span><span class="o">.</span><span class="n">eval_quoted</span><span class="p">(</span><span class="n">ast</span><span class="p">,</span> <span class="p">[</span><span class="ss">name:</span> <span class="n">name</span><span class="p">,</span> <span class="ss">value:</span> <span class="n">value</span><span class="p">],</span> <span class="n">__ENV__</span><span class="p">)</span>
<span class="k">end</span><span class="p">)</span>
<span class="c1"># TODO: def __all__</span>
<span class="no">StringNaming</span><span class="o">.</span><span class="no">H</span><span class="o">.</span><span class="n">nesteds</span><span class="p">(</span><span class="n">mod</span><span class="p">,</span> <span class="n">mods</span><span class="p">)</span> <span class="c1"># call back for the nesteds</span>
<span class="k">end</span>
</code></pre></div></div>
<p>I have posted a question on SO and
<a href="https://stackoverflow.com/a/44852119/2035262">Dogbert helped</a> me to make
the code clean with <a href="https://hexdocs.pm/elixir/Module.html#create/3"><code class="language-plaintext highlighter-rouge">Module.create/3</code></a>:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">ast</span> <span class="o">=</span> <span class="n">for</span> <span class="p">{</span><span class="n">name</span><span class="p">,</span> <span class="n">value</span><span class="p">}</span> <span class="o"><-</span> <span class="n">funs</span> <span class="k">do</span>
<span class="n">name</span> <span class="o">=</span> <span class="n">name</span> <span class="o">|></span> <span class="no">String</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="sr">~r/\A(\d)/</span><span class="p">,</span> <span class="s2">"N_</span><span class="se">\\</span><span class="s2">1"</span><span class="p">)</span> <span class="o">|></span> <span class="no">Macro</span><span class="o">.</span><span class="n">underscore</span> <span class="o">|></span> <span class="no">String</span><span class="o">.</span><span class="n">to_atom</span>
<span class="kn">quote</span> <span class="k">do</span><span class="p">:</span> <span class="k">def</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">name</span><span class="p">)(),</span> <span class="k">do</span><span class="p">:</span> <span class="o"><<</span><span class="no">String</span><span class="o">.</span><span class="n">to_integer</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">value</span><span class="p">),</span> <span class="mi">16</span><span class="p">)::</span><span class="n">utf8</span><span class="o">>></span>
<span class="k">end</span>
<span class="c1"># TODO: def __all__</span>
<span class="no">Module</span><span class="o">.</span><span class="n">create</span><span class="p">(</span><span class="no">Module</span><span class="o">.</span><span class="n">concat</span><span class="p">(</span><span class="n">mod</span><span class="p">),</span> <span class="n">ast</span><span class="p">,</span> <span class="no">Macro</span><span class="o">.</span><span class="no">Env</span><span class="o">.</span><span class="n">location</span><span class="p">(</span><span class="n">__ENV__</span><span class="p">))</span>
<span class="no">StringNaming</span><span class="o">.</span><span class="no">H</span><span class="o">.</span><span class="n">nesteds</span><span class="p">(</span><span class="n">mod</span><span class="p">,</span> <span class="n">mods</span><span class="p">)</span>
</code></pre></div></div>
<hr />
<p>That is basically it. The only thing left is to implement <code class="language-plaintext highlighter-rouge">__MODULE__.__all__/0</code> function
to return a keyword list of all the functions available, with their values:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="k">def</span> <span class="n">__all__</span> <span class="k">do</span>
<span class="ss">:functions</span>
<span class="o">|></span> <span class="bp">__MODULE__</span><span class="o">.</span><span class="n">__info__</span><span class="p">()</span>
<span class="o">|></span> <span class="no">Enum</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="k">fn</span>
<span class="p">{</span><span class="ss">:__all__</span><span class="p">,</span> <span class="mi">0</span><span class="p">}</span> <span class="o">-></span> <span class="no">nil</span>
<span class="p">{</span><span class="n">k</span><span class="p">,</span> <span class="mi">0</span><span class="p">}</span> <span class="o">-></span> <span class="p">{</span><span class="n">k</span><span class="p">,</span> <span class="n">apply</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="n">k</span><span class="p">,</span> <span class="p">[])}</span>
<span class="n">_</span> <span class="o">-></span> <span class="no">nil</span>
<span class="k">end</span><span class="p">)</span>
<span class="o">|></span> <span class="no">Enum</span><span class="o">.</span><span class="n">filter</span><span class="p">(</span><span class="o">&</span> <span class="nv">&1</span><span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Now we just call</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">StringNaming</span><span class="o">.</span><span class="no">H</span><span class="o">.</span><span class="n">nesteds</span><span class="p">([</span><span class="s2">"String"</span><span class="p">,</span> <span class="s2">"Naming"</span><span class="p">],</span> <span class="n">names_tree</span><span class="p">)</span>
</code></pre></div></div>
<p>on the top level, and the tree of modules is built under <code class="language-plaintext highlighter-rouge">StringNaming</code>
namespace. Enjoy:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">StringNaming</span><span class="o">.</span><span class="no">ChessSymbols</span><span class="o">.</span><span class="no">Black</span><span class="o">.</span><span class="no">Chess</span><span class="o">.</span><span class="n">king</span>
<span class="s2">"♚"</span>
</code></pre></div></div>
<hr />
<h2 id="get-the-pill">Get the pill</h2>
<p>There is not much more code in the package, besides the above,
but for those picky persons, we have
<a href="https://github.com/am-kantox/string_naming">string_naming @ github</a>, also the package
is <a href="https://hex.pm/packages/string_naming">string_naming @ hex.pm</a>.</p>
Define module in Elixir with initial binding2017-05-12T00:00:00+00:00https://rocket-science.ru/hacking/2017/05/12/defmodule-with-binding-implementation<p>This post is mostly a tutorial on Elixir macros, yet by the end we’ll have a
helper module that supports bound variables passed to <code class="language-plaintext highlighter-rouge">defmodule</code> built.</p>
<h3 id="the-problem">The problem</h3>
<p>Nested modules in <code class="language-plaintext highlighter-rouge">Elixir</code> derive the “context” if one treats context as
fully name qualification. In all other aspects, these two codepieces
are equivalent:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">A</span><span class="o">.</span><span class="no">B</span> <span class="k">do</span>
<span class="k">end</span>
<span class="c1"># versus</span>
<span class="k">defmodule</span> <span class="no">A</span> <span class="k">do</span>
<span class="k">defmodule</span> <span class="no">B</span> <span class="k">do</span>
<span class="k">end</span>
<span class="c1"># the only difference is that here</span>
<span class="c1"># we can refer B module as B</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Usually when one nests anything, they expect to derive <em>more</em> context. Like
module variables, or whatever. Even more, the context switch should probably
allow passing a binding through. At least, <code class="language-plaintext highlighter-rouge">quote</code> macro does:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">block</span> <span class="o">=</span> <span class="kn">quote</span> <span class="ss">bind_quoted:</span> <span class="p">[</span><span class="ss">var:</span> <span class="mi">42</span><span class="p">]</span> <span class="k">do</span>
<span class="n">var</span> <span class="o">==</span> <span class="mi">42</span>
<span class="k">end</span>
<span class="no">Code</span><span class="o">.</span><span class="n">eval_quoted</span> <span class="n">block</span>
<span class="c1">#⇒ {true, [{\{:var, Elixir}, 42}]}</span>
</code></pre></div></div>
<p>The above trick is specifically handy when writing macros:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">A</span> <span class="k">do</span>
<span class="k">defmacro</span> <span class="n">a</span><span class="p">(</span><span class="n">var</span><span class="p">)</span> <span class="k">do</span>
<span class="kn">quote</span> <span class="ss">bind_quoted:</span> <span class="p">[</span><span class="ss">var:</span> <span class="n">var</span><span class="p">]</span> <span class="k">do</span>
<span class="no">IO</span><span class="o">.</span><span class="n">puts</span> <span class="n">var</span> <span class="c1"># use it as is, without `unquote`</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="kn">require</span> <span class="no">A</span>
<span class="no">A</span><span class="o">.</span><span class="n">a</span><span class="p">(</span><span class="mi">42</span><span class="p">)</span>
<span class="c1">#⇒ 42</span>
</code></pre></div></div>
<p>OK, so we are to implement binding passing through for <code class="language-plaintext highlighter-rouge">defmodule</code>. The
code that might use our implementation, would look like:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">WithBinding</span> <span class="k">do</span>
<span class="c1"># our module, that implements custom `defmodule` macro</span>
<span class="kn">require</span> <span class="no">Atadura</span>
<span class="no">Atadura</span><span class="o">.</span><span class="k">defmodule</span> <span class="no">DynamicModule</span><span class="p">,</span> <span class="ss">status:</span> <span class="ss">:ok</span><span class="p">,</span> <span class="ss">message:</span> <span class="s2">"¡Yay!"</span> <span class="k">do</span>
<span class="k">def</span> <span class="n">response</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="p">[</span><span class="ss">status:</span> <span class="n">status</span><span class="p">,</span> <span class="ss">message:</span> <span class="n">message</span><span class="p">]</span>
<span class="no">IO</span><span class="o">.</span><span class="n">inspect</span> <span class="n">message</span><span class="p">,</span> <span class="ss">label:</span> <span class="s2">"Message (local)"</span>
<span class="c1">#⇒ Message (local): "¡Yay!"</span>
<span class="no">IO</span><span class="o">.</span><span class="n">inspect</span> <span class="nv">@message</span><span class="p">,</span> <span class="ss">label:</span> <span class="s2">"Message (attribute)"</span>
<span class="c1">#⇒ Message (attribute): "¡Yay!"</span>
<span class="no">IO</span><span class="o">.</span><span class="n">inspect</span> <span class="sx">~b|status message|</span><span class="p">,</span> <span class="ss">label:</span> <span class="s2">"Message (sigil)"</span>
<span class="c1">#⇒ Message (sigil): [:ok, "¡Yay!"]</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="no">WithBinding</span><span class="o">.</span><span class="no">DynamicModule</span><span class="o">.</span><span class="n">response</span><span class="p">()</span>
<span class="c1">#⇒ [status: :ok, message: "¡Yay!"]</span>
</code></pre></div></div>
<p>As one might see, both <code class="language-plaintext highlighter-rouge">status</code> and <code class="language-plaintext highlighter-rouge">message</code> variables were bound to
newly created module in three different ways:</p>
<ul>
<li>— as local functions;</li>
<li>— as module variables;</li>
<li>— as smart sigil.</li>
</ul>
<p>Also the module was granted with <code class="language-plaintext highlighter-rouge">bindings/0</code> function, returning the bindings
keyword list as is.</p>
<p>Under the hood, the nested module <code class="language-plaintext highlighter-rouge">WithBinding.DynamicModule.Bindings</code> was also
created, having two interesting macros, <code class="language-plaintext highlighter-rouge">bindings!/0</code> and <code class="language-plaintext highlighter-rouge">attributes!/0</code>.
The former populates the binding as local variables, the latter declares
modules attributes (it is internally used to declare module attributes for
the binding in the example above.)</p>
<h4 id="bindings0"><code class="language-plaintext highlighter-rouge">bindings!/0</code></h4>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">require</span> <span class="no">WithBinding</span><span class="o">.</span><span class="no">DynamicModule</span><span class="o">.</span><span class="no">Bindings</span>
<span class="no">WithBinding</span><span class="o">.</span><span class="no">DynamicModule</span><span class="o">.</span><span class="no">Bindings</span><span class="o">.</span><span class="n">bindings!</span>
<span class="c1">#⇒ [:ok, "¡Yay!"]</span>
<span class="n">status</span>
<span class="c1">#⇒ :ok</span>
</code></pre></div></div>
<h4 id="attributes0"><code class="language-plaintext highlighter-rouge">attributes!/0</code></h4>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">A</span> <span class="k">do</span>
<span class="kn">require</span> <span class="no">WithBinding</span><span class="o">.</span><span class="no">DynamicModule</span><span class="o">.</span><span class="no">Bindings</span>
<span class="no">WithBinding</span><span class="o">.</span><span class="no">DynamicModule</span><span class="o">.</span><span class="no">Bindings</span><span class="o">.</span><span class="n">attributes!</span>
<span class="k">def</span> <span class="n">status</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="nv">@status</span>
<span class="k">end</span>
<span class="no">A</span><span class="o">.</span><span class="n">status</span>
<span class="c1">#⇒ :ok</span>
</code></pre></div></div>
<h3 id="implementation">Implementation</h3>
<p>The implementation is simple, though it might look a bit cumbersome at the
first glance. Let’s start with <code class="language-plaintext highlighter-rouge">Atadura.defmodule/{2,3}</code>.</p>
<p>If no bindings were given, <code class="language-plaintext highlighter-rouge">Atadura.defmodule/2</code> would gracefully fallback
to <code class="language-plaintext highlighter-rouge">Kernel.defmodule/2</code>, without any impact on the compiled code.</p>
<p>Here is the complete code of this module, including some comments:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># function declaration; needed to allow importing w/out</span>
<span class="c1"># conflicts with `Kernel.defmodule/2` and fallback to it</span>
<span class="c1"># when no parameters are given.</span>
<span class="k">defmacro</span> <span class="k">defmodule</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">bindings</span> <span class="p">\\</span> <span class="p">[],</span> <span class="n">do_block</span><span class="p">)</span>
<span class="c1"># fallback to `Kernel.defmodule/2`</span>
<span class="k">defmacro</span> <span class="k">defmodule</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="p">[],</span> <span class="k">do</span><span class="p">:</span> <span class="n">block</span><span class="p">)</span> <span class="k">do</span>
<span class="kn">quote</span> <span class="k">do</span><span class="p">:</span> <span class="no">Kernel</span><span class="o">.</span><span class="k">defmodule</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">name</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">block</span><span class="p">))</span>
<span class="k">end</span>
<span class="c1"># handler for non-bracketed call (not available on import)</span>
<span class="k">defmacro</span> <span class="k">defmodule</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="p">[],</span> <span class="n">bindings_and_do</span><span class="p">)</span> <span class="k">do</span>
<span class="n">block</span> <span class="o">=</span> <span class="no">Keyword</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">bindings_and_do</span><span class="p">,</span> <span class="ss">:do</span><span class="p">,</span> <span class="no">nil</span><span class="p">)</span>
<span class="n">bindings</span> <span class="o">=</span> <span class="no">Keyword</span><span class="o">.</span><span class="n">delete</span><span class="p">(</span><span class="n">bindings_and_do</span><span class="p">,</span> <span class="ss">:do</span><span class="p">)</span>
<span class="kn">quote</span> <span class="k">do</span><span class="p">:</span> <span class="no">Atadura</span><span class="o">.</span><span class="k">defmodule</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">name</span><span class="p">),</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">bindings</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">block</span><span class="p">))</span>
<span class="k">end</span>
<span class="c1"># main handler</span>
<span class="k">defmacro</span> <span class="k">defmodule</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">bindings</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="n">block</span><span class="p">)</span> <span class="k">do</span>
<span class="c1"># we are to build the module name in the first place</span>
<span class="c1"># it’s done before `quote do` because `defmodule` below</span>
<span class="c1"># does not accept `Module.concat(unquote(name), Bindings)`</span>
<span class="n">binding_module</span> <span class="o">=</span> <span class="n">with</span> <span class="p">{</span><span class="ss">:__aliases__</span><span class="p">,</span> <span class="n">line</span><span class="p">,</span> <span class="n">names</span><span class="p">}</span> <span class="o"><-</span> <span class="n">name</span> <span class="k">do</span>
<span class="p">{</span><span class="ss">:__aliases__</span><span class="p">,</span> <span class="n">line</span><span class="p">,</span> <span class="n">names</span> <span class="o">++</span> <span class="p">[</span><span class="ss">:Bindings</span><span class="p">]}</span>
<span class="k">end</span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="c1"># `Bindings` nested module, passes `bindings` to</span>
<span class="c1"># `Atadura.Binder`, that will declare all the</span>
<span class="c1"># stuff in `__using__` callback</span>
<span class="k">defmodule</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">binding_module</span><span class="p">)</span> <span class="k">do</span>
<span class="kn">use</span> <span class="no">Atadura</span><span class="o">.</span><span class="no">Binder</span><span class="p">,</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">bindings</span><span class="p">)</span>
<span class="k">end</span>
<span class="c1"># OK, that is the exact reason we were quoting</span>
<span class="c1"># and unquoting everything: call to `use binding_module`</span>
<span class="c1"># is required to update (patch, extend, younameit) the</span>
<span class="c1"># module that is being created</span>
<span class="k">defmodule</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">name</span><span class="p">)</span> <span class="k">do</span>
<span class="kn">use</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">binding_module</span><span class="p">)</span>
<span class="kn">unquote</span><span class="p">(</span><span class="n">block</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<hr />
<p>That’s almost it. The only thing remains is to implement <code class="language-plaintext highlighter-rouge">__using__</code> there
in <code class="language-plaintext highlighter-rouge">Atadura.Binder</code>. Since it’s a bit of hardcore, let’s continue with
usage advises on that module. The post’s tail will cover the tech details
of the implementation, I promise.</p>
<h3 id="tips-and-tricks">Tips and tricks</h3>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="no">Atadura</span><span class="p">,</span> <span class="ss">only:</span> <span class="p">[</span><span class="ss">:defmodule</span><span class="p">,</span> <span class="mi">3</span><span class="p">]</span>
</code></pre></div></div>
<p>The above allows to still use default <code class="language-plaintext highlighter-rouge">Kernel.defmodule/2</code> unless
the keyword list is given as the second parameter:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="no">Atadura</span><span class="p">,</span> <span class="ss">only:</span> <span class="p">[</span><span class="ss">:defmodule</span><span class="p">,</span> <span class="mi">3</span><span class="p">]</span>
<span class="c1"># Plain old good `Kernel.defmodule/2` without bindings</span>
<span class="k">defmodule</span> <span class="no">A1</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="k">def</span> <span class="n">a</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="no">IO</span><span class="o">.</span><span class="n">puts</span> <span class="s2">"★★★"</span>
<span class="c1"># `Atadura.defmodule/3` with bindings</span>
<span class="k">defmodule</span> <span class="no">A2</span><span class="p">,</span> <span class="p">[</span><span class="ss">b:</span> <span class="mi">42</span><span class="p">],</span> <span class="k">do</span><span class="p">:</span> <span class="k">def</span> <span class="n">a</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="no">IO</span><span class="o">.</span><span class="n">puts</span> <span class="s2">"★★★"</span>
</code></pre></div></div>
<p>Without explicit <code class="language-plaintext highlighter-rouge">import</code>, <code class="language-plaintext highlighter-rouge">Atadura.defmodule/{2,3}</code> would gracefully
fallback to <code class="language-plaintext highlighter-rouge">Kernel.defmodule/2</code> if no bindings were given. Unfortunately,
there is no chance to distinguish between two following calls:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">A1</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="k">def</span> <span class="n">a</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="no">IO</span><span class="o">.</span><span class="n">puts</span> <span class="s2">"★⚐★"</span>
<span class="k">defmodule</span> <span class="no">A2</span><span class="p">,</span> <span class="ss">bound:</span> <span class="mf">3.14</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="k">def</span> <span class="n">a</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="no">IO</span><span class="o">.</span><span class="n">puts</span> <span class="s2">"★π★"</span>
</code></pre></div></div>
<p>because both examples are <code class="language-plaintext highlighter-rouge">defmodule/2</code> clauses, and we would be conflicting
with always imported <code class="language-plaintext highlighter-rouge">Kernel</code>. So, when <code class="language-plaintext highlighter-rouge">import Atadura</code> is used, the
brackets around the second argument in call to <code class="language-plaintext highlighter-rouge">defmodule</code> are mandatory.</p>
<h3 id="atadurabinder__using__1"><code class="language-plaintext highlighter-rouge">Atadura.Binder.__using__/1</code></h3>
<p>Dear José, thanks for still bearing with me, in case you wonder how ugly I had
this written, here are the implementation details.</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">Atadura</span><span class="o">.</span><span class="no">Binder</span> <span class="k">do</span>
<span class="nv">@moduledoc</span> <span class="sx">~S""</span><span class="s2">"
Since we want to hide the implementation details inside
`Bindings` nested module, we’ll do nested `__using__`.
See `Atadura.defmodule/3` for details.
"""</span>
<span class="k">defmacro</span> <span class="n">__using__</span><span class="p">(</span><span class="n">bindings</span><span class="p">)</span> <span class="k">do</span>
<span class="p">[</span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="c1"># bindings require two unquotes, on each subsequent quote</span>
<span class="n">bindings</span> <span class="o">=</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">bindings</span><span class="p">)</span>
<span class="c1"># this will be used in nested `Bindings` module, that will</span>
<span class="c1"># be in turn `use`d by `Atadura.defmodule/3`</span>
<span class="k">defmacro</span> <span class="n">__using__</span><span class="p">(</span><span class="n">_opts</span> <span class="p">\\</span> <span class="p">[])</span> <span class="k">do</span>
<span class="n">module</span> <span class="o">=</span> <span class="bp">__MODULE__</span> <span class="c1"># to get it properly inside quote</span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="kn">require</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">module</span><span class="p">),</span> <span class="ss">as:</span> <span class="no">Bindings</span>
<span class="kn">import</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">module</span><span class="p">)</span>
<span class="c1"># Bindings.bindings! # declare local variables: not very handy</span>
<span class="no">Bindings</span><span class="o">.</span><span class="n">attributes!</span> <span class="c1"># declare module attributes</span>
<span class="c1"># just an example of how module documentation</span>
<span class="c1"># might be constructed on the fly</span>
<span class="no">Module</span><span class="o">.</span><span class="n">add_doc</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="ss">:def</span><span class="p">,</span> <span class="p">{</span><span class="ss">:version</span><span class="p">,</span> <span class="mi">0</span><span class="p">},</span> <span class="p">[],</span>
<span class="sx">~s""</span><span class="s2">"
Returns a binding for this module, supplied when it was created.
This module </span><span class="si">#{</span><span class="bp">__MODULE__</span><span class="si">}</span><span class="s2"> was created with the following binding:
</span><span class="si">#{</span><span class="n">inspect</span> <span class="no">Bindings</span><span class="o">.</span><span class="n">bindings</span><span class="si">}</span><span class="s2">
Enjoy!
"""</span><span class="p">)</span>
<span class="c1"># the plain function, that returns all the bindings</span>
<span class="k">def</span> <span class="n">bindings</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="no">Bindings</span><span class="o">.</span><span class="n">bindings</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="c1"># back to `Bindings` module level, here it’s macro called 3 LOCs above</span>
<span class="k">defmacro</span> <span class="n">bindings</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">bindings</span><span class="p">)</span>
<span class="c1"># local variables producer, might be called from everywhere</span>
<span class="c1"># to populate the current context with binding</span>
<span class="k">defmacro</span> <span class="n">bindings!</span> <span class="k">do</span>
<span class="no">Enum</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">bindings</span><span class="p">),</span> <span class="k">fn</span> <span class="p">{</span><span class="n">attr</span><span class="p">,</span> <span class="n">val</span><span class="p">}</span> <span class="o">-></span>
<span class="p">{:</span><span class="o">=</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[{</span><span class="n">attr</span><span class="p">,</span> <span class="p">[],</span> <span class="no">nil</span><span class="p">},</span> <span class="n">val</span><span class="p">]}</span>
<span class="k">end</span><span class="p">)</span>
<span class="k">end</span>
<span class="c1"># module attribute producer, might be called from inside</span>
<span class="c1"># of any module to populate the current context with binding</span>
<span class="k">defmacro</span> <span class="n">attributes!</span><span class="p">()</span> <span class="k">do</span>
<span class="no">Enum</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">bindings</span><span class="p">),</span> <span class="k">fn</span> <span class="p">{</span><span class="n">attr</span><span class="p">,</span> <span class="n">val</span><span class="p">}</span> <span class="o">-></span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="no">Module</span><span class="o">.</span><span class="n">register_attribute</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">attr</span><span class="p">),</span> <span class="ss">accumulate:</span> <span class="no">false</span><span class="p">)</span>
<span class="no">Module</span><span class="o">.</span><span class="n">put_attribute</span><span class="p">(</span><span class="bp">__MODULE__</span><span class="p">,</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">attr</span><span class="p">),</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">val</span><span class="p">))</span>
<span class="k">end</span>
<span class="k">end</span><span class="p">)</span>
<span class="k">end</span>
<span class="c1"># sigils producer; when called with a single attribute name,</span>
<span class="c1"># e.g. `~b|status|`, returns it’s value as is.</span>
<span class="c1"># when called with many, returns a list of values.</span>
<span class="k">defmacro</span> <span class="n">sigil_b</span><span class="p">(</span><span class="n">keys</span><span class="p">,</span> <span class="n">_modifiers</span><span class="p">)</span> <span class="k">do</span>
<span class="n">bindings</span> <span class="o">=</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">bindings</span><span class="p">)</span>
<span class="p">{</span><span class="ss">:<</span><span class="o"><>></span><span class="p">,</span> <span class="n">_</span><span class="p">,</span> <span class="p">[</span><span class="n">key</span><span class="p">]}</span> <span class="o">=</span> <span class="n">keys</span>
<span class="k">case</span> <span class="no">String</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="sr">~r/\s+/</span><span class="p">)</span> <span class="k">do</span>
<span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="ow">when</span> <span class="n">is_binary</span><span class="p">(</span><span class="n">key</span><span class="p">)</span> <span class="o">-></span> <span class="c1"># single value requested</span>
<span class="kn">quote</span> <span class="k">do</span><span class="p">:</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">bindings</span><span class="p">)[</span><span class="no">String</span><span class="o">.</span><span class="n">to_atom</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">key</span><span class="p">))]</span>
<span class="n">keys</span> <span class="ow">when</span> <span class="n">is_list</span><span class="p">(</span><span class="n">keys</span><span class="p">)</span> <span class="o">-></span> <span class="c1"># multi values</span>
<span class="no">Enum</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">keys</span><span class="p">,</span> <span class="k">fn</span> <span class="n">key</span> <span class="o">-></span>
<span class="n">with</span> <span class="n">bindings</span> <span class="o"><-</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">bindings</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="kn">quote</span> <span class="k">do</span><span class="p">:</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">bindings</span><span class="p">)[</span><span class="no">String</span><span class="o">.</span><span class="n">to_atom</span><span class="p">(</span><span class="kn">unquote</span><span class="p">(</span><span class="n">key</span><span class="p">))]</span>
<span class="k">end</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span> <span class="o">|</span>
<span class="c1"># functions returning bindings by name, appended to the AST.</span>
<span class="no">Enum</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="n">bindings</span><span class="p">,</span> <span class="k">fn</span> <span class="p">{</span><span class="n">attr</span><span class="p">,</span> <span class="n">val</span><span class="p">}</span> <span class="o">-></span>
<span class="kn">quote</span> <span class="k">do</span>
<span class="k">def</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">attr</span><span class="p">)(),</span> <span class="k">do</span><span class="p">:</span> <span class="kn">unquote</span><span class="p">(</span><span class="n">val</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span><span class="p">)</span>
<span class="p">]</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Yes, that is it. There is no more code in the package, besides the above,
but for those picky persons, we have
<a href="https://github.com/am-kantox/atadura">atadura @ github</a>, also the package
is <a href="https://hex.pm/packages/atadura">available @ hex.pm</a>.</p>
Use `credo` Linter in Git `pre-commit` Hook2017-04-24T00:00:00+00:00https://rocket-science.ru/hacking/2017/04/24/git-commit-credo-linter-hook<p><a href="http://credo-ci.org/"><code class="language-plaintext highlighter-rouge">credo</code></a> is <a href="https://github.com/rrrene/credo">great</a>.</p>
<p>Every single Elixir project all around should use it. Besides fancy <code class="language-plaintext highlighter-rouge">mix</code> task
to report project styling issues, it has bindings to most of modern editors,
like <a href="https://www.dailydrip.com/topics/elixirsips/drips/neovim-for-elixir"><code class="language-plaintext highlighter-rouge">neovim</code></a>,
<a href="https://github.com/tonini/alchemist.el"><code class="language-plaintext highlighter-rouge">emacs</code> through alchemist.el</a> and even <a href="https://atom.io/packages/linter-elixir-credo"><code class="language-plaintext highlighter-rouge">Atom</code></a>.</p>
<p>The greatest thing about <code class="language-plaintext highlighter-rouge">credo</code> is it provides the <em>meaningful shell return codes</em>,
making it quite easy to plug it as a linter into <code class="language-plaintext highlighter-rouge">git</code> hooks toolchain. Let’s do it:</p>
<h3 id="include-credo-dependency-into-mixexs-project-file">Include <code class="language-plaintext highlighter-rouge">credo</code> dependency into <code class="language-plaintext highlighter-rouge">mix.exs</code> project file</h3>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="k">defp</span> <span class="n">deps</span> <span class="k">do</span>
<span class="p">[</span>
<span class="o">...</span>
<span class="p">{</span><span class="ss">:credo</span><span class="p">,</span> <span class="s2">"~> 0.7"</span><span class="p">,</span> <span class="ss">only:</span> <span class="p">[</span><span class="ss">:dev</span><span class="p">,</span> <span class="ss">:test</span><span class="p">]},</span>
<span class="o">...</span>
<span class="p">]</span>
<span class="k">end</span>
</code></pre></div></div>
<p>if you are reading this article from the future, immediately run</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ mix deps.update credo
</code></pre></div></div>
<p>to get the up-to-date version. It’s safe, since we’d use it in non-prod
environments only. Let’s try it in action.</p>
<h3 id="running-credo-for-the-first-time">Running <code class="language-plaintext highlighter-rouge">credo</code> for the first time</h3>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ mix credo
Checking 36 source files ...
Software Design
┃
┃ [D] ↗ Found a FIXME tag in a comment: # FIXME FIXME FIXME
┃ lib/my_app/file1.ex:194 (MyApp.File1.fun1)
┃ [D] → Found a TODO tag in a comment: # Todo: should round here?
┃ lib/my_app/file2.ex:53 (MyApp.File2.fun2)
Code Readability
┃
┃ [R] → Modules should have a @moduledoc tag.
┃ lib/my_app/file1.ex:4:13 (MyApp.File1)
┃ [R] → Modules should have a @moduledoc tag.
┃ lib/my_app/file3.ex:3:11 (MyApp.File3)
Refactoring opportunities
┃
┃ [F] → Function body is nested too deep (max depth is 2, was 4).
┃ lib/my_app/file1.ex:179:21 (MyApp.File1.fun1)
Warnings - please take a look
┃
┃ [W] ↗ Prefer lazy Logger calls.
┃ lib/my_app/file1.ex:91 (MyApp.File1.fun3)
┃ [W] ↗ Prefer lazy Logger calls.
┃ lib/my_app/file1.ex:82 (MyApp.File1.fun2)
┃ [W] ↗ Prefer lazy Logger calls.
┃ lib/my_app/file1.ex:169 (MyApp.File1.fun3)
Please report incorrect results: https://github.com/rrrene/credo/issues
Analysis took 1.1 seconds (0.02s to load, 1.1s running checks)
220 mods/funs, found 3 warnings,
1 refactoring opportunity,
2 code readability issues,
2 software design suggestions.
</code></pre></div></div>
<p>Here we go. First run report is always depressive and driving me bonkers.
I was writing this code today morning, it’s, you know, kinda perfect. I hope
you’ll get zero warnings and at most one <code class="language-plaintext highlighter-rouge">TODO</code> tag found, but I never get there.
First run turns me into a couple of hours of refactoring. The good thing is
from now on it’s fairly easy to keep the project in clean state, that
satisfies even such a captious judge as <code class="language-plaintext highlighter-rouge">credo</code>.</p>
<h3 id="configure-credo">Configure <code class="language-plaintext highlighter-rouge">credo</code></h3>
<p>The very straightforward way to make everything tuned would be to:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>cd config
# wget https://raw.githubusercontent.com/rrrene/credo/master/.credo.exs
mix credo gen.config
vim .credo.exs
cd -
</code></pre></div></div>
<p>The file is self-documented. Just go tweak it.</p>
<h3 id="making-it-happen-recurrently">Making it happen recurrently</h3>
<p>Well, I knew one guy who had put running linters into <code class="language-plaintext highlighter-rouge">crontab</code> at 2AM.
He started every morning with few cups of coffee and several hours of
refactoring. I am not as brave. I use git hooks to lint my code (save for editor
plugins.)</p>
<p>Let’s configure our <code class="language-plaintext highlighter-rouge">.git/config</code> in the first place. Just add
the following section to it:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[credo]
terminate = 16
</code></pre></div></div>
<p>The above will terminate the commit when there are warnings. Check <a href="https://github.com/rrrene/credo#exit-status">how <code class="language-plaintext highlighter-rouge">credo</code>
calculates an exit status</a> for details.</p>
<p>OK, now we are to create a hook itself. You might just put this in your
<code class="language-plaintext highlighter-rouge">.git/hooks/pre-commit</code> local hook (create this file if it’s not existing yet):</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/sh</span>
<span class="c">#</span>
<span class="c"># Linter Elixir files using Credo.</span>
<span class="c"># Called by "git receive-pack" with arguments: refname sha1-old sha1-new</span>
<span class="c">#</span>
<span class="c"># Config</span>
<span class="c"># ------</span>
<span class="c"># credo.terminate</span>
<span class="c"># The credo exit status level to be considered as “not passed”—to prevent</span>
<span class="c"># git commit until fixed.</span>
<span class="c"># Config</span>
<span class="nv">terminate_on</span><span class="o">=</span><span class="si">$(</span>git config <span class="nt">--int</span> credo.terminate<span class="si">)</span>
<span class="k">if</span> <span class="o">[[</span> <span class="nt">-z</span> <span class="s2">"</span><span class="nv">$terminate_on</span><span class="s2">"</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then </span><span class="nv">terminate_on</span><span class="o">=</span>16<span class="p">;</span> <span class="k">fi</span>
<span class="c"># test it :: run tests before commit (silently)</span>
mix <span class="nb">test </span>2>&1 <span class="o">></span>/dev/null
<span class="nv">TEST_RES</span><span class="o">=</span><span class="nv">$?</span>
<span class="k">if</span> <span class="o">[</span> <span class="nv">$TEST_RES</span> <span class="nt">-ne</span> 0 <span class="o">]</span>
<span class="k">then
</span><span class="nb">echo</span> <span class="s2">""</span>
<span class="nb">echo</span> <span class="s2">"☆ ==================================== ☆"</span>
<span class="nb">echo</span> <span class="s2">"☆ Some tests are failed. ☆"</span> <span class="o">></span>&2
<span class="nb">echo</span> <span class="s2">"☆ Please fix them before committing. ☆"</span> <span class="o">></span>&2
<span class="nb">echo</span> <span class="s2">"☆ ==================================== ☆"</span>
<span class="nb">echo</span> <span class="s2">""</span>
<span class="nb">exit</span> <span class="nv">$TEST_RES</span>
<span class="k">fi
</span><span class="nb">echo</span> <span class="s2">""</span>
<span class="nb">echo</span> <span class="s2">"★ ============================== ★"</span>
<span class="nb">echo</span> <span class="s2">"★ Tests passed successfully. ★"</span>
<span class="nb">echo</span> <span class="s2">"★ ============================== ★"</span>
<span class="nb">echo</span> <span class="s2">""</span>
<span class="c"># lint it :: credo checks before commit</span>
mix credo
<span class="nv">CREDO_RES</span><span class="o">=</span><span class="nv">$?</span>
<span class="k">if</span> <span class="o">[</span> <span class="nv">$CREDO_RES</span> <span class="nt">-ge</span> <span class="si">$(</span>terminate_on<span class="si">)</span> <span class="o">]</span><span class="p">;</span> <span class="k">then
</span><span class="nb">echo</span> <span class="s2">""</span>
<span class="nb">echo</span> <span class="s2">"☆ ============================================= ☆"</span>
<span class="nb">echo</span> <span class="s2">"☆ Credo found critical problems with your code ☆"</span> <span class="o">></span>&2
<span class="nb">echo</span> <span class="s2">"☆ and commit can not proceed. Please examine ☆"</span> <span class="o">></span>&2
<span class="nb">echo</span> <span class="s2">"☆ log above and fix issues before committing. ☆"</span> <span class="o">></span>&2
<span class="nb">echo</span> <span class="s2">"☆ ============================================= ☆"</span>
<span class="nb">echo</span> <span class="s2">""</span>
<span class="nb">exit</span> <span class="nv">$CREDO_RES</span>
<span class="k">fi
if</span> <span class="o">[</span> <span class="nv">$CREDO_RES</span> <span class="nt">-le</span> 9 <span class="o">]</span><span class="p">;</span> <span class="k">then </span><span class="nv">CREDO_RES</span><span class="o">=</span><span class="s2">" </span><span class="nv">$CREDO_RES</span><span class="s2">"</span><span class="p">;</span> <span class="k">fi
</span><span class="nb">echo</span> <span class="s2">""</span>
<span class="nb">echo</span> <span class="s2">"★ ============================== ★"</span>
<span class="nb">echo</span> <span class="s2">"★ Credo passed successfully. ★"</span>
<span class="nb">echo</span> <span class="s2">"★ Exit value is: (total: </span><span class="nv">$CREDO_RES</span><span class="s2">). ★"</span>
<span class="nb">echo</span> <span class="s2">"★ ============================== ★"</span>
<span class="nb">echo</span> <span class="s2">""</span>
<span class="c"># Finished</span>
<span class="nb">exit </span>0
</code></pre></div></div>
<p>Before <code class="language-plaintext highlighter-rouge">credo</code> we also run tests. If you are like me and want to commit
code even while all the tests are failing (hopefully providing the comment
like “:bomb: [BROKEN] Evening commit! Do not use at home or school!”,)
just supply <code class="language-plaintext highlighter-rouge">--no-verify</code> option to <code class="language-plaintext highlighter-rouge">git commit</code>.</p>
<p>And yes, we are done. Try to <code class="language-plaintext highlighter-rouge">git commit</code> and you’ll see as your code will
be tested and credo’ed for you automagically.</p>
`is_empty` Guard for Binaries in Elixir2017-03-28T00:00:00+00:00https://rocket-science.ru/hacking/2017/03/28/is-empty-guard-for-binaries<p>I have many times heard the question “How would one implement <code class="language-plaintext highlighter-rouge">is_empty</code> guard
for binaries in <code class="language-plaintext highlighter-rouge">Elixir</code>.”</p>
<p>It is impossible out of the box, basically because guards are compile-time beasts.
The documentation on <a href="https://hexdocs.pm/elixir/master/guards.html"><code class="language-plaintext highlighter-rouge">Guards</code></a>
states:</p>
<blockquote>
<p>Guards are a way to augment pattern matching with more complex checks;
<strong>they are allowed in a predefined set of constructs</strong> where pattern matching is allowed.</p>
</blockquote>
<p>That said, one can not just put <code class="language-plaintext highlighter-rouge">String.trim(s) == ""</code> because call to <code class="language-plaintext highlighter-rouge">String.trim</code>
is not allowed in guard:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">|</span><span class="mi">1</span> <span class="err">▶</span> <span class="k">defmodule</span> <span class="no">TestEmpty</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="k">def</span> <span class="n">test</span><span class="p">(</span><span class="n">s</span><span class="p">)</span> <span class="ow">when</span> <span class="no">String</span><span class="o">.</span><span class="n">trim</span><span class="p">(</span><span class="n">s</span><span class="p">)</span> <span class="o">==</span> <span class="s2">""</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="ss">:ok</span>
<span class="o">**</span> <span class="p">(</span><span class="no">CompileError</span><span class="p">)</span> <span class="ss">iex:</span><span class="mi">1</span><span class="p">:</span> <span class="n">cannot</span> <span class="n">invoke</span> <span class="n">remote</span> <span class="n">function</span> <span class="no">String</span><span class="o">.</span><span class="n">trim</span><span class="o">/</span><span class="mi">1</span> <span class="n">inside</span> <span class="n">guard</span>
</code></pre></div></div>
<p>So far, so bad. But there is a nifty workaround: one might explicitly <code class="language-plaintext highlighter-rouge">trim_leading</code> the
<code class="language-plaintext highlighter-rouge">String</code> with pattern match:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">TestEmpty</span> <span class="k">do</span>
<span class="k">def</span> <span class="n">trimmer</span><span class="p">(</span><span class="o"><<</span><span class="s2">" "</span> <span class="p">::</span> <span class="n">binary</span><span class="p">,</span> <span class="n">rest</span> <span class="p">::</span> <span class="n">binary</span><span class="o">>></span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="n">trimmer</span><span class="p">(</span><span class="n">rest</span><span class="p">)</span>
<span class="k">def</span> <span class="n">trimmer</span><span class="p">(</span><span class="n">string</span><span class="p">)</span> <span class="ow">when</span> <span class="n">string</span> <span class="o">==</span> <span class="s2">""</span><span class="p">,</span>
<span class="k">do</span><span class="p">:</span> <span class="no">IO</span><span class="o">.</span><span class="n">puts</span> <span class="s2">"Empty input"</span>
<span class="k">def</span> <span class="n">trimmer</span><span class="p">(</span><span class="n">string</span><span class="p">)</span> <span class="ow">when</span> <span class="n">is_binary</span><span class="p">(</span><span class="n">string</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="no">IO</span><span class="o">.</span><span class="n">puts</span> <span class="s2">"Left trimmed string is: [</span><span class="si">#{</span><span class="n">string</span><span class="si">}</span><span class="s2">]"</span>
<span class="k">end</span>
<span class="no">TestEmpty</span><span class="o">.</span><span class="n">trimmer</span> <span class="s2">" "</span>
<span class="c1">#⇒ Empty input</span>
<span class="no">TestEmpty</span><span class="o">.</span><span class="n">trimmer</span> <span class="s2">" ☆ "</span>
<span class="c1">#⇒ Left trimmed string is: [☆ ]</span>
</code></pre></div></div>
<p>This approach won’t work if one needs to deal with the original input
unless it is “empty,” but in most cases it would do the trick. After all,
we for some reason already treat spaces as empties.</p>
Hack vs Kludge2017-03-08T00:00:00+00:00https://rocket-science.ru/hacking/2017/03/08/hack-vs-kludge<p>I would like to make an example of what is the difference between an <em>elegant hack</em> and a <em>nasty kludge</em>.
In software development, using an allegory from the linguistics.</p>
<p>We love to brag to our wives that “we are hackers.” Many great solutions in the computer industry
were not even possible without hacks. We say “hack” whenever the undocumented or even not widely
known feature of the language/tool is used. When we blame colleagues for using hacks,
we do that with a shade of rapture. Those studying embroidery in the university, should be
familiar with the term by fashion magazines covers, that love to use the word
in contamination with “life.”</p>
<p>Unfortunately, what we tend to call hack is usually a kludge. A <em>chapuza</em>, if your mother
tongue is Spanish, or a <em>костыль</em>, if you, like me, were born in Saint-Petersburg.
No admiration, just a plain shame.</p>
<p>What is the difference and where is the border between? That’s easy. Let me explain
by example.</p>
<p>There in Catalan language the double “l” is spelled as “y” in “York.” The Mallorquin language
(that is quite similar to Catalan) is pronounced as “Mah-york-in,” not “Mal-or-keen.”
So far so good, but there are words, that, you know, were borrowed from other languages,
like “parallel,” or “installación.” In these words that <em>doble ele</em> is to be pronounced as is,
as double “l.” Here we have a middle dot to denote this “paral·lel.”</p>
<p><img src="/img/parallel.jpg" alt="paral·lel ⇒ parallllllel" /></p>
<p>Voilà—no doubt, it’s to be pronounced “ˈpa-rə-, -ləl”. That is <strong>the hack</strong>.</p>
<hr />
<p>In Spanish, on the other hand, that <em>double ele</em> is apparently to be
pronounced as… “y” in ”York” as well. And there are borrowed words in Spanish, as well.
All those “parallel”s and “installación”s. And it’s still to be “l” not “y” there.
In Spanish, we just cut the second “l” off. Shave it. Trim out. We write “paralel” with
a single “l” for it to be pronounced with “l”. Despite the broken compatibility
with all other Romance languages.</p>
<p>That is <strong>the kludge</strong>.</p>
Either Monad in Elixir2017-02-17T00:00:00+00:00https://rocket-science.ru/hacking/2017/02/17/either-monad-in-elixir<p>In the beginning was the Hype, and the Hype was with Monads, and the Hype was Monad.</p>
<p>Either this hype makes sense or not. Try it yourself. Right? Left unclear? Anyway.
Yes, <code class="language-plaintext highlighter-rouge">Anyway</code> monad were my favorite one, if it were existing.</p>
<p>Monads are known to be good in handling side effects and dealing with side effects.
Monads are very simple (despite their cryptographogastroenterologic name.) Monads
are after all handy, resulting in an <em>easy-to-read</em> code, which is nowadays considered
to be even more valuable then <em>producing-expected-result</em> and even <em>correct</em> code.</p>
<p>OK, jokes aside.</p>
<p>There are many 3rd party libraries, implementing the basic monadic behavior, but
the truth is <strong>Elixir comes with monads onboard</strong>. Out of the box. I am not kidding you.</p>
<p>Our today’s special guest is
<a href="https://hexdocs.pm/elixir/Kernel.SpecialForms.html#with/1"><code class="language-plaintext highlighter-rouge">Kernel.SpecialForms.with/1</code></a>.</p>
<p>It is widely used to chain the simple atomic functions returning <code class="language-plaintext highlighter-rouge">{:ok, result}</code>
tuples:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">with</span> <span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="p">[</span><span class="n">file</span> <span class="o">|</span> <span class="n">_tail</span><span class="p">]}</span> <span class="o"><-</span> <span class="no">File</span><span class="o">.</span><span class="n">ls</span><span class="p">,</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">content</span><span class="p">}</span> <span class="o"><-</span> <span class="no">File</span><span class="o">.</span><span class="n">read</span><span class="p">(</span><span class="n">file</span><span class="p">),</span>
<span class="k">do</span><span class="p">:</span> <span class="no">IO</span><span class="o">.</span><span class="n">puts</span> <span class="n">content</span>
</code></pre></div></div>
<p>And you know what? It behaves as simple <code class="language-plaintext highlighter-rouge">Either</code> monad. That said,
<em>if the match on any subsequent step failed, the execution will immediately stop
and RHO will be returned</em>.</p>
<p>Let’s check this trivial contrived example:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">iex</span><span class="o">|</span><span class="mi">1</span> <span class="err">▶</span> <span class="n">with</span> <span class="mi">1</span> <span class="o"><-</span> <span class="mi">1</span><span class="p">,</span>
<span class="o">...|</span><span class="mi">1</span> <span class="err">▶</span> <span class="mi">2</span> <span class="o"><-</span> <span class="s2">"HERE WE GO"</span><span class="p">,</span>
<span class="o">...|</span><span class="mi">1</span> <span class="err">▶</span> <span class="mi">3</span> <span class="o"><-</span> <span class="mi">3</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="s2">"HERE WE WON’T GO"</span>
<span class="s2">"HERE WE GO"</span>
</code></pre></div></div>
<p>The second match failed and <em>not matched term had beed returned</em>. It’s amazing,
is not it?</p>
<p>Let’s turn back to the first example with files. What would be returned in case
we failed to list files? Yes: <code class="language-plaintext highlighter-rouge">{:error, :enoent}</code> (or similar, and we would
safely skip all the subsequent steps, directly returning <code class="language-plaintext highlighter-rouge">{:error, :enoent}</code>
to the caller!) What is this?—this is an <code class="language-plaintext highlighter-rouge">Either</code> monad.</p>
<p>That simple.</p>
<hr />
<h3 id="bonus-track">Bonus Track</h3>
<p>It’s not mentioned anywhere in the documentation, or at least I could not find it,
but <code class="language-plaintext highlighter-rouge">with</code> special form <em>apparently supports <code class="language-plaintext highlighter-rouge">else</code> with matching clauses</em>.
Consider the following code:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">with</span> <span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="p">[</span><span class="n">file</span> <span class="o">|</span> <span class="n">_tail</span><span class="p">]}</span> <span class="o"><-</span> <span class="no">File</span><span class="o">.</span><span class="n">ls</span><span class="p">(</span><span class="s2">"NOWAY"</span><span class="p">),</span>
<span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="n">content</span><span class="p">}</span> <span class="o"><-</span> <span class="no">File</span><span class="o">.</span><span class="n">read</span><span class="p">(</span><span class="n">file</span><span class="p">)</span> <span class="k">do</span>
<span class="no">IO</span><span class="o">.</span><span class="n">puts</span> <span class="n">content</span>
<span class="k">else</span>
<span class="p">{</span><span class="ss">:error</span><span class="p">,</span> <span class="ss">:enoent</span><span class="p">}</span> <span class="o">-></span> <span class="no">IO</span><span class="o">.</span><span class="n">puts</span> <span class="s2">"NO SUCH FILE"</span>
<span class="n">unexpected</span> <span class="o">-></span> <span class="no">IO</span><span class="o">.</span><span class="n">puts</span> <span class="s2">"</span><span class="si">#{</span><span class="n">inspect</span> <span class="n">unexpected</span><span class="si">}</span><span class="s2"> happened"</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Yes, it would spit <code class="language-plaintext highlighter-rouge">"NO SUCH FILE"</code> message out to the console. Awesome!</p>
Iterating Tuples in Elixir2016-12-27T00:00:00+00:00https://rocket-science.ru/hacking/2016/12/27/iterating-tuples-in-elixir<p>According to <a href="http://erlang.org/doc/reference_manual/data_types.html#id68683">erlang docs</a>,</p>
<blockquote>
<p>A tuple is a compound data type with a fixed number of terms:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{Term1,...,TermN}
</code></pre></div> </div>
</blockquote>
<blockquote>
<p>Each term <code class="language-plaintext highlighter-rouge">Term</code> in the tuple is called an <strong>element</strong>.
The number of elements is said to be the <strong>size</strong> of the tuple.</p>
</blockquote>
<p><code class="language-plaintext highlighter-rouge">Tuple</code>s are known to be faster than <code class="language-plaintext highlighter-rouge">List</code>s (and even <code class="language-plaintext highlighter-rouge">Map</code>s)
for elements’ direct access. Also, some libraries/packages often format
a response as tuples.</p>
<p>So, <code class="language-plaintext highlighter-rouge">Tuple</code>s are good. Save for one single glitch: they are not iterable.
That said, <code class="language-plaintext highlighter-rouge">Tuple</code>s do not implement <code class="language-plaintext highlighter-rouge">Enumerable</code> protocol. The goal of this
small post is to show how this protocol might be implemented for tuples
in non-naïve way (the naïve way would be to just convert tuples to lists.)</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defmodule</span> <span class="no">Tuple</span><span class="o">.</span><span class="no">Enumerable</span> <span class="k">do</span>
<span class="k">defimpl</span> <span class="no">Enumerable</span><span class="p">,</span> <span class="ss">for:</span> <span class="no">Tuple</span> <span class="k">do</span>
<span class="nv">@max_items</span> <span class="mi">42</span>
<span class="k">def</span> <span class="n">count</span><span class="p">(</span><span class="n">tuple</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="n">tuple_size</span><span class="p">(</span><span class="n">tuple</span><span class="p">)</span>
<span class="c1"># member? implementation is done through casting tuple to the list</span>
<span class="c1"># it’s not required for iteration, and building all those matched</span>
<span class="c1"># clauses seems to be an overkill here</span>
<span class="k">def</span> <span class="n">member?</span><span class="p">([],</span> <span class="n">_</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="no">false</span><span class="p">}</span>
<span class="k">def</span> <span class="n">member?</span><span class="p">(</span><span class="n">tuple</span><span class="p">,</span> <span class="n">value</span><span class="p">)</span> <span class="ow">when</span> <span class="n">is_tuple</span><span class="p">(</span><span class="n">tuple</span><span class="p">)</span> <span class="k">do</span>
<span class="n">tuple</span> <span class="o">|></span> <span class="no">Tuple</span><span class="o">.</span><span class="n">to_list</span> <span class="o">|></span> <span class="n">member?</span><span class="p">(</span><span class="n">value</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="n">member?</span><span class="p">(</span><span class="n">tuple</span><span class="p">,</span> <span class="n">value</span><span class="p">)</span> <span class="ow">when</span> <span class="n">is_list</span><span class="p">(</span><span class="n">tuple</span><span class="p">)</span> <span class="k">do</span>
<span class="n">for</span> <span class="p">[</span><span class="n">h</span> <span class="o">|</span> <span class="n">t</span><span class="p">]</span> <span class="o"><-</span> <span class="n">tuple</span> <span class="k">do</span>
<span class="k">if</span> <span class="n">h</span> <span class="o">==</span> <span class="n">value</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="ss">:ok</span><span class="p">,</span> <span class="no">true</span><span class="p">},</span> <span class="k">else</span><span class="p">:</span> <span class="n">member?</span><span class="p">(</span><span class="n">t</span><span class="p">,</span> <span class="n">value</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="n">reduce</span><span class="p">(</span><span class="n">tuple</span><span class="p">,</span> <span class="n">acc</span><span class="p">,</span> <span class="n">fun</span><span class="p">)</span> <span class="k">do</span>
<span class="n">do_reduce</span><span class="p">(</span><span class="n">tuple</span><span class="p">,</span> <span class="n">acc</span><span class="p">,</span> <span class="n">fun</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">defp</span> <span class="n">do_reduce</span><span class="p">(</span><span class="n">_</span><span class="p">,</span> <span class="p">{</span><span class="ss">:halt</span><span class="p">,</span> <span class="n">acc</span><span class="p">},</span> <span class="n">_fun</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="ss">:halted</span><span class="p">,</span> <span class="n">acc</span><span class="p">}</span>
<span class="k">defp</span> <span class="n">do_reduce</span><span class="p">(</span><span class="n">tuple</span><span class="p">,</span> <span class="p">{</span><span class="ss">:suspend</span><span class="p">,</span> <span class="n">acc</span><span class="p">},</span> <span class="n">fun</span><span class="p">)</span> <span class="k">do</span>
<span class="p">{</span><span class="ss">:suspended</span><span class="p">,</span> <span class="n">acc</span><span class="p">,</span> <span class="o">&</span><span class="n">do_reduce</span><span class="p">(</span><span class="n">tuple</span><span class="p">,</span> <span class="nv">&1</span><span class="p">,</span> <span class="n">fun</span><span class="p">)}</span>
<span class="k">end</span>
<span class="k">defp</span> <span class="n">do_reduce</span><span class="p">({},</span> <span class="p">{</span><span class="ss">:cont</span><span class="p">,</span> <span class="n">acc</span><span class="p">},</span> <span class="n">_fun</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="p">{</span><span class="ss">:done</span><span class="p">,</span> <span class="n">acc</span><span class="p">}</span>
<span class="k">defp</span> <span class="n">do_reduce</span><span class="p">({</span><span class="n">value</span><span class="p">},</span> <span class="p">{</span><span class="ss">:cont</span><span class="p">,</span> <span class="n">acc</span><span class="p">},</span> <span class="n">fun</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="n">do_reduce</span><span class="p">({},</span> <span class="n">fun</span><span class="o">.</span><span class="p">(</span><span class="n">value</span><span class="p">,</span> <span class="n">acc</span><span class="p">),</span> <span class="n">fun</span><span class="p">)</span>
<span class="no">Enum</span><span class="o">.</span><span class="n">each</span><span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="nv">@max_items</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="k">fn</span> <span class="n">tot</span> <span class="o">-></span>
<span class="n">tail</span> <span class="o">=</span> <span class="no">Enum</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="no">Enum</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="n">tot</span><span class="p">,</span> <span class="o">&</span> <span class="s2">"e_</span><span class="si">#{</span><span class="nv">&1</span><span class="si">}</span><span class="s2">"</span><span class="p">),</span> <span class="s2">","</span><span class="p">)</span>
<span class="n">match</span> <span class="o">=</span> <span class="no">Enum</span><span class="o">.</span><span class="n">join</span><span class="p">([</span><span class="s2">"value"</span><span class="p">]</span> <span class="o">++</span> <span class="p">[</span><span class="n">tail</span><span class="p">],</span> <span class="s2">","</span><span class="p">)</span>
<span class="no">Code</span><span class="o">.</span><span class="n">eval_string</span><span class="p">(</span>
<span class="s2">"defp do_reduce({</span><span class="si">#{</span><span class="n">match</span><span class="si">}</span><span class="s2">}, {:cont, acc}, fun), do: do_reduce({</span><span class="si">#{</span><span class="n">tail</span><span class="si">}</span><span class="s2">}, fun.(value, acc), fun)"</span><span class="p">,</span> <span class="p">[],</span> <span class="n">__ENV__</span>
<span class="p">)</span>
<span class="k">end</span><span class="p">)</span>
<span class="c1"># list fallback for huge tuples</span>
<span class="k">defp</span> <span class="n">do_reduce</span><span class="p">([</span><span class="n">h</span> <span class="o">|</span> <span class="n">t</span><span class="p">],</span> <span class="p">{</span><span class="ss">:cont</span><span class="p">,</span> <span class="n">acc</span><span class="p">},</span> <span class="n">fun</span><span class="p">)</span> <span class="k">do</span>
<span class="n">do_reduce</span><span class="p">((</span><span class="k">if</span> <span class="no">Enum</span><span class="o">.</span><span class="n">count</span><span class="p">(</span><span class="n">t</span><span class="p">)</span> <span class="o"><=</span> <span class="nv">@max_items</span><span class="p">,</span> <span class="k">do</span><span class="p">:</span> <span class="no">List</span><span class="o">.</span><span class="n">to_tuple</span><span class="p">(</span><span class="n">t</span><span class="p">),</span> <span class="k">else</span><span class="p">:</span> <span class="n">t</span><span class="p">),</span> <span class="n">fun</span><span class="o">.</span><span class="p">(</span><span class="n">h</span><span class="p">,</span> <span class="n">acc</span><span class="p">),</span> <span class="n">fun</span><span class="p">)</span>
<span class="k">end</span>
<span class="c1"># fallback to list for huge tuples</span>
<span class="k">defp</span> <span class="n">do_reduce</span><span class="p">(</span><span class="n">huge</span><span class="p">,</span> <span class="p">{</span><span class="ss">:cont</span><span class="p">,</span> <span class="n">acc</span><span class="p">},</span> <span class="n">fun</span><span class="p">)</span> <span class="ow">when</span> <span class="n">huge</span> <span class="o">></span> <span class="nv">@max_items</span> <span class="k">do</span>
<span class="n">do_reduce</span><span class="p">(</span><span class="no">Tuple</span><span class="o">.</span><span class="n">to_list</span><span class="p">(</span><span class="n">huge</span><span class="p">),</span> <span class="p">{</span><span class="ss">:cont</span><span class="p">,</span> <span class="n">acc</span><span class="p">},</span> <span class="n">fun</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The code above builds <code class="language-plaintext highlighter-rouge">@max_items</code> function match clauses for <code class="language-plaintext highlighter-rouge">Tuple</code>s
having not more than <code class="language-plaintext highlighter-rouge">42</code> members. That guarantees no conversion to <code class="language-plaintext highlighter-rouge">list</code>
happens for those.</p>
<p>Huge <code class="language-plaintext highlighter-rouge">Tuple</code>s, having more than <code class="language-plaintext highlighter-rouge">42</code> members, will be converted to lists
before operating.</p>
<p>The approach above might be useful in the case when one needs to use
<code class="language-plaintext highlighter-rouge">Tuple</code>s for quick access: changing <code class="language-plaintext highlighter-rouge">@max_items</code> to super high value
would drastically increase <em>the compilation time</em>, while the <em>execution</em> time
will remain very impressive, comparing to <code class="language-plaintext highlighter-rouge">List</code>s.</p>
Queso al Romero2016-12-25T00:00:00+00:00https://rocket-science.ru/hobby/2016/12/25/queso-al-romero<p>A friend of my wife, visiting us in Barcelona, accidentally asked me “Where should
I buy the famous cheese with rosmarinus?”—In fact, she did not literally said
“famous,” but I got her speaking that way. We’d been living in Barcelona for
almost two years already and it was a bit offensive that I had never heard about
“cheese with rosmarinus.”</p>
<p>I went to the <a href="http://www.boqueria.info/mercat-benvinguts.php">Boqueria Mercat</a>
and bought some for the guest of Barcelona.</p>
<hr />
<p>We live near <a href="https://en.wikipedia.org/wiki/El_Putget_i_Farr%C3%B3">El Putxet</a>,
where rosmarinus is growing up in the natural wild thickets. The last time I
had a promenade there, I eventually understood I should give a try to producing
this “cheese with rosmarinus” at home. Don’t ask; I love crazy ideas and I tend
to accomplish any coming to my head.</p>
<p>There are basically two mainstream kinds of this cheese:
<a href="https://www.gourmet-food.com/spanish-cheese/cabra-al-romero-cheese-101143.aspx">Cabra al Romero</a>
and <a href="https://www.gourmet-food.com/spanish-cheese/romao-cheese-1000289.aspx">Romao Cheese</a>.
The difference is the former is produced from goat milk and the latter—from cow milk.
So I went to a supermarket and bought the one made from the mixed milk, sheep’s included.</p>
<p>Also I bought rosmarinus, estragon and dill, a pergament paper and bags with a zipper.
Here I started:</p>
<p><img src="/img/cheese/1.jpg" alt="Preparation: Ingredients" /></p>
<p>Honestly, this cheese was as tasty as the one the human being can buy for money,
even better. So I suggest you to write this recipe down and try yourself: it’s
easy and it is worth the efforts put into.</p>
<hr />
<p>Get three sheets of the pergament paper, sprinkle it abundantly with species
(more rosmarinus, less estragon, less dill.) If you have the grains of coriander,
put them there as well.</p>
<p><img src="/img/cheese/2.jpg" alt="Preparation: sprinkling pergament sheets" /></p>
<p>Place the cheese head in the middle of that spicy lawn.</p>
<p><img src="/img/cheese/3.jpg" alt="Preparation: putting the cheese head" /></p>
<p>Roll the cheese head in the species. It should be covered with as thick layer
of dried herbs, as possible.</p>
<p><img src="/img/cheese/4.jpg" alt="Preparation: covering the cheese with species" /></p>
<p>As thick as possible, I have said!</p>
<p><img src="/img/cheese/5.jpg" alt="Preparation: more covering" /></p>
<p>Wrap the cheese with pergament sheets; 7–8 sheets would be enough.</p>
<p><img src="/img/cheese/6.jpg" alt="Preparation: wrapping the cheese" /></p>
<p>Put it into the bag, zip it, and put the pack into another one zipped bag.</p>
<p><img src="/img/cheese/7.jpg" alt="Preparation: putting the cheese in the zipped bag" /></p>
<p>You are all done! Now place the bag to the fridge for one month. To get
the cheese “more aged,” one might change the pergament papers past two weeks,
they will collect the moisture out of the cheese and become wet; decreasing
the humidity in the package will make the cheese tasting more cured.</p>
<p>After one month (6 weeks are even better,) your “famous cheese with rosmarinus”
is ready. ¡Que aproveche!</p>
<p><img src="/img/cheese/8.jpg" alt="Ready & sliced" /></p>
Open Graph Protocol …and Her Friends2016-12-22T00:00:00+00:00https://rocket-science.ru/hacking/2016/12/22/open-graph-protocol-and-friends<p>When one copy-pastes a link to <em>twitter</em> and/or <em>facebook</em>, it might either become
a nifty image with a fancy title and description, or remain an orphan lonely link.</p>
<p><img src="/img/afisha.png" alt="Afisha Open Graph / Twitter" /></p>
<p>The tweet above happened to be built by twitter itself. The tweet author just
copy-pasted the link to their webpage from her browser’s address. This wow-tech
is named “Twitter Cards”; it is <a href="https://dev.twitter.com/cards/overview">described in details here</a>.</p>
<p>That said, as soon as any link is posted to twitter, it’s engine crawls the content,
parses it for some specific <code class="language-plaintext highlighter-rouge">meta</code>s and renders the pretty preview as shown above.
The very basic example of the site providing tweet rendering layout would be
<a href="https://github.com">GitHub</a>:</p>
<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt"><meta</span> <span class="na">name=</span><span class="s">"twitter:card"</span> <span class="na">content=</span><span class="s">"summary"</span> <span class="nt">/></span>
<span class="nt"><meta</span> <span class="na">name=</span><span class="s">"twitter:image:src"</span> <span class="na">content=</span><span class="s">"https://avatars2..."</span> <span class="nt">/></span>
<span class="nt"><meta</span> <span class="na">name=</span><span class="s">"twitter:site"</span> <span class="na">content=</span><span class="s">"@github"</span> <span class="nt">/></span>
<span class="nt"><meta</span> <span class="na">name=</span><span class="s">"twitter:title"</span> <span class="na">content=</span><span class="s">"TITLE"</span> <span class="nt">/></span>
<span class="nt"><meta</span> <span class="na">name=</span><span class="s">"twitter:description"</span> <span class="na">content=</span><span class="s">"DESCRIPTION"</span> <span class="nt">/></span>
</code></pre></div></div>
<p>The layout type would be <code class="language-plaintext highlighter-rouge">"summary"</code> <a href="https://dev.twitter.com/cards/types">out of four possible</a>:</p>
<blockquote>
<p><strong>Card Types</strong></p>
<ul>
<li><em>Summary Card:</em> Title, description, thumbnail, and Twitter account attribution.</li>
<li><em>Summary Card with</em> Large Image: Similar to a Summary Card, but with a prominently featured image.</li>
<li><em>App Card:</em> A Card to detail a mobile app with direct download.</li>
<li><em>Player Card:</em> A Card to provide video/audio/media.</li>
</ul>
</blockquote>
<p>So far, so good. What about facebook? Well, these guys came up with their own
solution. Welcome <a href="http://opengraphprotocol.org/">The Open Graph Protocol</a>.
As stated in the very first para of Introduction to it:</p>
<blockquote>
<p>The Open Graph protocol enables any web page to become a rich object in a social graph.
For instance, this is used on Facebook to allow any web page to have the same
functionality as any other object on Facebook.</p>
</blockquote>
<blockquote>
<p>While many different technologies and schemas exist and could be combined together, there isn’t a single technology which provides enough information to richly represent any web page within the social graph. The Open Graph protocol builds on these existing technologies and gives developers one thing to implement. Developer simplicity is a key goal of the Open Graph protocol which has informed many of the technical design decisions.</p>
</blockquote>
<p>It sounds like a galaxy conquest, but at the very moment is basically used to
embed trendy rendered links into facebook feed. GitHub surely has OGP implemented
as well:</p>
<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt"><meta</span> <span class="na">property=</span><span class="s">"og:type"</span> <span class="na">content=</span><span class="s">"object"</span> <span class="nt">/></span>
<span class="nt"><meta</span> <span class="na">property=</span><span class="s">"og:image"</span> <span class="na">content=</span><span class="s">"https://avatars2...."</span> <span class="nt">/></span>
<span class="nt"><meta</span> <span class="na">property=</span><span class="s">"og:site_name"</span> <span class="na">content=</span><span class="s">"GitHub"</span> <span class="nt">/></span>
<span class="nt"><meta</span> <span class="na">property=</span><span class="s">"og:title"</span> <span class="na">content=</span><span class="s">"TITLE"</span> <span class="nt">/></span>
<span class="nt"><meta</span> <span class="na">property=</span><span class="s">"og:description"</span> <span class="na">content=</span><span class="s">"DESCRIPTION"</span> <span class="nt">/></span>
<span class="nt"><meta</span> <span class="na">property=</span><span class="s">"og:url"</span> <span class="na">content=</span><span class="s">"https://github.com/..."</span> <span class="nt">/></span>
</code></pre></div></div>
<p>Also, OGP appreciates the respective namespaces to be explicitly specified in the <code class="language-plaintext highlighter-rouge"><html></code>
tag:</p>
<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt"><html</span> <span class="na">prefix=</span><span class="s">"og: http://ogp.me/ns#"</span><span class="nt">></span>
</code></pre></div></div>
<p>GitHub bothers about in it’s own way (note <code class="language-plaintext highlighter-rouge">head</code> tag):</p>
<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt"><head</span> <span class="na">prefix=</span><span class="s">"og: http://ogp.me/ns#
fb: http://ogp.me/ns/fb#
object: http://ogp.me/ns/object#
article: http://ogp.me/ns/article#
profile: http://ogp.me/ns/profile#"</span><span class="nt">></span>
</code></pre></div></div>
<p>In any case, it’s safe to omit these: consumers will <em>parse</em>, not <em>build XML tree</em>.</p>
<hr />
<p>The profit of it would be: one might add a great-looking “share” buttons to her site
without any significant effort. Just combine these tags (see, 5 of them have same <code class="language-plaintext highlighter-rouge">content</code>
value) and get the fancy mentions in social networks, supporting OGP.</p>
<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt"><meta</span> <span class="na">name=</span><span class="s">"twitter:card"</span> <span class="na">content=</span><span class="s">"summary"</span> <span class="nt">/></span>
<span class="nt"><meta</span> <span class="na">property=</span><span class="s">"og:type"</span> <span class="na">content=</span><span class="s">"article"</span> <span class="nt">/></span>
<span class="nt"><meta</span> <span class="na">name=</span><span class="s">"twitter:image:src"</span> <span class="na">property=</span><span class="s">"og:image"</span> <span class="na">content=</span><span class="s">"https://avatars2..."</span> <span class="nt">/></span>
<span class="nt"><meta</span> <span class="na">name=</span><span class="s">"twitter:site"</span> <span class="na">property=</span><span class="s">"og:site_name"</span> <span class="na">content=</span><span class="s">"@rocket-science.ru"</span> <span class="nt">/></span>
<span class="nt"><meta</span> <span class="na">name=</span><span class="s">"twitter:url"</span> <span class="na">property=</span><span class="s">"og:url"</span> <span class="na">content=</span><span class="s">"https://rocket-s..."</span> <span class="nt">/></span>
<span class="nt"><meta</span> <span class="na">name=</span><span class="s">"twitter:creator"</span> <span class="na">content=</span><span class="s">"@mudasobwa"</span> <span class="nt">/></span>
<span class="nt"><meta</span> <span class="na">name=</span><span class="s">"twitter:title"</span> <span class="na">property=</span><span class="s">"og:title"</span> <span class="na">content=</span><span class="s">"TITLE"</span> <span class="nt">/></span>
<span class="nt"><meta</span> <span class="na">name=</span><span class="s">"twitter:description"</span> <span class="na">property=</span><span class="s">"og:description"</span> <span class="na">content=</span><span class="s">"DESCRIPTION"</span> <span class="nt">/></span>
</code></pre></div></div>
<p>We are done. Happy embedding!</p>
Fixing Broken Lightbulbs in a Nutshell2016-12-21T00:00:00+00:00https://rocket-science.ru/fun/2016/12/21/fixing-broken-lightbulbs-in-a-nutshell<blockquote>
<p>inspired by <a href="https://blog.toggl.com/2016/12/developers-explained-with-lightbulbs/">IT Jobs explained with a broken lightbulb</a></p>
</blockquote>
<ul>
<li><em>Javascripter:</em> casts many cumbersome async promises to make <code class="language-plaintext highlighter-rouge">this</code> refer to sales room’s lightbulb;</li>
<li><em>Rubyist:</em> monkeypatches lightbulb to use a teapot spiral as a glower;</li>
<li><em>COBOL grandma:</em> copies an office with many lightbulbs from the upper floor;</li>
<li><em>PHP junior:</em> dies with the message “lightbulb: ошиб�”;</li>
<li><em>Pythonist:</em> indents a lightbulb from the teapot with eight spaces, it lights back up;</li>
<li><em>Erlanger:</em> crashes a lightbulb with a baseball bit, the lightbuld is respawned shining;</li>
<li><em>Elixir:</em> crashes a lightbulb with garbage disposal machine, the lightbuld is respawned shining;</li>
<li><em>С/С++ guru:</em> solders a lampholder out, patches the glower with a paperclip;</li>
<li><em>Rust:</em> solders a lampholder out, patches the glower with a brand new firmware glower;</li>
<li><em>LISP loyal user:</em> throws the bulb out of the window to escape a surveillance;</li>
<li><em>Java Enterprise Senior:</em> buys a floodlight and settles it outside of the window;</li>
<li><em>Go-fella:</em> hires 42 juniors and creates a department on the problem “fixing lightbulbs.”</li>
</ul>
Dry Behaviour aka Protocol Pattern in Ruby2016-12-19T00:00:00+00:00https://rocket-science.ru/hacking/2016/12/19/dry-behaviour<p>Elixir introduced the concept of behaviours. The quote from the <a href="http://elixir-lang.org/getting-started/protocols.html">official docs</a>:</p>
<blockquote>
<p>Protocols are a mechanism to achieve polymorphism in Elixir.
Dispatching on a protocol is available to any data type as long as it implements the protocol.</p>
</blockquote>
<p>What is it all about? Well, Elixir entities, aka “terms,” are all immutable.
While in ruby we tend to declare methods on objects, that simply mutate the
objects, in Elixir it is impossible. Everybody had seen the <code class="language-plaintext highlighter-rouge">Animal</code> example
explaining the polymorphism in a nutshell for any of so-called OO languages:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Animal</span>
<span class="k">def</span> <span class="nf">sound</span>
<span class="k">raise</span> <span class="s2">"I am an abstract animal, I keep silence (and mystery.)"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">class</span> <span class="nc">Dog</span> <span class="o"><</span> <span class="no">Animal</span>
<span class="k">def</span> <span class="nf">sound</span>
<span class="nb">puts</span> <span class="s2">"[LOG] I’m a dog, I bark"</span>
<span class="s2">"woof"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">class</span> <span class="nc">Cat</span> <span class="o"><</span> <span class="no">Animal</span>
<span class="k">def</span> <span class="nf">sound</span>
<span class="nb">puts</span> <span class="s2">"[LOG] I’m a cat, I am meowing"</span>
<span class="s2">"meow"</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Now we are safe to call <code class="language-plaintext highlighter-rouge">sound</code> method on any animal, without bothering
to determine what exact type of animal we are facing. In Elixir, on the other
hand, we do not have “methods defined on objects.” The approach to achieve
more or less same functionality (the most typical example of where it’s really
handy is, for instance, the string interpolation,) would be to declare
the protocol.</p>
<p><em>Sidenote:</em> another approach would be to use <a href="http://elixir-lang.org/getting-started/typespecs-and-behaviours.html#behaviours">behaviours</a>,
but for the sake of our task we would stick to protocols in this post.</p>
<p>The protocol is a pure interface, declared with <code class="language-plaintext highlighter-rouge">defprotocol</code> keyword.
For the animalistic example above it would be:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defprotocol</span> <span class="no">Noisy</span> <span class="k">do</span>
<span class="nv">@doc</span> <span class="s2">"Produces a sound for the animal given"</span>
<span class="k">def</span> <span class="n">sound</span><span class="p">(</span><span class="n">animal</span><span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The implementation goes into <code class="language-plaintext highlighter-rouge">defimpl</code> clause:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">defimpl</span> <span class="no">Noisy</span><span class="p">,</span> <span class="ss">for:</span> <span class="no">Dog</span> <span class="k">do</span>
<span class="k">def</span> <span class="n">sound</span><span class="p">(</span><span class="n">animal</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="s2">"woof"</span>
<span class="k">end</span>
<span class="k">defimpl</span> <span class="no">Noisy</span><span class="p">,</span> <span class="ss">for:</span> <span class="no">Cat</span> <span class="k">do</span>
<span class="k">def</span> <span class="n">sound</span><span class="p">(</span><span class="n">animal</span><span class="p">),</span> <span class="k">do</span><span class="p">:</span> <span class="s2">"meow"</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Now we can use the protocol, without actual care who the animal we have:</p>
<div class="language-elixir highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">ExtrernalSource</span><span class="o">.</span><span class="n">animal</span>
<span class="o">|></span> <span class="no">Noisy</span><span class="o">.</span><span class="n">sound</span>
</code></pre></div></div>
<hr />
<p>OK. Why would we want to have this pattern in ruby? We indeed already have
polymorphism, right? Yes. And no. The most evident example would be classes,
coming from different external third-party sources, but still having
something in common. The <em>rails</em> approach, widely spread into ruby world by DHH,
would be to monkeypatch everything. The irony here is that <em>I personally love
monkeypatching</em>. Yet in some cases I find the <code class="language-plaintext highlighter-rouge">protocol</code> approach being more
robust. That way, instead of re-opening <code class="language-plaintext highlighter-rouge">Integer</code> class for declaring date-aware
methods, one might declare the protocol, having <code class="language-plaintext highlighter-rouge">to_days</code> method.</p>
<p>It in turn might be used as <code class="language-plaintext highlighter-rouge">DateGuru.to_days(something)</code> instead of
<code class="language-plaintext highlighter-rouge">something.to_days</code>. That way all the code, responsible for the date
conversions/operations, would be placed together, providing sorta guarantee
that there are no conflicts, no accidental unintended monkeypatches etc.</p>
<p>I am not advocating this approach is better; it is just different.</p>
<p>To try it, we would need to provide some DSL to make it easy to declare
protocols in pure ruby. Let’s do it. We are to start with tests.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nn">Protocols::Arithmetics</span>
<span class="kp">include</span> <span class="no">Dry</span><span class="o">::</span><span class="no">Protocol</span>
<span class="n">defprotocol</span> <span class="k">do</span>
<span class="n">defmethod</span> <span class="ss">:add</span><span class="p">,</span> <span class="ss">:this</span><span class="p">,</span> <span class="ss">:other</span>
<span class="n">defmethod</span> <span class="ss">:subtract</span><span class="p">,</span> <span class="ss">:this</span><span class="p">,</span> <span class="ss">:other</span>
<span class="n">defmethod</span> <span class="ss">:to_s</span><span class="p">,</span> <span class="ss">:this</span>
<span class="k">def</span> <span class="nf">multiply</span><span class="p">(</span><span class="n">this</span><span class="p">,</span> <span class="n">other</span><span class="p">)</span>
<span class="k">raise</span> <span class="s2">"We can multiply by integers only"</span> <span class="k">unless</span> <span class="n">other</span><span class="p">.</span><span class="nf">is_a?</span><span class="p">(</span><span class="no">Integer</span><span class="p">)</span>
<span class="p">(</span><span class="mi">1</span><span class="o">...</span><span class="n">other</span><span class="p">).</span><span class="nf">inject</span><span class="p">(</span><span class="n">this</span><span class="p">)</span> <span class="p">{</span> <span class="o">|</span><span class="n">memo</span><span class="p">,</span><span class="o">|</span> <span class="n">memo</span> <span class="o">+</span> <span class="n">this</span> <span class="p">}</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="n">defimpl</span> <span class="no">Protocols</span><span class="o">::</span><span class="no">Arithmetics</span><span class="p">,</span> <span class="ss">target: </span><span class="no">String</span> <span class="k">do</span>
<span class="k">def</span> <span class="nf">add</span><span class="p">(</span><span class="n">this</span><span class="p">,</span> <span class="n">other</span><span class="p">)</span>
<span class="n">this</span> <span class="o">+</span> <span class="n">other</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">subtract</span><span class="p">(</span><span class="n">this</span><span class="p">,</span> <span class="n">other</span><span class="p">)</span>
<span class="n">this</span><span class="p">.</span><span class="nf">gsub</span> <span class="sr">/</span><span class="si">#{</span><span class="n">other</span><span class="si">}</span><span class="sr">/</span><span class="p">,</span> <span class="s1">''</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">to_s</span>
<span class="n">this</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="n">defimpl</span> <span class="ss">target: </span><span class="p">[</span><span class="no">Integer</span><span class="p">,</span> <span class="no">Float</span><span class="p">],</span> <span class="ss">delegate: :to_s</span><span class="p">,</span> <span class="ss">map: </span><span class="p">{</span> <span class="ss">add: </span><span class="p">:</span><span class="o">+</span><span class="p">,</span> <span class="ss">subtract: </span><span class="p">:</span><span class="o">-</span> <span class="p">}</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Let’s dig a bit into the code above. We have declared the protocol <code class="language-plaintext highlighter-rouge">Arithmetics</code>,
responsible for adding and subtracting values. Once two operations above
are implemented for instances of some class, we have <code class="language-plaintext highlighter-rouge">multiply</code> method for granted.
The usage of this protocol would be <code class="language-plaintext highlighter-rouge">Arithmetics.add(42, 3) #⇒ 45</code>.
Our DSL support <em>method delegation</em>, <em>mapping</em> and explicit declaration.</p>
<p>This contrived example does not make much sense as is, but it provides a good
test case for our DSL. Let’s write tests.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">expect</span><span class="p">(</span><span class="no">Protocols</span><span class="o">::</span><span class="no">Adder</span><span class="p">.</span><span class="nf">add</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="mi">3</span><span class="p">)).</span><span class="nf">to</span> <span class="n">eq</span><span class="p">(</span><span class="mi">8</span><span class="p">)</span>
<span class="n">expect</span><span class="p">(</span><span class="no">Protocols</span><span class="o">::</span><span class="no">Adder</span><span class="p">.</span><span class="nf">add</span><span class="p">(</span><span class="mf">5.5</span><span class="p">,</span> <span class="mi">3</span><span class="p">)).</span><span class="nf">to</span> <span class="n">eq</span><span class="p">(</span><span class="mf">8.5</span><span class="p">)</span>
<span class="n">expect</span><span class="p">(</span><span class="no">Protocols</span><span class="o">::</span><span class="no">Adder</span><span class="p">.</span><span class="nf">subtract</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="mi">10</span><span class="p">)).</span><span class="nf">to</span> <span class="n">eq</span><span class="p">(</span><span class="o">-</span><span class="mi">5</span><span class="p">)</span>
<span class="n">expect</span><span class="p">(</span><span class="no">Protocols</span><span class="o">::</span><span class="no">Adder</span><span class="p">.</span><span class="nf">multiply</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="mi">3</span><span class="p">)).</span><span class="nf">to</span> <span class="n">eq</span><span class="p">(</span><span class="mi">15</span><span class="p">)</span>
<span class="n">expect</span> <span class="k">do</span>
<span class="no">Protocols</span><span class="o">::</span><span class="no">Adder</span><span class="p">.</span><span class="nf">multiply</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="mf">3.5</span><span class="p">)</span>
<span class="k">end</span><span class="p">.</span><span class="nf">to</span> <span class="n">raise_error</span><span class="p">(</span><span class="no">RuntimeException</span><span class="p">,</span> <span class="s2">"We can multiply by integers only"</span><span class="p">)</span>
</code></pre></div></div>
<p>Yay, it’s time to finally implement this DSL. This is easy.</p>
<hr />
<p>The whole implementation fits one single module. We would call it <code class="language-plaintext highlighter-rouge">BlackTie</code>,
since it’s all about protocols. In the first place tt will hold the maps of
declared protocols to their implementations.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nn">BlackTie</span>
<span class="k">class</span> <span class="o"><<</span> <span class="nb">self</span>
<span class="k">def</span> <span class="nf">protocols</span>
<span class="vi">@protocols</span> <span class="o">||=</span> <span class="no">Hash</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span> <span class="o">|</span><span class="n">h</span><span class="p">,</span> <span class="n">k</span><span class="o">|</span> <span class="n">h</span><span class="p">[</span><span class="n">k</span><span class="p">]</span> <span class="o">=</span> <span class="n">h</span><span class="p">.</span><span class="nf">dup</span><span class="p">.</span><span class="nf">clear</span> <span class="p">}</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">implementations</span>
<span class="vi">@implementations</span> <span class="o">||=</span> <span class="no">Hash</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span> <span class="o">|</span><span class="n">h</span><span class="p">,</span> <span class="n">k</span><span class="o">|</span> <span class="n">h</span><span class="p">[</span><span class="n">k</span><span class="p">]</span> <span class="o">=</span> <span class="n">h</span><span class="p">.</span><span class="nf">dup</span><span class="p">.</span><span class="nf">clear</span> <span class="p">}</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p><em>Sidenote:</em> the trick with <code class="language-plaintext highlighter-rouge">default_proc</code> in hash declarations
(<code class="language-plaintext highlighter-rouge">Hash.new { |h, k| h[k] = h.dup.clear }</code>) produces the hash that has
a deep <code class="language-plaintext highlighter-rouge">default_proc</code>, returning an empty hash.</p>
<p><code class="language-plaintext highlighter-rouge">defmethod</code> is the most trivial method here, it simply stores the
declaration under respective name in the global <code class="language-plaintext highlighter-rouge">@protocols</code> hash:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">defmethod</span><span class="p">(</span><span class="nb">name</span><span class="p">,</span> <span class="o">*</span><span class="n">params</span><span class="p">)</span>
<span class="no">BlackTie</span><span class="p">.</span><span class="nf">protocols</span><span class="p">[</span><span class="nb">self</span><span class="p">][</span><span class="nb">name</span><span class="p">]</span> <span class="o">=</span> <span class="n">params</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Declaration of the <code class="language-plaintext highlighter-rouge">protocol</code> is a bit more cumbersome (some details are
omitted here for the sake of clarity, see
<a href="https://github.com/am-kantox/dry-behaviour/blob/master/lib/dry/behaviour/black_tie.rb#L19">the full code here</a>.)</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">defprotocol</span>
<span class="k">raise</span> <span class="k">if</span> <span class="no">BlackTie</span><span class="p">.</span><span class="nf">protocols</span><span class="p">.</span><span class="nf">key?</span><span class="p">(</span><span class="nb">self</span><span class="p">)</span> <span class="o">||</span> <span class="o">!</span><span class="nb">block_given?</span>
<span class="n">ims</span> <span class="o">=</span> <span class="nb">instance_methods</span><span class="p">(</span><span class="kp">false</span><span class="p">)</span>
<span class="nb">class_eval</span><span class="p">(</span><span class="o">&</span><span class="no">Proc</span><span class="p">.</span><span class="nf">new</span><span class="p">)</span>
<span class="p">(</span><span class="nb">instance_methods</span><span class="p">(</span><span class="kp">false</span><span class="p">)</span> <span class="o">-</span> <span class="n">ims</span><span class="p">).</span><span class="nf">each</span> <span class="p">{</span> <span class="o">|</span><span class="n">m</span><span class="o">|</span> <span class="nb">class_eval</span> <span class="p">{</span> <span class="kp">module_function</span> <span class="n">m</span> <span class="p">}</span> <span class="p">}</span>
<span class="n">singleton_class</span><span class="p">.</span><span class="nf">send</span> <span class="ss">:define_method</span><span class="p">,</span> <span class="ss">:method_missing</span> <span class="k">do</span> <span class="o">|</span><span class="nb">method</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="o">|</span>
<span class="k">raise</span> <span class="no">Dry</span><span class="o">::</span><span class="no">Protocol</span><span class="o">::</span><span class="no">NotImplemented</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="ss">:method</span><span class="p">,</span> <span class="nb">self</span><span class="p">.</span><span class="nf">inspect</span><span class="p">,</span> <span class="nb">method</span><span class="p">)</span>
<span class="k">end</span>
<span class="no">BlackTie</span><span class="p">.</span><span class="nf">protocols</span><span class="p">[</span><span class="nb">self</span><span class="p">].</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="nb">method</span><span class="p">,</span> <span class="o">*|</span>
<span class="n">singleton_class</span><span class="p">.</span><span class="nf">send</span> <span class="ss">:define_method</span><span class="p">,</span> <span class="nb">method</span> <span class="k">do</span> <span class="o">|</span><span class="n">receiver</span> <span class="o">=</span> <span class="kp">nil</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="o">|</span>
<span class="n">impl</span> <span class="o">=</span> <span class="n">receiver</span><span class="p">.</span><span class="nf">class</span><span class="p">.</span><span class="nf">ancestors</span><span class="p">.</span><span class="nf">lazy</span><span class="p">.</span><span class="nf">map</span> <span class="k">do</span> <span class="o">|</span><span class="n">c</span><span class="o">|</span>
<span class="no">BlackTie</span><span class="p">.</span><span class="nf">implementations</span><span class="p">[</span><span class="nb">self</span><span class="p">].</span><span class="nf">fetch</span><span class="p">(</span><span class="n">c</span><span class="p">,</span> <span class="kp">nil</span><span class="p">)</span>
<span class="k">end</span><span class="p">.</span><span class="nf">reject</span><span class="p">(</span><span class="o">&</span><span class="ss">:nil?</span><span class="p">).</span><span class="nf">first</span>
<span class="k">raise</span> <span class="no">Dry</span><span class="o">::</span><span class="no">Protocol</span><span class="o">::</span><span class="no">NotImplemented</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="ss">:protocol</span><span class="p">,</span> <span class="nb">self</span><span class="p">.</span><span class="nf">inspect</span><span class="p">,</span> <span class="n">receiver</span><span class="p">.</span><span class="nf">class</span><span class="p">)</span> <span class="k">unless</span> <span class="n">impl</span>
<span class="n">impl</span><span class="p">[</span><span class="nb">method</span><span class="p">]</span><span class="o">.</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">.</span><span class="nf">unshift</span><span class="p">(</span><span class="n">receiver</span><span class="p">))</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Basically, the code above has four block. First of all, we check the conditions
the protocol must meet. Then we execute a block given, recording what methods
were added by this block, and exposing them with <code class="language-plaintext highlighter-rouge">module_function</code>.
In the third block we declare the generic <code class="language-plaintext highlighter-rouge">method_missing</code> to provide
meaningful error messages on erroneous calls. And, lastly, we declare methods,
either delegating them to respective implementation (when exists,) or throwing
the descriptive exception is there is no implementation for this particular
receiver.</p>
<p>OK, the only thing left is to declare <code class="language-plaintext highlighter-rouge">defimpl</code> DSL. The code below is a bit
simplified.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">defimpl</span><span class="p">(</span><span class="n">protocol</span> <span class="o">=</span> <span class="kp">nil</span><span class="p">,</span> <span class="ss">target: </span><span class="kp">nil</span><span class="p">,</span> <span class="ss">delegate: </span><span class="p">[],</span> <span class="ss">map: </span><span class="p">{})</span>
<span class="k">raise</span> <span class="k">if</span> <span class="n">target</span><span class="p">.</span><span class="nf">nil?</span> <span class="o">||</span> <span class="o">!</span><span class="nb">block_given?</span> <span class="o">&&</span> <span class="n">delegate</span><span class="p">.</span><span class="nf">empty?</span> <span class="o">&&</span> <span class="n">map</span><span class="p">.</span><span class="nf">empty?</span>
<span class="c1"># builds the simple map out of both delegates and map</span>
<span class="n">mds</span> <span class="o">=</span> <span class="n">normalize_map_delegates</span><span class="p">(</span><span class="n">delegate</span><span class="p">,</span> <span class="n">map</span><span class="p">)</span>
<span class="no">Module</span><span class="p">.</span><span class="nf">new</span> <span class="k">do</span>
<span class="n">mds</span><span class="p">.</span><span class="nf">each</span><span class="p">(</span><span class="o">&</span><span class="no">DELEGATE_METHOD</span><span class="p">.</span><span class="nf">curry</span><span class="p">[</span><span class="n">singleton_class</span><span class="p">])</span> <span class="c1"># delegation impl</span>
<span class="n">singleton_class</span><span class="p">.</span><span class="nf">class_eval</span><span class="p">(</span><span class="o">&</span><span class="no">Proc</span><span class="p">.</span><span class="nf">new</span><span class="p">)</span> <span class="k">if</span> <span class="nb">block_given?</span> <span class="c1"># block takes precedence</span>
<span class="k">end</span><span class="p">.</span><span class="nf">tap</span> <span class="k">do</span> <span class="o">|</span><span class="n">mod</span><span class="o">|</span>
<span class="n">mod</span><span class="p">.</span><span class="nf">methods</span><span class="p">(</span><span class="kp">false</span><span class="p">).</span><span class="nf">tap</span> <span class="k">do</span> <span class="o">|</span><span class="n">meths</span><span class="o">|</span>
<span class="p">(</span><span class="no">BlackTie</span><span class="p">.</span><span class="nf">protocols</span><span class="p">[</span><span class="n">protocol</span> <span class="o">||</span> <span class="nb">self</span><span class="p">].</span><span class="nf">keys</span> <span class="o">-</span> <span class="n">meths</span><span class="p">).</span><span class="nf">each_with_object</span><span class="p">(</span><span class="n">meths</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">m</span><span class="p">,</span> <span class="n">acc</span><span class="o">|</span>
<span class="n">logger</span><span class="p">.</span><span class="nf">warn</span><span class="p">(</span><span class="s2">"Implicit delegate </span><span class="si">#{</span><span class="p">(</span><span class="n">protocol</span> <span class="o">||</span> <span class="nb">self</span><span class="p">).</span><span class="nf">inspect</span><span class="si">}</span><span class="s2">#</span><span class="si">#{</span><span class="n">m</span><span class="si">}</span><span class="s2"> to </span><span class="si">#{</span><span class="n">target</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
<span class="no">DELEGATE_METHOD</span><span class="o">.</span><span class="p">(</span><span class="n">mod</span><span class="p">.</span><span class="nf">singleton_class</span><span class="p">,</span> <span class="p">[</span><span class="n">m</span><span class="p">]</span> <span class="o">*</span> <span class="mi">2</span><span class="p">)</span>
<span class="n">acc</span> <span class="o"><<</span> <span class="n">m</span>
<span class="k">end</span>
<span class="k">end</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">m</span><span class="o">|</span>
<span class="p">[</span><span class="o">*</span><span class="n">target</span><span class="p">].</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">tgt</span><span class="o">|</span>
<span class="no">BlackTie</span><span class="p">.</span><span class="nf">implementations</span><span class="p">[</span><span class="n">protocol</span> <span class="o">||</span> <span class="nb">self</span><span class="p">][</span><span class="n">tgt</span><span class="p">][</span><span class="n">m</span><span class="p">]</span> <span class="o">=</span> <span class="n">mod</span><span class="p">.</span><span class="nf">method</span><span class="p">(</span><span class="n">m</span><span class="p">).</span><span class="nf">to_proc</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="kp">module_function</span> <span class="ss">:defimpl</span>
</code></pre></div></div>
<p>Despite the amount of LOCs, the code above is fairly simple: we create an
anonymous module, declare methods on it and supply it as the target of
method delegation from the main protocol class methods. Once we have called
<code class="language-plaintext highlighter-rouge">Arithmetics.add(5, 3)</code>, the receiver (<code class="language-plaintext highlighter-rouge">5</code>) would be used to lookup the
respective implementation (<code class="language-plaintext highlighter-rouge">defimpl Arithmetics, target: Integer</code>) and
it’s method <code class="language-plaintext highlighter-rouge">:+</code> (because of <code class="language-plaintext highlighter-rouge">defimpl target: [Integer, ...], ..., map: { add: :+, ... }</code>,
<code class="language-plaintext highlighter-rouge">add</code> is mapped to <code class="language-plaintext highlighter-rouge">:+</code>) would be called. That’s it.</p>
<hr />
<p>Whether you still think, this is a redundant of-no-practival-use garbage,
imagine the <code class="language-plaintext highlighter-rouge">Tax</code> protocol. That might be implemented for: <code class="language-plaintext highlighter-rouge">ItemToSell</code>,
<code class="language-plaintext highlighter-rouge">Shipment</code>, <code class="language-plaintext highlighter-rouge">Employee</code>, <code class="language-plaintext highlighter-rouge">Lunch</code> etc.</p>
<p>❖ <a href="https://github.com/am-kantox/dry-behaviour/"><code class="language-plaintext highlighter-rouge">dry-behaviour</code> repo</a>. Enjoy!</p>
Struct With Hash-like Default Proc2016-12-13T00:00:00+00:00https://rocket-science.ru/hacking/2016/12/13/struct-with-default-proc<p>This question was originally asked on <a href="http://stackoverflow.com/questions/41114667/how-to-implement-autovivification-for-ruby-structs/41115465#41115465">StackOverflow</a>,
and the answer is more-or-less trivial; in any case I find myself explaining
some of this metaprogramming techniques as often, as I decided to finally
write this short post on the topic.</p>
<p><strong><em>Q.</em> How to implement autovivification for Ruby structs?</strong></p>
<p>Everybody programming ruby at least three months should have met the
<a href="http://ruby-doc.org/core/Hash.html#method-i-default_proc"><code class="language-plaintext highlighter-rouge">Hash#default_proc</code></a>
behaviour. This <code class="language-plaintext highlighter-rouge">default_proc</code> might be also passed directly to the constructor
of the hash:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">▶</span> <span class="nb">hash</span> <span class="o">=</span> <span class="no">Hash</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span> <span class="o">|</span><span class="n">h</span><span class="p">,</span> <span class="n">k</span><span class="o">|</span> <span class="n">h</span><span class="p">[</span><span class="n">k</span><span class="p">]</span> <span class="o">=</span> <span class="mi">42</span> <span class="p">}</span>
<span class="c1">#⇒ {}</span>
<span class="err">▶</span> <span class="nb">hash</span><span class="p">[</span><span class="ss">:answer</span><span class="p">]</span>
<span class="c1">#⇒ 42</span>
</code></pre></div></div>
<p>One might require the sane functionality from structs (why not, after all?)
Indeed, this is easy.</p>
<p>Let’s start with the complete working example.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nn">StructVivificator</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">prepended</span><span class="p">(</span><span class="n">base</span><span class="p">)</span>
<span class="k">raise</span> <span class="s1">'Sorry, structs only!'</span> <span class="k">unless</span> <span class="n">base</span> <span class="o"><</span> <span class="no">Struct</span>
<span class="n">base</span><span class="p">.</span><span class="nf">singleton_class</span><span class="p">.</span><span class="nf">prepend</span><span class="p">(</span><span class="no">Module</span><span class="p">.</span><span class="nf">new</span> <span class="k">do</span>
<span class="k">def</span> <span class="nf">new</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">&</span><span class="err">λ</span><span class="p">)</span> <span class="c1"># override `new` to accept block</span>
<span class="k">super</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">).</span><span class="nf">tap</span> <span class="p">{</span> <span class="err">@λ</span> <span class="o">=</span> <span class="err">λ</span> <span class="p">}</span>
<span class="k">end</span>
<span class="k">end</span><span class="p">)</span>
<span class="n">base</span><span class="p">.</span><span class="nf">send</span><span class="p">(</span><span class="ss">:define_method</span><span class="p">,</span> <span class="ss">:default_proc</span><span class="o">=</span><span class="p">)</span> <span class="p">{</span> <span class="o">|</span><span class="err">λ</span><span class="o">|</span> <span class="err">@λ</span> <span class="o">=</span> <span class="err">λ</span> <span class="p">}</span>
<span class="n">base</span><span class="p">.</span><span class="nf">send</span><span class="p">(</span><span class="ss">:define_method</span><span class="p">,</span> <span class="ss">:default_proc</span><span class="p">)</span> <span class="p">{</span> <span class="o">|&</span><span class="err">λ</span><span class="o">|</span> <span class="err">λ</span> <span class="p">?</span> <span class="err">@λ</span> <span class="o">=</span> <span class="err">λ</span> <span class="p">:</span> <span class="err">@λ</span> <span class="p">}</span>
<span class="c1"># override accessors (additional advantage: performance/clarity)</span>
<span class="n">base</span><span class="p">.</span><span class="nf">members</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">m</span><span class="o">|</span>
<span class="n">base</span><span class="p">.</span><span class="nf">send</span><span class="p">(</span><span class="ss">:define_method</span><span class="p">,</span> <span class="n">m</span><span class="p">)</span> <span class="p">{</span> <span class="nb">self</span><span class="p">[</span><span class="n">m</span><span class="p">]</span> <span class="p">}</span>
<span class="n">base</span><span class="p">.</span><span class="nf">send</span><span class="p">(</span><span class="ss">:define_method</span><span class="p">,</span> <span class="s2">"</span><span class="si">#{</span><span class="n">m</span><span class="si">}</span><span class="s2">="</span><span class="p">)</span> <span class="p">{</span> <span class="o">|</span><span class="n">value</span><span class="o">|</span> <span class="nb">self</span><span class="p">[</span><span class="n">m</span><span class="p">]</span> <span class="o">=</span> <span class="n">value</span> <span class="p">}</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">[]</span><span class="p">(</span><span class="nb">name</span><span class="p">)</span>
<span class="k">super</span> <span class="o">||</span> <span class="n">default_proc</span> <span class="o">&&</span> <span class="n">default_proc</span><span class="o">.</span><span class="p">(</span><span class="nb">name</span><span class="p">)</span> <span class="c1"># or more sophisticated checks</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="c1">############################################</span>
<span class="c1">##### usage example</span>
<span class="c1">############################################</span>
<span class="no">Foo</span> <span class="o">=</span> <span class="no">Struct</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="ss">:bar</span><span class="p">,</span> <span class="ss">:baz</span><span class="p">)</span> <span class="k">do</span>
<span class="n">prepend</span> <span class="no">StructVivificator</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Here we declared the module to <code class="language-plaintext highlighter-rouge">prepend</code>. Once prepended, it checks
whether it was prepended to <code class="language-plaintext highlighter-rouge">Struct</code>, and declares two methods
on the base class: getter and setter for <code class="language-plaintext highlighter-rouge">default_proc</code>. Also, it overwrites
the default <a href="https://ruby-doc.org/core/Struct.html#method-i-5B-5D"><code class="language-plaintext highlighter-rouge">Struct#[]</code></a>
<em>property getter</em>, trying to call the superior method and gracefully falling
back to the call to <code class="language-plaintext highlighter-rouge">default_proc</code>, if declared.</p>
<p>Since the <code class="language-plaintext highlighter-rouge">.property</code> access calls <code class="language-plaintext highlighter-rouge">[:property]</code> under the hood
through <code class="language-plaintext highlighter-rouge">method_missing</code> magic, the only thing to overwrite is a <code class="language-plaintext highlighter-rouge">Struct#[]</code> method.</p>
<p>So far so good. Let’s test it.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">foo</span> <span class="o">=</span> <span class="no">Foo</span><span class="p">.</span><span class="nf">new</span>
<span class="n">foo</span><span class="p">.</span><span class="nf">default_proc</span> <span class="o">=</span> <span class="o">-></span><span class="p">(</span><span class="nb">name</span><span class="p">)</span> <span class="p">{</span> <span class="nb">name</span> <span class="o">==</span> <span class="ss">:bar</span> <span class="p">?</span> <span class="mi">42</span> <span class="p">:</span> <span class="mi">0</span> <span class="p">}</span>
<span class="nb">puts</span> <span class="n">foo</span><span class="p">.</span><span class="nf">bar</span> <span class="c1"># => 42</span>
<span class="nb">puts</span> <span class="n">foo</span><span class="p">[</span><span class="ss">:bar</span><span class="p">]</span> <span class="o">+=</span> <span class="mi">1</span> <span class="c1"># => 43</span>
<span class="nb">puts</span> <span class="n">foo</span><span class="p">.</span><span class="nf">bar</span> <span class="o">+=</span> <span class="mi">1</span> <span class="c1"># => 44</span>
<span class="nb">puts</span> <span class="n">foo</span><span class="p">[</span><span class="ss">:baz</span><span class="p">]</span> <span class="o">+=</span> <span class="mi">1</span> <span class="c1"># => 1</span>
</code></pre></div></div>
Progress Bar in Console for Rake Tasks2016-11-24T00:00:00+00:00https://rocket-science.ru/hacking/2016/11/24/progress-bar-for-dummies<p>Yesterday I found myself writing a long-running migration rake task.</p>
<p>It was supposed to walk through one big database table and update records.
One by one. There was no way to write a clever SQL query: 3rd party service
was called for each record and the updated column value depended on
the result of the request. I do not like starring at the stale console screen
since 90s, when I had a 486 machine with 2MB of RAM. There is no typo above:
my working computer, I used heavily to earn money as freelancer had 2MB of ROM.
20 years ago. Anyway, there is no room for nostalgia now and I feel no
nostalgia at all for 14Kb modem connection, ROM counted in megabytes and
processors, doing less flops than my cat chasing a fly.</p>
<p>I decided I need a progress bar for this rake task to show—well—progress.
I went to <a href="https://duckduckgo.com"><code class="language-plaintext highlighter-rouge">DuckDuckGo</code></a> (which is by all means better
than Google for developers) and asked for a ruby gem providing console status
bar functionality. There are many. With all respect to their authors, they must
serve coffee and massage your back for this amount of code they contain.</p>
<p>Damn it, I decided. It would be faster to write the progress bar on my own,
than to download one of these gems and digdoc there interfaces. After all,
everything I need from a progress bar is to draw—well—progress bar.</p>
<p>Below is the source code of it. I put this file into my <code class="language-plaintext highlighter-rouge">lib/tasks</code> directory
and I <code class="language-plaintext highlighter-rouge">require_relative</code> it from tasks where I need to draw a meter. That simple.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">ProgressBar</span>
<span class="no">RUNNING</span> <span class="o">=</span> <span class="sx">%W{👆 👇 👈 👉 👊}</span><span class="p">.</span><span class="nf">freeze</span>
<span class="nb">attr_reader</span> <span class="ss">:count</span><span class="p">,</span> <span class="ss">:position</span>
<span class="k">def</span> <span class="nf">initialize</span> <span class="n">caption</span><span class="p">,</span> <span class="n">count</span><span class="p">,</span> <span class="n">running</span> <span class="o">=</span> <span class="no">RUNNING</span>
<span class="vi">@count</span> <span class="o">=</span> <span class="n">count</span>
<span class="vi">@caption</span> <span class="o">=</span> <span class="s2">" </span><span class="si">#{</span><span class="n">caption</span><span class="si">}</span><span class="s2"> "</span>
<span class="vi">@running</span> <span class="o">=</span> <span class="n">running</span><span class="p">.</span><span class="nf">cycle</span>
<span class="vi">@position</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">tick</span>
<span class="vi">@now</span> <span class="o">=</span> <span class="no">Time</span><span class="p">.</span><span class="nf">now</span> <span class="k">if</span> <span class="vi">@position</span><span class="p">.</span><span class="nf">zero?</span>
<span class="vi">@position</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="n">closing</span>
<span class="n">opening</span>
<span class="n">meter</span>
<span class="k">end</span>
<span class="c1">##########################################################</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">test</span> <span class="n">caption</span> <span class="o">=</span> <span class="s1">'Hello'</span><span class="p">,</span> <span class="n">count</span> <span class="o">=</span> <span class="mi">1024</span>
<span class="no">ProgressBar</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">caption</span><span class="p">,</span> <span class="n">count</span><span class="p">).</span><span class="nf">tap</span> <span class="k">do</span> <span class="o">|</span><span class="n">pb</span><span class="o">|</span>
<span class="n">pb</span><span class="p">.</span><span class="nf">count</span><span class="p">.</span><span class="nf">times</span> <span class="k">do</span>
<span class="nb">sleep</span> <span class="mf">0.01</span>
<span class="n">pb</span><span class="p">.</span><span class="nf">tick</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="c1">##########################################################</span>
<span class="kp">private</span>
<span class="k">def</span> <span class="nf">opening</span><span class="p">(</span><span class="n">sym</span> <span class="o">=</span> <span class="s1">'|'</span><span class="p">)</span>
<span class="nb">print</span> <span class="s2">"</span><span class="se">\e</span><span class="s2">[1G</span><span class="si">#{</span><span class="vi">@caption</span><span class="si">}#{</span><span class="n">sym</span><span class="si">}</span><span class="s2">"</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">closing</span><span class="p">(</span><span class="n">sym</span> <span class="o">=</span> <span class="s1">'|'</span><span class="p">)</span>
<span class="nb">print</span> <span class="s2">"</span><span class="se">\e</span><span class="s2">[</span><span class="si">#{</span><span class="vg">$stdin</span><span class="p">.</span><span class="nf">winsize</span><span class="p">.</span><span class="nf">first</span><span class="si">}</span><span class="s2">G</span><span class="si">#{</span><span class="n">sym</span><span class="si">}</span><span class="s2">"</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">meter</span><span class="p">(</span><span class="n">sym</span> <span class="o">=</span> <span class="s1">'='</span><span class="p">)</span>
<span class="n">space</span> <span class="o">=</span> <span class="vg">$stdin</span><span class="p">.</span><span class="nf">winsize</span><span class="p">.</span><span class="nf">first</span> <span class="o">-</span> <span class="vi">@caption</span><span class="p">.</span><span class="nf">length</span> <span class="o">-</span> <span class="mi">2</span>
<span class="nb">print</span> <span class="n">sym</span> <span class="o">*</span> <span class="p">(</span><span class="vi">@position</span><span class="p">.</span><span class="nf">to_f</span> <span class="o">*</span> <span class="n">space</span> <span class="o">/</span> <span class="vi">@count</span><span class="p">)</span>
<span class="k">if</span> <span class="vi">@position</span> <span class="o">==</span> <span class="vi">@count</span>
<span class="nb">puts</span>
<span class="c1"># rubocop:disable Style/FormatString</span>
<span class="nb">puts</span> <span class="s2">"It took %s min %s sec"</span> <span class="o">%</span> <span class="p">(</span><span class="no">Time</span><span class="p">.</span><span class="nf">now</span> <span class="o">-</span> <span class="vi">@now</span><span class="p">).</span><span class="nf">ceil</span><span class="p">.</span><span class="nf">divmod</span><span class="p">(</span><span class="mi">60</span><span class="p">)</span>
<span class="c1"># rubocop:enable Style/FormatString</span>
<span class="k">else</span>
<span class="nb">print</span> <span class="vi">@running</span><span class="p">.</span><span class="nf">next</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>This code is provided as is, take it, play with it, but don’t write me back
saying it could’ve been done better. It could have. But I needed to have it up
and running in half of an hour: now it’s running and I am writing this post,
keeping one of my eyes on fancy progress meter.</p>
Use Hooks in Riak to Create Views2016-09-09T00:00:00+00:00https://rocket-science.ru/hacking/2016/09/09/riak-hooks-for-quick-views<blockquote>
<p><a href="http://docs.basho.com/riak/kv/2.1.4/downloads/">Riak KV</a> is a distributed
NoSQL database designed to deliver maximum data availability by distributing
data across multiple servers. As long as your Riak KV client can reach one
Riak server, it should be able to write data.</p>
</blockquote>
<p>Riak is written in Erlang, that’s why. Also, Riak supports
<a href="http://docs.basho.com/riak/kv/2.1.4/developing/usage/commit-hooks/">hooks, written in Erlang</a>.</p>
<p>I love hooks. When it comes to a voodoo magic, I am a big fun of
<a href="https://en.wikipedia.org/wiki/Domino_effect">domino effect</a>. One simply
inserts a value into the database, and complicated business logic processes
do their job transparently. All these schema transformations, history, logging,
versioning, emailing, coffeemaking, tequiladrinking… Sorry, was distracted.</p>
<p>So, in one of our projects we have a stream of changing data; think of
a temperature value in hundred of different cities. Updates come every second,
or even more often. I need to keep a history, but also I need a clean pure
view with last up-to-date values. Like:</p>
<table>
<thead>
<tr>
<th>City</th>
<th>Temperature</th>
</tr>
</thead>
<tbody>
<tr>
<td>Barcelona </td>
<td>23°</td>
</tr>
<tr>
<td>London </td>
<td>22°</td>
</tr>
<tr>
<td>Moscow </td>
<td>-6°</td>
</tr>
<tr>
<td>New York </td>
<td>0°</td>
</tr>
</tbody>
</table>
<hr />
<p>So far so good. Riak handles inserts into “raw” data bucket perfectly,
but how would I keep my view up-to-date? Easy.</p>
<p>Let’s introduce a hook on Riak’s insert into “raw” data bucket.
Hooks are to be written in Erlang, so we’ll produce a simple module:</p>
<div class="language-erlang highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">-</span><span class="ni">module</span><span class="p">(</span><span class="n">backend_hooks</span><span class="p">).</span>
<span class="p">-</span><span class="ni">export</span><span class="p">([</span><span class="n">update_current</span><span class="o">/</span><span class="mi">1</span><span class="p">]).</span>
<span class="c">%% When the `raw` value is changed, we are to update the `current`
%% bucket, that collects all up-to-date values.
</span><span class="nf">update_current</span><span class="p">(</span><span class="nv">RiakObject</span><span class="p">)</span> <span class="o">-></span>
<span class="p">{_,</span> <span class="nv">Key</span><span class="p">}</span> <span class="o">=</span> <span class="nn">riak_object</span><span class="p">:</span><span class="nf">bucket</span><span class="p">(</span><span class="nv">RiakObject</span><span class="p">),</span>
<span class="nv">Bucket</span> <span class="o">=</span> <span class="p">{</span><span class="o"><<</span><span class="s">"current"</span><span class="o">>></span><span class="p">,</span> <span class="o"><<</span><span class="s">"spot"</span><span class="o">>></span><span class="p">},</span>
<span class="nv">MetaData</span> <span class="o">=</span> <span class="nn">riak_object</span><span class="p">:</span><span class="nf">get_metadata</span><span class="p">(</span><span class="nv">RiakObject</span><span class="p">),</span>
<span class="k">case</span> <span class="nn">dict</span><span class="p">:</span><span class="nf">find</span><span class="p">(</span><span class="o"><<</span><span class="s">"X-Riak-Deleted"</span><span class="o">>></span><span class="p">,</span> <span class="nv">MetaData</span><span class="p">)</span> <span class="k">of</span>
<span class="p">{</span><span class="n">ok</span><span class="p">,</span> <span class="p">_}</span> <span class="o">-></span>
<span class="c">% do nothing, this is a deletion
</span> <span class="nn">io</span><span class="p">:</span><span class="nf">fwrite</span><span class="p">(</span><span class="s">"!! DELETED AN OBJECT FROM RAW: </span><span class="si">~w~n</span><span class="s">"</span><span class="p">,</span> <span class="p">[</span><span class="nv">RiakObject</span><span class="p">]);</span>
<span class="p">_</span> <span class="o">-></span>
<span class="p">{</span><span class="n">struct</span><span class="p">,</span> <span class="nv">Json</span><span class="p">}</span> <span class="o">=</span> <span class="nn">mochijson2</span><span class="p">:</span><span class="nf">decode</span><span class="p">(</span><span class="nn">riak_object</span><span class="p">:</span><span class="nf">get_value</span><span class="p">(</span><span class="nv">RiakObject</span><span class="p">)),</span>
<span class="p">{</span><span class="o"><<</span><span class="s">"value"</span><span class="o">>></span><span class="p">,</span> <span class="nv">Value</span><span class="p">}</span> <span class="o">=</span> <span class="nn">lists</span><span class="p">:</span><span class="nf">keyfind</span><span class="p">(</span><span class="o"><<</span><span class="s">"value"</span><span class="o">>></span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="nv">Json</span><span class="p">)</span>
<span class="p">{</span><span class="n">ok</span><span class="p">,</span> <span class="nv">C</span><span class="p">}</span> <span class="o">=</span> <span class="nn">riak</span><span class="p">:</span><span class="nf">local_client</span><span class="p">(),</span>
<span class="k">case</span> <span class="nv">C</span><span class="p">:</span><span class="nb">get</span><span class="p">(</span><span class="nv">Bucket</span><span class="p">,</span> <span class="nv">Key</span><span class="p">)</span> <span class="k">of</span>
<span class="p">{</span><span class="n">ok</span><span class="p">,</span> <span class="nv">Old</span><span class="p">}</span> <span class="o">-></span>
<span class="k">case</span> <span class="nn">riak_object</span><span class="p">:</span><span class="nf">get_value</span><span class="p">(</span><span class="nv">Old</span><span class="p">)</span> <span class="k">of</span>
<span class="nv">Value</span> <span class="o">-></span> <span class="c">% Value has not changed, skipping update
</span> <span class="c">% io:fwrite("LEFT INSTACT: ~w~n", [{Key, Value}]);
</span> <span class="n">ok</span><span class="p">;</span>
<span class="p">_</span> <span class="o">-></span>
<span class="c">% io:fwrite("UPDATED: ~w~n", [{Key, Value}]),
</span> <span class="nv">C</span><span class="p">:</span><span class="nb">put</span><span class="p">(</span><span class="nn">riak_object</span><span class="p">:</span><span class="nf">update_value</span><span class="p">(</span><span class="nv">Old</span><span class="p">,</span> <span class="nv">Value</span><span class="p">))</span>
<span class="k">end</span><span class="p">;</span>
<span class="p">{</span><span class="n">error</span><span class="p">,</span> <span class="n">notfound</span><span class="p">}</span> <span class="o">-></span>
<span class="c">% io:fwrite("CREATED: ~w~n", [{Key, Value}]),
</span> <span class="nv">C</span><span class="p">:</span><span class="nb">put</span><span class="p">(</span><span class="nn">riak_object</span><span class="p">:</span><span class="nf">new</span><span class="p">(</span><span class="nv">Bucket</span><span class="p">,</span> <span class="nv">Key</span><span class="p">,</span> <span class="nv">Value</span><span class="p">))</span>
<span class="k">end</span>
<span class="k">end</span><span class="p">,</span>
<span class="nv">RiakObject</span><span class="p">.</span>
</code></pre></div></div>
<p>Cool. Now we have to install this hook. To do so, one should do three things.
Riak hooks might be attached directly to specific buckets, but I prefer to keep things
clear and introduce the <code class="language-plaintext highlighter-rouge">bucket-type</code> for it:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>riak-admin bucket-type create raw <span class="se">\</span>
<span class="s1">'{"props":{"precommit":[{"mod":"backend_hooks","fun":"update_current"}]}}'</span>
<span class="nb">sudo </span>riak-admin bucket-type activate raw
<span class="nb">sudo </span>riak-admin bucket-type create current
<span class="nb">sudo </span>riak-admin bucket-type activate current
</code></pre></div></div>
<p>To install a hook, one should compile the Erlang module and copy the resulting
<code class="language-plaintext highlighter-rouge">beam</code> into the directory, Riak is aknowledged of. First of all, let’s tell
Riak about our hook directory:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">cat</span> /etc/riak/advanced.config
<span class="o">[</span>
<span class="o">{</span>riak_kv, <span class="o">[</span>
<span class="o">{</span>add_paths, <span class="o">[</span><span class="s2">"/usr/lib/riak/hooks"</span><span class="o">]}</span>
<span class="o">]}</span>
<span class="o">]</span><span class="nb">.</span>
</code></pre></div></div>
<p>On the fresh installation, this file is empty/non-existent. Put the content
above into it. This will add <code class="language-plaintext highlighter-rouge">"/usr/lib/riak/hooks"</code> to the Riak paths.</p>
<p>Now, copy the <code class="language-plaintext highlighter-rouge">beam</code> into this directory:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span><span class="nb">sudo mkdir</span> <span class="nt">-p</span> /usr/lib/riak/hooks
<span class="nv">$ </span>erlc backend_hooks.erl <span class="o">&&</span> <span class="se">\</span>
<span class="nb">sudo cp </span>backend_hooks.beam /usr/lib/riak/hooks
</code></pre></div></div>
<p>Restart Riak and we are all set. From now on every insert into <code class="language-plaintext highlighter-rouge">raw</code> bucket,
will result in Riak to gracefully update the “nested” up-to-date
<code class="language-plaintext highlighter-rouge">{current, spot}</code> bucket.</p>
<p>And all that was done with a dozen of lines of code. Cute, isn’t it?</p>
Howto Read Stack Overflow Comments2016-08-11T00:00:00+00:00https://rocket-science.ru/culture/2016/08/11/how-to-read-so-comments<table>
<thead>
<tr>
<th>Comment</th>
<th>Meaning</th>
</tr>
</thead>
<tbody>
<tr>
<td>this makes no sense</td>
<td>I did not understand this</td>
</tr>
<tr>
<td>this is not idiomatic ruby</td>
<td>this violates my sense of beauty</td>
</tr>
<tr>
<td>in idiomatic ruby that would be written as…</td>
<td>I would code it this way</td>
</tr>
<tr>
<td> </td>
<td> </td>
</tr>
<tr>
<td> </td>
<td> </td>
</tr>
<tr>
<td> </td>
<td> </td>
</tr>
<tr>
<td> </td>
<td> </td>
</tr>
<tr>
<td> </td>
<td> </td>
</tr>
<tr>
<td> </td>
<td> </td>
</tr>
<tr>
<td> </td>
<td> </td>
</tr>
<tr>
<td> </td>
<td> </td>
</tr>
</tbody>
</table>
Thou shalt not make unto thee any graven image2016-07-30T00:00:00+00:00https://rocket-science.ru/culture/2016/07/30/thou-shalt-not-make-unto-thee-any-graven-image<p>When I was a kid, we had too few influential opinions: “Dad says,”
“Britannica states” and “I’ve tried, it hurts.”</p>
<p>Seriously, teachers were not just throwing their private opinions on things onto us;
Newton laws were not shared through 140-symbols message one Saturday’s evening.
There were lemmas, theorems and they all were proven. The only thing that was
given to us as a must, was an axiomatics. And you know what? The amount of
axioms, needed to derive and prove <em>everything else</em> in Euclid’s geometry, is five.</p>
<p>Nowadays we have many more authorities. Many more axiomatics. Many more
postulates in each.</p>
<p>Since currently I mostly work with Ruby, I will provide the examples from the Ruby
world, but I believe they are applicable to almost any other area of choice.
Including fashion, copywriting, food, younameit.</p>
<p>There is a tendency to listen to what people we respect use to say. Which is good.
In the modern era, any random phrase, occasionally dropped in any bloated
discussion immediately becomes public. Which is bad.</p>
<p>This happens because arguing is being done through the web, in shared access mode.
When an arbitrary Einstein had accidentally had said gibberish (which I believe
happened to him quite often,) it was not spread immediately over the millions
of followers.</p>
<p>When an arbitrary DHH has had behaved the same way, it is. Less famous people
take any word from there as a word of Holy Truth, carved in stone. There is
the only problem with this approach: people can not constantly say wise things.
Those who do rare <em>do</em> wise things and they never become opinion makers.</p>
<p>Turning back to Ruby, now there is a trend to talk about “monkeypatching is an evil,”
“the code must be as clear as possible,” “never use <code class="language-plaintext highlighter-rouge">eval</code>, it hurts” etc. These
things are being repeated as a mantra, as a spell to cast to call the Gods of
the great code.</p>
<p>The above is a bullshit.</p>
<p>Whether the developer is so dumb, that it needs to be guarded by artificial
restrictions, she should go switch to go (excuse me :) or use any other technique
to improve the skills. Cutting ruby’s natural beauty by suppressing the greatest
features like monkeypatching, re-opening classes and family is kinda
<em>security by obscurity</em>. It is hazardous, because developers become even more
stupid than they are currently. The only way to improve the code quality is
to choose the language that one feels comfortable with and then use all the features
it provides. That’s how things work. Nobody buys a Ferrary F-50 to drive at most
50 MpH just because <em>it’s dangerous</em>. Those <em>bewarers</em> buy Fiat 500, and that
is the right choice.</p>
<p>But wait, one meticulous reader should ask me here. John A., Jack B. and even
Joanne R. said we should not use <code class="language-plaintext highlighter-rouge">eval</code>. Well, <code class="language-plaintext highlighter-rouge">GOTO 1</code>, please.</p>
Monkeypatch It!2016-06-27T00:00:00+00:00https://rocket-science.ru/hacking/2016/06/27/monkeypatch-it<p>More than one month ago, <a href="http://solnic.eu/about.html">Piotr Solnica</a> wrote
a manifesto <a href="http://solnic.eu/2016/05/22/my-time-with-rails-is-up.html">My time with Rails is up</a>.</p>
<p>I felt myself agitated. I waited—while the foam settles—on purpose: I wanted to express
my opinion, unbiased and calm. I kinda agreed completely, but as time was passing,
I was getting an impression that I was zombied. After all I love all that mess,
I love open classes and monkeypatching for it’s incomparable ability to bring
elegance to where it never has belonged.</p>
<p>Once one month elapsed, I am ready to articulate my opinion: in Ruby,
<strong>monkeypatching is one of the best things all around and should be used extensively</strong>.
Yes, I said that. I do and I will use monkeypatching (refinements where it’s appropriate,
and monkeypatching elsewhere.) An axe is a great tool, but when one wants to
install the screw, she’d better use a hammer. Or a screwdriver, whether she
is graduated in cabinetry.</p>
<p>Ruby is all about <em>elegance</em>, <em>intuitively clear solutions</em>, <em>fast ans pleasant
code creation</em>. Nobody sane would use Ruby as is there in low-memory devices,
real-time systems, high-load message processing etc. Once twitter succeeded,
guys left Ruby. Not because Ruby is bad, not at all. Because Ruby is just
not about that. Failure of an axe to install screws [in robust way] does
not make an axe the bad tool in general.</p>
<p>Cooking a dinner for myself, I fry steaks <em>and</em> seethe potatoes. I have salted
cucumbers, pickled pepper, sauces and drinks: all that <em>frontend</em>, you know. But
even for <em>backend</em> I have the steaks and the potatoes.</p>
<p>I have beach shoes and office shoes. I have desktop and notebook. I use Ubuntu
at home and Gentoo at work. I believe you got the point.</p>
<p>What LOC is more readable, elegant and <em>easier in general</em>:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">elapsed</span> <span class="o">=</span> <span class="mi">2</span><span class="p">.</span><span class="nf">business_days</span><span class="p">(</span><span class="s1">'EUR'</span><span class="p">,</span> <span class="s1">'USD'</span><span class="p">).</span><span class="nf">after</span><span class="p">(</span><span class="no">Time</span><span class="p">.</span><span class="nf">zone</span><span class="p">.</span><span class="nf">now</span><span class="p">)</span>
<span class="c1"># or</span>
<span class="n">elapsed</span> <span class="o">=</span> <span class="no">BusinessDaysFactory</span><span class="p">.</span><span class="nf">for_currencies</span><span class="p">(</span><span class="s1">'EUR'</span><span class="p">,</span> <span class="s1">'USD'</span><span class="p">)</span>
<span class="p">.</span><span class="nf">add</span><span class="p">(</span><span class="no">Time</span><span class="p">.</span><span class="nf">zone</span><span class="p">.</span><span class="nf">now</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span>
</code></pre></div></div>
<hr />
<p>Ruby yielded it’s popularity because of Rails. I <em>am not</em> a fan of Rails. But
I am mature enough to face facts as they are: unless Rails existed, we’ve got
two popular scripting languages only: <code class="language-plaintext highlighter-rouge">php</code> and <code class="language-plaintext highlighter-rouge">python</code>.</p>
<p>Rails yielded it’s popularity because of monkeypatching. It is proven to work,
to be robust, to be maintainable and to continue producing success stories for
startups.</p>
<p>So, unless you are an academician, writing your seventh PhD thesis on functional
patterns in Haskell, please, get your hands off monkeypatching. It is by no means
bad pattern. Even more: <em>it is a great pattern</em>, while used where it is
applicable to.</p>
<hr />
<p>I’ve had a colleague, who was geeky on pattern matching. Pattern matching is a
great feature, that is not a part of Ruby. That simple. He tried to bring
pattern matching to Ruby, he even implemented sorta working prototype… The only
problem is that Ruby has an aversion to it. Ruby existing syntax just rejected it.
Ruby is a perfect language, but whether you need pattern matching, please,
consider to switch to Erlang/Elixir/±20 other languages that have it natively
supported. Ruby just has not.</p>
<p>On the other hand, as you have chosen Ruby, please, stop tinkering anything else
out of it. Use monkeypatching. Use open classes. Use criptic variables. Create your
own <code class="language-plaintext highlighter-rouge">.rubocop.yml</code> file, allowing non-ascii method names, increased cyclomatic
complexity and multiline block chains.</p>
<p>Ruby does not need a plastic surgery, it is beautiful enough as it was conceived.</p>
StackOverflow Achievements2016-04-22T00:00:00+00:00https://rocket-science.ru/hacking/2016/04/22/stack-overflow-achievements<p>This post should have been named “Open Letter to <code class="language-plaintext highlighter-rouge">@tadman</code>”. I decided to write it
when I got a “don’t do it” comment under my very correct answer on StackOverflow.</p>
<p><a href="http://stackoverflow.com/a/36775404/2035262">Here is a link</a>, if anybody is curious.</p>
<p>For those, who does not like surfing, here goes the <strong>tl;dr</strong>:</p>
<blockquote>
<p><strong>Q.</strong> I need to construct a 3D matrix out of existing data.</p>
</blockquote>
<blockquote>
<p><strong>A.</strong> One might use the <a href="http://ruby-doc.org/core-2.3.0/Hash.html#method-i-default_proc"><code class="language-plaintext highlighter-rouge">Hash#default_proc</code></a>
in the following manner:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">result</span> <span class="o">=</span> <span class="no">Hash</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span> <span class="o">|</span><span class="n">h1</span><span class="p">,</span> <span class="n">k1</span><span class="o">|</span>
<span class="p">(</span><span class="mi">0</span><span class="o">...</span><span class="n">col1</span><span class="p">.</span><span class="nf">size</span><span class="p">)</span> <span class="o">===</span> <span class="n">k1</span> <span class="p">?</span> <span class="n">h1</span><span class="p">[</span><span class="n">k1</span><span class="p">]</span> <span class="o">=</span> <span class="no">Hash</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span> <span class="o">|</span><span class="n">h2</span><span class="p">,</span> <span class="n">k2</span><span class="o">|</span>
<span class="p">(</span><span class="mi">0</span><span class="o">...</span><span class="n">col2</span><span class="p">.</span><span class="nf">size</span><span class="p">)</span> <span class="o">===</span> <span class="n">k2</span> <span class="p">?</span> <span class="n">h2</span><span class="p">[</span><span class="n">k2</span><span class="p">]</span> <span class="o">=</span> <span class="no">Hash</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span> <span class="o">|</span><span class="n">h3</span><span class="p">,</span> <span class="n">k3</span><span class="o">|</span> <span class="k">do</span>
<span class="n">o3</span> <span class="o">=</span> <span class="n">col3</span><span class="p">.</span><span class="nf">detect</span> <span class="p">{</span> <span class="o">|</span><span class="n">o</span><span class="o">|</span> <span class="n">o</span><span class="p">.</span><span class="nf">id</span> <span class="o">==</span> <span class="n">k3</span> <span class="p">}</span>
<span class="n">o3</span> <span class="p">?</span> <span class="n">h3</span><span class="p">[</span><span class="n">k3</span><span class="p">]</span> <span class="o">=</span> <span class="no">Obj</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">col1</span><span class="p">[</span><span class="n">k1</span><span class="p">].</span><span class="nf">att</span><span class="p">,</span> <span class="n">col2</span><span class="p">[</span><span class="n">k2</span><span class="p">].</span><span class="nf">att</span><span class="p">,</span> <span class="n">o3</span><span class="p">.</span><span class="nf">att</span><span class="p">)</span> <span class="p">:</span> <span class="kp">nil</span>
<span class="p">}</span> <span class="p">:</span> <span class="kp">nil</span>
<span class="p">}</span> <span class="p">:</span> <span class="kp">nil</span>
<span class="p">}</span>
</code></pre></div> </div>
<p><strong>C.</strong> This is one crazy intense nugget of code.</p>
</blockquote>
<p>The comment above was given by the person, I know as being experienced,
highly professional, talented ruby developer. I put the disclaimer into my answer,
saying “please do not do it at home.” That’s it.</p>
<p>From this very moment I feel myself frustrated. I am convinced that:</p>
<ul>
<li>this code is not complex by no means;</li>
<li>it is very maintainable, since it delegates everything to ruby internal
implementation of <code class="language-plaintext highlighter-rouge">default_proc</code>, which I trust;</li>
<li>it has an added value.</li>
</ul>
<p>BTW, I would not ever have decided to write a post on the topic, just because
somebody thinks my code is ugly. I was mostly disappointed by indirect claim
that we should stick to giving answers in the “ruby for dummies” style.</p>
<p>I volunteer to StackOverflow for nearly five years. I deserve (and desire) some
payback. This is not about achievements, reputation, badges and +10s. This is
about my willingness to understand that I help people to learn. I am there not
because I want to write the code for others for free. I don’t. That simple.</p>
<p>I feel pity that nowadays the development process is not about R&D anymore.
It’s mostly about S&A (aka Search’N’Apply.) I don’t think I bring any goodness
to the universe by providing another piece of code, that might be copy-pasted
into some application without any thought.</p>
<p>Rails framework is so great that it is awful. It brings an ability to stop
thinking at all, just constantly apply patterns found in the internet. An
average Rails programmer requires neither research nor development. It is
sufficiently enough to <code class="language-plaintext highlighter-rouge">google || ask_on_so && copy_paste</code>.</p>
<p>And—let me repeat that—I am there on StackOverflow not to multiply the amount
of dumb snippets, that are to be blindmindly applied.</p>
<p>While I have an ability to demonstrate the technique, pattern, younameit—I am to
keep doing that. In hope, that maybe experienced and talented programmers won’t
consider the trivial codepieces like the one above as being “rather complicated
thing to throw at someone.”</p>
New Hash Syntax for the Rescue2016-03-09T00:00:00+00:00https://rocket-science.ru/hacking/2016/03/09/new-hash-syntax-for-the-rescue<p>Ruby 1.9 introduced new hash syntax: instead of cool hashrockets there came something barely
looking as native Ruby. It was more like a javascript injection (or should I’ve called it infection?)</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="c1"># { :foo => 'bar' } # legacy syntax, do not use</span>
<span class="p">{</span> <span class="ss">foo: </span><span class="s1">'bar'</span> <span class="p">}</span> <span class="c1"># cool new syntax, inspired by, well, json</span>
</code></pre></div></div>
<p>I am not a slave of my habits and customs. When I join new development team, I study
the style guide book (or, frankly, it is usually style guide page, or even style guide link.)
Whether my colleagues are to put the opening curly bracket on the same line as function
declaration, I would do it (though it is obviously unreadable, misleading and insane.)</p>
<p>So, years ago I switched from fancy hashrockets to silly wonky colons. After all,
it saves a keystroke per hash key. Only yesterday I realized, why using colons is safer
and leads to the better, less error-prone code in general.</p>
<p>Everybody who ever used Rails, should be aware of <code class="language-plaintext highlighter-rouge">with_indifferent_access</code> helper, that
Rails brings to hash instances. It allows us to not bother whether the keys are strings,
or atoms (symbols.)</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nb">hash</span> <span class="o">=</span> <span class="p">{</span> <span class="s1">'foo'</span> <span class="o">=></span> <span class="mi">42</span><span class="p">,</span> <span class="ss">:bar</span> <span class="o">=></span> <span class="s1">'baz'</span> <span class="p">}</span>
<span class="nb">hash</span><span class="p">[</span><span class="s1">'foo'</span><span class="p">]</span> <span class="c1">#⇒ 42</span>
<span class="nb">hash</span><span class="p">[</span><span class="ss">:foo</span><span class="p">]</span> <span class="c1">#⇒ nil</span>
<span class="nb">hash</span><span class="p">.</span><span class="nf">with_indifferent_access</span><span class="p">[</span><span class="ss">:foo</span><span class="p">]</span> <span class="c1">#⇒ 42</span>
</code></pre></div></div>
<p>That is cool, but once we’ve forgotten to call this magic 23-symbols-in-name helper method, we
lose. The performance penalties on using this methods are also obvious.</p>
<p>With the new syntax, though, one would never make such a silly mistake. We are now forced
to use symbols as keys, unless the intent of using string / object is clearly stated. No more
pain in the ass grepping for “did I make this key a string, or a symbol?”</p>
<p>And you know what? Ruby 2.3.0 makes this even more standard, allowing strings as hash keys
in the new syntax. They will just be converted to symbols:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="err">▶</span> <span class="nb">hash</span> <span class="o">=</span> <span class="p">{</span> <span class="ss">a: </span><span class="mi">5</span><span class="p">,</span> <span class="s1">'b'</span><span class="p">:</span> <span class="mi">6</span><span class="p">,</span> <span class="ss">:'c'</span> <span class="o">=></span> <span class="mi">7</span> <span class="p">}</span>
<span class="c1">#⇒ { :a => 5, :b => 6, :c => 7 }</span>
</code></pre></div></div>
<p>Farewell, my dear Rails helper friend, <code class="language-plaintext highlighter-rouge">with_indifferent_access</code>. You are out of business.</p>
Testing migrations2015-11-19T00:00:00+00:00https://rocket-science.ru/hacking/2015/11/19/testing-migrations<p>I am not aware of any good, quick and robust way of testing migrations in ruby applications. Well,
whether the migrations just creates a new table it’s usually safe to rely on those guys in
Rails team who implemented <code class="language-plaintext highlighter-rouge">create_table</code>.</p>
<p>But in real life we often have a legacy data to be transposed into new structure in some
cumbersome way. Imagine the address field to be splitted into three brand new fields:
a zip, a street name and the number of a house. OK, we write a migration, that
introduces new three fields in the respective table, converts data using regular
expressions (hey, we have 100K records, if you know the better, please drop me an email,)
drops the original field.</p>
<p>So far, so good. The migration looks like a brilliant piece of code, it runs smoothly
on fixtures etc. Well done? Or is it still medium rare?</p>
<p>Of course, we need to test the migration. Nobody wants to send 100K of commercials next
month to irrelevant addresses, right? How to do it?</p>
<p>There is quick and dirty approach. If one knows “potentially problematic” addresses,
she might <strong>create a simple fixture with that data and run the migration <code class="language-plaintext highlighter-rouge">down</code> and
then back <code class="language-plaintext highlighter-rouge">up</code></strong>. That simple.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">require_relative</span> <span class="no">File</span><span class="p">.</span><span class="nf">join</span> <span class="no">Rails</span><span class="p">.</span><span class="nf">root</span><span class="p">,</span>
<span class="s1">'db'</span><span class="p">,</span>
<span class="s1">'migrate'</span><span class="p">,</span>
<span class="s1">'20151119020000_convert_addresses_to_new_format.rb'</span>
<span class="n">context</span> <span class="s1">'#up'</span> <span class="k">do</span>
<span class="n">before</span> <span class="k">do</span>
<span class="no">ConvertAddressesToNewFormat</span><span class="p">.</span><span class="nf">new</span><span class="p">.</span><span class="nf">down</span>
<span class="n">apply_legacy_data_fixtures</span>
<span class="n">load_new_data_fixtures</span>
<span class="k">end</span>
<span class="n">it</span> <span class="s1">'should convert data properly'</span> <span class="k">do</span>
<span class="no">ConvertAddressesToNewFormat</span><span class="p">.</span><span class="nf">new</span><span class="p">.</span><span class="nf">up</span>
<span class="no">Address</span><span class="p">.</span><span class="nf">all</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">a</span><span class="o">|</span>
<span class="n">expect</span><span class="p">(</span><span class="n">a</span><span class="p">.</span><span class="nf">zip</span><span class="p">).</span><span class="nf">to</span> <span class="n">be_present</span>
<span class="o">...</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The same might be easily done to test <code class="language-plaintext highlighter-rouge">#down</code> migration.</p>
Recursion Without Explicit Method2015-10-29T00:00:00+00:00https://rocket-science.ru/hacking/2015/10/29/recursion-without-explicit-method<p>Recursion is being associated with evaluating factorial. There are well-known common approaches to
implement is in ruby:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="n">num</span><span class="p">).</span><span class="nf">reduce</span><span class="p">(</span><span class="o">&</span><span class="p">:</span><span class="o">*</span><span class="p">)</span> <span class="o">||</span> <span class="mi">1</span>
</code></pre></div></div>
<p>There are more sophisticated methods, like one presented <a href="https://bugs.ruby-lang.org/issues/9528">here</a>
(AFIUK, the patch was not accepted.) OK, ruby is friendly in building recursive functions, unless
one tries to accomplish something a bit less trivial than factorial calculation. In other words,
the complexity comes with unknown iterations count.</p>
<p>Let’s imagine we want to build the breadcrumbs for one deeply nested page on a website. For unknown
reason, we have a page structure stored as hash:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">page</span> <span class="o">=</span> <span class="p">{</span>
<span class="ss">title: </span><span class="s1">'Contacts'</span><span class="p">,</span>
<span class="ss">parent: </span><span class="p">{</span>
<span class="ss">title: </span><span class="s1">'Everything'</span><span class="p">,</span>
<span class="ss">parent: </span><span class="p">{</span>
<span class="ss">title: </span><span class="s1">'Additional'</span><span class="p">,</span>
<span class="ss">parent: </span><span class="p">{</span>
<span class="ss">title: </span><span class="s1">'Landing'</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Common approach would be to declare a method, receiving hash and recursively call it subsequently,
until the nested hash is absent. I could imagine something like that:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">navigate</span> <span class="nb">hash</span><span class="p">,</span> <span class="n">memo</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">navigate</span><span class="p">(</span><span class="nb">hash</span><span class="p">[</span><span class="ss">:parent</span><span class="p">],</span> <span class="n">memo</span><span class="p">)</span> <span class="k">if</span> <span class="nb">hash</span><span class="p">[</span><span class="ss">:parent</span><span class="p">]</span>
<span class="n">memo</span> <span class="o"><<</span> <span class="nb">hash</span><span class="p">[</span><span class="ss">:title</span><span class="p">]</span>
<span class="k">end</span>
<span class="n">navigate</span> <span class="n">page</span>
<span class="c1">#⇒ [ 'Landing', 'Additional', 'Everything', 'Contacts' ]</span>
</code></pre></div></div>
<p>But wait, we already have memo! Can we get rid of redundant method? Sure.</p>
<p>Infinity is cool. Ruby has an infinite method <a href="http://ruby-doc.org/core-2.2.0/Kernel.html#method-i-loop"><code class="language-plaintext highlighter-rouge">Kernel#loop</code></a>.
Let’s take advantage of it, since we can not predict how many iterations we’ll need after all:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kp">loop</span><span class="p">.</span><span class="nf">inject</span><span class="p">({</span><span class="ss">result: </span><span class="p">[],</span> <span class="ss">hash: </span><span class="n">page</span><span class="p">})</span> <span class="k">do</span> <span class="o">|</span><span class="n">memo</span><span class="o">|</span>
<span class="n">memo</span><span class="p">[</span><span class="ss">:result</span><span class="p">]</span> <span class="o"><<</span> <span class="n">memo</span><span class="p">[</span><span class="ss">:hash</span><span class="p">][</span><span class="ss">:title</span><span class="p">]</span>
<span class="k">break</span> <span class="n">memo</span><span class="p">[</span><span class="ss">:result</span><span class="p">]</span> <span class="k">unless</span> <span class="n">memo</span><span class="p">[</span><span class="ss">:hash</span><span class="p">]</span> <span class="o">=</span> <span class="n">memo</span><span class="p">[</span><span class="ss">:hash</span><span class="p">][</span><span class="ss">:parent</span><span class="p">]</span>
<span class="n">memo</span>
<span class="k">end</span><span class="p">.</span><span class="nf">reverse</span>
<span class="c1">#⇒ [ 'Landing', 'Additional', 'Everything', 'Contacts' ]</span>
</code></pre></div></div>
<p>One might claim, that the former variant with a method is cleaner, and I must admit: yes, it is.
I just wanted to demonstrate the technique and prove, that infinite <code class="language-plaintext highlighter-rouge">loop</code>s are useful. Sometimes.
And yes, I know, this is not a canonical recursion. Anyway, I like it.</p>
<p>Turning back to factorial evaluation:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kp">loop</span><span class="p">.</span><span class="nf">reduce</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">6</span><span class="p">])</span> <span class="k">do</span> <span class="o">|</span><span class="n">memo</span><span class="o">|</span>
<span class="n">memo</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">*=</span> <span class="n">memo</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="n">memo</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">-=</span> <span class="mi">1</span>
<span class="k">break</span> <span class="n">memo</span><span class="p">.</span><span class="nf">first</span> <span class="k">if</span> <span class="n">memo</span><span class="p">.</span><span class="nf">last</span><span class="p">.</span><span class="nf">zero?</span>
<span class="n">memo</span>
<span class="k">end</span>
<span class="c1">#⇒ 720</span>
</code></pre></div></div>
<p>Not the best example to advertise the power of infinite loops, huh?</p>
Two ways to write ruby code2015-10-14T00:00:00+00:00https://rocket-science.ru/hacking/2015/10/14/two-ways-to-write-ruby-code<blockquote>
<p>vi editor has two modes: one to beep whenever you hit a key and one to delete all the work you’ve done</p>
</blockquote>
<p>There are mainly two modes to write rails (and, well, any ruby) code.</p>
<p>In Mode I, everything has clean, thought-out interfaces, each method performs
a well-defined, human understandable task (nowadays it’s fashionable to call
those <em>mutations</em>) and …inside the methods, between those <code class="language-plaintext highlighter-rouge">def</code> and <code class="language-plaintext highlighter-rouge">end</code>
there is a hell load of gibberish.</p>
<p>In Mode II, every damn line of code is clean and understandable. Besides that,
nothing might be comprehended. Neither interfaces in general, nor methods
themselves seem to be of any good.</p>
<p>Nah, I know, there are gurus (somewhere in the land of pink fairies and unicorns,)
who produce clean readable rails code in real projects 24×7. Unfortunately, I live
in the parallel galaxy. With deadlines “yesterday,” with hotfixes, with business
rules changing on the fly and all that shit, always accompanying the bleeding
edge of commercial development. If I were refactoring every shitty piece of
our code, I’d be doing it till now, unpaid and probably unemployed. Business
sometimes is growing faster than the development department. And it, after all, dictates.</p>
<p>Turning back to our modes, I would reveal a couple of examples. I do not want to
have little force. I’m going to bring everything out clearly.</p>
<h3 id="mode-i-bond-james-bond">Mode I. Bond. James Bond.</h3>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">SPECIAL_CASING_FIELDS</span><span class="p">.</span><span class="nf">each</span> <span class="p">{</span> <span class="o">|</span><span class="nb">method</span><span class="o">|</span>
<span class="n">define_method</span><span class="p">(</span><span class="s2">"filter_</span><span class="si">#{</span><span class="nb">method</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span> <span class="p">{</span> <span class="o">|</span><span class="n">cp</span><span class="p">,</span> <span class="n">filters</span> <span class="o">=</span> <span class="p">[]</span><span class="o">|</span>
<span class="nb">hash</span><span class="p">[</span><span class="n">ncp</span> <span class="o">=</span> <span class="n">__to_code_point</span><span class="p">(</span><span class="n">cp</span><span class="p">)].</span><span class="nf">nil?</span> <span class="p">?</span> <span class="p">\</span>
<span class="kp">nil</span> <span class="p">:</span> <span class="p">[</span><span class="o">*</span><span class="nb">hash</span><span class="p">[</span><span class="n">ncp</span><span class="p">]].</span><span class="nf">select</span> <span class="p">{</span> <span class="o">|</span><span class="n">h</span><span class="o">|</span>
<span class="n">filters</span><span class="p">.</span><span class="nf">inject</span><span class="p">(</span><span class="kp">true</span><span class="p">)</span> <span class="p">{</span> <span class="o">|</span><span class="n">memo</span><span class="p">,</span> <span class="n">f</span><span class="o">|</span>
<span class="n">memo</span> <span class="o">&&=</span> <span class="n">h</span><span class="p">[</span><span class="nb">method</span><span class="p">.</span><span class="nf">to_sym</span><span class="p">].</span><span class="nf">match</span> <span class="n">f</span>
<span class="p">}</span>
<span class="p">}</span> <span class="o">||</span>
<span class="p">[</span><span class="o">*</span><span class="nb">hash</span><span class="p">[</span><span class="n">ncp</span><span class="p">]].</span><span class="nf">select</span> <span class="p">{</span> <span class="o">|</span><span class="n">h</span><span class="o">|</span> <span class="n">h</span><span class="p">[</span><span class="nb">method</span><span class="p">.</span><span class="nf">to_sym</span><span class="p">].</span><span class="nf">vacant?</span> <span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This is a piece from my <a href="https://github.com/mudasobwa/forkforge">Unicode lightweight library</a>.
It produces six helper methods for filtering and selecting different unicode
entities from the consortium description file (like <em>code_point</em>,
<em>lowercase_mapping</em>, <em>condition_list</em> etc.) It is absolutely unreadable, but
once thoroughly tested it is known to do it’s job perfectly. I do not envy anybody
who will be obliged to read this code, though.</p>
<h3 id="mode-ii-mary-poppins">Mode II. Mary Poppins.</h3>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">check_validity_for_supplier</span> <span class="n">supplier</span><span class="p">,</span> <span class="ss">additional: </span><span class="kp">false</span>
<span class="k">if</span> <span class="o">!</span><span class="n">supplier</span> <span class="k">return</span> <span class="kp">false</span>
<span class="k">if</span> <span class="o">!</span><span class="n">supplier</span><span class="p">.</span><span class="nf">valid?</span> <span class="k">return</span> <span class="kp">false</span>
<span class="k">if</span> <span class="n">additional</span> <span class="o">&&</span> <span class="o">!</span><span class="n">supplier</span><span class="p">.</span><span class="nf">additional</span> <span class="k">return</span> <span class="kp">false</span>
<span class="kp">true</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Everything here is clear. But in terms of the whole picture this method is
a fiction. Even being declared as <code class="language-plaintext highlighter-rouge">private</code>, it populates the screen area,
padding out the code clarity. Yeah, I know, they say “methods should not be
longer than 10 lines” and “use helpers.” Sometimes it is a good advise.
Sometimes it is not.</p>
<p>There is nothing wrong with the method, that has 50 lines of code, if it was
thoroughly tested and corked up. Pass integer to it, and it will return it’s square
(plus all these input checks gives exactly 50 lines.) Who ever cares what this method has inside?
Ugly code? Well, maybe. Take a look at spline interpolation. After all,
we use all these <code class="language-plaintext highlighter-rouge">inject</code> and even <code class="language-plaintext highlighter-rouge">permutation</code> without any fear that they
contain unreadable code inside.</p>
<p>Using the advise to split everything into tiny methods, it’s equally easy to
make the code either more readable, or the best tasted spaghetti ever. Method
declarations are <em>goto</em> s, at least for future readers. This must be admitted.</p>
<p>I do not propose to write ugly unreadable code, of course. But the idealists are
usually mistaken. In the real life, sometimes, with a dozen of reserves, blah-blah,
you know, it’s better to write one well-tested, complicated helper and lock it
in the chest, than to engender twenty one-liner noodles. After all, your code
does not have pesto to dress these methods with.</p>
Quotation Marks in XXI Century2015-10-13T00:00:00+00:00https://rocket-science.ru/hacking/2015/10/13/quotation-marks-in-XXI-century<p>There is a common problem we inherited from <code class="language-plaintext highlighter-rouge">VT-100</code> terminal times: there is
insufficient amount of buttons on a keyboard. While
<a href="https://en.wikipedia.org/wiki/Latin_alphabet">Latin alphabet</a> contains 23 letters,
modern variation has 26 letters (<code class="language-plaintext highlighter-rouge">J</code>, <code class="language-plaintext highlighter-rouge">U</code> and <code class="language-plaintext highlighter-rouge">W</code> instilled,) and some languages
using it have added “diacriticsized” letters (like umlauts <code class="language-plaintext highlighter-rouge">Ö</code> in German and Swedish,
“island” <code class="language-plaintext highlighter-rouge">Ø</code> and “ångström” <code class="language-plaintext highlighter-rouge">Å</code> in Danish and Norwegian, etc.)</p>
<p>On Cyrillic keyboards (33 letters in the alphabet,) we have made a sacrifice of
square brackets and backtick, to admit additional letters into the house. I am
afraid to imagine how Archaic Egyptian keyboard should look like.</p>
<p>Due to the lack of place on our laptops and cellulars, we get accustomed to use
neutral straight dumbs instead of singular, double quotes, and even instead of
an apostrophe. Especially in programming languages.</p>
<p>Whoever cares about a typography rules in error messages, formatted as</p>
<blockquote>
<p>err: there is no such entity in container (#42)</p>
</blockquote>
<hr />
<p>Well, first of all since one has a localized application, she probably does not
want to scare these
<a href="http://arstechnica.com/information-technology/2013/07/linus-torvalds-defends-his-right-to-shame-linux-kernel-developers/">touchy Finns</a>
with wrong typography. And, you know, different languages have different
<a href="https://en.wikipedia.org/wiki/Quotation_mark">typography rules</a>.</p>
<p>In fact, using proper typography quotes (well, hang the Finns, even simple
English quotation marks,) one yields a ton of profit. Typographically correct
quotation marks are:</p>
<ul>
<li>idempotent, left and right differ;</li>
<li>easy to grep in the huge codebase;</li>
<li>do not need any escaping anywhere in the code.</li>
</ul>
<p><strong>The latter is the silver bullet.</strong> Matz even invented <code class="language-plaintext highlighter-rouge">%Q{}</code> literal to
simplify the dealing with strings, containing both single and double quotation
marks, but what this string is converted to json? passed to remote service?
encoded and decoded?</p>
<p>Bah.</p>
<p>There is just no problem with strings, containing typographically correct
quotation marks. Not. At. All.</p>
<p>Pass them everywhere (I doubt about COBOL, but any other language in 2015 would
gracefully understand the unicode,) convert it back and forth, escape them,
unescape them, you’ll always have the correct result.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err"> </span><span class="n">str</span> <span class="o">=</span> <span class="s1">'And God said “We’ll call it ‘typography’,” and there was light'</span>
</code></pre></div></div>
<p>Actually, I came to this conclusion after a couple of hours of hard debug
session, where quoted string was passed into javascript from ruby with a
superfluous escaping, ruining json content, but still providing the well-formed
json.</p>
Pry :: breakpoint or ARGF?2015-09-02T00:00:00+00:00https://rocket-science.ru/hacking/2015/09/02/pry-and-stdin<p><a href="https://github.com/pry/pry"><code class="language-plaintext highlighter-rouge">Pry</code></a> is fantastic. If you are still tumbling in IRB,
go get Pry a chance and you’ll never roll back.</p>
<p>Besides all it’s buns and cookies, pry provides an ability to debug your ruby code.
Just put <code class="language-plaintext highlighter-rouge">binding.pry</code> whereever in your code and voilà, you are done. Run it and
see an execution stopped on this line of code:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err"> </span><span class="mi">5</span><span class="p">:</span> <span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="mi">42</span><span class="p">).</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">i</span><span class="o">|</span>
<span class="err"> </span><span class="mi">6</span><span class="p">:</span> <span class="nb">puts</span> <span class="s2">"Iteration #</span><span class="si">#{</span><span class="n">i</span><span class="si">}</span><span class="s2">."</span>
<span class="err"> </span><span class="mi">7</span><span class="p">:</span> <span class="nb">puts</span> <span class="s2">"This code does nothing useful."</span>
<span class="err"> </span><span class="o">=></span><span class="err"> </span><span class="mi">8</span><span class="p">:</span> <span class="nb">binding</span><span class="p">.</span><span class="nf">pry</span>
<span class="err"> </span><span class="mi">9</span><span class="p">:</span> <span class="k">end</span>
</code></pre></div></div>
<p>Everything works like a charm until one tries to make a script accepting
standard input. Unix way, you know.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="n">ps</span> <span class="o">-</span><span class="no">Ao</span> <span class="n">pid</span><span class="p">,</span><span class="n">command</span> <span class="o">|</span> <span class="n">grep</span> <span class="n">proc_of_interest</span> <span class="o">|</span> <span class="n">awk</span> <span class="s1">'{ print $1 }'</span> <span class="o">|</span> <span class="n">my_script</span>
</code></pre></div></div>
<p>In <code class="language-plaintext highlighter-rouge">my_script</code> one would utilize <code class="language-plaintext highlighter-rouge">ARGF</code> power:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">input</span> <span class="o">=</span> <span class="no">ARGF</span><span class="p">.</span><span class="nf">read</span>
<span class="c1"># binding.pry</span>
<span class="n">do_stuff</span> <span class="n">input</span>
</code></pre></div></div>
<p>Have you noticed <code class="language-plaintext highlighter-rouge">binding.pry</code>? Well, it was not commented out, since I met
a problem with my data and wanted to examine them in debugger session.</p>
<p>Unfortunately, <code class="language-plaintext highlighter-rouge">pry</code> just <em>won’t stop here</em>. It will silently ignore this
breakpoint. WTF?</p>
<p>Well, there is no bug in pry and no magic around. As soon as you get input piped
to your script, ruby internals <code class="language-plaintext highlighter-rouge">$stdin</code> and <code class="language-plaintext highlighter-rouge">STDIN</code> are not <code class="language-plaintext highlighter-rouge">tty</code> anymore. That
said, <code class="language-plaintext highlighter-rouge">$stdin</code> is assigned to the standard input we just passed to it through
pipe. And yes, pry is clever enough to check for <code class="language-plaintext highlighter-rouge">$stdin.tty?</code> and switch to
non-interactive mode, since no interaction is possible. It’s silly to wait for
user input when there is neither user no input available, right? So pry does.</p>
<p>How could one overcome this? Pretty easy. Let’s cheat <code class="language-plaintext highlighter-rouge">pry</code>:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">pry_stdin</span> <span class="o">=</span> <span class="no">IO</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="no">IO</span><span class="p">.</span><span class="nf">sysopen</span><span class="p">(</span><span class="s1">'/dev/tty'</span><span class="p">),</span> <span class="s1">'r'</span><span class="p">)</span>
<span class="c1"># load pry and cheat it with our stdio</span>
<span class="nb">require</span> <span class="s1">'pry'</span>
<span class="no">Pry</span><span class="p">.</span><span class="nf">config</span><span class="p">.</span><span class="nf">input</span> <span class="o">=</span> <span class="n">pry_stdin</span>
<span class="c1"># ...</span>
<span class="nb">binding</span><span class="p">.</span><span class="nf">pry</span>
</code></pre></div></div>
<p>We just opened a <code class="language-plaintext highlighter-rouge">tty</code> device for reading from and passed it as input stream to
<code class="language-plaintext highlighter-rouge">pry</code>. All internal <code class="language-plaintext highlighter-rouge">pry</code> checks are now happy and we yield good old breakpoint.</p>
CASE-WHEN :: another trick for N+1 problem2015-08-29T00:00:00+00:00https://rocket-science.ru/hacking/2015/08/29/active-record-case-when<p>Everybody working with Rails project, slightly more complicated, than
To-Do list, is aware of <a href="http://www.sitepoint.com/silver-bullet-n1-problem/">N+1 Query Problem</a>.
It is awful, it might drastically decrease the performance of an application.
I saw pages, performing over 4000 queries against database on a single load.</p>
<p>There are great diagnostics gems, like aforementioned <a href="https://github.com/flyerhzm/bullet"><code class="language-plaintext highlighter-rouge">bullet</code></a>
and/or <a href="https://github.com/nesquena/query_reviewer"><code class="language-plaintext highlighter-rouge">query_reviewer</code></a> There are plenty of
hints, tips and tricks on how to overcome it (<strong>TL;DR</strong>: use <code class="language-plaintext highlighter-rouge">includes</code> eager loading.)</p>
<p>Unfortunately, there is no clean solution on an opposite problem: multiple updates.
Imagine you have to update a table, setting a column value basing on the value in another column.</p>
<p>MySQL (and most other dialects) provides a single query for it:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">UPDATE</span> <span class="nv">`profiles`</span> <span class="k">SET</span> <span class="nv">`yay`</span> <span class="o">=</span> <span class="k">CASE</span> <span class="nv">`workflow_state`</span>
<span class="k">WHEN</span> <span class="s1">'approved'</span> <span class="k">THEN</span> <span class="s1">'yes'</span>
<span class="k">WHEN</span> <span class="s1">'cancelled'</span> <span class="k">THEN</span> <span class="s1">'no'</span>
<span class="k">ELSE</span> <span class="nv">`yay`</span>
<span class="k">END</span>
</code></pre></div></div>
<p>Till now Rails had no nifty wrapper for it. Now it has:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nn">ActiveRecord</span>
<span class="k">class</span> <span class="nc">Base</span> <span class="c1"># :nodoc:</span>
<span class="c1"># Updates multiple rows in table using prepared hash as input</span>
<span class="c1">#</span>
<span class="c1"># @param by_field [String] name of field to be used as `id`</span>
<span class="c1"># @param prepared_updates [Hash] batched update, hash consisting of</span>
<span class="c1"># field_names ⇒ hash of maps {id ⇒ value}</span>
<span class="c1">#</span>
<span class="c1"># The query to be prepared and executed:</span>
<span class="c1">#</span>
<span class="c1"># UPDATE `profiles` SET `yay` = CASE `workflow_state`</span>
<span class="c1"># WHEN 'approved' THEN 'yes'</span>
<span class="c1"># WHEN 'cancelled' THEN 'no'</span>
<span class="c1"># ELSE `yay`</span>
<span class="c1"># END</span>
<span class="c1">#</span>
<span class="c1"># Code for that:</span>
<span class="c1">#</span>
<span class="c1"># update_multiple 'id', { yay: { approved: :yes, cancelled: 'no' } }</span>
<span class="c1">#</span>
<span class="c1"># Real life example:</span>
<span class="c1">#</span>
<span class="c1"># Profile.update_multiple :id, {</span>
<span class="c1"># address: { 9 => 'Avda Success', 287 => 'Avda Failure' },</span>
<span class="c1"># province: { 9 => 'Siberia', 287 => 'Siberia II' } }</span>
<span class="c1">#</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">update_multiple</span> <span class="n">by_field</span><span class="p">,</span> <span class="n">prepared_updates</span>
<span class="n">query</span> <span class="o">=</span> <span class="p">[</span>
<span class="s2">"UPDATE `</span><span class="si">#{</span><span class="nb">self</span><span class="p">.</span><span class="nf">table_name</span><span class="si">}</span><span class="s2">`"</span><span class="p">,</span>
<span class="s2">"SET"</span><span class="p">,</span>
<span class="n">prepared_updates</span><span class="p">.</span><span class="nf">map</span> <span class="k">do</span> <span class="o">|</span><span class="n">field</span><span class="p">,</span> <span class="n">prepared_update</span><span class="o">|</span>
<span class="s2">"`</span><span class="si">#{</span><span class="n">field</span><span class="si">}</span><span class="s2">` = </span><span class="si">#{</span><span class="n">__case_block_of_multiple_rows_update_query</span> <span class="n">by_field</span><span class="p">,</span> <span class="n">field</span><span class="p">,</span> <span class="n">prepared_update</span><span class="si">}</span><span class="s2">"</span>
<span class="k">end</span><span class="p">.</span><span class="nf">join</span><span class="p">(</span><span class="s2">",</span><span class="si">#{</span><span class="vg">$/</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
<span class="p">].</span><span class="nf">join</span> <span class="vg">$/</span>
<span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span><span class="p">.</span><span class="nf">connection</span><span class="p">.</span><span class="nf">execute</span> <span class="n">query</span>
<span class="k">end</span>
<span class="kp">private</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">__case_block_of_multiple_rows_update_query</span> <span class="n">by_field</span><span class="p">,</span> <span class="n">field</span><span class="p">,</span> <span class="n">prepared_update</span>
<span class="p">[</span>
<span class="s2">"CASE `</span><span class="si">#{</span><span class="n">by_field</span><span class="si">}</span><span class="s2">`"</span><span class="p">,</span>
<span class="n">prepared_update</span><span class="p">.</span><span class="nf">map</span> <span class="k">do</span> <span class="o">|</span><span class="nb">id</span><span class="p">,</span> <span class="n">val</span><span class="o">|</span>
<span class="n">val</span> <span class="o">=</span> <span class="s2">"'</span><span class="si">#{</span><span class="n">val</span><span class="si">}</span><span class="s2">'"</span> <span class="k">unless</span> <span class="n">val</span><span class="p">.</span><span class="nf">is_a?</span><span class="p">(</span><span class="no">Numeric</span><span class="p">)</span> <span class="c1"># Date??</span>
<span class="s2">"</span><span class="se">\t</span><span class="s2">WHEN '</span><span class="si">#{</span><span class="nb">id</span><span class="si">}</span><span class="s2">' THEN </span><span class="si">#{</span><span class="n">val</span><span class="si">}</span><span class="s2">"</span>
<span class="k">end</span><span class="p">,</span>
<span class="s2">"</span><span class="se">\t</span><span class="s2">ELSE `</span><span class="si">#{</span><span class="n">field</span><span class="si">}</span><span class="s2">`"</span><span class="p">,</span>
<span class="s1">'END'</span>
<span class="p">].</span><span class="nf">join</span> <span class="vg">$/</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The above code will update a table with single query, according to the hash given. Enjoy!</p>
YAML Parser Tuning2015-04-14T00:00:00+00:00https://rocket-science.ru/hacking/2015/04/14/yaml-parser-tuning<p>YAML files are good. They are [likely] human readable, the syntax in more or
less minimalistic and they are nearly a standard for configuration files in
ruby world. One day, though, everybody faces up a necessity to read YAML in
some unusual, often weird manner.</p>
<p>Yesterday I was participating in answering a
<a href="http://stackoverflow.com/questions/29462856/loading-yaml-with-line-number-for-each-key/29595013">question on StackOverflow</a>.
The YAML file was to be parsed as usual, but with a tiny improvement: instead
of leaves values there should be placed hashes like:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span> <span class="ss">value: </span><span class="n">value</span><span class="p">,</span> <span class="ss">line: </span><span class="n">line</span> <span class="p">}</span>
</code></pre></div></div>
<p>where line is a line in original YAML file this leaf was met. The technique
below actually is not stuck with this particular case; it demonstrates the
common approach on how to parse YAML in a non-standard way.</p>
<p>The default parser in Ruby is <code class="language-plaintext highlighter-rouge">Psych</code>. It is a good old AST builder. To improve
(read: change) it’s behaviour, one needs to bring three things on the table.</p>
<h4 id="node">Node</h4>
<p>Patching the node is pretty straightforward. We would store a line, so here we go:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Psych::Nodes::Node</span>
<span class="nb">attr_accessor</span> <span class="ss">:line</span>
<span class="k">end</span>
</code></pre></div></div>
<h4 id="treebuilder">TreeBuilder</h4>
<p><code class="language-plaintext highlighter-rouge">TreeBuilder</code> uses visitor pattern to build a syntax tree. In general, it has
the only method of interest, <code class="language-plaintext highlighter-rouge">TreeBuilder#scalar</code>, which is invoked on every
node. Lets’s deal with it a bit.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">EnchancedBuilder</span> <span class="o"><</span> <span class="no">Psych</span><span class="o">::</span><span class="no">TreeBuilder</span>
<span class="c1"># Line numbers are available to parser, not to builder; we need a backreference</span>
<span class="nb">attr_accessor</span> <span class="ss">:parser</span>
<span class="c1"># Main handler in TreeBuilder</span>
<span class="c1"># @param value [String] the value met</span>
<span class="c1"># @style [Integer] the type of entity met (scalar/int/array/etc)</span>
<span class="k">def</span> <span class="nf">scalar</span> <span class="n">value</span><span class="p">,</span> <span class="n">anchor</span><span class="p">,</span> <span class="n">tag</span><span class="p">,</span> <span class="n">plain</span><span class="p">,</span> <span class="n">quoted</span><span class="p">,</span> <span class="n">style</span>
<span class="n">s</span> <span class="o">=</span> <span class="k">super</span>
<span class="c1"># using the mark from a previous hit to handle multilined values</span>
<span class="n">s</span><span class="p">.</span><span class="nf">line</span> <span class="o">=</span> <span class="vi">@line</span> <span class="o">||</span> <span class="mi">1</span>
<span class="vi">@line</span> <span class="o">=</span> <span class="n">parser</span><span class="p">.</span><span class="nf">mark</span><span class="p">.</span><span class="nf">line</span> <span class="o">+</span> <span class="mi">1</span> <span class="c1"># marks are zero-based</span>
<span class="n">s</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Here we set the prepared <code class="language-plaintext highlighter-rouge">Node.line</code> attribute and store the current value
of line of current entity.</p>
<h4 id="toruby">ToRuby</h4>
<p>The only thing left is to spit the newly introduced <code class="language-plaintext highlighter-rouge">line</code> attribute to
generated ruby properly.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Psych::Visitors::ToRuby</span>
<span class="c1"># There may be problems with Yaml mappings that have tags.</span>
<span class="c1"># @author @matt</span>
<span class="k">def</span> <span class="nf">revive_hash</span> <span class="nb">hash</span><span class="p">,</span> <span class="n">o</span>
<span class="n">o</span><span class="p">.</span><span class="nf">children</span><span class="p">.</span><span class="nf">each_slice</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span> <span class="p">{</span> <span class="o">|</span><span class="n">k</span><span class="p">,</span><span class="n">v</span><span class="o">|</span>
<span class="n">key</span> <span class="o">=</span> <span class="n">accept</span><span class="p">(</span><span class="n">k</span><span class="p">)</span>
<span class="n">val</span> <span class="o">=</span> <span class="n">accept</span><span class="p">(</span><span class="n">v</span><span class="p">)</span>
<span class="c1"># This is the important bit. If the value is a scalar,</span>
<span class="c1"># we replace it with the desired hash.</span>
<span class="k">if</span> <span class="n">v</span><span class="p">.</span><span class="nf">is_a?</span> <span class="o">::</span><span class="no">Psych</span><span class="o">::</span><span class="no">Nodes</span><span class="o">::</span><span class="no">Scalar</span>
<span class="n">val</span> <span class="o">=</span> <span class="p">{</span> <span class="s2">"value"</span> <span class="o">=></span> <span class="n">val</span><span class="p">,</span> <span class="s2">"line"</span> <span class="o">=></span> <span class="n">v</span><span class="p">.</span><span class="nf">line</span> <span class="p">}</span>
<span class="k">end</span>
<span class="c1"># Code dealing with << (for merging hashes) omitted.</span>
<span class="c1"># If you need this you will probably need to copy it</span>
<span class="c1"># in here. See the method:</span>
<span class="c1"># https://github.com/tenderlove/psych/blob/v2.0.13/lib/psych/visitors/to_ruby.rb#L333-L365</span>
<span class="nb">hash</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="o">=</span> <span class="n">val</span>
<span class="p">}</span>
<span class="nb">hash</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>That’s it. Now we are able to produce hashes as shown below from YAML.</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">key1</span><span class="pi">:</span> <span class="s">value1</span>
<span class="na">key2</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">value21</span>
<span class="pi">-</span> <span class="s">value22</span>
</code></pre></div></div>
<p>would become</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="s">hash = {</span>
<span class="s">'key1' => { 'value' => 'value1', 'line' => 1 },</span>
<span class="s">'key2' => [</span>
<span class="s">'value' => 'value21', 'line' => 3,</span>
<span class="s">'value' => 'value22', 'line' => </span><span class="m">4</span>
<span class="err">]</span>
<span class="err">}</span>
</code></pre></div></div>
Log With Pleasure2015-04-11T00:00:00+00:00https://rocket-science.ru/hacking/2015/04/11/log-with-pleasure<p>One half of my working time I spend staring like a stuck pig at my logs. (In case
anybody’s wondering: another half I gaze on pry’s debugger stack; in pauses I
keep my eyes fixed on code itself.) Logs are great. You may run an application,
go drink a couple ristrettos, get back to your table and reflectively read what
that huge clumsy sluggish piece of code, having words ‘innovative’ and
‘revolutionary’ in it‘s name had spitted out to console.</p>
<p>The main problem is that log is like dormitory. Unfortunately, my colleagues
get used to utilize the same working routine; rails and other plugged in engines
seem to feel themselves lonely without communication as well. The log looks like
a trash can. Grokking logs is a hard task, requiring from the brave log explorer
such skills like “Mouse Wheel Scrolling Master,” “Eye Scanner Level 80” and
“Winner of World Gotcha Championship.”</p>
<p>I want to share a couple of tricks helping me to utilize the log in cosy manner.</p>
<h4 id="intercepting-standard-rails-log">Intercepting standard Rails log</h4>
<p>First of all, to tune the log we have to gain an access to it. Let’s do it.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="vi">@logdevice</span> <span class="o">=</span> <span class="nb">const_defined?</span> <span class="s1">'Rails'</span> <span class="p">?</span>
<span class="o">::</span><span class="no">Rails</span><span class="p">.</span><span class="nf">logger</span>
<span class="p">.</span><span class="nf">instance_variable_get</span><span class="p">(:</span><span class="vi">@logger</span><span class="p">)</span>
<span class="p">.</span><span class="nf">instance_variable_get</span><span class="p">(</span><span class="ss">:@log</span><span class="p">)</span> <span class="p">:</span>
<span class="no">Logger</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="vg">$stdout</span><span class="p">)</span>
<span class="vi">@tty</span> <span class="o">=</span> <span class="vi">@logdevice</span><span class="p">.</span><span class="nf">instance_variable_get</span><span class="p">(</span><span class="ss">:@logdev</span><span class="p">)</span>
<span class="p">.</span><span class="nf">instance_variable_get</span><span class="p">(</span><span class="ss">:@dev</span><span class="p">).</span><span class="nf">tty?</span> <span class="o">||</span>
<span class="nb">const_defined?</span><span class="p">(</span><span class="s1">'Rails'</span><span class="p">)</span> <span class="o">&&</span> <span class="o">::</span><span class="no">Rails</span><span class="p">.</span><span class="nf">env</span><span class="p">.</span><span class="nf">development?</span>
</code></pre></div></div>
<p>The latter condition in <code class="language-plaintext highlighter-rouge">@tty</code> check is required to force <code class="language-plaintext highlighter-rouge">@tty</code> be set to <code class="language-plaintext highlighter-rouge">true</code>
in <em>Rails</em> development environment, where logs are going through <code class="language-plaintext highlighter-rouge">development.log</code>
file, yet most purpose is to review them on the fly in the console.</p>
<p>Now we should not forget about our colleagues, who probably would not admit
the accidental log format change as long-awaited Christmas gift.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="vi">@keep_out</span> <span class="o">=</span> <span class="no">Kernel</span><span class="p">.</span><span class="nf">const_defined?</span><span class="p">(</span><span class="s1">'Rails'</span><span class="p">)</span> <span class="o">&&</span> <span class="o">::</span><span class="no">Rails</span><span class="p">.</span><span class="nf">env</span><span class="p">.</span><span class="nf">production?</span> <span class="o">||</span>
<span class="no">ENV</span><span class="p">[</span><span class="s1">'RAILS_PRETTY_LOG'</span><span class="p">]</span> <span class="o">!=</span> <span class="s1">'42'</span>
</code></pre></div></div>
<p>OK, my log modifications would never affect neither production env, nor alien
environments not having <code class="language-plaintext highlighter-rouge">RAILS_PRETTY_LOG</code> environment variable set to <code class="language-plaintext highlighter-rouge">42</code>.
Fine. Let’s hack a log a bit. First of all, let’s add colors.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">SEV_COLORS</span> <span class="o">=</span> <span class="p">{</span>
<span class="s1">'INFO'</span> <span class="o">=></span> <span class="p">[</span><span class="s1">'01;38;05;21'</span><span class="p">,</span> <span class="s1">'00;38;05;152'</span><span class="p">],</span>
<span class="s1">'WARN'</span> <span class="o">=></span> <span class="p">[</span><span class="s1">'01;38;05;226'</span><span class="p">,</span> <span class="s1">'00;38;05;222'</span><span class="p">],</span>
<span class="s1">'ERROR'</span> <span class="o">=></span> <span class="p">[</span><span class="s1">'01;38;05;196'</span><span class="p">,</span> <span class="s1">'01;38;05;174'</span><span class="p">],</span>
<span class="s1">'DEBUG'</span> <span class="o">=></span> <span class="p">[</span><span class="s1">'01;38;05;242'</span><span class="p">,</span> <span class="s1">'00;38;05;246'</span><span class="p">],</span>
<span class="s1">'ANY'</span> <span class="o">=></span> <span class="p">[</span><span class="s1">'01;38;05;222;48;05;238'</span><span class="p">,</span> <span class="s1">'01;38;05;253;48;05;238'</span><span class="p">]</span>
<span class="p">}</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">clrz</span> <span class="n">txt</span><span class="p">,</span> <span class="n">clr</span>
<span class="k">return</span> <span class="n">txt</span> <span class="k">unless</span> <span class="vi">@tty</span>
<span class="s2">"</span><span class="se">\e</span><span class="s2">[</span><span class="si">#{</span><span class="n">clr</span><span class="si">}</span><span class="s2">m</span><span class="si">#{</span><span class="n">txt</span><span class="p">.</span><span class="nf">gsub</span><span class="p">(</span><span class="sr">/«(.*?)»/</span><span class="p">,</span> <span class="s2">"</span><span class="se">\e</span><span class="s2">[01;38;05;51m</span><span class="se">\\</span><span class="s2">1</span><span class="se">\e</span><span class="s2">[</span><span class="si">#{</span><span class="n">clr</span><span class="si">}</span><span class="s2">m"</span><span class="p">)</span><span class="si">}</span><span class="se">\e</span><span class="s2">[0m"</span>
<span class="k">end</span>
</code></pre></div></div>
<p>We will colorize different types of messages (red errors, blue infos, darkgrayed
debugs, all that stuff.) Whether the message contains text in guillemets, it
will be automatically highlighted.</p>
<p>Fine. Now let’s introduce the stopwords.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="vi">@stopwords</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">def</span> <span class="nf">logger_stopwords</span> <span class="n">file</span>
<span class="vi">@stopwords</span> <span class="o">+=</span> <span class="no">File</span><span class="p">.</span><span class="nf">read</span><span class="p">(</span><span class="n">file</span><span class="p">).</span><span class="nf">split</span><span class="p">(</span><span class="vg">$/</span><span class="p">).</span><span class="nf">map</span> <span class="p">{</span> <span class="o">|</span><span class="n">l</span><span class="o">|</span> <span class="no">Regexp</span><span class="p">.</span><span class="nf">new</span> <span class="n">l</span> <span class="p">}</span> <span class="k">rescue</span> <span class="kp">nil</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Once the text file, having list of regexps one by line, is loaded with
aforementioned <code class="language-plaintext highlighter-rouge">logger_stopwords</code> function, matching messages would be removed
from log. This is quite useful for filtering <code class="language-plaintext highlighter-rouge">ActiveRecords</code>’s debug littered
with infinite SQL statements.</p>
<p>It sounds like everything is ready. Let’s implement our own
<a href="http://ruby-doc.org/stdlib-2.1.5/libdoc/logger/rdoc/Logger/Formatter.html">formatter</a>.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">SEV_SYMBOLS</span> <span class="o">=</span> <span class="p">{</span>
<span class="s1">'INFO'</span> <span class="o">=></span> <span class="s1">'✔'</span><span class="p">,</span>
<span class="s1">'WARN'</span> <span class="o">=></span> <span class="s1">'✗'</span><span class="p">,</span>
<span class="s1">'ERROR'</span> <span class="o">=></span> <span class="s1">'✘'</span><span class="p">,</span>
<span class="s1">'DEBUG'</span> <span class="o">=></span> <span class="s1">'✓'</span><span class="p">,</span>
<span class="s1">'ANY'</span> <span class="o">=></span> <span class="s1">'▷'</span>
<span class="p">}</span>
<span class="k">unless</span> <span class="vi">@keep_out</span>
<span class="vi">@logdevice</span><span class="p">.</span><span class="nf">formatter</span> <span class="o">=</span> <span class="nb">proc</span> <span class="p">{</span> <span class="o">|</span><span class="n">severity</span><span class="p">,</span> <span class="n">datetime</span><span class="p">,</span> <span class="n">progname</span><span class="p">,</span> <span class="n">message</span><span class="o">|</span>
<span class="n">message</span><span class="p">.</span><span class="nf">strip!</span> <span class="c1"># strip</span>
<span class="n">message</span><span class="p">.</span><span class="nf">gsub!</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">,</span> <span class="s2">"</span><span class="se">\n</span><span class="si">#{</span><span class="s1">' '</span> <span class="o">*</span> <span class="mi">31</span><span class="si">}</span><span class="s2">"</span> <span class="c1"># align new lines pretty</span>
<span class="k">if</span> <span class="n">message</span><span class="p">.</span><span class="nf">empty?</span> <span class="o">||</span> <span class="vi">@stopwords</span><span class="p">.</span><span class="nf">any?</span> <span class="p">{</span> <span class="o">|</span><span class="n">sw</span><span class="o">|</span> <span class="n">sw</span> <span class="o">=~</span> <span class="n">message</span> <span class="p">}</span>
<span class="kp">nil</span> <span class="c1"># skip stopwords</span>
<span class="k">else</span>
<span class="s1">''</span> <span class="o"><<</span> <span class="n">clrz</span><span class="p">(</span><span class="n">clrz</span><span class="p">(</span><span class="s2">"</span><span class="si">#{</span><span class="no">SEV_SYMBOLS</span><span class="p">[</span><span class="n">severity</span><span class="p">]</span><span class="si">}</span><span class="s2"> "</span><span class="p">,</span> <span class="no">SEV_COLORS</span><span class="p">[</span><span class="n">severity</span><span class="p">].</span><span class="nf">first</span><span class="p">)</span> <span class="p">\</span>
<span class="o"><<</span> <span class="n">clrz</span><span class="p">(</span><span class="n">severity</span><span class="p">[</span><span class="mi">0</span><span class="o">..</span><span class="mi">2</span><span class="p">],</span> <span class="no">SEV_COLORS</span><span class="p">[</span><span class="n">severity</span><span class="p">].</span><span class="nf">first</span><span class="p">)</span> <span class="p">\</span>
<span class="o"><<</span> <span class="s1">' | '</span> <span class="p">\</span>
<span class="o"><<</span> <span class="n">clrz</span><span class="p">(</span><span class="n">datetime</span><span class="p">.</span><span class="nf">strftime</span><span class="p">(</span><span class="s1">'%Y%m%d-%H%M%S.%3N'</span><span class="p">),</span> <span class="s1">'01;38;05;238'</span><span class="p">)</span> <span class="p">\</span>
<span class="o"><<</span> <span class="s1">' | '</span> <span class="p">\</span>
<span class="o"><<</span> <span class="n">clrz</span><span class="p">(</span><span class="n">message</span><span class="p">,</span> <span class="no">SEV_COLORS</span><span class="p">[</span><span class="n">severity</span><span class="p">].</span><span class="nf">last</span><span class="p">)</span> <span class="p">\</span>
<span class="o"><<</span> <span class="s2">"</span><span class="se">\n</span><span class="s2">"</span>
<span class="k">end</span>
<span class="p">}</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The above will filter everything matching stopwords, format multiline strings
in more human-readable manner and colorize output when printing on console.
Nifty symbols will be prepended to every message, making it even easier to
navigate eyes through bundles of lines on the screen.</p>
<p>The only thing left to mention is <code class="language-plaintext highlighter-rouge">ANY</code> severity: it will be spitted out despite
current log level. This is useful, e.g., to print out some values quite
temporarily, not switching the context.</p>
<p>Well, we are to define wrappers:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="sx">%i(warn info error debug)</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">m</span><span class="o">|</span>
<span class="nb">class_eval</span> <span class="s2">"
def </span><span class="si">#{</span><span class="n">m</span><span class="si">}</span><span class="s2"> message
logger.</span><span class="si">#{</span><span class="n">m</span><span class="si">}</span><span class="s2">(message)
end
module_function :</span><span class="si">#{</span><span class="n">m</span><span class="si">}</span><span class="s2">
"</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Let’s run it and go take another ristretto.</p>
<p><img src="/img/log-tricks.png" alt="Log output" /></p>
Don’t be a language slave2015-03-03T00:00:00+00:00https://rocket-science.ru/hacking/2015/03/03/dont-be-a-language-slave<p>Last week I had an intriguing discussion with some Python guru (he got a nickname “<a href="https://en.wikipedia.org/wiki/Python_reticulatus">reticulatus</a>” a long ago.) I complained to him that the last year I was oblidged to code in PHP and now I am anxious about my IQ being decreased significally. That language virtually forces one to write an enormously nasty code. On the other hand, I finally feigned that PHP code might be written in functional manner as well. My friend archly grinned and suggested me to run benchmarks on functional-ity.</p>
<p>Well, I did.</p>
<p>The result is embarassing. I can’t even surmise how much scorn is to be felt to produce the interpreter that acts in the following way. Let’s take a look at the benchmarks. NB I wrote <a href="https://github.com/mudasobwa/screwdrivers/blob/master/src/Mudasobwa/Screwdrivers/YardStick.php">the tiny wrapper</a> for calls to <code class="language-plaintext highlighter-rouge">microtime</code>. It does nothing but calculates times and memory usage between subsequent calls. Consider it a handy transparent utility.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$count</span> <span class="o">=</span> <span class="mi">1000000</span><span class="p">;</span> <span class="c1">// an amount of iterations</span>
<span class="c1">// test input, it will be mapped and reduced</span>
<span class="nv">$array</span> <span class="o">=</span> <span class="k">array</span><span class="p">();</span>
<span class="k">for</span> <span class="p">(</span><span class="nv">$i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="nv">$i</span> <span class="o"><</span> <span class="nv">$count</span><span class="p">;</span> <span class="nv">$i</span><span class="o">++</span><span class="p">)</span> <span class="p">{</span>
<span class="nv">$array</span><span class="p">[]</span> <span class="o">=</span> <span class="nb">rand</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="nv">$count</span><span class="p">);</span>
<span class="p">}</span>
<span class="c1">// the wrapper to benchmark easily</span>
<span class="nv">$ys</span> <span class="o">=</span> <span class="k">new</span> <span class="err">\</span><span class="nf">Mudasobwa\Screwdrivers\YardStick</span><span class="p">(</span><span class="kc">true</span><span class="p">);</span>
</code></pre></div></div>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// Mapping</span>
<span class="nv">$ys</span><span class="o">-></span><span class="nf">milestone</span><span class="p">(</span><span class="s1">'MAP#Start'</span><span class="p">);</span>
<span class="nv">$countew_array1</span> <span class="o">=</span> <span class="k">array</span><span class="p">();</span>
<span class="k">foreach</span> <span class="p">(</span><span class="nv">$array</span> <span class="k">as</span> <span class="nv">$a</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$countew_array1</span><span class="p">[]</span> <span class="o">=</span> <span class="nv">$a</span> <span class="o">/</span> <span class="mi">2</span><span class="p">;</span> <span class="p">}</span>
<span class="nv">$ys</span><span class="o">-></span><span class="nf">milestone</span><span class="p">(</span><span class="s1">'MAP#FOREACH'</span><span class="p">);</span>
<span class="nv">$countew_array2</span> <span class="o">=</span> <span class="k">array</span><span class="p">();</span>
<span class="k">for</span> <span class="p">(</span><span class="nv">$i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="nv">$i</span> <span class="o"><</span> <span class="nv">$count</span><span class="p">;</span> <span class="nv">$i</span><span class="o">++</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$countew_array2</span><span class="p">[]</span> <span class="o">=</span> <span class="nv">$array</span><span class="p">[</span><span class="nv">$i</span><span class="p">]</span> <span class="o">/</span> <span class="mi">2</span><span class="p">;</span> <span class="p">}</span>
<span class="nv">$ys</span><span class="o">-></span><span class="nf">milestone</span><span class="p">(</span><span class="s1">'MAP#FOR'</span><span class="p">);</span>
<span class="nv">$countew_array3</span> <span class="o">=</span> <span class="nb">array_map</span><span class="p">(</span><span class="k">function</span><span class="p">(</span><span class="nv">$a</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nv">$a</span> <span class="o">/</span> <span class="mi">2</span><span class="p">;</span> <span class="p">},</span> <span class="nv">$array</span><span class="p">);</span>
<span class="nv">$ys</span><span class="o">-></span><span class="nf">milestone</span><span class="p">(</span><span class="s1">'MAP#ARRAY_MAP'</span><span class="p">);</span>
<span class="nv">$ys</span><span class="o">-></span><span class="nf">report</span><span class="p">(</span><span class="s1">'MAP.+'</span><span class="p">);</span> <span class="c1">// report measures for milestones `MAP*`</span>
</code></pre></div></div>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// Reducing</span>
<span class="nv">$ys</span><span class="o">-></span><span class="nf">milestone</span><span class="p">(</span><span class="s1">'REDUCE#Start'</span><span class="p">);</span>
<span class="nv">$average1</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
<span class="k">foreach</span> <span class="p">(</span><span class="nv">$array</span> <span class="k">as</span> <span class="nv">$a</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$average1</span> <span class="o">+=</span> <span class="nv">$a</span> <span class="o">/</span> <span class="nv">$count</span><span class="p">;</span> <span class="p">}</span>
<span class="nv">$ys</span><span class="o">-></span><span class="nf">milestone</span><span class="p">(</span><span class="s1">'REDUCE#FOREACH'</span><span class="p">);</span>
<span class="nv">$average2</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
<span class="k">for</span> <span class="p">(</span><span class="nv">$i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="nv">$i</span> <span class="o"><</span> <span class="nv">$count</span><span class="p">;</span> <span class="nv">$i</span><span class="o">++</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$average2</span> <span class="o">+=</span> <span class="nv">$array</span><span class="p">[</span><span class="nv">$i</span><span class="p">]</span> <span class="o">/</span> <span class="nv">$count</span><span class="p">;</span> <span class="p">}</span>
<span class="nv">$ys</span><span class="o">-></span><span class="nf">milestone</span><span class="p">(</span><span class="s1">'REDUCE#FOR'</span><span class="p">);</span>
<span class="nv">$average3</span> <span class="o">=</span> <span class="nb">array_reduce</span><span class="p">(</span><span class="nv">$array</span><span class="p">,</span> <span class="k">function</span><span class="p">(</span><span class="nv">$memo</span><span class="p">,</span> <span class="nv">$a</span><span class="p">)</span> <span class="k">use</span><span class="p">(</span><span class="nv">$count</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="nv">$memo</span> <span class="o">+</span> <span class="nv">$a</span> <span class="o">/</span> <span class="nv">$count</span><span class="p">;</span>
<span class="p">},</span> <span class="mi">0</span><span class="p">);</span>
<span class="nv">$ys</span><span class="o">-></span><span class="nf">milestone</span><span class="p">(</span><span class="s1">'REDUCE#ARRAY_REDUCE'</span><span class="p">);</span>
<span class="nv">$ys</span><span class="o">-></span><span class="nf">report</span><span class="p">(</span><span class="s1">'REDUCE.+'</span><span class="p">);</span> <span class="c1">// report measures for milestones `REDUCE*`</span>
</code></pre></div></div>
<p>The results are shown below:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">====</span> Diff <span class="k">for </span>tags: <span class="o">[</span>MAP#Start :: MAP#FOREACH]
<span class="nt">--</span> ⌚ Time: ⇒ 0.223850 sec
<span class="nt">--</span> ⌛ Memory: ⇒ 141,009.3 KB
<span class="o">====</span> Diff <span class="k">for </span>tags: <span class="o">[</span>MAP#FOREACH :: MAP#FOR]
<span class="nt">--</span> ⌚ Time: ⇒ 0.240624 sec
<span class="nt">--</span> ⌛ Memory: ⇒ 141,009.2 KB
<span class="o">====</span> Diff <span class="k">for </span>tags: <span class="o">[</span>MAP#FOR :: MAP#ARRAY_MAP]
<span class="nt">--</span> ⌚ Time: ⇒ 0.374310 sec
<span class="nt">--</span> ⌛ Memory: ⇒ 141,009.2 KB
——————————————————————————————————————
<span class="o">====</span> Diff <span class="k">for </span>tags: <span class="o">[</span>REDUCE#Start :: REDUCE#FOREACH]
<span class="nt">--</span> ⌚ Time: ⇒ 0.072303 sec
<span class="nt">--</span> ⌛ Memory: ⇒ 4.6 KB
<span class="o">====</span> Diff <span class="k">for </span>tags: <span class="o">[</span>REDUCE#FOREACH :: REDUCE#FOR]
<span class="nt">--</span> ⌚ Time: ⇒ 0.086437 sec
<span class="nt">--</span> ⌛ Memory: ⇒ 4.6 KB
<span class="o">====</span> Diff <span class="k">for </span>tags: <span class="o">[</span>REDUCE#FOR :: REDUCE#ARRAY_REDUCE]
<span class="nt">--</span> ⌚ Time: ⇒ 0.276434 sec
<span class="nt">--</span> ⌛ Memory: ⇒ 4.6 KB
</code></pre></div></div>
<p>Memory consumption is the same, but <code class="language-plaintext highlighter-rouge">array_map</code> execution time is almost twice as much as <code class="language-plaintext highlighter-rouge">foreach</code>, and <code class="language-plaintext highlighter-rouge">array_reduce</code> is almost four times slower! WUT? Let’s turn back to ruby (I’m not as reptile as my friend, I like jewels more than snakes.)</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">require</span> <span class="s1">'benchmark'</span>
<span class="n">n</span> <span class="o">=</span> <span class="mi">1_000_000</span>
<span class="n">prng</span> <span class="o">=</span> <span class="no">Random</span><span class="p">.</span><span class="nf">new</span>
<span class="n">array</span> <span class="o">=</span> <span class="mi">1</span><span class="p">.</span><span class="nf">upto</span><span class="p">(</span><span class="n">n</span><span class="p">).</span><span class="nf">map</span> <span class="p">{</span> <span class="o">|</span><span class="n">_</span><span class="o">|</span> <span class="mf">1.0</span> <span class="o">*</span> <span class="p">(</span><span class="n">prng</span><span class="p">.</span><span class="nf">rand</span> <span class="n">n</span><span class="p">)</span> <span class="p">}</span>
<span class="n">array_new1</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">array_new2</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">array_new3</span> <span class="o">=</span> <span class="p">[]</span>
<span class="no">Benchmark</span><span class="p">.</span><span class="nf">bm</span> <span class="k">do</span> <span class="o">|</span><span class="n">x</span><span class="o">|</span>
<span class="n">x</span><span class="p">.</span><span class="nf">report</span> <span class="p">{</span> <span class="k">for</span> <span class="n">i</span> <span class="k">in</span> <span class="mi">1</span><span class="o">...</span><span class="n">n</span><span class="p">;</span> <span class="n">array_new1</span> <span class="o"><<</span> <span class="n">array</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">/</span> <span class="mi">2</span><span class="p">;</span> <span class="k">end</span> <span class="p">}</span>
<span class="n">x</span><span class="p">.</span><span class="nf">report</span> <span class="p">{</span> <span class="n">array</span><span class="p">.</span><span class="nf">each</span> <span class="p">{</span> <span class="o">|</span><span class="n">a</span><span class="o">|</span> <span class="n">array_new2</span> <span class="o"><<</span> <span class="n">a</span> <span class="o">/</span> <span class="mi">2</span> <span class="p">}</span> <span class="p">}</span>
<span class="n">x</span><span class="p">.</span><span class="nf">report</span> <span class="p">{</span> <span class="n">array_new3</span> <span class="o">=</span> <span class="n">array</span><span class="p">.</span><span class="nf">map</span> <span class="p">{</span> <span class="o">|</span><span class="n">a</span><span class="o">|</span> <span class="n">a</span> <span class="o">/</span> <span class="mi">2</span> <span class="p">}</span> <span class="p">}</span>
<span class="k">end</span>
<span class="n">reduce1</span> <span class="o">=</span> <span class="n">reduce2</span> <span class="o">=</span> <span class="n">reduce3</span> <span class="o">=</span> <span class="mi">0</span>
<span class="no">Benchmark</span><span class="p">.</span><span class="nf">bm</span> <span class="k">do</span> <span class="o">|</span><span class="n">x</span><span class="o">|</span>
<span class="n">x</span><span class="p">.</span><span class="nf">report</span> <span class="p">{</span> <span class="k">for</span> <span class="n">i</span> <span class="k">in</span> <span class="mi">1</span><span class="o">...</span><span class="n">n</span><span class="p">;</span> <span class="n">reduce1</span> <span class="o">+=</span> <span class="n">array</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">/</span> <span class="n">n</span><span class="p">;</span> <span class="k">end</span> <span class="p">}</span>
<span class="n">x</span><span class="p">.</span><span class="nf">report</span> <span class="p">{</span> <span class="n">array</span><span class="p">.</span><span class="nf">each</span> <span class="p">{</span> <span class="o">|</span><span class="n">a</span><span class="o">|</span> <span class="n">reduce2</span> <span class="o">+=</span> <span class="n">a</span> <span class="o">/</span> <span class="n">n</span> <span class="p">}</span> <span class="p">}</span>
<span class="n">x</span><span class="p">.</span><span class="nf">report</span> <span class="p">{</span> <span class="n">reduce3</span> <span class="o">=</span> <span class="n">array</span><span class="p">.</span><span class="nf">reduce</span><span class="p">(</span><span class="o">&</span><span class="p">:</span><span class="o">+</span><span class="p">)</span> <span class="o">/</span> <span class="n">n</span> <span class="p">}</span>
<span class="k">end</span>
</code></pre></div></div>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code> user system total real
0.130000 0.000000 0.130000 <span class="o">(</span> 0.130952<span class="o">)</span>
0.110000 0.000000 0.110000 <span class="o">(</span> 0.104156<span class="o">)</span>
0.090000 0.010000 0.100000 <span class="o">(</span> 0.097225<span class="o">)</span>
user system total real
0.120000 0.000000 0.120000 <span class="o">(</span> 0.115846<span class="o">)</span>
0.150000 0.000000 0.150000 <span class="o">(</span> 0.155771<span class="o">)</span>
0.090000 0.000000 0.090000 <span class="o">(</span> 0.090643<span class="o">)</span>
</code></pre></div></div>
<p>Ooh! Here we got what we expected from the modern language: mapping and reducing might be optimized comparing to dumb iteration and they occasionally were optimized. Both <code class="language-plaintext highlighter-rouge">map</code> and <code class="language-plaintext highlighter-rouge">reduce</code> are predictably <em>faster</em> than iterations. QED.</p>
<p>The difference in ruby is not significant, but the language is likely nudging us to use functional approach. Besides this code is a way more readable, it is just <em>faster</em>. While PHP… Well, thanks God my PHP diving season is over.</p>
Using local sources in Gemfile2014-01-24T00:00:00+00:00https://rocket-science.ru/hacking/2014/01/24/using-local-sources-in-gemfile<p>Let’s imagine we develop application, depending on a couple of our own gems.
Or, say, we decide to break a functionality into several gems. The main
application <code class="language-plaintext highlighter-rouge">Gemfile</code> thus contains references to our gems among all others.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">gem</span> <span class="s1">'mycutegem'</span>
</code></pre></div></div>
<p>The above uses the gem from <a href="http://rubygems.org">Ruby Gems</a>, which is quite
inconvenient in our case: we modify it simultaneously. There is option to
supply path to load the source locally:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">gem</span> <span class="s1">'mycutegem'</span><span class="p">,</span> <span class="ss">:path</span> <span class="o">=></span> <span class="s1">'../mycutegem'</span>
</code></pre></div></div>
<p>This is hardly acceptable because <code class="language-plaintext highlighter-rouge">Gemfile</code> became inconsistent with remote
repository. Unfortunately, we cannot specify rules for different groups
in the <code class="language-plaintext highlighter-rouge">Gemfile</code>, like:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">gem</span> <span class="s1">'mycutegem'</span><span class="p">,</span> <span class="ss">:path</span> <span class="o">=></span> <span class="s1">'../mycutegem'</span><span class="p">,</span> <span class="ss">:group</span> <span class="o">=></span> <span class="ss">:development</span>
<span class="n">gem</span> <span class="s1">'mycutegem'</span><span class="p">,</span> <span class="s1">'~> 0.9.3'</span><span class="p">,</span> <span class="ss">:group</span> <span class="o">=></span> <span class="ss">:production</span>
</code></pre></div></div>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">You</span> <span class="n">cannot</span> <span class="n">specify</span> <span class="n">the</span> <span class="n">same</span> <span class="n">gem</span> <span class="n">twice</span> <span class="n">with</span> <span class="n">different</span> <span class="n">version</span> <span class="n">requirements</span><span class="o">.</span>
</code></pre></div></div>
<p>The solution is simple. There is an ability to inform bundler about
our <em>local</em> copy of repository:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">$</span> <span class="n">bundle</span> <span class="n">config</span> <span class="n">local</span><span class="p">.</span><span class="nf">mycutegem</span> <span class="sr">/home/</span><span class="n">am</span><span class="o">/</span><span class="no">Projects</span><span class="o">/</span><span class="n">mycutegem</span>
</code></pre></div></div>
<p>The latter should be a path to <em>local <strong>git</strong> repository</em>. Now the
following code will use our local version of repository (with all the
hot changes,) while all others will use the public source (<code class="language-plaintext highlighter-rouge">:branch</code> is
mandatory):</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">gem</span> <span class="s1">'mycutegem'</span><span class="p">,</span>
<span class="ss">:git</span> <span class="o">=></span> <span class="s1">'git://github.com/mudasobwa/mycutegem'</span><span class="p">,</span>
<span class="ss">:branch</span> <span class="o">=></span> <span class="s1">'master'</span>
</code></pre></div></div>
Ruby Memory Pitfalls2013-12-17T00:00:00+00:00https://rocket-science.ru/hacking/2013/12/17/ruby-memory-pitfalls<p>Ruby has an automatic memory management. In most cases this is good; sometimes it becomes sad.</p>
<p>Ruby memory management is both elegant and cumbersome. It stores objects (named <code class="language-plaintext highlighter-rouge">RVALUE</code>s)
in so-called heaps of size of approx 16KB. On a low level, <code class="language-plaintext highlighter-rouge">RVALUE</code> is a <code class="language-plaintext highlighter-rouge">c</code>-struct, containing
a union of different standard ruby object representations.</p>
<p>So, heaps store <code class="language-plaintext highlighter-rouge">RVALUE</code> objects, which size is not more than 40 bytes. For such
objects as <code class="language-plaintext highlighter-rouge">String</code>, <code class="language-plaintext highlighter-rouge">Array</code>, <code class="language-plaintext highlighter-rouge">Hash</code> etc. this means that small objects can fit in
the heap, but as soon as they reach a threshold, an extra memory outside of the
Ruby heaps will be allocated.</p>
<p><strong>This extra memory is flexible; is will be freed as soon as an object became GC’ed.
But the heaps themselves are not released to OS anymore.</strong></p>
<p>Let’s take a look at the simple example:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="k">def</span> <span class="nf">report</span>
<span class="nb">puts</span> <span class="s1">'Memory '</span> <span class="o">+</span> <span class="sb">`ps ax -o pid,rss | grep -E "^[[:space:]]*</span><span class="si">#{</span><span class="vg">$$</span><span class="si">}</span><span class="sb">"`</span>
<span class="p">.</span><span class="nf">strip</span><span class="p">.</span><span class="nf">split</span><span class="p">.</span><span class="nf">map</span><span class="p">(</span><span class="o">&</span><span class="ss">:to_i</span><span class="p">)[</span><span class="mi">1</span><span class="p">].</span><span class="nf">to_s</span> <span class="o">+</span> <span class="s1">'KB'</span>
<span class="k">end</span>
<span class="n">report</span>
<span class="n">big_var</span> <span class="o">=</span> <span class="s2">" "</span> <span class="o">*</span> <span class="mi">10_000_000</span>
<span class="n">report</span>
<span class="n">big_var</span> <span class="o">=</span> <span class="kp">nil</span>
<span class="n">report</span>
<span class="no">ObjectSpace</span><span class="p">.</span><span class="nf">garbage_collect</span>
<span class="nb">sleep</span> <span class="mi">1</span>
<span class="n">report</span>
<span class="c1"># ⇒ Memory 11788KB</span>
<span class="c1"># ⇒ Memory 65188KB</span>
<span class="c1"># ⇒ Memory 65188KB</span>
<span class="c1"># ⇒ Memory 11788KB</span>
</code></pre></div></div>
<p>Here we allocate the huge amount of memory, use it somehow and then release back to OS.
Everything seems to be fine. Let’s now slightly change the source code:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">-</span> <span class="n">big_var</span> <span class="o">=</span> <span class="s2">" "</span> <span class="o">*</span> <span class="mi">10_000_000</span>
<span class="o">+</span> <span class="n">big_var</span> <span class="o">=</span> <span class="mi">1_000_000</span><span class="p">.</span><span class="nf">times</span><span class="p">.</span><span class="nf">map</span><span class="p">(</span><span class="o">&</span><span class="ss">:to_s</span><span class="p">)</span>
</code></pre></div></div>
<p>That was a humdrum modification, wasn’t it? But wait:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="c1"># ⇒ Memory 11788KB</span>
<span class="c1"># ⇒ Memory 65188KB</span>
<span class="c1"># ⇒ Memory 65188KB</span>
<span class="c1"># ⇒ Memory 57448KB</span>
</code></pre></div></div>
<p>WTF? The memory is not released to OS anymore. That’s because each element
of the array we introduced <em>suits</em> the <code class="language-plaintext highlighter-rouge">RVALUE</code> size and is stored in the <em>ruby heap</em>.</p>
<p>In most cases this is OK. There are more empty slots in ruby heap now; code
re-run will not eat any additional memory; <code class="language-plaintext highlighter-rouge">GC[:heap_used]</code> value is decreased
as expected every time we dispose <code class="language-plaintext highlighter-rouge">big_var</code> and a lot of empty heaps, ready
for operation are returned back to Ruby. To <em>Ruby</em> that said, not to OS.</p>
<p>So, be careful with creating a lot of temporary variables suiting the 40 bytes:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="n">big_var</span> <span class="o">=</span> <span class="s2">" "</span> <span class="o">*</span> <span class="mi">10_000_000</span>
<span class="n">big_var</span><span class="p">.</span><span class="nf">gsub</span><span class="p">(</span><span class="sr">/\s/</span><span class="p">)</span> <span class="p">{</span> <span class="o">|</span><span class="n">c</span><span class="o">|</span> <span class="s1">'-'</span> <span class="p">}</span>
</code></pre></div></div>
<p>results in growth of memory guzzled by Ruby as well. And this memory will not
be returned back to OS during the whole long run:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="c1"># ⇒ Memory 10156KB</span>
<span class="c1"># ⇒ Memory 13788KB</span>
<span class="c1"># ⇒ Memory 13788KB</span>
<span class="c1"># ⇒ Memory 12808KB</span>
</code></pre></div></div>
<p>Not so crucial, but noteworthy enough.</p>
StackOverflow questions ⇒ Diverse Worlds2013-12-09T00:00:00+00:00https://rocket-science.ru/fun/2013/12/09/stackoverflow-questions--diverse-worlds<h3 id="linux">Linux</h3>
<p><strong>Q.</strong> I’m stuck with installation of <code class="language-plaintext highlighter-rouge">best-lib-ever</code> of version <code class="language-plaintext highlighter-rouge">1.2.3</code>, needed to
convert integer value to string? I have tried
to install library from sources, following to <em>this</em> and <em>this</em> blog posts with no success
(error message follows). I also have <em>this</em>, <em>this</em> and <em>this</em> related posts read.
I have tried to uninstall/reinstall <code class="language-plaintext highlighter-rouge">better-lib-ever</code> with no luck. According to <em>this</em>,
I might have a problem with <code class="language-plaintext highlighter-rouge">another-best-lib</code>, but that’s not my case (test follows.)
As a last try I created a fully fresh installation within virtual machine (config follows,)
but still have the same problem (memory dump follows.)</p>
<p><strong>A.</strong> Where the f×ck did you dig that piece of shit? Why not use the standard <code class="language-plaintext highlighter-rouge">int_to_str</code>?</p>
<h3 id="macosx">MacOSX</h3>
<p><strong>Q.</strong> I met the following problem on my <em>OSX Lion</em>. After I have successfuly installed
<code class="language-plaintext highlighter-rouge">best-lib-ever</code>, the <code class="language-plaintext highlighter-rouge">web-server-of-choice</code> prints warning messages during startup (messages
follow.) I tried to remove <code class="language-plaintext highlighter-rouge">best-lib-ever</code> and now server starts without problems.
I still need <code class="language-plaintext highlighter-rouge">best-lib-ever</code>, so the question is: how am I supposed to install <code class="language-plaintext highlighter-rouge">best-lib-ever</code>
so that the web server will not be damaged.</p>
<p><strong>A.</strong> <em>Here</em> is a solution (summing up: you are not supposed to install <code class="language-plaintext highlighter-rouge">best-lib-ever</code> near
the <code class="language-plaintext highlighter-rouge">web-server-of-choice</code>.)</p>
<h3 id="windows">Windows</h3>
<p><strong>Q.</strong> I have tried to install <code class="language-plaintext highlighter-rouge">best-development-tool</code> and nothing works. Please help!</p>
<p><strong>A.</strong> Install <code class="language-plaintext highlighter-rouge">mingw</code> and follow the instructions <em>here</em>.</p>
Command line application wrappers problem2013-11-06T00:00:00+00:00https://rocket-science.ru/hacking/2013/11/06/command-line-application-wrappers-problem<p>Well, everybody knows, that there is no locale but <code class="language-plaintext highlighter-rouge">C</code>. That’s why all the wrappers
bar none use simple <code class="language-plaintext highlighter-rouge">Popen3::popen3</code> calls and then parse the output selfless. For
instance whether we are to count the total number of file system blocks, including indirect
blocks, used by the files in directory, we would write the following wrapper:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="k">def</span> <span class="nf">get_dir_total</span> <span class="n">dir</span>
<span class="n">stdin</span><span class="p">,</span> <span class="n">stdout</span><span class="p">,</span> <span class="n">stderr</span> <span class="o">=</span> <span class="no">Open3</span><span class="p">.</span><span class="nf">popen3</span> <span class="s2">"ls -la </span><span class="si">#{</span><span class="n">dir</span><span class="si">}</span><span class="s2">"</span>
<span class="c1"># log the errors or whatever</span>
<span class="n">stdout</span><span class="p">.</span><span class="nf">read</span><span class="p">.</span><span class="nf">split</span><span class="p">(</span><span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">).</span><span class="nf">select</span> <span class="p">{</span> <span class="o">|</span><span class="n">line</span><span class="o">|</span> <span class="n">line</span><span class="p">[</span><span class="s1">'total'</span><span class="p">]</span> <span class="p">}.</span><span class="nf">gsub</span><span class="p">(</span><span class="sr">/\D/</span><span class="p">,</span> <span class="s1">''</span><span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>
<p>And this code will pass all the tests. And we’ll put it in production, and for all
the colleagues it will work like a charm. Until some trainee from the adjacent
department has a bugreport filled. Damn europeans, will probably think we about,
reading an email from <em>Łukash Poręba</em>.</p>
<p>Aha. He has likely polished locale set on his laptop. As well as me, having the russian
one. Which forces <code class="language-plaintext highlighter-rouge">ls</code> to print <code class="language-plaintext highlighter-rouge">итого</code> instead of <code class="language-plaintext highlighter-rouge">total</code>, breaking so cute-promising code.</p>
<p>So, I have monkeypatched my <code class="language-plaintext highlighter-rouge">Popen3</code> with</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="err">…</span>
<span class="n">cmd</span><span class="p">.</span><span class="nf">prepend</span> <span class="s2">"LC_ALL=C "</span>
<span class="n">original</span><span class="p">.</span><span class="nf">popen3</span> <span class="n">cmd</span>
<span class="err">…</span>
</code></pre></div></div>
<p>And I would like to ask wrapper-writers “Please, don’t rely on standard locale on target computers.”</p>
Debug inplace2013-11-04T00:00:00+00:00https://rocket-science.ru/hacking/2013/11/04/debug-inplace<p>Most valuable feature of Ruby is, <abbr title="‘Meiner demütig Meinung nach’ is the german equivalent for ‘IMHO’">MDMN</abbr>,
the ability to overwrite virtually all the default behaviour. The swiss knife of hacking is <a href="http://pryrepl.org">pry</a> which is,
according to official site, “a <strong>powerful</strong> alternative to the standard IRB shell for Ruby. It features <strong>syntax highlighting</strong>,
a flexible <strong>plugin architecture</strong>, <strong>runtime invocation</strong> and <strong>source and documentation browsing</strong>.”</p>
<p>Though <code class="language-plaintext highlighter-rouge">pry</code> is better than the standard IRB console in all the aspects around, I personally am totally amused with
it’s “runtime invocation” feature. It works in the following way: anywhere within your code scope you simply drop the line</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">binding</span><span class="p">.</span><span class="nf">pry</span>
</code></pre></div></div>
<p>and—voilá—the execution flow is stopped here, putting you in <code class="language-plaintext highlighter-rouge">pry</code> session with the context specified. Let’s say we have
the code:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">#!/usr/bin/env ruby</span>
<span class="nb">require</span> <span class="s1">'pry'</span>
<span class="k">def</span> <span class="nf">iterate</span>
<span class="mi">40</span><span class="p">.</span><span class="nf">times</span> <span class="p">{</span> <span class="o">|</span><span class="n">i</span><span class="o">|</span>
<span class="c1"># ⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓ HERE GOES MAGIC</span>
<span class="nb">binding</span><span class="p">.</span><span class="nf">pry</span> <span class="n">or</span> <span class="k">break</span> <span class="k">if</span> <span class="nb">rand</span><span class="p">(</span><span class="mi">40</span><span class="o">-</span><span class="n">i</span><span class="p">).</span><span class="nf">zero?</span>
<span class="nb">print</span> <span class="s1">'='</span>
<span class="p">}</span>
<span class="nb">puts</span>
<span class="k">end</span>
<span class="n">iterate</span>
</code></pre></div></div>
<p>We’ll yield an amount of equal signs in the output, following by accidental zero in <code class="language-plaintext highlighter-rouge">rand</code> call, leading us
to the <code class="language-plaintext highlighter-rouge">pry</code> instance within current context:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err">〉</span><span class="n">ruby</span> <span class="n">pry</span><span class="p">.</span><span class="nf">rb</span>
<span class="o">===============</span>
<span class="no">From</span><span class="p">:</span> <span class="sr">/tmp/</span><span class="n">pry</span><span class="p">.</span><span class="nf">rb</span> <span class="err">@</span> <span class="n">line</span> <span class="mi">7</span> <span class="no">Object</span><span class="c1">#iterate:</span>
<span class="mi">5</span><span class="p">:</span> <span class="k">def</span> <span class="nf">iterate</span>
<span class="mi">6</span><span class="p">:</span> <span class="mi">40</span><span class="p">.</span><span class="nf">times</span> <span class="p">{</span> <span class="o">|</span><span class="n">i</span><span class="o">|</span>
<span class="o">=></span> <span class="mi">7</span><span class="p">:</span> <span class="nb">binding</span><span class="p">.</span><span class="nf">pry</span> <span class="n">or</span> <span class="k">break</span> <span class="k">if</span> <span class="nb">rand</span><span class="p">(</span><span class="mi">40</span><span class="o">-</span><span class="n">i</span><span class="p">).</span><span class="nf">zero?</span>
<span class="mi">8</span><span class="p">:</span> <span class="nb">print</span> <span class="s1">'='</span>
<span class="mi">9</span><span class="p">:</span> <span class="p">}</span>
<span class="mi">10</span><span class="p">:</span> <span class="nb">puts</span>
<span class="mi">11</span><span class="p">:</span> <span class="k">end</span>
<span class="mf">2.1</span><span class="o">.</span><span class="mi">0</span> <span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mi">0</span> <span class="o">></span>
</code></pre></div></div>
<p>Here goes the whole stuff in action (thanks to brilliant <a href="http://showterm.io">showterm.io</a> service):</p>
<iframe src="http://showterm.io/d542cd31224acaa84549f" width="640" height="480"> </iframe>
EMACS SHIT2013-10-24T00:00:00+00:00https://rocket-science.ru/fun/2013/10/24/emacs-shit<p>The manhole below was found in Toledo, Spain. The legend states “EMACSA alcantarillado” for “emacs is shit” in spanish.</p>
<p><img src="/img/emacs-alcantarillada.jpg" alt="EMACSA alcantarillado" /></p>
Sequentional execution: example of Reactor pattern impl2013-09-05T00:00:00+00:00https://rocket-science.ru/hacking/2013/09/05/sequentional-execution-example-of-reactor-pattern-impl<p><a href="http://en.wikipedia.org/wiki/Reactor_pattern">Reactor pattern</a> is an event handling pattern
for operating service requests delivered concurrently to a service handler by one or more inputs. All
the ready-to-use ruby implementations (like <a href="http://rubyeventmachine.com/">EventMachine</a> and family,)
although are very smart built and quite helpful, hide all the details. I decided to write down a short
example of how the task may be accomplished in pure ruby.</p>
<p>Let’s say we are interested in running an echo-service-like application. We don’t need any servers,
just pure “wait-for-input ⇒ reply” app. The CLI wrapper might be a good example. All we want is to have
a running instance somewhere that will accept internal calls like <code class="language-plaintext highlighter-rouge">@inst.cmd("ls")</code> and reply with
a result of the command run.</p>
<p>We decide to have two independent threads, which are to be synchronized in the following manner: just
after the <code class="language-plaintext highlighter-rouge">prior</code> function is executed, the <code class="language-plaintext highlighter-rouge">posterior</code> function wakes up, does something and falls sleep
back until the next call to <code class="language-plaintext highlighter-rouge">prior</code>. Here we go:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nn">SeqExec</span>
<span class="k">class</span> <span class="nc">Seqs</span>
<span class="nb">attr_reader</span> <span class="ss">:init</span>
<span class="k">def</span> <span class="nf">synch_prior</span> <span class="n">mx</span><span class="p">,</span> <span class="n">cv</span>
<span class="no">Thread</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span>
<span class="n">mx</span><span class="p">.</span><span class="nf">synchronize</span> <span class="p">{</span>
<span class="vi">@init</span><span class="p">[</span><span class="ss">:prior</span><span class="p">]</span> <span class="o">=</span> <span class="kp">true</span>
<span class="kp">loop</span> <span class="k">do</span>
<span class="n">cv</span><span class="p">.</span><span class="nf">wait</span> <span class="n">mx</span>
<span class="k">yield</span> <span class="k">if</span> <span class="nb">block_given?</span>
<span class="n">cv</span><span class="p">.</span><span class="nf">broadcast</span>
<span class="k">end</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">synch_posterior</span> <span class="n">mx</span><span class="p">,</span> <span class="n">cv</span>
<span class="no">Thread</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span>
<span class="n">mx</span><span class="p">.</span><span class="nf">synchronize</span> <span class="p">{</span>
<span class="vi">@init</span><span class="p">[</span><span class="ss">:posterior</span><span class="p">]</span> <span class="o">=</span> <span class="kp">true</span>
<span class="kp">loop</span> <span class="k">do</span>
<span class="n">cv</span><span class="p">.</span><span class="nf">wait</span> <span class="n">mx</span>
<span class="k">yield</span> <span class="k">if</span> <span class="nb">block_given?</span>
<span class="n">cv</span><span class="p">.</span><span class="nf">broadcast</span>
<span class="k">end</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">synch</span> <span class="err">λ</span><span class="mi">1</span><span class="p">,</span> <span class="err">λ</span><span class="mi">2</span>
<span class="vi">@init</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">mx</span> <span class="o">=</span> <span class="no">Mutex</span><span class="p">.</span><span class="nf">new</span>
<span class="n">cv</span> <span class="o">=</span> <span class="no">ConditionVariable</span><span class="p">.</span><span class="nf">new</span>
<span class="n">synch_prior</span><span class="p">(</span><span class="n">mx</span><span class="p">,</span> <span class="n">cv</span><span class="p">,</span> <span class="o">&</span><span class="err">λ</span><span class="mi">1</span><span class="p">)</span> <span class="c1"># prior function</span>
<span class="no">Thread</span><span class="p">.</span><span class="nf">pass</span> <span class="k">until</span> <span class="p">{</span><span class="ss">:prior</span><span class="o">=></span><span class="kp">true</span><span class="p">}</span> <span class="o">==</span> <span class="vi">@init</span>
<span class="n">synch_posterior</span><span class="p">(</span><span class="n">mx</span><span class="p">,</span> <span class="n">cv</span><span class="p">,</span> <span class="o">&</span><span class="err">λ</span><span class="mi">2</span><span class="p">)</span> <span class="c1"># posterior function</span>
<span class="no">Thread</span><span class="p">.</span><span class="nf">pass</span> <span class="k">until</span> <span class="p">{</span><span class="ss">:prior</span><span class="o">=></span><span class="kp">true</span><span class="p">,</span><span class="ss">:posterior</span><span class="o">=></span><span class="kp">true</span><span class="p">}</span> <span class="o">==</span> <span class="vi">@init</span>
<span class="n">cv</span><span class="p">.</span><span class="nf">signal</span> <span class="c1"># we are ready to start</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Here we produce two threads, waiting one for another until the <code class="language-plaintext highlighter-rouge">yield</code> clause
(which may be blocking, if necessary) occurs to initiate the <em>ping-pong</em> mechanism.</p>
<p>Let’s now add some syntactic sugar:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nn">SeqExec</span>
<span class="no">Thread</span><span class="p">.</span><span class="nf">abort_on_exception</span> <span class="o">=</span> <span class="kp">true</span>
<span class="k">def</span> <span class="nf">pre</span> <span class="o">&</span><span class="n">cb</span>
<span class="vi">@prior</span> <span class="o">=</span> <span class="n">cb</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">post</span> <span class="o">&</span><span class="n">cb</span>
<span class="vi">@posterior</span> <span class="o">=</span> <span class="n">cb</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">run</span> <span class="err">λ</span><span class="mi">1</span> <span class="o">=</span> <span class="kp">nil</span><span class="p">,</span> <span class="err">λ</span><span class="mi">2</span> <span class="o">=</span> <span class="kp">nil</span>
<span class="n">pre</span> <span class="o">&</span><span class="err">λ</span><span class="mi">1</span> <span class="k">if</span> <span class="err">λ</span><span class="mi">1</span>
<span class="n">post</span> <span class="o">&</span><span class="err">λ</span><span class="mi">2</span> <span class="k">if</span> <span class="err">λ</span><span class="mi">2</span>
<span class="k">raise</span> <span class="no">ArgumentError</span><span class="p">.</span><span class="nf">new</span> <span class="s2">"Cannot run sequential execution, lambdas are not set"</span> <span class="p">\</span>
<span class="k">unless</span> <span class="p">(</span><span class="vi">@prior</span> <span class="o">&&</span> <span class="vi">@posterior</span><span class="p">)</span>
<span class="no">Seqs</span><span class="p">.</span><span class="nf">new</span><span class="p">.</span><span class="nf">synch</span> <span class="vi">@prior</span><span class="p">,</span> <span class="vi">@posterior</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Now it’s time to play with:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kp">include</span> <span class="no">SeqExec</span>
<span class="vi">@i</span><span class="o">=</span><span class="mi">0</span>
<span class="vi">@stack</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">pre</span> <span class="p">{</span> <span class="nb">sleep</span> <span class="mf">0.3</span><span class="p">;</span> <span class="nb">print</span> <span class="s2">"-</span><span class="si">#{</span><span class="vi">@i</span> <span class="o">+=</span> <span class="mi">1</span><span class="si">}</span><span class="s2">-"</span><span class="p">;</span> <span class="vi">@stack</span><span class="p">.</span><span class="nf">push</span><span class="p">(</span><span class="vi">@i</span><span class="p">)</span> <span class="p">}</span>
<span class="n">post</span> <span class="p">{</span> <span class="nb">print</span> <span class="s2">"|</span><span class="si">#{</span><span class="vi">@stack</span><span class="p">.</span><span class="nf">pop</span><span class="si">}</span><span class="s2">|"</span> <span class="p">}</span>
<span class="n">run</span>
<span class="mi">10</span><span class="p">.</span><span class="nf">times</span> <span class="p">{</span> <span class="nb">print</span> <span class="s2">"#"</span><span class="p">;</span> <span class="nb">sleep</span> <span class="mf">0.1</span> <span class="p">}</span>
<span class="nb">sleep</span> <span class="mi">3</span>
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">prior</code> function pushes the incremented integer to the stack, the <code class="language-plaintext highlighter-rouge">posterior</code> reacts
by printing it to the terminal:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># ⇒ ####-1-|1|###-2-|2|###-3-|3|-4-|4|-5-|5|-6-|6|-7-|7|-8-|8|-9-|9|-10-|10|-11-|11|-12-|12|-13-|13|</span>
</code></pre></div></div>
Collage directory preview with RMagick2013-08-15T00:00:00+00:00https://rocket-science.ru/hacking/2013/08/15/collage-directory-preview-with-rmagick<p>Just before my last trip I have finally decided to issue daily reports. Not to forget the impressions
as well as to share my experiences with friends. The scenario I foresaw was: all the day I make photos,
then reach a hotel with Wifi internet, pick out a dozen of best views and publish.</p>
<p>It’s worth to mention that I have chosen a jekyll successor, the <a href="http://ruhoh.com">ruhoh</a> publishing
system. I only needed a handy script to quickly produce a new blog entry by the directory with today photos.
Plus I wanted a collage to be put as a post preview.</p>
<p>There is an <a href="http://imagemagick.org">ImageMagick</a> wrapper for Ruby: <a href="http://rmagick.rubyforge.org">RMagick</a>.
The only problem remained: I still needed to produce the collage by hands. So, I decided to monkeypatch the library.
Below goes the code of the patch:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">require</span> <span class="s1">'RMagick'</span>
<span class="k">module</span> <span class="nn">Magick</span>
<span class="k">class</span> <span class="nc">ImageList</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">preview</span> <span class="n">files</span><span class="p">,</span> <span class="n">options</span><span class="o">=</span><span class="p">{}</span>
<span class="n">options</span> <span class="o">=</span> <span class="p">{</span>
<span class="ss">:columns</span> <span class="o">=></span> <span class="mi">5</span><span class="p">,</span> <span class="c1"># number of columns in collage</span>
<span class="ss">:scale_range</span> <span class="o">=></span> <span class="mf">0.1</span><span class="p">,</span> <span class="c1"># ± to the thumb width</span>
<span class="ss">:thumb_width</span> <span class="o">=></span> <span class="mi">120</span><span class="p">,</span> <span class="c1"># the width of thumbnail</span>
<span class="ss">:rotate_angle</span> <span class="o">=></span> <span class="mi">20</span><span class="p">,</span> <span class="c1"># maximal rotate angle</span>
<span class="ss">:background</span> <span class="o">=></span> <span class="s1">'white'</span><span class="p">,</span> <span class="c1"># background of the collage</span>
<span class="ss">:border</span> <span class="o">=></span> <span class="s1">'gray20'</span> <span class="c1"># border color</span>
<span class="p">}.</span><span class="nf">merge</span><span class="p">(</span><span class="n">options</span><span class="p">)</span>
<span class="c1"># Produce collage from all the files in directory</span>
<span class="n">files</span> <span class="o">=</span> <span class="s2">"</span><span class="si">#{</span><span class="n">files</span><span class="si">}</span><span class="s2">/*"</span> <span class="k">if</span> <span class="no">File</span><span class="p">.</span><span class="nf">directory?</span><span class="p">(</span><span class="n">files</span><span class="p">)</span>
<span class="n">imgs</span> <span class="o">=</span> <span class="no">ImageList</span><span class="p">.</span><span class="nf">new</span>
<span class="c1"># The placeholder for collage borders</span>
<span class="n">imgnull</span> <span class="o">=</span> <span class="no">Image</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">options</span><span class="p">[</span><span class="ss">:thumb_width</span><span class="p">],</span><span class="n">options</span><span class="p">[</span><span class="ss">:thumb_width</span><span class="p">])</span> <span class="p">{</span>
<span class="nb">self</span><span class="p">.</span><span class="nf">background_color</span> <span class="o">=</span> <span class="s1">'transparent'</span>
<span class="p">}</span>
<span class="c1"># Top row</span>
<span class="p">(</span><span class="n">options</span><span class="p">[</span><span class="ss">:columns</span><span class="p">]</span><span class="o">+</span><span class="mi">2</span><span class="p">).</span><span class="nf">times</span> <span class="p">{</span> <span class="n">imgs</span> <span class="o"><<</span> <span class="n">imgnull</span><span class="p">.</span><span class="nf">dup</span> <span class="p">}</span>
<span class="no">Dir</span><span class="p">.</span><span class="nf">glob</span><span class="p">(</span><span class="s2">"</span><span class="si">#{</span><span class="n">files</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span> <span class="p">{</span> <span class="o">|</span><span class="n">f</span><span class="o">|</span>
<span class="no">Image</span><span class="o">::</span><span class="n">read</span><span class="p">(</span><span class="n">f</span><span class="p">).</span><span class="nf">each</span> <span class="p">{</span> <span class="o">|</span><span class="n">i</span><span class="o">|</span>
<span class="n">scale</span> <span class="o">=</span> <span class="p">(</span><span class="mf">1.0</span> <span class="o">+</span> <span class="n">options</span><span class="p">[</span><span class="ss">:scale_range</span><span class="p">]</span><span class="o">*</span><span class="no">Random</span><span class="o">::</span><span class="nb">rand</span><span class="p">(</span><span class="o">-</span><span class="mf">1.0</span><span class="o">..</span><span class="mf">1.0</span><span class="p">))</span><span class="o">*</span><span class="n">options</span><span class="p">[</span><span class="ss">:thumb_width</span><span class="p">]</span><span class="o">/</span><span class="p">[</span><span class="n">i</span><span class="p">.</span><span class="nf">columns</span><span class="p">,</span> <span class="n">i</span><span class="p">.</span><span class="nf">rows</span><span class="p">].</span><span class="nf">max</span>
<span class="c1"># Placeholder if that’s the first column</span>
<span class="n">imgs</span> <span class="o"><<</span> <span class="n">imgnull</span><span class="p">.</span><span class="nf">dup</span> <span class="k">if</span> <span class="p">(</span><span class="n">imgs</span><span class="p">.</span><span class="nf">size</span> <span class="o">%</span> <span class="p">(</span><span class="n">options</span><span class="p">[</span><span class="ss">:columns</span><span class="p">]</span><span class="o">+</span><span class="mi">2</span><span class="p">)).</span><span class="nf">zero?</span>
<span class="n">imgs</span> <span class="o"><<</span> <span class="n">i</span><span class="p">.</span><span class="nf">auto_orient</span><span class="p">.</span><span class="nf">thumbnail</span><span class="p">(</span><span class="n">scale</span><span class="p">).</span><span class="nf">polaroid</span><span class="p">(</span>
<span class="no">Random</span><span class="o">::</span><span class="nb">rand</span><span class="p">(</span><span class="o">-</span><span class="n">options</span><span class="p">[</span><span class="ss">:rotate_angle</span><span class="p">]</span><span class="o">..</span><span class="n">options</span><span class="p">[</span><span class="ss">:rotate_angle</span><span class="p">])</span>
<span class="p">)</span>
<span class="c1"># Placeholder if that’s the last columns</span>
<span class="n">imgs</span> <span class="o"><<</span> <span class="n">imgnull</span><span class="p">.</span><span class="nf">dup</span> <span class="k">if</span> <span class="p">(</span><span class="n">imgs</span><span class="p">.</span><span class="nf">size</span> <span class="o">%</span> <span class="p">(</span><span class="n">options</span><span class="p">[</span><span class="ss">:columns</span><span class="p">]</span><span class="o">+</span><span class="mi">2</span><span class="p">))</span> <span class="o">==</span> <span class="n">options</span><span class="p">[</span><span class="ss">:columns</span><span class="p">]</span><span class="o">+</span><span class="mi">1</span>
<span class="p">}</span> <span class="k">rescue</span> <span class="nb">puts</span> <span class="s2">"Skipping error: </span><span class="si">#{</span><span class="vg">$!</span><span class="si">}</span><span class="s2">"</span> <span class="c1"># simply skip non-image files</span>
<span class="p">}</span>
<span class="c1"># Fill the last row</span>
<span class="p">(</span><span class="mi">2</span><span class="o">*</span><span class="n">options</span><span class="p">[</span><span class="ss">:columns</span><span class="p">]</span><span class="o">+</span><span class="mi">4</span><span class="o">-</span><span class="p">(</span><span class="n">imgs</span><span class="p">.</span><span class="nf">size</span> <span class="o">%</span> <span class="p">(</span><span class="n">options</span><span class="p">[</span><span class="ss">:columns</span><span class="p">]</span><span class="o">+</span><span class="mi">2</span><span class="p">))).</span><span class="nf">times</span> <span class="p">{</span> <span class="n">imgs</span> <span class="o"><<</span> <span class="n">imgnull</span><span class="p">.</span><span class="nf">dup</span> <span class="p">}</span>
<span class="n">imgs</span><span class="p">.</span><span class="nf">montage</span> <span class="p">{</span>
<span class="nb">self</span><span class="p">.</span><span class="nf">tile</span> <span class="o">=</span> <span class="no">Magick</span><span class="o">::</span><span class="no">Geometry</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">options</span><span class="p">[</span><span class="ss">:columns</span><span class="p">]</span><span class="o">+</span><span class="mi">2</span><span class="p">)</span>
<span class="nb">self</span><span class="p">.</span><span class="nf">geometry</span> <span class="o">=</span> <span class="s2">"-</span><span class="si">#{</span><span class="n">options</span><span class="p">[</span><span class="ss">:thumb_width</span><span class="p">]</span><span class="o">/</span><span class="mi">5</span><span class="si">}</span><span class="s2">-</span><span class="si">#{</span><span class="n">options</span><span class="p">[</span><span class="ss">:thumb_width</span><span class="p">]</span><span class="o">/</span><span class="mi">4</span><span class="si">}</span><span class="s2">"</span>
<span class="nb">self</span><span class="p">.</span><span class="nf">background_color</span> <span class="o">=</span> <span class="n">options</span><span class="p">[</span><span class="ss">:background</span><span class="p">]</span>
<span class="p">}.</span><span class="nf">trim</span><span class="p">(</span><span class="kp">true</span><span class="p">).</span><span class="nf">border</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span><span class="mi">10</span><span class="p">,</span><span class="n">options</span><span class="p">[</span><span class="ss">:background</span><span class="p">]).</span><span class="nf">border</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="mi">1</span><span class="p">,</span><span class="n">options</span><span class="p">[</span><span class="ss">:border</span><span class="p">])</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The only noticable piece of code here is the stuff with <code class="language-plaintext highlighter-rouge">imgnull</code>. Since I rotate the thumbnails, I need to
preserve a sufficient place around the resulting montaged image. The function may be called with:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Magick</span><span class="o">::</span><span class="no">ImageList</span><span class="o">::</span><span class="n">preview</span><span class="p">(</span><span class="no">IMGS_DIR</span><span class="p">,</span> <span class="ss">:thumb_width</span><span class="o">=></span><span class="mi">200</span><span class="p">).</span><span class="nf">write</span> <span class="no">COLLAGE</span><span class="p">.</span><span class="nf">jpg</span>
</code></pre></div></div>
<p>Resulting in smth like:</p>
<p><img src="/img/collage.jpg" alt="Collage" /></p>
Developer Omniboxes for Chrome2013-04-26T00:00:00+00:00https://rocket-science.ru/hacking/2013/04/26/developer-omniboxes-for-chrome<p>A couple of months ago I finally switched from Firefox to Chromium. As I mentioned before, I’m a keyboard addict. I mostly use the browser
to find some information over the internet. That’s not click-through pattern. I know what am I interested in and I know where am I to find it.
So, half of the time I was wasting on opening new browser tabs and entering the respective search engine. Firefox gave me an opportunity
to use the small search box on the right of the usual one (sorry for not naming it properly, I likely do not bother how it’s called.)
Chromium, on the other hand, has no such ability (or I was not able to find it at a glance.)</p>
<p>Past a week I found myself searching for a “developing chrome extension extended search.” That was how I came to
<a href="http://developer.chrome.com/extensions/omnibox.html">Omnibox</a> how-tos. It has actually an already existing and growing bundle of
<a href="https://code.google.com/p/developer-omniboxes-for-chrome/">search tools for developers</a>, including handy documentation lookup for
virtually speaking all the programming languages. E. g. to find a documentation on Ruby’s <code class="language-plaintext highlighter-rouge">Array</code> one does simply type in the
address bar: <code class="language-plaintext highlighter-rouge">rb Arr</code> and scrolls down the most suitable suggestion:</p>
<p><img src="/img/omnibox-ruby.png" alt="Search against Ruby documentation" /></p>
<p>The omnibox extension is so simple, that I immediately decided to have my own written. I chose <code class="language-plaintext highlighter-rouge">»</code> as the “keyword” for omnibox.
Depending on the input language, it suggests search against either different common engines (like youtube, wikipedia and bing,) or
against russian analogues (yandex, russian wiki, lingvo translation service.) For <code class="language-plaintext highlighter-rouge">» текст</code> it suggests the following engines:</p>
<p><img src="/img/omnibox-lingvo.png" alt="Search against Russian engines" /></p>
<p>The source code is rather simple and may be found at <a href="https://github.com/mudasobwa/searchisk">github</a> as usual.</p>
ABC for Fluent Speakers (NM Level)2013-04-06T00:00:00+00:00https://rocket-science.ru/fun/2013/04/06/abc-for-fluent-speakers-nm-level<p><img src="/img/ABC.png" alt="ABC for linguists" /></p>
YADR for Dummies2013-04-04T00:00:00+00:00https://rocket-science.ru/shell/2013/04/04/yadr-for-dummies<p>I spend a half of my life in the terminal window (another half is being wasted even sillier.)
A couple of years ago I switched to <code class="language-plaintext highlighter-rouge">zsh</code>, then I met <a href="https://github.com/robbyrussell/oh-my-zsh"><code class="language-plaintext highlighter-rouge">oh-my-zsh</code></a>,
I even wrote my own theme for it, with blackjack and <a href="/shell/2013/01/20/zsh-weird-right-prompt">battery charge indicator</a>.
From time to time I tuned the theme up, played with newly investigated vim tricks etc. I switched from <code class="language-plaintext highlighter-rouge">oh-my-zsh</code> to
<a href="https://github.com/sorin-ionescu/prezto"><code class="language-plaintext highlighter-rouge">prezto</code></a>, but I anyway felt myself a little bit deprived.</p>
<p>Accidentally I stumbled upon <a href="https://github.com/skwp/dotfiles"><code class="language-plaintext highlighter-rouge">YADR</code></a>. The project headline states
“YADR is the best vim, git, zsh plugins and the cleanest vimrc you’ve ever seen.” And you know what?—That’s true.
Try it youself and I swear you never decide to turn back to your crude homebred dotfiles.</p>
<p>An installation is as easy (you already have ruby installed, haven’t you?) as:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git clone https://github.com/skwp/dotfiles ~/.yadr
<span class="nb">cd</span> ~/.yadr <span class="o">&&</span> rake <span class="nb">install</span>
</code></pre></div></div>
<p>The only polish required (in my opinion) is the proper theme. So, here we go. I teached the pretty
<a href="https://gist.github.com/agnoster/3712874">Agnoster</a> theme to show proper right prompt with current gemset,
current ruby version etc. If you are not as to Ruby as me, you’ll find a better application for it.</p>
<p>To install the theme you’ll need:</p>
<ul>
<li>install a <a href="https://gist.github.com/1595572">Powerline-patched font</a> for the theme special symbols
to render correctly</li>
<li>grab the <a href="https://gist.github.com/mudasobwa/5308070">theme file</a> and put in into <code class="language-plaintext highlighter-rouge">~/.zsh.prompts/prompt_mudasobwa_setup</code></li>
<li>put the following three lines in the end of your <code class="language-plaintext highlighter-rouge">~/.zshrc</code> file:</li>
</ul>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nv">$ </span>autoload <span class="nt">-Uz</span> promptinit
<span class="nv">$ </span>promptinit
<span class="nv">$ </span>prompt mudasobwa
</code></pre></div></div>
<ul>
<li>restart <code class="language-plaintext highlighter-rouge">zsh</code></li>
</ul>
<p>The screenshots below show only the visual effectiveness of the theme; the cymes is in details. Everyone
is to be exalted about it’s power and scared kinda “why didn’t I switched to it yesterday.”</p>
<p><strong>Inside git branch:</strong></p>
<p><img src="/img/yadr-mudasobwa-git-prompt.png" alt="YADR prompt inside git branch" /></p>
<p><strong>With background jobs (the blue gear on the left) and non-zero exit status from the previously run command:</strong></p>
<p><img src="/img/yadr-mudasobwa-full-prompt.png" alt="YADR prompt with running process" /></p>
<p><strong>VIM status line:</strong></p>
<p><img src="/img/yadr-mudasobwa-vim-prompt.png" alt="YADR prompt for VIM" /></p>
Ruby 2.0 Refinements: Totally Useless Crap2013-03-06T00:00:00+00:00https://rocket-science.ru/hacking/2013/03/06/ruby-20-refinements-totally-useless-crap<p>There were so-called refinements introduced in Ruby 2.0. I was playing with them and now I’m totally cajoled.
Let me explain, what’s wrong with ’em and why I consider nobody actually wants to use them.</p>
<h4 id="the-main-declared-advantage-of-refines-is-that-they-are-not-global-scoped-bah">The main declared advantage of refines is that they are not global scoped. Bah.</h4>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nn">MyModule</span>
<span class="k">class</span> <span class="o">::</span><span class="no">String</span>
<span class="k">def</span> <span class="nf">my_locally_needed_func</span>
<span class="c1"># do smth </span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="c1"># here I need it</span>
<span class="nb">require</span> <span class="s1">'mymodule'</span>
<span class="s2">""</span><span class="p">.</span><span class="nf">my_locally_needed_func</span>
</code></pre></div></div>
<p>is isolated not worse.</p>
<h4 id="refinements-do-not-support-class-methods-bah">Refinements do not support class methods. Bah.</h4>
<p>Of course they do through a hack (remember, everything is an object:)</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nn">VoidRefinements</span>
<span class="n">refine</span> <span class="no">String</span> <span class="k">do</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">singleton_method_for_string_class</span>
<span class="nb">puts</span> <span class="s2">"inside singleton_method_for_string_class"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">module</span> <span class="nn">VoidRefinementsOK</span>
<span class="n">refine</span> <span class="no">Class</span> <span class="k">do</span>
<span class="k">def</span> <span class="nf">singleton_method_for_string_class</span>
<span class="n">err_msg</span> <span class="o">=</span> <span class="s2">"NoMethodError: undefined method"</span> <span class="o">+</span> <span class="p">\</span>
<span class="s2">"‘</span><span class="si">#{</span><span class="n">__method__</span><span class="si">}</span><span class="s2">’ for ‘</span><span class="si">#{</span><span class="nb">self</span><span class="si">}</span><span class="s2">:</span><span class="si">#{</span><span class="nb">self</span><span class="p">.</span><span class="nf">class</span><span class="si">}</span><span class="s2">’"</span>
<span class="k">raise</span> <span class="no">NoMethodError</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">err_msg</span><span class="p">)</span> <span class="k">unless</span> <span class="no">String</span> <span class="o">==</span> <span class="nb">self</span>
<span class="nb">puts</span> <span class="s2">"inside proper singleton_method_for_string_class"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="n">using</span> <span class="no">VoidRefinements</span>
<span class="no">String</span><span class="p">.</span><span class="nf">singleton_method_for_string_class</span> <span class="k">rescue</span> <span class="nb">puts</span> <span class="vg">$!</span>
<span class="c1"># ⇒ undefined method `singleton_method_for_string_class' for String:Class</span>
<span class="n">using</span> <span class="no">VoidRefinementsOK</span>
<span class="no">String</span><span class="p">.</span><span class="nf">singleton_method_for_string_class</span> <span class="k">rescue</span> <span class="nb">puts</span> <span class="vg">$!</span>
<span class="c1"># ⇒ inside proper singleton_method_for_string_class</span>
</code></pre></div></div>
<p>The latter is not even resulting in performance penalties, since nobody would call <code class="language-plaintext highlighter-rouge">Fixnum.substr</code> on purpose.</p>
<h4 id="refinements-are-executed-through-eval">Refinements are executed through eval.</h4>
<p><code class="language-plaintext highlighter-rouge">refine</code> is not a keyword. Bah. (well, “bah!” again.)</p>
<p>Plus I have had some weird unpredicted errors with non-ascii method names in refinements. But that does actually
make already no sense after all.</p>
<p>Am I missing smth or everyone sees no advantages in the newly introduced feature?</p>
Ruby Shorthand to Yield Within Blocks2013-03-03T00:00:00+00:00https://rocket-science.ru/hacking/2013/03/03/ruby-shorthand-to-yield-within-blocks<p>I’m definitely a <em>lambda-guy</em>. I’m addicted to simplicity, beauty and intelligibility of closures.
He who claims there is syntax clearer than <code class="language-plaintext highlighter-rouge">arr.each { |e| … }</code>, let him throw the first stone at me.
After all yielding is much more of human nature than jumping.</p>
<p>However there are two cases when <code class="language-plaintext highlighter-rouge">yield</code> seems to be hardly used. First is the <code class="language-plaintext highlighter-rouge">&</code>-shorthand to method
within another codeblock:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">uglify</span> <span class="n">items</span>
<span class="k">raise</span> <span class="no">ArgumentException</span><span class="p">.</span><span class="nf">new</span> <span class="k">unless</span> <span class="n">items</span><span class="p">.</span><span class="nf">respond_to?</span> <span class="ss">:each</span>
<span class="n">items</span><span class="p">.</span><span class="nf">each</span> <span class="p">{</span> <span class="o">|</span><span class="n">e</span><span class="o">|</span> <span class="k">yield</span> <span class="n">e</span> <span class="p">}</span>
<span class="k">end</span>
</code></pre></div></div>
<p>If we try to shorten the <code class="language-plaintext highlighter-rouge">|e| yield e</code> clause with <code class="language-plaintext highlighter-rouge">&</code>-notation we’ll gain no success, since <code class="language-plaintext highlighter-rouge">yield</code> is
a keyword rather than method of a very superclass (like <code class="language-plaintext highlighter-rouge">Object</code> or <code class="language-plaintext highlighter-rouge">Kernel</code>.) But who cares?</p>
<p>Well, I do. Because there is a second, much more vital situation when <code class="language-plaintext highlighter-rouge">yield</code> sucks. It is re-passing a
codeblock to a subsequent method:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">cleanup</span>
<span class="c1"># do some cleanup if block_given?</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">evaluate_lua</span>
<span class="n">cleanup</span> <span class="k">yield</span> <span class="k">if</span> <span class="nb">block_given?</span>
<span class="c1"># do actual evaluate</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The latter will throw an odd error <code class="language-plaintext highlighter-rouge">ArgumentError: wrong number of arguments (1 for 0)</code>.
The <code class="language-plaintext highlighter-rouge">cleanup &yield</code> neither works, if one were curious.</p>
<p>What’s the problem to use explicit <code class="language-plaintext highlighter-rouge">&cb</code> param here? There are two as usual. First is
an aesthetics, which matters a lot in Ruby. The last and not the least is that instantiating
a new <code class="language-plaintext highlighter-rouge">Proc</code> object leads to a surprisingly heavy performance penalty (≈ five times slower.)
Happily, there is a not wide-known <a href="http://www.ruby-doc.org/core-2.0/Proc.html#method-c-new">feature</a>
of <code class="language-plaintext highlighter-rouge">Proc.new</code> constructor. Being called <em>without a block within a method with an attached block</em>, it
<em>converts</em> that block to the <code class="language-plaintext highlighter-rouge">Proc</code> object. It means that both</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">uglify</span> <span class="n">items</span>
<span class="k">raise</span> <span class="no">ArgumentException</span><span class="p">.</span><span class="nf">new</span> <span class="k">unless</span> <span class="n">items</span><span class="p">.</span><span class="nf">respond_to?</span> <span class="ss">:each</span>
<span class="n">items</span><span class="p">.</span><span class="nf">each</span><span class="p">(</span><span class="o">&</span><span class="no">Proc</span><span class="p">.</span><span class="nf">new</span><span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>
<p>and</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">cleanup</span>
<span class="k">if</span> <span class="nb">block_given?</span>
<span class="c1"># do some cleanup if block_given?</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">evaluate_lua</span>
<span class="n">cleanup</span> <span class="o">&</span><span class="no">Proc</span><span class="p">.</span><span class="nf">new</span>
<span class="c1"># do actual evaluate</span>
<span class="k">end</span>
</code></pre></div></div>
<p>will work as expected without extra performance penalty.</p>
Ruby Predefined Globals2013-02-26T00:00:00+00:00https://rocket-science.ru/tips/2013/02/26/ruby-predefined-globals<h2 id="below-is-the-summary-table-for-the-predefined-globals-in-ruby">Below is the summary table for the predefined globals in ruby.</h2>
<table class="table table-bordered code">
<tbody>
<tr>
<td><a href="#bang" class="btn"><strong>$!</strong></a></td>
<td><a href="#plus" class="btn"><strong>$+</strong></a></td>
<td><a href="#minus-w" class="btn"><strong>$-W</strong></a></td>
<td><a href="#minus-v-small" class="btn"><strong>$-v</strong></a></td>
<td><a href="#digit" class="btn"><strong>$2</strong></a></td>
<td><a href="#digit" class="btn"><strong>$8</strong></a></td>
<td><a href="#greater" class="btn"><strong>$></strong></a></td>
</tr>
<tr>
<td><a href="#double-quotes" class="btn"><strong>$"</strong></a></td>
<td><a href="#comma" class="btn"><strong>$,</strong></a></td>
<td><a href="#minus-a-small" class="btn"><strong>$-a</strong></a></td>
<td><a href="#minus-w-small" class="btn"><strong>$-w</strong></a></td>
<td><a href="#digit" class="btn"><strong>$3</strong></a></td>
<td><a href="#digit" class="btn"><strong>$9</strong></a></td>
<td><a href="#qmark" class="btn"><strong>$?</strong></a></td>
</tr>
<tr>
<td><a href="#dollar" class="btn"><strong>$$</strong></a></td>
<td><a href="#minus-zero" class="btn"><strong>$-0</strong></a></td>
<td><a href="#minus-d-small" class="btn"><strong>$-d</strong></a></td>
<td><a href="#dot" class="btn"><strong>$.</strong></a></td>
<td><a href="#digit" class="btn"><strong>$4</strong></a></td>
<td><a href="#colon" class="btn"><strong>$:</strong></a></td>
<td><a href="#at" class="btn"><strong>$@</strong></a></td>
</tr>
<tr>
<td><a href="#ampersand" class="btn"><strong>$&</strong></a></td>
<td><a href="#minus-f" class="btn"><strong>$-F</strong></a></td>
<td><a href="#minus-i-small" class="btn"><strong>$-i</strong></a></td>
<td><a href="#slash" class="btn"><strong>$/</strong></a></td>
<td><a href="#digit" class="btn"><strong>$5</strong></a></td>
<td><a href="#semicolon" class="btn"><strong>$;</strong></a></td>
<td><a href="#backslash" class="btn"><strong>$\</strong></a></td>
</tr>
<tr>
<td><a href="#end-quote" class="btn"><strong>$'</strong></a></td>
<td><a href="#minus-i" class="btn"><strong>$-I</strong></a></td>
<td><a href="#minus-l-small" class="btn"><strong>$-l</strong></a></td>
<td><a href="#digit" class="btn"><strong>$0</strong></a></td>
<td><a href="#digit" class="btn"><strong>$6</strong></a></td>
<td><a href="#less" class="btn"><strong>$<</strong></a></td>
<td><a href="#underscore" class="btn"><strong>$_</strong></a></td>
</tr>
<tr>
<td><a href="#splat" class="btn"><strong>$*</strong></a></td>
<td><a href="#minus-k" class="btn"><strong>$-K</strong></a></td>
<td><a href="#minus-p-small" class="btn"><strong>$-p</strong></a></td>
<td><a href="#digit" class="btn"><strong>$1</strong></a></td>
<td><a href="#digit" class="btn"><strong>$7</strong></a></td>
<td><a href="#tilde" class="btn"><strong>$~</strong></a></td>
<td><a href="#backtick" class="btn"><strong>$`</strong></a></td>
</tr>
</tbody>
</table>
<div class="table-wrapper">
<table class="table">
<thead><tr><th>Variable</th><th>Description</th></tr></thead>
<tbody>
<tr>
<td class="tooltipable">
<a id="bang"></a>
<code class="btn btn-large disabled"><strong>$!</strong></code>
<span class="code">Exception === $! unless $!.nil?</span>
</td>
<td>
<p><span class="label label-important">read-only</span> <span class="label label-info">thread-local</span> <strong>The last exception, that was <em>thrown and rescued</em> in the current context.</strong><br />
Locals to the <code>rescue</code> clause.</p>
```ruby
> 0 / 0 rescue $!
# ⇒ <ZeroDivisionError: divided by 0>
> begin
> 0 / 0
> rescue
> $!
> end
# ⇒ <ZeroDivisionError: divided by 0>
> 0 / 0
# ZeroDivisionError: divided by 0
# from (pry):67:in `/'
> $!
# ⇒ nil
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="double-quotes"></a>
<code class="btn btn-large disabled"><strong>$"</strong></code>
<span class="code">Array === $"</span>
</td>
<td>
<p><span class="label label-important">read-only</span> <strong>Array of strings,
containing absolute paths to loaded files.</strong><br />
Files, loaded within both <code>Kernel’s</code> <code>load</code>,
<code>require</code>, <code>require_relative</code> directives <strong><em>and</em></strong> platform-specific
<a href="http://www.ruby-doc.org/stdlib-1.9.3/libdoc/dl/rdoc/DL.html">DL</a>/<a href="http://www.ruby-doc.org/stdlib-1.9.3/libdoc/fiddle/rdoc/Fiddle.html">Fiddle</a>,
are shown in the list.
</p>
```ruby
> $"
# ⇒ [
# [ 0] "enumerator.so",
# [ 1] "/…/lib/ruby/2.0.0/x86_64-linux/enc/encdb.so",
# [ 2] "/…/lib/ruby/2.0.0/x86_64-linux/enc/trans/transdb.so",
# [ 3] "/…/lib/ruby/site_ruby/2.0.0/rubygems/defaults.rb",
# [ 4] "/…/lib/ruby/2.0.0/x86_64-linux/rbconfig.rb",
# …
# [227] "/…/lib/ruby/2.0.0/dl.rb"
# ]
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="dollar"></a>
<code class="btn btn-large disabled"><strong>$$</strong></code>
<span class="code">Fixnum === $$</span>
</td>
<td>
<p><span class="label label-important">read-only</span> <strong>Current process ID.</strong><br /></p>
```ruby
> $"
# ⇒ 8603
> Process.pid
# ⇒ 8603
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="ampersand"></a>
<code class="btn btn-large disabled"><strong>$&</strong></code>
<span class="code">String === $& unless $&.nil?</span>
</td>
<td>
<p><span class="label label-important">read-only</span> <span class="label label-info">thread-local</span> <strong>The matched string from the previous successful pattern match.</strong><br />
Locals to the pattern match scope.</p>
```ruby
> "foo bar baz".match /foo|bar|baz/
# ⇒ <MatchData "foo">
> $&
# ⇒ "foo"
> "foo bar baz".gsub /foo|bar|baz/, "ggg"
# ⇒ "ggg ggg ggg"
> $&
# ⇒ "baz"
> "foo bar baz".match /foobarbaz/
# ⇒ nil
> $&
# ⇒ nil
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="end-quote"></a>
<code class="btn btn-large disabled"><strong>$'</strong></code>
<span class="code">String === $' unless $'.nil?</span>
</td>
<td>
<p><span class="label label-important">read-only</span> <span class="label label-info">thread-local</span> <strong>The rest of the string <em>after the matched</em> substring in the previous successful pattern match.</strong><br />
Locals to the pattern match scope.</p>
```ruby
> "foo bar baz".match /foo|bar|baz/
# ⇒ <MatchData "foo">
> $'
# ⇒ " bar baz"
> "foo bar baz".gsub /foo|bar|baz/, 'ggg'
# ⇒ "ggg ggg ggg"
> $'
# ⇒ ""
> "foo bar baz".match /foobarbaz/
# ⇒ nil
> $'
# ⇒ nil
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="splat"></a>
<code class="btn btn-large disabled"><strong>$*</strong></code>
<span class="code">Array === $*</span>
</td>
<td>
<p><span class="label label-important">read-only</span> <strong>The command line arguments passed to the currently executed ruby script.</strong><br />
An alias for <code>ARGV</code>.</p>
```ruby
~ pry --simple-prompt
> $*
# ⇒ [
# [0] "--simple-prompt"
# ]
> ARGV
# ⇒ [
# [0] "--simple-prompt"
# ]
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="plus"></a>
<code class="btn btn-large disabled"><strong>$+</strong></code>
<span class="code">String === $+ unless $+.nil?</span>
</td>
<td>
<p><span class="label label-important">read-only</span> <span class="label label-info">thread-local</span> <strong>The last <em>captured</em> match from the previous successful pattern match.</strong><br />
Locals to the pattern match scope. Contains <code>nil</code> if there was no capture (even while match was successful.)</p>
```ruby
> "foo bar baz".match /(foo) (bar) baz/
# ⇒ <MatchData "foo bar baz" 1:"foo" 2:"bar">
> $+
# ⇒ "bar"
> "foo bar baz".scan (/a(\S)/) { |m| puts m }
# r
# z
# ⇒ "foo bar baz"
> $+
# ⇒ "z"
> "foo bar baz".match /foo bar baz/
# ⇒ <MatchData "foo bar baz">
> $+
# ⇒ nil
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="comma"></a>
<code class="btn btn-large disabled"><strong>$,</strong></code>
<span class="code">String === $, unless $,.nil?</span>
</td>
<td>
<p><span class="label label-success">read-write</span> <strong>Separator for both
<code>Kernel#print</code> and <code>Array#join</code>.</strong><br />
Defaults to nil.</p>
```ruby
> print "foo", "bar", "baz"
# foobarbaz⇒ nil
> %w[foo bar baz].join
# ⇒ "foobarbaz"
> $,='%'
# ⇒ "%"
> print "foo", "bar", "baz"
# foo%bar%baz⇒ nil
> %w[foo bar baz].join
# ⇒ "foo%bar%baz"
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="minus-zero"></a>
<code class="btn btn-large disabled"><strong>$-0</strong></code><br /><br />
<a id="slash"></a>
<code class="btn btn-large disabled"><strong>$/</strong></code>
<span class="code">String === $-0 unless $-0.nil?<br />String === $/ unless $/.nil?</span>
</td>
<td>
<p><span class="label label-success">read-write</span> <strong>Input record separator.</strong><br />
Defaults to <code>\n</code>. May be set with the <code>-0</code> command line parameter (as octal value.)</p>
```ruby
> $-0='%'
# ⇒ "%"
> gets
f
o
o
%
# ⇒ "f\no\no\n%"
~ ruby -045 /…/bin/pry
> gets
f
o
o
%
# ⇒ "f\no\no\n%"
>
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="minus-f"></a>
<code class="btn btn-large disabled"><strong>$-F</strong></code><br /><br />
<a id="semicolon"></a>
<code class="btn btn-large disabled"><strong>$;</strong></code>
<span class="code">String === $-F unless $-F.nil?<br />String === $: unless $:.nil?</span>
</td>
<td>
<p><span class="label label-success">read-write</span> <strong>Default field separator for <code>String#split</code>.</strong><br />
Defaults to <code>nil</code>. May be set with the <code>-F</code> command line parameter.</p>
```ruby
> $-F
# ⇒ nil
> "foo bar baz".split
# ⇒ [
# [0] "foo",
# [1] "bar",
# [2] "baz"
# ]
> $-F='%'
# ⇒ "%"
> "foo bar baz".split
# ⇒ [
# [0] "foo bar baz"
# ]
> "foo%bar%baz".split
# ⇒ [
# [0] "foo",
# [1] "bar",
# [2] "baz"
# ]
> $-F == $;
# ⇒ true
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="minus-i"></a>
<code class="btn btn-large disabled"><strong>$-I</strong></code><br /><br />
<a id="colon"></a>
<code class="btn btn-large disabled"><strong>$:</strong></code>
<span class="code">Array === $-I<br />Array === $:</span>
</td>
<td>
<p><span class="label label-important">read-only</span> <strong>Array of include paths.</strong><br />
Absolute paths to be searched by <code>Kernel.load</code> and/or <code>Kernel.require</code>.</p>
```ruby
> $-I
# ⇒ [
# [ 0] "/…/gems/rubygems-bundler-1.1.0/lib",
# [ 1] "/…/gems/bundler-1.2.3/lib",
# [ 2] "/…/gems/coderay-1.0.8/lib",
# [ 3] "/…/gems/slop-3.4.3/lib",
# …
# [14] "/…/lib/ruby/2.0.0/x86_64-linux"
# ]
> $-I == $:
# ⇒ true
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="minus-k"></a>
<code class="btn btn-large disabled"><strong>$-K</strong></code>
<span class="code">String === $-K unless $-K.nil?</span>
</td>
<td>
<p><span class="label label-inverse">obsolete</span> <span class="label label-success">read-write</span> <strong>Determines the encoding to be used whilst parsing <code>.rb</code> files.</strong><br />
One should avoid using that variable since <code>ruby-1.9</code>.</p>
```ruby
> $-K
# (pry):13: warning: variable $KCODE is no longer effective
# ⇒ nil
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="minus-w"></a>
<code class="btn btn-large disabled"><strong>$-W</strong></code>
<span class="code">Fixnum === $-W</span>
</td>
<td>
<p><span class="label label-important">read-only</span> <strong>Current verbosity level.</strong><br />
Defaults to <code>1</code>. May be set to <code>0</code> with the <code>-W0</code> command line arg
and to <code>2</code> with one of <code>-w</code>, <code>-v</code>, or <code>--verbose</code> switches.</p>
```ruby
> $-W
# ⇒ 1
> exit
~ ruby -v /…/bin/pry
# ruby 2.0.0dev (2012-12-01 trunk 38126) [x86_64-linux]
# /…/gems/method_source-0.8.1/lib/method_source/code_helpers.rb:38: warning: assigned but unused variable - e2
# …
# /…/gems/pry-0.9.11.4/lib/pry/pry_class.rb:446: warning: instance variable @critical_section not initialized
> $-W
# /…/gems/awesome_print-1.1.0/lib/awesome_print/inspector.rb:114: warning: instance variable @force_colors not initialized
# ⇒ 2
> exit
~ ruby -W0 /…/bin/pry
> $-W
# ⇒ 0
>
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="minus-a-small"></a>
<code class="btn btn-large disabled"><strong>$-a</strong></code>
<span class="code">∈ [true, false]</span>
</td>
<td>
<p><span class="label label-important">read-only</span> <strong>Denotes whether the auto-split mode was enabled with <code>-a</code> command line argument.</strong><br />
<code>-a</code> switch makes sense when used together with either <a href="#minus-p-small"><code>-p</code></a>
or <a href="#minus-n-small"><code>-n</code></a> args.
In auto-split mode, Ruby executes <code>$F = <a href="#underscore">$_</a>.split</code> at the beginning of each loop.</p>
```ruby
~ cat > test-a-switch.rb 〈〈EOF
# heredoc> # encoding: utf-8
# heredoc> puts "LINE=[#{\$_.strip}]"
# heredoc> puts "\$F=#{\$F}"
# heredoc> EOF
~ ruby -a -n test-a-switch.rb test-a-switch.rb
# LINE=[# encoding: utf-8]
# $F=["#", "encoding:", "utf-8"]
# LINE=[puts "LINE=#{$_.strip}"]
# $F=["puts", "\"LINE=\#{$_.strip},"]
# LINE=[puts "$F=#{$F}"]
# $F=["puts", "$F=\#{$F}\""]
~ ruby -n test-a-switch.rb test-a-switch.rb
# LINE=# encoding: utf-8
# $F=
# LINE=puts "LINE=#{$_.strip}"
# $F=
# LINE=puts "$F=#{$F}"
# $F=
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="minus-d-small"></a>
<code class="btn btn-large disabled"><strong>$-d</strong></code>
<span class="code">∈ [true, false]</span>
</td>
<td>
<p><span class="label label-important">read-only</span> <strong>Denotes whether the debug mode was enabled with <code>-d</code> switch.</strong><br /></p>
```ruby
~ cat > test-d-switch.rb 〈〈EOF
# heredoc> puts "DEBUG MODE: #{\$-d}"
# heredoc> EOF
~ ruby -d test-d-switch.rb
# Exception `LoadError' at /…/lib/ruby/site_ruby/2.0.0/rubygems.rb:1264 - cannot load such file -- rubygems/defaults/operating_system
# Exception `LoadError' at /…/lib/ruby/site_ruby/2.0.0/rubygems.rb:1273 - cannot load such file -- rubygems/defaults/ruby
# DEBUG MODE: true
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="minus-i-small"></a>
<code class="btn btn-large disabled"><strong>$-i</strong></code>
<span class="code">∈ [true, false]</span>
</td>
<td>
<p><span class="label label-important">read-only</span> <strong>Value of the <code>-i</code> command line argument, if given.</strong><br />
Defaults to <code>nil</code>. When provided, this argument enables inplace-edit mode; the parameter specifies an extension of
backup file to be created.</p>
```ruby
~ cat > test-i-switch.rb << EOF
\$_.upcase!
puts "i-switch: #{\$-i}"
EOF
~ ruby -p -i.bak test-i-switch.rb test-i-switch.rb
~ cat test-i-switch.rb
# i-switch: .bak
# $_.UPCASE!
# i-switch: .bak
# PUTS "I-SWITCH: #{$-I}"
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="minus-l-small"></a>
<code class="btn btn-large disabled"><strong>$-l</strong></code>
<span class="code">∈ [true, false]</span>
</td>
<td>
<p><span class="label label-important">read-only</span> <strong>Denotes whether the automatic
line-ending processing mode was enabled with <code>-l</code> switch.</strong><br />
Automatic line-endings sets <a href="#backslash"><code>$\</code></a> to
the value of <a href="#slash"><code>$/</code></a>, and (when used with either <a href="#minus-p-small"><code>-p</code></a>
or <a href="#minus-n-small"><code>-n</code></a> args) chops every line read using <code>chop!</code>.</p>
```ruby
# Could not invent a meaningful example :-(
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="minus-p-small"></a>
<a id="minus-n-small"></a>
<code class="btn btn-large disabled"><strong>$-p</strong></code>
<span class="code">∈ [true, false]</span>
</td>
<td>
<p><span class="label label-important">read-only</span> <strong>Denotes whether the “gets loop around” mode was enabled with <code>-p</code> switch.</strong><br />
<code>-p</code> switch acts mostly like <code>-n</code> sibling with one exception: it prints the value
of variable <code>$_</code> at the each end of the loop.<br />For some unknown reason there is no internal global
to get aknowledged whether the <code>-n</code> command line switch was given. In case it were, Ruby is to assume
the following loop around your script, which provides an iteration over filename arguments somewhat like
<code>sed -n</code> and <code>awk</code> do.</p>
```ruby
while gets
…
end
```
<p>An example of usage follows:</p>
```ruby
~ echo "foo bar baz" | ruby -p -e '$_.tr! "o-z", "O-Z"'
# ⇒ fOO baR baZ
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="minus-v-small"></a>
<code class="btn btn-large disabled"><strong>$-v</strong></code><br /><br />
<a id="minus-w-small"></a>
<code class="btn btn-large disabled"><strong>$-w</strong></code>
<span class="code">∈ [true, false]</span>
</td>
<td>
<p><span class="label label-important">read-only</span> <strong>Denotes whether the verbose mode
was enabled with either <code>-v</code> or <code>-w</code> switch.</strong><br /></p>
```ruby
~ cat > test-v-switch.rb 〈〈EOF
# heredoc> puts "VERBOSE MODE: #{\$-v}"
# heredoc> EOF
~ ruby -v test-v-switch.rb
# ruby 2.0.0dev (2012-12-01 trunk 38126) [x86_64-linux]
# VERBOSE MODE: true
~
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="dot"></a>
<code class="btn btn-large disabled"><strong>$.</strong></code>
<span class="code">Fixnum === $.</span>
</td>
<td>
<p><span class="label label-important">read-only</span> <strong>Number of the last line read from the current input file <a href="#less"><code>ARGF</code></a>.</strong><br />
<a href="http://www.ruby-doc.org/core-2.0/ARGF.html">ARGF</a> is a stream designed to use in scripts that process files given as command-line arguments or passed in via <code>STDIN</code>.</p>
```ruby
ARGV.replace ["file1"] # file1 ≡ 'a\nb\nc\nd'
ARGF.readlines # Returns the contents of file1 as an Array
$.
# ⇒ 4
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="digit"></a>
<code class="btn btn-large disabled"><strong>$0…$9</strong></code>
<span class="code">String === $0 unless $0.nil?<br />
String === $1 unless $1.nil?<br />
String === $2 unless $2.nil?<br />
String === $3 unless $3.nil?<br />
String === $4 unless $4.nil?<br />
String === $5 unless $5.nil?<br />
String === $6 unless $6.nil?<br />
String === $7 unless $7.nil?<br />
String === $8 unless $8.nil?<br />
String === $9 unless $9.nil?</span>
</td>
<td>
<p><span class="label label-important">read-only</span> <strong>The <code>N-th</code> capture of the previous successful pattern match.</strong><br />
Defaults to <code>nil</code> if the match failed or if <code>N</code> is greater than an amount of captured groups.</p>
```ruby
> "foo bar baz".match(/(foo) (bar)/)
# ⇒ <MatchData:0x18cb9f4>
> $1
# ⇒ "foo"
> $2
# ⇒ "bar"
> $3
# ⇒ nil
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="less"></a>
<code class="btn btn-large disabled"><strong>$<</strong></code>
<span class="code">ARGF === $<</span>
</td>
<td>
<p><span class="label label-important">read-only</span> <strong>Read‐only alias for ARGF.</strong><br /></p>
```ruby
> $<.class
# ⇒ ARGF.class < Object
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="equals"></a>
<code class="btn btn-large disabled"><strong>$=</strong></code>
<span class="code">∈ [true, false]</span>
</td>
<td>
<p><span class="label label-inverse">obsolete</span> <br /></p>
```ruby
> $=
# (pry):23: warning: variable $= is no longer effective
# ⇒ false
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="greater"></a>
<code class="btn btn-large disabled"><strong>$></strong></code>
<span class="code">IO === $></span>
</td>
<td>
<p><span class="label label-success">read-write</span> <strong>Standard output stream.</strong><br /></p>
```ruby
> $> = File.new('/tmp/foo', 'w')
# ⇒ <File:/tmp/foo>
# -rw-r--r-- 1 am users 0 Feb 27 12:41 /tmp/foo
> puts "bar baz"
# ⇒ nil
> exit
~ cat /tmp/foo
# bar baz
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="qmark"></a>
<code class="btn btn-large disabled"><strong>$?</strong></code>
<span class="code">Process::Status === $? unless $?.nil?</span>
</td>
<td>
<p><span class="label label-important">read-only</span> <strong>Exit status of the last terminated process within current context.</strong><br /></p>
```ruby
> `ls FooBarBaz`
# ls: невозможно получить доступ к FooBarBaz: Нет такого файла или каталога
# ⇒ ""
> $?
# ⇒ <Process::Status: pid 31718 exit 2>
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="at"></a>
<code class="btn btn-large disabled"><strong>$@</strong></code>
<span class="code">Array === $@ unless $@.nil?</span>
</td>
<td>
<p><span class="label label-important">read-only</span> <span class="label label-info">thread-local</span> <strong>Shorthand to <code>$!.backtrace</code></strong><br />
Locals to the <code>rescue</code> clause.</p>
```ruby
> 0 / 0 rescue $@
# ⇒ [
# [ 0] "(pry):7:in `/'",
# [ 1] "(pry):7:in `__pry__'",
# …
# [23] "/…/bin/ruby_noexec_wrapper:14:in `<main>'"
# ]
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="backslash"></a>
<code class="btn btn-large disabled"><strong>$\</strong></code>
<span class="code">String === $\ unless $\.nil?</span>
</td>
<td>
<p><span class="label label-success">read-write</span> <strong>Appended to <code>Kernel.print</code> output.</strong><br />
Defaults to <code>nil</code>, or to <a href="#slash"><code>$/</code></a> if the <code>-l</code> switch was given.</p>
```ruby
> $\='%'
# ⇒ "%"
> print 'foo', 'bar', 'baz'
# ⇒ %foobarbaz%=> nil
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="underscore"></a>
<code class="btn btn-large disabled"><strong>$_</strong></code>
<span class="code">String === $_ unless $_.nil?</span>
</td>
<td>
<p><span class="label label-important">read-only</span> <span class="label label-info">thread-local</span> <strong>Last <code>String</code> read from <code>IO</code>
by one of <code>Kernel.gets</code>, <code>Kernel.readline</code> or siblings.</strong><br />
Widely used with <a href="#minus-p-small"><code>-p</code></a> and <a href="#minus-n-small"><code>-n</code></a> switches.</p>
```ruby
> gets
foo
# ⇒ "foo\n"
> $_
# ⇒ "foo\n"
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="backtick"></a>
<code class="btn btn-large disabled"><strong>$`</strong></code>
<span class="code">String === $` unless $`.nil?</span>
</td>
<td>
<p><span class="label label-important">read-only</span> <span class="label label-info">thread-local</span> <strong>The
rest of the string <em>before</em> the matched substring in the previous successful pattern match.</strong><br />
Locals to the pattern match scope.</p>
```ruby
> "foo bar baz".match /bar|baz/
# ⇒ <MatchData "bar">
> $`
# ⇒ "foo "
> "foo bar baz".gsub /foo|bar|baz/, 'ggg'
# ⇒ "ggg ggg ggg"
> $`
# ⇒ "foo bar "
> "foo bar baz".match /foobarbaz/
# ⇒ nil
> $`
# ⇒ nil
```
</td>
</tr>
<tr>
<td class="tooltipable">
<a id="tilde"></a>
<code class="btn btn-large disabled"><strong>$~</strong></code>
<span class="code">MatchData === $~ unless $~.nil?</span>
</td>
<td>
<p><span class="label label-important">read-only</span> <span class="label label-info">thread-local</span> <strong>The
<code>MatchData</code> from the previous successful pattern match.</strong><br /></p>
```ruby
> "abc12def34ghijklmno567pqrs".gsub (/\d+/) { |m| p $~ }
# ⇒ <MatchData "12">
# ⇒ <MatchData "34">
# ⇒ <MatchData "567">
```
</td>
</tr>
</tbody>
</table>
</div>
Credits to [Jim Neath](http://jimneath.org/2010/01/04/cryptic-ruby-global-variables-and-their-meanings.html) and
[runpaint](http://ruby.runpaint.org/globals), some examples came from these posts.
</main></td></tr></tbody></table></div>
Ruby Blocks: Do-end vs. Braces2013-02-21T00:00:00+00:00https://rocket-science.ru/tips/2013/02/21/ruby-blocks-do-end-vs-braces<p>Ruby tries to add a spoon of sugar to each damn line of your code any time she thinks it to be possible. That’s nice.
Sometimes she became meddlesome. That used to drive me bonkers. A newbie is to be that rara avis, to understand that
there is actually a huge difference between braces and <code class="language-plaintext highlighter-rouge">do-end</code> constructs to define a block.</p>
<p>Well, everybody may spend a life successfully writing tons lines of ruby code and never get stuck in weird exceptions,
coming from 3rd party libs, which in case were induced by negligent mess between <code class="language-plaintext highlighter-rouge">{}</code> and <code class="language-plaintext highlighter-rouge">do-end</code> in domestic code.
Let’s digress from our dramatic story and take a look at the following snippet. Somebody was set bringing benchmarks
to an application running at a snail’s pace. Well, the task looks straightforward. Install <code class="language-plaintext highlighter-rouge">benchmark</code> gem, read the
<a href="http://ruby-doc.org/stdlib-1.9.3/libdoc/benchmark/rdoc/Benchmark.html">documentation</a>, try an example:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">require</span> <span class="s1">'benchmark'</span>
<span class="nb">puts</span> <span class="no">Benchmark</span><span class="p">.</span><span class="nf">measure</span> <span class="p">{</span> <span class="s2">"a"</span><span class="o">*</span><span class="mi">1_000_000</span> <span class="p">}</span>
</code></pre></div></div>
<p>Works fine! Let’s try smth more complicated:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">require</span> <span class="s1">'benchmark'</span>
<span class="nb">puts</span> <span class="no">Benchmark</span><span class="p">.</span><span class="nf">measure</span> <span class="k">do</span>
<span class="k">while</span> <span class="kp">true</span>
<span class="nb">print</span> <span class="s1">'a'</span>
<span class="k">break</span> <span class="k">if</span> <span class="no">Random</span><span class="p">.</span><span class="nf">rand</span><span class="p">(</span><span class="mi">100</span><span class="p">)</span> <span class="o">===</span> <span class="mi">1</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Ooooups…</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">irb</span><span class="p">:</span><span class="mo">010</span> <span class="o">></span>
<span class="c1"># LocalJumpError: no block given (yield)</span>
<span class="c1"># from IRRELEVANT_PATH_TO_RVM/lib/ruby/2.0.0/benchmark.rb:281:in `measure'</span>
<span class="c1"># from (irb):9`</span>
</code></pre></div></div>
<p>WTF? That’s the syntax sugar, ruby mixed into your tea. That’s a time to take a look at an operator precedence table.
<code class="language-plaintext highlighter-rouge">{}</code> binds tighter than <code class="language-plaintext highlighter-rouge">do-end</code>. Furthermore, <code class="language-plaintext highlighter-rouge">do-end</code> clause has lowest precedence at all. In other words, the latter
codepiece is similar to:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">require</span> <span class="s1">'benchmark'</span>
<span class="p">(</span><span class="nb">puts</span> <span class="no">Benchmark</span><span class="p">.</span><span class="nf">measure</span><span class="p">)</span> <span class="k">do</span>
<span class="c1"># irrelevant code</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Surely, the aforementioned hitch never befuddled a guru, but there is a style guideline which may get rid of the
possibility even to stuck with this. Use braces <code class="language-plaintext highlighter-rouge">{}</code> when a block returnes a value. Use <code class="language-plaintext highlighter-rouge">do-end</code> clause otherwise.</p>
Traceroute to Episode IV2013-02-11T00:00:00+00:00https://rocket-science.ru/fun/2013/02/11/traceroute-to-episode-iv<p>Kinda eastern egg in DNS (gaze at the trace starting with 13<sup>th</sup> node.)</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>traceroute 216.81.59.173
</code></pre></div></div>
<p><img src="http://img-fotki.yandex.ru/get/5647/63225775.0/0_95d68_f19957fe_L.jpg" alt="Traceroute to 216.81.59.173" /></p>
Match or Not Die2013-02-11T00:00:00+00:00https://rocket-science.ru/hacking/2013/02/11/match-or-not-die<p>Let’s imagine we want to regexp a string and print the capitalized match out.
We start with one of the followings:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">str</span><span class="p">.</span><span class="nf">match</span><span class="p">(</span><span class="sr">/(regexp)/</span><span class="p">)[</span><span class="mi">1</span><span class="p">].</span><span class="nf">capitalize</span>
<span class="sr">/(regexp)/</span><span class="p">.</span><span class="nf">match</span><span class="p">(</span><span class="n">str</span><span class="p">)[</span><span class="mi">1</span><span class="p">].</span><span class="nf">capitalize</span>
</code></pre></div></div>
<p>This works fine on matched strings. Being called on the input with no matches
it indeed results in annoying:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># ⇒ NoMethodError: undefined method `[]' for nil:NilClass</span>
</code></pre></div></div>
<p>So we come to spaghetti <code class="language-plaintext highlighter-rouge">if-then</code> checks like:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">m</span> <span class="o">=</span> <span class="n">str</span><span class="p">.</span><span class="nf">match</span><span class="p">(</span><span class="sr">/(regexp)/</span><span class="p">)</span>
<span class="n">m1</span> <span class="o">=</span> <span class="n">m</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="k">if</span> <span class="n">m</span>
<span class="n">cap</span> <span class="o">=</span> <span class="n">m1</span><span class="p">.</span><span class="nf">cap</span> <span class="k">if</span> <span class="n">m1</span>
</code></pre></div></div>
<p>We got bogged down in checking while the only goal was to write a oneliner to
either capitalize match or gracefully keep silent. Happily, ruby has a not
wide well-known way to perform exactly the task we wanted. It’s the
<code class="language-plaintext highlighter-rouge">String#[regexp, fixnum]</code> method:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">(</span><span class="n">str</span><span class="p">.</span><span class="nf">match</span><span class="p">[</span><span class="sr">/(regexp)/</span><span class="p">,</span> <span class="mi">1</span><span class="p">]</span> <span class="o">||</span> <span class="s1">''</span><span class="p">).</span><span class="nf">capitalize</span>
</code></pre></div></div>
Oll Korrect Cartoon2013-02-11T00:00:00+00:00https://rocket-science.ru/fun/2013/02/11/all-correct-cartoon<p>Genious retrospection on “Cartoon of Atonement.”</p>
<blockquote><p><img src="http://www.newyorker.com/images/2012/10/01/p465/121001_cn-politically-correct_p465.jpg" alt="Please enjoy this culturally, ethnically, religiously and politically correct cartoon responsibly. Thank you. 〈blank frame〉" /></p>
<small>via <a href="http://www.newyorker.com/online/blogs/cartoonists/2012/09/cartoon-of-atonement.html">NewYorker</a></small></blockquote>
Delightful Logging2013-02-09T00:00:00+00:00https://rocket-science.ru/hacking/2013/02/09/delightful-logging<p>Many ruby project scaffolders (like <a href="https://github.com/dkastner/bueller">bueller</a> &Co.) produce a file tree
consisting of main <code class="language-plaintext highlighter-rouge">module</code> file (<code class="language-plaintext highlighter-rouge">MODULE_NAME.rb</code> needed to syntax-sugaring the project stuff requires,)
some additional garbage and a <code class="language-plaintext highlighter-rouge">lib/MODULE_NAME/version.rb</code> file. The latter looks like:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># encoding: utf-8</span>
<span class="k">module</span> <span class="nn">MyModule</span>
<span class="no">VERSION</span> <span class="o">=</span> <span class="s2">"0.0.1"</span>
<span class="k">end</span>
</code></pre></div></div>
<p>I get used to utilize this <code class="language-plaintext highlighter-rouge">version.rb</code> to define all the project-wide constants etc. If the project requires
config, there is a code to read <code class="language-plaintext highlighter-rouge">config.yml</code> placed, as well as other predefines.</p>
<p>Nowadays I rejoice in even better approach. There is mere usual but very effective way to make a <code class="language-plaintext highlighter-rouge">Logger</code> instance
(and others someone likely needs all over the project) accessible from anywhere around the code. To achieve
that I simply create a <code class="language-plaintext highlighter-rouge">module Kernel</code> section inside the <code class="language-plaintext highlighter-rouge">version.rb</code> and put there all the stuff I might need
everywhere:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># encoding: utf-8</span>
<span class="k">module</span> <span class="nn">MyModule</span>
<span class="no">VERSION</span> <span class="o">=</span> <span class="s2">"0.0.1"</span>
<span class="no">LOGGER_STREAM</span><span class="p">,</span> <span class="no">LOGGER_LEVEL</span> <span class="o">=</span> <span class="n">load_config</span> <span class="o">||</span> <span class="no">STDOUT</span><span class="p">,</span> <span class="no">Logger</span><span class="o">::</span><span class="no">INFO</span>
<span class="k">end</span>
<span class="k">module</span> <span class="nn">Kernel</span>
<span class="vc">@@logger</span> <span class="o">=</span> <span class="no">Logger</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="no">MyModule</span><span class="o">::</span><span class="no">LOGGER_STREAM</span><span class="p">)</span>
<span class="vc">@@logger</span><span class="p">.</span><span class="nf">level</span> <span class="o">=</span> <span class="no">MyModule</span><span class="o">::</span><span class="no">LOGGER_LEVEL</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The trick is that <code class="language-plaintext highlighter-rouge">Kernel</code> module <em>is being included within <code class="language-plaintext highlighter-rouge">Object</code> class</em> and since all the classes have <code class="language-plaintext highlighter-rouge">Object</code> as
their superclass, all of them obtain an access to <code class="language-plaintext highlighter-rouge">@@logger</code> class variable transparently. Now anywhere within the project
scope we may write:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="vc">@@logger</span><span class="p">.</span><span class="nf">info</span> <span class="s2">"Hello, I’m logger"</span>
</code></pre></div></div>
<p>yielding:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># ⇒ I, [2013-02-09T18:44:18.208111 #27511] INFO -- : Hello, I’m logger</span>
</code></pre></div></div>
Make Hash Element Access Painless2013-02-05T00:00:00+00:00https://rocket-science.ru/hacking/2013/02/05/make-hash-element-access-painless<p>We often need to deal with a data from very unreliable sources. Over unreliable networks. Using unreliable foreign libraries.</p>
<p>Typically, parsing XML or JSON returns a hash, array, or combination of them. These hashes and/or arrays are likely to be
partially fulfilled or even broken. Parsing through an invalid array leads to curly handling of all sorts of <code class="language-plaintext highlighter-rouge">Error</code>s and
unexpected nils. I like spaghetti if and only it’s paired with bolognese.</p>
<p>Actually, the behaviour I expect is quite straightforward: return value requsted if the key provided contains it,
or smth, having <code class="language-plaintext highlighter-rouge">smth.empty? === true</code> otherwise.</p>
<p>For example, I have a ramified hash and want to look it up current user’s rights:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">myhash</span><span class="p">[</span><span class="s1">'users'</span><span class="p">][</span><span class="vi">@current</span><span class="p">][</span><span class="s1">'policy'</span><span class="p">][</span><span class="s1">'groups'</span><span class="p">][</span><span class="s1">'admin'</span><span class="p">]</span>
</code></pre></div></div>
<p>The user may not being exist, policies might be not set yet, it surely may participate in no groups and, finally, <code class="language-plaintext highlighter-rouge">admin</code>
privilege may be either set to <code class="language-plaintext highlighter-rouge">false</code> or completely unset. And guess what? I want to write:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">show_admin_menu</span> <span class="p">\</span>
<span class="k">if</span> <span class="n">myhash</span><span class="p">[</span><span class="s1">'users'</span><span class="p">][</span><span class="vi">@current</span><span class="p">][</span><span class="s1">'policy'</span><span class="p">][</span><span class="s1">'groups'</span><span class="p">][</span><span class="s1">'admin'</span><span class="p">]</span> <span class="o">===</span> <span class="s1">'granted'</span>
</code></pre></div></div>
<p>Occasionally, there is a way to hack around. To be as accurate as possible, we will not interfere the standard classes
like <code class="language-plaintext highlighter-rouge">Hash</code> and <code class="language-plaintext highlighter-rouge">Array</code>. Instead of that we will produce a static function, that will inject the proper behaviour
(return empty Hash/Array rather than throw an exception) in the instances. The complete code is shown below:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">weaken_checks_for_brackets_accessor</span> <span class="n">inst</span>
<span class="n">inst</span><span class="p">.</span><span class="nf">instance_variable_set</span><span class="p">(</span><span class="ss">:@original_get_element_method</span><span class="p">,</span> <span class="n">inst</span><span class="p">.</span><span class="nf">method</span><span class="p">(</span><span class="ss">:[]</span><span class="p">))</span> <span class="p">\</span>
<span class="k">unless</span> <span class="n">inst</span><span class="p">.</span><span class="nf">instance_variable_get</span><span class="p">(</span><span class="ss">:@original_get_element_method</span><span class="p">)</span>
<span class="n">singleton_class</span> <span class="o">=</span> <span class="k">class</span> <span class="o"><<</span> <span class="n">inst</span><span class="p">;</span> <span class="nb">self</span><span class="p">;</span> <span class="k">end</span>
<span class="n">singleton_class</span><span class="p">.</span><span class="nf">send</span><span class="p">(</span><span class="ss">:define_method</span><span class="p">,</span> <span class="ss">:[]</span><span class="p">)</span> <span class="k">do</span> <span class="o">|*</span><span class="n">keys</span><span class="o">|</span>
<span class="k">begin</span>
<span class="n">res</span> <span class="o">=</span> <span class="p">(</span><span class="n">inst</span><span class="p">.</span><span class="nf">instance_variable_get</span><span class="p">(</span><span class="ss">:@original_get_element_method</span><span class="p">).</span><span class="nf">call</span> <span class="o">*</span><span class="n">keys</span><span class="p">)</span>
<span class="k">rescue</span>
<span class="k">end</span>
<span class="n">weaken_checks_for_brackets_accessor</span><span class="p">(</span><span class="n">res</span><span class="p">.</span><span class="nf">nil?</span> <span class="p">?</span> <span class="n">inst</span><span class="p">.</span><span class="nf">class</span><span class="p">.</span><span class="nf">new</span> <span class="p">:</span> <span class="n">res</span><span class="p">)</span>
<span class="k">end</span>
<span class="n">inst</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Let’s dig up roots. Being called on the instance of Hash (Array is OK as all the other classes, having <code class="language-plaintext highlighter-rouge">#[]</code> defined),
this method stores the original <code class="language-plaintext highlighter-rouge">Hash#[]</code> method unless it is already substituted (that’s needed to prevent
stack overflow during multiple calls.) Then it injects the custom implementation of <code class="language-plaintext highlighter-rouge">#[]</code> method, returning empty
class instead of nil/exception. To use the safe value retrieval:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">a</span> <span class="o">=</span> <span class="p">{</span> <span class="s1">'foo'</span> <span class="o">=></span> <span class="p">{</span> <span class="s1">'bar'</span> <span class="o">=></span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]</span> <span class="p">}</span> <span class="p">}</span>
<span class="nb">p</span> <span class="p">(</span><span class="n">weaken_checks_for_brackets_accessor</span> <span class="n">a</span><span class="p">)[</span><span class="s1">'foo'</span><span class="p">][</span><span class="s1">'bar'</span><span class="p">]</span>
<span class="nb">p</span> <span class="s2">"1 </span><span class="si">#{</span><span class="n">a</span><span class="p">[</span><span class="s1">'foo'</span><span class="p">]</span><span class="si">}</span><span class="s2">"</span>
<span class="nb">p</span> <span class="s2">"2 </span><span class="si">#{</span><span class="n">a</span><span class="p">[</span><span class="s1">'foo'</span><span class="p">][</span><span class="s1">'bar'</span><span class="p">]</span><span class="si">}</span><span class="s2">"</span>
<span class="nb">p</span> <span class="s2">"3 </span><span class="si">#{</span><span class="n">a</span><span class="p">[</span><span class="s1">'foo'</span><span class="p">][</span><span class="s1">'bar'</span><span class="p">][</span><span class="s1">'ghgh'</span><span class="p">]</span><span class="si">}</span><span class="s2">"</span>
<span class="nb">p</span> <span class="s2">"4 </span><span class="si">#{</span><span class="n">a</span><span class="p">[</span><span class="s1">'foo'</span><span class="p">][</span><span class="s1">'bar'</span><span class="p">][</span><span class="s1">'ghgh'</span><span class="p">][</span><span class="mi">0</span><span class="p">]</span><span class="si">}</span><span class="s2">"</span>
<span class="nb">p</span> <span class="s2">"5 </span><span class="si">#{</span><span class="n">a</span><span class="p">[</span><span class="s1">'foo'</span><span class="p">][</span><span class="s1">'bar'</span><span class="p">][</span><span class="s1">'ghgh'</span><span class="p">][</span><span class="mi">0</span><span class="p">][</span><span class="s1">'olala'</span><span class="p">]</span><span class="si">}</span><span class="s2">"</span>
</code></pre></div></div>
<p>Yielding:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err"> </span><span class="c1">#⇒ [1, 2, 3]</span>
<span class="err"> </span><span class="c1">#⇒ "1 {\"bar\"=>[1, 2, 3]}"</span>
<span class="err"> </span><span class="c1">#⇒ "2 [1, 2, 3]"</span>
<span class="err"> </span><span class="c1">#⇒ "3 []"</span>
<span class="err"> </span><span class="c1">#⇒ "4 []"</span>
<span class="err"> </span><span class="c1">#⇒ "5 []"</span>
</code></pre></div></div>
<p>Nice, isn’t it?</p>
Ruby Logger :: Temporary Enable Debug for One Class2013-02-04T00:00:00+00:00https://rocket-science.ru/hacking/2013/02/04/ruby-logger-temporary-enable-debug-for-one-class<p>In our application we may occasionally have a heavy usage of <code class="language-plaintext highlighter-rouge">logger.debug</code> calls. It appears
to be helpful to examine the fails, especially in multithreading environment. That’s why we
do not want to get rid of these calls. We set the <code class="language-plaintext highlighter-rouge">logger.level = Logger::INFO</code> in production instead.</p>
<p>But what if we need to print debug messages for only one or two of over 9000 files? There is no elegant
solution on hand, as far as I know. Let me show an ugly hack to provide such a functionality.</p>
<p>First of all, let’s prepare the function to retrieve the caller’s filename and/or method. This would
be a static function somewhere in top-level of our module:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nn">MyModule</span>
<span class="k">def</span> <span class="nf">parse_caller</span>
<span class="c1"># magic number 7 below is the amount of calls </span>
<span class="c1"># on stack to unwind to get your caller</span>
<span class="k">if</span> <span class="sr">/^(?<file>.+?):(?<line>\d+)(?::in `(?<method>.*)')?/</span> <span class="o">=~</span> <span class="nb">caller</span><span class="p">(</span><span class="mi">7</span><span class="p">).</span><span class="nf">first</span>
<span class="n">file</span> <span class="o">=</span> <span class="no">Regexp</span><span class="p">.</span><span class="nf">last_match</span><span class="p">[</span><span class="ss">:file</span><span class="p">]</span>
<span class="n">line</span> <span class="o">=</span> <span class="no">Regexp</span><span class="p">.</span><span class="nf">last_match</span><span class="p">[</span><span class="ss">:line</span><span class="p">].</span><span class="nf">to_i</span>
<span class="nb">method</span> <span class="o">=</span> <span class="no">Regexp</span><span class="p">.</span><span class="nf">last_match</span><span class="p">[</span><span class="ss">:method</span><span class="p">]</span>
<span class="p">[</span><span class="n">file</span><span class="p">,</span> <span class="n">line</span><span class="p">,</span> <span class="nb">method</span><span class="p">]</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The magic number “7” there states for an amount of subcalls between our call and resulting
<code class="language-plaintext highlighter-rouge">Logger#add</code>. We will examine the original caller of <code class="language-plaintext highlighter-rouge">logger.debug</code> method and reject all
calls except of interesting ones.</p>
<p>The other thing we need is to override the formatter of our logger:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># put this after logger initialization</span>
<span class="n">logger</span><span class="p">.</span><span class="nf">level</span> <span class="o">=</span> <span class="no">Logger</span><span class="o">::</span><span class="no">DEBUG</span>
<span class="n">logger</span><span class="p">.</span><span class="nf">formatter</span> <span class="o">=</span> <span class="nb">lambda</span> <span class="k">do</span> <span class="o">|</span><span class="n">sev</span><span class="p">,</span> <span class="n">dt</span><span class="p">,</span> <span class="n">prog</span><span class="p">,</span> <span class="n">msg</span><span class="o">|</span>
<span class="n">f</span><span class="p">,</span><span class="n">l</span><span class="p">,</span><span class="n">m</span> <span class="o">=</span> <span class="n">parse_caller</span>
<span class="err"> </span><span class="c1">#↓↓↓↓↓↓↓↓↓↓↓↓ here goes the check ↓↓↓↓↓↓↓↓↓↓↓↓↓</span>
<span class="k">if</span> <span class="p">(</span><span class="n">f</span> <span class="o">=~</span> <span class="sr">/newclass/i</span><span class="p">)</span> <span class="o">&&</span> <span class="p">(</span><span class="n">m</span> <span class="o">=~</span> <span class="sr">/failed_method/</span><span class="p">)</span>
<span class="n">original_formatter</span><span class="p">.</span><span class="nf">call</span><span class="p">(</span><span class="n">sev</span><span class="p">,</span> <span class="n">dt</span><span class="p">,</span> <span class="n">prog</span><span class="p">,</span> <span class="n">msg</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>I know it looks a weird hack, but it works fine. As soon as we don’t need debug logging at all, simply
turn back to <code class="language-plaintext highlighter-rouge">INFO</code> level (and don’t forget to switch back to regular <code class="language-plaintext highlighter-rouge">Logger#formatter</code>):</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">logger</span><span class="p">.</span><span class="nf">level</span> <span class="o">=</span> <span class="no">Logger</span><span class="o">::</span><span class="no">INFO</span>
<span class="n">logger</span><span class="p">.</span><span class="nf">formatter</span> <span class="o">=</span> <span class="no">Logger</span><span class="o">::</span><span class="no">Formatter</span><span class="p">.</span><span class="nf">new</span>
</code></pre></div></div>
Shorthands in Ruby Code Blocks2013-02-03T00:00:00+00:00https://rocket-science.ru/hacking/2013/02/03/shorthands-in-ruby-code-blocks<p>Sometimes Ruby, being like a functional language, unobtrusively forces us to use
code blocks within iterators (such as <a href="http://www.ruby-doc.org/core-1.9.3/Array.html#method-i-map">map</a>,
<a href="http://www.ruby-doc.org/core-1.9.3/Array.html#method-i-each">each</a>, etc.) More than offen, these codeblock
are kinda one-liners and woo-<code class="language-plaintext highlighter-rouge">do-|o|-end</code> magic makes a code looking overdriven:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">arr</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">x</span><span class="o">|</span>
<span class="n">x</span><span class="p">.</span><span class="nf">name</span>
<span class="k">end</span><span class="p">.</span><span class="nf">join</span><span class="p">(</span><span class="s2">" "</span><span class="p">)</span>
</code></pre></div></div>
<p>Well, there is curly-brackets-notation available, hence we may rewrite the code above within one line.
But there is still a lot of absolutely unnecessary garbage hiding the core of what’s being actually done.</p>
<p>The good news is: ruby provides us with a syntactic sugar for that stuff. Let’s look at this:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">arr</span><span class="p">.</span><span class="nf">each</span> <span class="p">(</span><span class="o">&</span><span class="ss">:name</span><span class="p">).</span><span class="nf">join</span><span class="p">(</span><span class="s2">" "</span><span class="p">)</span>
</code></pre></div></div>
<p>It is fully equivalent to the <code class="language-plaintext highlighter-rouge">do-|o|-end</code> codepiece above, but the readability is drastically improved.</p>
<p>How the hell does it work?</p>
<p>The <code class="language-plaintext highlighter-rouge">&obj</code> is evaluated in ruby in the following way:</p>
<ul>
<li>if object is a <code class="language-plaintext highlighter-rouge">block</code>, it converts the block into a simple <code class="language-plaintext highlighter-rouge">proc</code>.</li>
<li>if object is a <code class="language-plaintext highlighter-rouge">Proc</code>, it converts the object into a block while preserving the <code class="language-plaintext highlighter-rouge">lambda?</code> status of the object.</li>
<li>if object is not a <code class="language-plaintext highlighter-rouge">Proc</code>, it first calls <code class="language-plaintext highlighter-rouge">#to_proc</code> on the object and then converts it into a block.</li>
</ul>
<p>In our case
the method is <code class="language-plaintext highlighter-rouge">#to_proc</code> on a <code class="language-plaintext highlighter-rouge">Symbol</code>’s instance (because <code class="language-plaintext highlighter-rouge">:name.class == Symbol</code>). The <code class="language-plaintext highlighter-rouge">Symbol#to_proc</code> method
was originally added by <code class="language-plaintext highlighter-rouge">ActiveSupport</code> but has been integrated into Ruby since 1.8.7.</p>
<p>To enable this shorthand for classes other than <code class="language-plaintext highlighter-rouge">Symbol</code>, e. g. for an <code class="language-plaintext highlighter-rouge">Array</code>:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Array</span>
<span class="k">def</span> <span class="nf">to_proc</span>
<span class="nb">lambda</span> <span class="p">{</span> <span class="o">|</span><span class="n">recv</span><span class="o">|</span> <span class="n">recv</span><span class="p">.</span><span class="nf">send</span> <span class="o">*</span><span class="nb">self</span> <span class="p">}</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Now we can write:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span> <span class="s2">"Apple"</span><span class="p">,</span> <span class="s2">"Orange"</span><span class="p">,</span> <span class="s2">"Pear"</span> <span class="p">].</span><span class="nf">map</span> <span class="o">&</span><span class="p">[</span> <span class="p">:</span><span class="o">+</span><span class="p">,</span> <span class="s2">" is a fruit."</span> <span class="p">]</span>
<span class="p">[</span> <span class="s2">"Apple"</span><span class="p">,</span> <span class="s2">"Orange"</span><span class="p">,</span> <span class="s2">"Pear"</span> <span class="p">].</span><span class="nf">map</span> <span class="o">&</span><span class="p">[</span> <span class="ss">:match</span><span class="p">,</span> <span class="s2">"[a-z]e"</span> <span class="p">]</span>
</code></pre></div></div>
<p>yielding:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="err"> </span><span class="c1">#⇒ ["Apple is a fruit.", "Orange is a fruit.", "Pear is a fruit."]</span>
<span class="err"> </span><span class="c1">#⇒ [#<MatchData "le">, #<MatchData "ge">, nil]</span>
</code></pre></div></div>
<p>Methods are being called on array elements (on <code class="language-plaintext highlighter-rouge">String</code>s in the example above.)</p>
<hr />
<p>Kinda same trick may be done for external methods using <code class="language-plaintext highlighter-rouge">&method</code> shorthand. Let’s say we have:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">arr</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">x</span><span class="o">|</span>
<span class="n">get_fullname</span> <span class="n">x</span>
<span class="k">end</span><span class="p">.</span><span class="nf">join</span><span class="p">(</span><span class="s2">" "</span><span class="p">)</span>
</code></pre></div></div>
<p>Thus, assuming we have the <code class="language-plaintext highlighter-rouge">get_fullname</code> method defined, we can rewrite it as:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">arr</span><span class="p">.</span><span class="nf">each</span> <span class="o">&</span><span class="nb">method</span><span class="p">(</span><span class="ss">:get_fullname</span><span class="p">)</span>
</code></pre></div></div>
<p>In other words,</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="sx">%w{ first second third }</span><span class="p">.</span><span class="nf">map</span> <span class="o">&</span><span class="nb">method</span><span class="p">(</span><span class="ss">:puts</span><span class="p">)</span>
</code></pre></div></div>
<p>will print the array content out (expanding to <code class="language-plaintext highlighter-rouge">{ |s| puts s }</code>).</p>
Multiple Match in Ruby2013-02-02T00:00:00+00:00https://rocket-science.ru/hacking/2013/02/02/multiple-match-in-ruby<p>When somebody needs to find all occerences of a pattern in a string, she usually
does either <code class="language-plaintext highlighter-rouge">scan</code> or repeatitive <code class="language-plaintext highlighter-rouge">match</code> calls. I was unable to google a quick solution
of getting all the <code class="language-plaintext highlighter-rouge">MatchData</code> instances. Even
<a href="http://stackoverflow.com/questions/6804557/how-do-i-get-the-match-data-for-all-occurrences-of-a-ruby-regular-expression-in">StackOverflow provides</a>
a strange answers on a topic, like that one below:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">str</span><span class="p">.</span><span class="nf">to_enum</span><span class="p">(</span><span class="ss">:scan</span><span class="p">,</span> <span class="sr">/PATTERN/</span><span class="p">).</span><span class="nf">map</span> <span class="p">{</span> <span class="no">Regexp</span><span class="p">.</span><span class="nf">last_match</span> <span class="p">}</span>
</code></pre></div></div>
<p>Actually, the <a href="http://en.wikipedia.org/wiki/Twin_Peaks">owls are easier than they seem</a>. All we
need is to use one of a cryptic <code class="language-plaintext highlighter-rouge">$</code> <a href="http://jimneath.org/2010/01/04/cryptic-ruby-global-variables-and-their-meanings.html">ruby globals</a>:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">input</span> <span class="o">=</span> <span class="s2">"abc12def34ghijklmno567pqrs"</span>
<span class="n">numbers</span> <span class="o">=</span> <span class="sr">/\d+/</span>
<span class="n">input</span><span class="p">.</span><span class="nf">gsub</span><span class="p">(</span><span class="n">numbers</span><span class="p">)</span> <span class="p">{</span> <span class="o">|</span><span class="n">m</span><span class="o">|</span> <span class="nb">p</span> <span class="vg">$~</span> <span class="p">}</span>
</code></pre></div></div>
<p>prints all the <code class="language-plaintext highlighter-rouge">MatchData</code>s:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># ⇒ <MatchData "12"></span>
<span class="c1"># ⇒ <MatchData "34"></span>
<span class="c1"># ⇒ <MatchData "567"></span>
</code></pre></div></div>
<p>Voilà.</p>
Internet Explorer vs Murder Rate2013-01-27T00:00:00+00:00https://rocket-science.ru/fun/2013/01/27/internet-explorer-vs-murder-rate<blockquote>
<p><img src="http://24.media.tumblr.com/782442f5e24703d89c7176938f305556/tumblr_mh2v5eR7Ao1qzpsuoo1_1280.jpg" alt="http://meme-meme.org/image/41312255083" />
— via <a href="http://meme-meme.org/image/41312255083">meme-meme.org</a></p>
</blockquote>
Zsh :: Weird Right Prompt2013-01-20T00:00:00+00:00https://rocket-science.ru/shell/2013/01/20/zsh-weird-right-prompt<p>Tinkering at <code class="language-plaintext highlighter-rouge">zsh</code> settings, I accidentally came across a very interesting <a href="http://stevelosh.com/blog/2010/02/my-extravagant-zsh-prompt/">solution</a> for the right-side prompt. Everybody who runs zsh on her notebook should definitely take a glance. The right-side prompt usually holds an unnecessary garbage, like either clock or <code class="language-plaintext highlighter-rouge">tty id</code>. That’s why I really like an idea to put there a battery charge indicator. Depending on the battery level, the notifier is to be shown in green, yellow or red.</p>
<p>The inventor of the solution used the far-fetched python to retrieve the battery charge level. I decided to rewrite the whole stuff using native shell. The result looks like this:</p>
<p><img src="/img/shell-color-prompt-battery.png" alt="Shell terminal window with battery charge indicator in the prompt" /></p>
<p>The code below has been tested on Ubuntu, though it should work fine on any linux distribution and even MacOSX under zsh. Well, code talks while bullshit walks. Let’s proceed.</p>
<p>Right prompt in zsh is being set with an environment variable <code class="language-plaintext highlighter-rouge">RPROMPT</code>. Here is a piece of code that is to be put in the end of <code class="language-plaintext highlighter-rouge">~ /.zshrc</code> file (or <code class="language-plaintext highlighter-rouge">~/.oh-my-zsh/themes/THEME_OF_CHOICE</code> if you are already addicted to <a href="https://github.com/robbyrussell/oh-my-zsh">oh-my-zsh</a> ):</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">function </span>battery_charge <span class="o">{</span>
<span class="c"># Battery 0: Discharging, 94%, 03:46:34 remaining</span>
<span class="nv">bat_percent</span><span class="o">=</span><span class="sb">`</span>acpi | <span class="nb">awk</span> <span class="nt">-F</span> <span class="s1">':'</span> <span class="o">{</span><span class="s1">'print $2;'</span><span class="o">}</span> | <span class="nb">awk</span> <span class="nt">-F</span> <span class="s1">','</span> <span class="o">{</span><span class="s1">'print $2;'</span><span class="o">}</span> | <span class="nb">sed</span> <span class="nt">-e</span> <span class="s2">"s/</span><span class="se">\s</span><span class="s2">//"</span> <span class="nt">-e</span> <span class="s2">"s/%.*//"</span><span class="sb">`</span>
<span class="k">if</span> <span class="o">[</span> <span class="nv">$bat_percent</span> <span class="nt">-lt</span> 20 <span class="o">]</span><span class="p">;</span> <span class="k">then </span><span class="nv">cl</span><span class="o">=</span><span class="s1">'%F{red}'</span>
<span class="k">elif</span> <span class="o">[</span> <span class="nv">$bat_percent</span> <span class="nt">-lt</span> 50 <span class="o">]</span><span class="p">;</span> <span class="k">then </span><span class="nv">cl</span><span class="o">=</span><span class="s1">'%F{yellow}'</span>
<span class="k">else </span><span class="nv">cl</span><span class="o">=</span><span class="s1">'%F{green}'</span>
<span class="k">fi
</span><span class="nv">filled</span><span class="o">=</span><span class="k">${</span><span class="p">(l</span>:<span class="sb">`</span><span class="nb">expr</span> <span class="nv">$bat_percent</span> / 10<span class="sb">`</span>::▸:<span class="p">)</span><span class="k">}</span>
<span class="nv">empty</span><span class="o">=</span><span class="k">${</span><span class="p">(l</span>:<span class="sb">`</span><span class="nb">expr </span>10 - <span class="nv">$bat_percent</span> / 10<span class="sb">`</span>::▹:<span class="p">)</span><span class="k">}</span>
<span class="nb">echo</span> <span class="nv">$cl$filled$empty</span><span class="s1">'%F{default}'</span>
<span class="o">}</span>
<span class="nv">RPROMPT</span><span class="o">=</span><span class="s1">'[%*] $(battery_charge)'</span>
</code></pre></div></div>
<p>The algorithm is simplier than an amoeba in it’s extramarital period:</p>
<ul>
<li>Receive and parse the battery from <code class="language-plaintext highlighter-rouge">acpi</code> (you may need to install it with <code class="language-plaintext highlighter-rouge">sudo apt-get install acpi</code>);</li>
<li>Specify a color to display (red, if less than 20%, yellow is under 50%, green otherwise);</li>
<li>Paint filled triangles for the enegry left, unpainted for the rest;</li>
<li>Put the result into the right prompt.</li>
</ul>
<p>This post is just an illustration of the principle “Spend three minutes and make your life more comfortable for years.” If you are, of course, an adept of the console, like me. I know that I didn’t make the code shine, probably because I don’t spend most of my life watching at my shell configs.</p>
<p>Just in case I have one more trick <em>als Nachspeise</em>. The following snippet prints a return code of a previosly executed command:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">RPROMPT</span><span class="o">=</span><span class="s1">'%{$fg[red]%} ⏎ $? %{$reset_color%} '</span><span class="nv">$RPROMPT</span>
</code></pre></div></div>
256 Color Term (Nightmare Level)2013-01-20T00:00:00+00:00https://rocket-science.ru/shell/2013/01/20/256-color-terminal<p><img src="/img/256-color-terminal.png" alt="Screenshot of escape-sentense builder" /></p>
<p>I’m currently reinventing a wheel, driving a long debug output to <code class="language-plaintext highlighter-rouge">stdout</code>. I finally came to conclusion that the only way not to get my eyes broken leads to colorizing all the output. Modern terminals like <code class="language-plaintext highlighter-rouge">xterm</code> are able to show 256 colors, they do even like it. But those mastodons who are still giving their <code class="language-plaintext highlighter-rouge">VT100</code> a dust, have curly escape-sequences chosen to show a rainbow up. I revere a backward compatibility; I even have a latent tendency to cryptographic empiricism… You know, it took me a couple of hours to sort the stuff out. Internet gave not a rich detailed explanation, as well.</p>
<p>Nowadays a text in the X-terminal may appear in an ornate way. That’s why I came with jotting down a web-service. It’s likely a WYSIWYG for escape-sequence generation. One choses colors, font styles etc and finally gets a bundle of flourishes.</p>
<h3 id="colors-in-xterm">Colors in xterm</h3>
<p>For a terminal to realize that it can show 256 colors, we should aknowledge it about:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">case</span> <span class="s2">"</span><span class="nv">$TERM</span><span class="s2">"</span> <span class="k">in</span>
<span class="s1">'xterm'</span><span class="p">)</span> <span class="nv">TERM</span><span class="o">=</span>xterm-256color<span class="p">;;</span>
<span class="s1">'screen'</span><span class="p">)</span> <span class="nv">TERM</span><span class="o">=</span>screen-256color<span class="p">;;</span>
<span class="s1">'Eterm'</span><span class="p">)</span> <span class="nv">TERM</span><span class="o">=</span>Eterm-256color<span class="p">;;</span>
<span class="k">esac</span>
</code></pre></div></div>
<p>The color itself is encoded in quite breathtaking manner. Escape-sequence is starting with the traditional <code class="language-plaintext highlighter-rouge">\e[</code> and ending with <code class="language-plaintext highlighter-rouge">m</code>. It consists of <em>flags</em>, the <em>color of the background</em> and the <em>text color</em>. Flags for bold, italic, underline and inverse (fg ⇐ ⇒ bg) are <code class="language-plaintext highlighter-rouge">01</code>, <code class="language-plaintext highlighter-rouge">03</code>, <code class="language-plaintext highlighter-rouge">04</code> and <code class="language-plaintext highlighter-rouge">07</code> respectively (there is a flag for the flashing as well, but what if there are children reading this?) Flags to cancel the style are <code class="language-plaintext highlighter-rouge">22</code>, <code class="language-plaintext highlighter-rouge">23</code>, <code class="language-plaintext highlighter-rouge">24</code>, <code class="language-plaintext highlighter-rouge">27</code>. Flags can be written in simultaneous order, one after another, separated by semicolons. There must be no semicolon just before the final <code class="language-plaintext highlighter-rouge">m</code>.</p>
<p>The text color signature is <code class="language-plaintext highlighter-rouge">38, 05; COLOR;</code>. The same for background color is <code class="language-plaintext highlighter-rouge">48, 05; COLOR;</code>. The color here is an integer ∈ [1, 255]. The first sixteen items are well-known old-school terminal colors, the last 24 are shades of gray, and the rest… well, these are the remainder colors.</p>
<p>Something like that (thanks FedoraProject for the picture):</p>
<p><img src="/img/256-color-encoding.png" alt="Encoding of 156 used for terminals" /></p>
<p>It is easy to see that the sequence <code class="language-plaintext highlighter-rouge">\e[01;04;38;05;196;48;05;232m</code> turns the mode of a bold underlined red text on a black background. Haha.</p>
<p>###How to get a color, huh?</p>
<p>It turns put that colors are encoded in the remaining 256 - 16 - 24 = 216 choices using a simple and straightforward algorithm. A range of tones is calibrated against division modulo 6. That number is treated as RGB-constituent in 6-based notation with “zero in sixteen”. For orange (<code class="language-plaintext highlighter-rouge">#ff9900</code>) it gives <code class="language-plaintext highlighter-rouge">16 + 5 * 6² + 3 * 6 + 0 = 214</code>). There are exceptions; they will be there any time. Those “standard” old-school colors and grayscale. Yeah.</p>
<h3 id="who-the-hell-does-need-that">Who the hell does need that?</h3>
<p>Well, first of all, I was curious. Secondly, three hours are three hours. Furthermore I have now my logfile so weird-colored that it’s became totally ununderstandable. Plus <code class="language-plaintext highlighter-rouge">PS1</code> is surely re-written from scratch.</p>
<p>In general, if you need an escape-sequence for some color—here’s a <a href="http://terminal-color-builder.mudasobwa.ru/">WYSIWYG</a>. If somebody likes to examine spaghetti-like write-only javascript code, welcome @<a href="https://github.com/mudasobwa/TermColorBuilder">github</a>.</p>
EventMachine :: Nested Calls2013-01-19T00:00:00+00:00https://rocket-science.ru/hacking/2013/01/19/event-machine-nested<p>The easiest way to drop a brick is to use asynchronous threads. Few friends of mine, while proved themselves as a strong professional hackers, were literally surrendering to the multithreading. For starters I would tell my favorite parable of the deadlock (sorry for the dupe, but it is too nice to omit it here.) Ten years ago <a href="http://ap.org/">Associated Press</a> told the world as, in the Swedish city airport Krisianstad, a pilot was trying to land a passenger plane, but none of air traffic controllers did respond to his requests. It turned out that the controller had not yet returned from his vacation. As a result, the plane circled the airport until an urgently summoned reserve air traffic controller landed the plane in half an hour. Debriefing revealed that the problem was caused by a delay of the airplane. The air traffic controller, who had to land an airplane, hurried to his workplace from a vacation on it’s board.</p>
<p>Well, when we bump into the asynchrony, we have to break the usual picture in the head: a subjective world around us is single threaded. If we had sent a letter and have got an answer past a week, everything happened in a single flow. We are not responsible for actions of our respondent and/or a postman. While our code is.</p>
<p>To make a life easier for the programmer, man can use the <a href="http://en.wikipedia.org/wiki/Reactor_pattern">Reactor</a> pattern. I swear that the best its implementation for ruby is <a href="https://github.com/eventmachine/eventmachine/wiki">EventMachine</a>. But there is no perfect thing under the sun and there are some not obvious things with it. I plan to briefly tell about one of them.</p>
<h3 id="eventmachine">EventMachine</h3>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gem <span class="nb">install </span>eventmachine
</code></pre></div></div>
<p>An <code class="language-plaintext highlighter-rouge">EventMachine</code> class is more or less documented. Dealing with simple queries is straightforward. Usually the stuff looks somehow like below (here and after <code class="language-plaintext highlighter-rouge">EM</code> is an alias for <code class="language-plaintext highlighter-rouge">EventMachine</code>):</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">begin</span>
<span class="no">EM</span><span class="p">.</span><span class="nf">run</span> <span class="k">do</span>
<span class="err">…</span> <span class="c1"># all the meaningful code, e. g. EM.connect (…)</span>
<span class="c1"># we are to print a weird message infinitely</span>
<span class="no">EM</span><span class="p">.</span><span class="nf">add_periodic_timer</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> <span class="p">{</span> <span class="nb">puts</span> <span class="s2">"on tick"</span> <span class="p">}</span>
<span class="k">end</span>
<span class="k">ensure</span>
<span class="no">EM</span><span class="p">.</span><span class="nf">stop</span> <span class="c1"># this could be omitted since the destructor nevertheless calls it</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Reactor is well-hooked (e. g. <code class="language-plaintext highlighter-rouge">EventMachine.add_shutdown_hook { puts «Exiting…» }</code>.) Asynchronous connection surely may be served on the fly. A documentation is, again, presented. Sometimes it is even clearly.</p>
<p>But let’s cease humdrum.</p>
<p>###Reaping a harvest</p>
<p>While a task is limited to “request processing → response,” there is no problem. But what if we need to send the next query basing on the result of the previous one? Not to make a post too long, let us get right to the problem:</p>
<blockquote>
<p><strong><em>look up jabber-server for a Discovery component and then communicate with it asynchronously</em></strong></p>
</blockquote>
<p>We are to send a request to Discovery, to get a list of components, to ask each component for its capabilities. If the list of features is ours, perform our job. Gracefully quit, if it is not.</p>
<p>Here’s how it looks like with <code class="language-plaintext highlighter-rouge">EventMachine</code> (I removed everything that has nothing to do with <code class="language-plaintext highlighter-rouge">EM</code> directly):</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="vi">@stream</span><span class="p">.</span><span class="nf">write_with_handler</span><span class="p">(</span><span class="n">disco</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">result</span><span class="o">|</span>
<span class="err">…</span>
<span class="c1"># iterate thru disco results and wait until all collected</span>
<span class="no">EM</span><span class="o">::</span><span class="no">Iterator</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">result</span><span class="p">.</span><span class="nf">items</span><span class="p">).</span><span class="nf">map</span> <span class="nb">proc</span><span class="p">{</span> <span class="o">|</span><span class="n">c</span><span class="p">,</span> <span class="n">it_disco</span><span class="o">|</span>
<span class="err">…</span>
<span class="vi">@stream</span><span class="p">.</span><span class="nf">write_with_handler</span><span class="p">(</span><span class="n">info</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">reply</span><span class="o">|</span>
<span class="err">…</span>
<span class="c1"># iterate thru discoinfo results and wait until all collected,</span>
<span class="c1"># then switch the parent’s iterator</span>
<span class="no">EM</span><span class="o">::</span><span class="no">Iterator</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">reply</span><span class="p">.</span><span class="nf">features</span><span class="p">).</span><span class="nf">map</span> <span class="nb">proc</span><span class="p">{</span> <span class="o">|</span><span class="n">f</span><span class="p">,</span> <span class="n">it_info</span><span class="o">|</span>
<span class="n">it_info</span><span class="p">.</span><span class="nf">return</span> <span class="err">…</span>
<span class="p">},</span> <span class="nb">proc</span><span class="p">{</span> <span class="o">|</span><span class="n">comps</span><span class="o">|</span>
<span class="c1"># one more disco item was utilized</span>
<span class="n">it_disco</span><span class="p">.</span><span class="nf">return</span> <span class="err">…</span>
<span class="p">}</span>
<span class="err">…</span>
<span class="k">end</span>
<span class="err">…</span>
<span class="p">},</span> <span class="nb">proc</span><span class="p">{</span> <span class="o">|</span><span class="n">compss</span><span class="o">|</span>
<span class="c1"># yielding</span>
<span class="c1"># compss.init or smth like that</span>
<span class="err">…</span>
<span class="p">}</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Everything is done for us by iterators and their magic function <a href="http://eventmachine.rubyforge.org/EventMachine/Iterator.html#map-instance_method"><code class="language-plaintext highlighter-rouge">map</code></a>. Lambda code under the last bracket (near the comment “yielding”) will be executed if and only we have collected all the discoinfos for all components.</p>
<p>I apologize if someone does seem the snippet above obvious, but Google has suggested to me no quick solution, while romping through Fiber’s here turns into pure hell.</p>