Blog

I'm Russell Cousineau. This is a blog about my programming adventures.

This is built using mdBook.

Source for the site is on Gitlab.

Rust Warrior - Beginner Extraordinaire

October 14, 2019

The 9th and final beginner level of Rust Warrior is at last completed! And it happened on the 100th commit! So far the gameplay has stayed almost entirely true to the original Ruby Warrior, with some minor tweaks here and there. At PDX Rust a couple weeks ago I was able to demo Rust Warrior and that was a lot of fun!

Since this feels like a nice milestone, I'll review some of the things that make Rust Warrior different from its predecessor and some things I've learned while working on it. Some of what you're about to read can also be found in commits and releases on Github. I tend to write mini blogs in my commit messages and release notes, and they're great notes to pull from for this recap of the project!

Porting a game engine written in Ruby -- which appears to be the most free-spirited programming language ever -- to Rust -- which some claim has a high learning curve because of its unique restrictiveness -- was interesting. I haven't written a ton of Ruby in the past, but I know enough to read it and have a general understanding of what's happening. Much of the code for Ruby Warrior required multiple readings in order to grasp what was really happening. If I can be critical of the Ruby syntax very briefly, why must it be allowed for method names to be bare words? In my opinion it greatly hurts readability when you're not sure if something is a local variable or a method...

Going a little deeper than the surface syntax, the first obvious difference I encountered between Ruby and Rust is that one is interpreted and one is compiled. Basically, the player is not going to be able to fire up an interpreter and load in their Rust code at run time. I had to devise a new way for players to write Rust code and interact with this game engine. I liked the way Ruby Warrior could be run from the command line and would generate the files needed to start a game, so I set up Rust Warrior so that the main.rs would do exactly that. I theorized that the player's Rust code could import the game engine as a library. I knew it was possible to set up a Rust project this way, so I tried it out. By generating a Cargo.toml and src/main.rs (just like cargo new), I was able to produce a working Rust Warrior player program that depended on the Rust Warrior crate. Much of the Rust Warrior code that generates the new player files, starts up the game, and transitions between levels is noticeably different from the Ruby Warrior code that it's based on.

Another thing that made this port particularly challenging was, wait for it... the BORROW CHECKER!!! Ha!! Ruby doesn't care at all about shared mutable state! More specifically, Ruby doesn't care about ownership. It's totally fine for two objects to point at each other. This hurdle appeared very early on. While trying to add support for level 2, I struggled to manage the state of the warrior and the sludge. The sludge needed a way to remove health from the warrior, and vice versa. Basically, I coded myself into a corner where I could only proceed by either having two mutable references to one piece of state or a mutable and immutable reference to a piece of state. I don't recall exactly what state I was juggling at the time, but if I recall there was Level and it had references to each Unit and would mutate them to move them, etc. I decided to incorporate ECS into the game to help with managing state, and there's a meaty commit 64a7241 where I implemented that. Shortly after, I was able to (relatively) easily finish coding level 2!

One thing probably nobody would ever notice if I didn't point it out is that level 4 of Rust Warrior is not the same size as level 4 of Ruby Warrior. Around the time when I was working on level 7, I was play testing earlier levels and died while playing level 4. I was trying to back away from the archer and ended up with my back against the wall while still in range of his attacks! So rather than changing my player code I altered the level so that doesn't happen, by moving the archer back one space.

Level 4 was hard to beat in Rust Warrior. Another reason I was dying while play testing is that Ruby Warrior unfairly prevents damage to the player when they first approach an enemy. The game is turn-based, and a critical element in turn-based combat is deciding turn order. Based on all other actions that had occurred in the game thus far, the player always goes first. But in Ruby Warrior if the player moves into an enemy's attack range, when that enemy takes its turn it does not attack. What?!! This meant that in Rust Warrior, approaching the thick sludge always meant death because it had more health than the warrior and because there was no special rule about skipping the first attack. When I reduced this enemy's health from 24 to 18, the level became just barely survivable.

A very noticeable difference between Ruby Warrior and Rust Warrior appears at level 6. In Ruby, you can omit arguments to a method (like direction). But in Rust Warrior, I needed to come up with a way to allow the player to move forwards with walk() and also move backwards. For directional actions, I decided to add walk_toward(Direction) as well as a directional counterpart for every other action method. It's also worth mentioning that the Warrior interface in Rust Warrior is defined once for all levels. In Ruby Warrior, however, there are different methods available at each level. Not long ago I finally implemented some restriction on which of those methods can be used, depending on which level is being played. So at the end of the day, although the Warrior in Rust Warrior has a lot more methods on it, and some of them are named differently, it's basically the same functionality as Ruby Warrior.

As I've worked on this project, I've leveled up my development tools a bit here and there. I thoroughly enjoy enabling formatOnSave in VS Code and always having my code formatted (via rustfmt). I also have clippy integrated with VS Code, and one of the lints that I encountered was new_without_default. If you write new(), clippy says you should just make that default() instead. I wanted to get rid of yellow squiggly lines in my editor, so I followed those instructions. But eventually I decided that I didn't like always doing Struct::default() when Struct::new() seemed like a more clear method name. Nowadays, if I see this warning I just add #[derive(Default)] and have a method like this:

fn new() -> Self {
    Self::default()
}

I'm curious if anyone else has run into that...

Recently I've been learning quite a bit about macros (including derive macros), and I saw an example of using serde_derive for serialization and deserialization between a type and a text format. Somehow it got me thinking about some TOML serialization and deserialization I've done in the past. One of my very first Rust projects (about 2 years ago) used the toml crate to read from and write to a config file. I remembered it being a little tricky, so when I wanted to do the same thing in Rust Warrior I just copied that code. But this serde_derive example made me wonder why I was manually constructing and deconstructing this struct instead of just deriving that functionality. It turns out I had no reason! I greatly simplified the config file reading and writing by simply adding #[derive(Deserialize, Serialize)]!

There is probably a lot more that I've learned while doing this project, but I think I've covered the most significant stuff. I hope any Rust beginners who stumble into Rust Warrior are able to use it to learn more about this wonderful language and ecosystem. There is still more that can be added to Rust Warrior (check out the issues page!), and I'm hoping to open it up to contributors soon.

🤩 Rust Hype Intensifies 🤩

August 30, 2019

I had the marvelous fortune of being able to attend RustConf 2019 a week ago, and it was a pretty incredible experience. I got to shake hands with Steve Klabnik and Sean Griffin, which made my inner Rust fanboy giggle uncontrollably. But the conference also just had a very cool atmosphere. I've heard it said so many times that the Rust community is welcoming, but that feeling got solidified in seeing the diversity of not just attendees but also the speakers. The talks were pretty great. I can't wait for them to hit Youtube so I can re-watch them, and so I can see the other half of the talks from the conference.

Steve Klabnik is co-author of The Rust Programming Language (a.k.a. "the book"), which is an absolute pleasure to read. There are really good explanations and examples throughout. I'm well overdue for my third full read-through of it. It was awesome to meet him in person and thank him for the work he put into the book.

Sean Griffin has hosted the great podcasts The Bike Shed and The Yak Shave and appeared on a few episodes of New Rustacean. He's also the creator of diesel. I love listening to him on podcasts because he constantly thinks out loud about different programming challenges and discusses them with other folks on the air.

Between going to a few PDX Rust meetups recently and now getting this chance to attend RustConf, my excitement about Rust just keeps growing and growing. For about a month, I've been working on a fun project to implement a Rust introductory tutorial game called Rust Warrior. It's a spiritual successor to Ruby Warrior, which I played many years ago while I was in college. It's been a fun project, and I've learned quite a bit along the way.

One thing I've learned is that Rust is a joy to work with on a real project. I intentionally started the project dead simple, despite having a perfect view of the finished product. One reason for this was that I wanted to leave myself room for taking it in a different direction than its predecessor. But also, each time I wanted to add a new feature it required refactoring, sometimes in non-trivial ways. Each time, I was confidently able to use the compiler to steer me towards successful completion.

This project allowed me to scratch an itch I've had for a while. This is a weird and obscure itch, but I've always wanted to generate one of those gifs of a program demo in a terminal. I was able to accomplish this relatively easily with Terminalizer. Check out the project readme.

While working on this project, I became acquainted with crates.io and docs.rs. The default behavior of cargo is to publish your crate (Rust project) to crates.io when you do cargo publish. In addition to your crate being available there, your documentation is also automatically made available on docs.rs.

My heightened degree of acquaintance with docs.rs came from me noticing that, after publishing a release to crates.io and checking in regularly... 13 hours later my documentation was still not updated. The docs.rs team is usually fairly responsive in discord, and I was kindly informed that there is a build queue which can sometimes get blocked by crates whose docs take a very long time to build. I have since seen that queue reach a size of over 200 crates, though at this exact moment the queue is empty. I was also told that one aspect of the slowness is that all of the files are uploaded to S3 sequentially. There was some talk about parallelizing the upload, which I offered to implement. At the time of writing this, my pull request is still open, but it's nearly approved and merged! This is an exciting accomplishment because I've been wanting an opportunity to work with async Rust.

One of my favorite talks from RustConf was Ferris Explores Rustc, partly because it was so entertaining (again, I can't wait for this and the other talks to all hit Youtube!); but also because it amplified my desire to contribute to the Rust language and compiler.

For the last several weeks, I have been trying to learn everything I can about macros because I want to do a presentation at an upcoming PDX Rust meetup. My notes and slides are here. This feels like a nice warm up for jumping in to work on the compiler, since procedural macros (formerly known as compiler extensions) give you a glimpse at some internals of the Rust language. So my plan is to wrap up this presentation and set sail for the rustc!

THANK YOU to everyone at RustConf and everyone in this awesome community.

May the Rust be With You


P.S.

Totally unrelated to Rust, I recently built my first ever Firefox extension! It is quite simple: it lists all of your open tabs (across all Firefox windows) in a sidebar panel. You can click an item in the list to switch to that tab. I use it constantly because I always have so many tabs open that their titles are truncated down to just a few letters and an ellipsis, so I can't tell which is which. If this is a struggle for you as well, check out Tab List Sidebar.

Some other rad extensions I use:

Spaghetti

July 23, 2019

Know what really grinds my gears? A git DAG that looks like a pile of spaghetti.

spaghetti

That's a screenshot from Atlassian Sourcetree for a real repository at my work (I rotated it 90 degrees for better layout in this post). Here's another (much more mild) real example:

*   1d50ef2     O
|\
| * d718abb     N         <-- should have been rebased on 520009a (M)
* |   520009a   M
|\ \
| |/
| * e117172     L         <-- should have been rebased on 28c9618 (K)
* |   28c9618   K
|\ \
| |/
| * b498d5d     J         <-- should have been rebased on fdde87d (I)
* | fdde87d     I
* |   a7afa18   H
|\ \
| * | 3dcd450   G
| * | b654da1   F
| * | 0895247   E
|/ /
* |   8b7931d   D
|\ \
| |/
| * b0fb1ae     C
| * a27228f     B
|/
* f9959b4       A

This is what you might see when doing git log --graph --oneline. But --graph is actually hiding part of the problem. Here's the same tree without the --graph option:

1d50ef2   O
d718abb   N
520009a   M
e117172   L
28c9618   K  <---+  this commit merged b498d5d (J) to master
fdde87d   I      |
a7afa18   H      |
b498d5d   J -----+
3dcd450   G
b654da1   F
0895247   E
8b7931d   D
b0fb1ae   C
a27228f   B
f9959b4   A

Note how the history is not really in order. HOW is it meaningful to say that b498d5d (J) occurred before a7afa18 (H) and fdde87d (I) if it was merged after? If you were trying to figure out what to revert, this would be at least a little bit misleading or confusing. This is just one problem caused by allowing Spaghetti Merges. I'm a bit anal retentive about this, but keeping a clean git history really does help when tracking down bugs, etc.

Another thing to consider is isolated testing. Whether testing some changes manually or running CI on a pull request, there is some significance to the current branch base at the time when the testing is done. In the example above, b498d5d (J) was tested with a branch base of b0fb1ae (C). That means that each of 0895247 (E), b654da1 (F), 3dcd450 (G), and fdde87d (I) were opportunities for semantic conflicts (the other commits are just merges).

The bors project was created to prevent semantic conflicts. This illustration shows an example of how isolated testing and an unlucky merge can lead to a broken build on master. On large projects, it can be nigh impossible to successfully enforce manually rebasing before every merge. The chances are simply too high that another merge will have occurred between your most recent rebase and the completion of automated or manual tests afterwards. This is why bors is invaluable on large projects (such as servo).

It's a little unfortunate that bors is specifically designed for Github, since many enterprise software engineers use Bitbucket or a self-hosted git server, and Gitlab has been growing in popularity. I'll be keeping my eye out for tools that can accomplish the same thing without requiring Github. At my previous job we typically did a pretty good job of rebasing before merging. Since we had a rather time consuming CI pipeline, we implemented a merge queue for preventing that frustrating cycle of rebase + retest. You only have to rebase once: when it's your turn to merge.

I think we can all agree that it's very nice to have a clean history:

                                    __ooooooooo__
                               oOOOOOOOOOOOOOOOOOOOOOo
                           oOOOOOOOOOOOOOOOOOOOOOOOOOOOOOo
                        oOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOo
*   ef533e1           oOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOo
|\                  oOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOo
| * a8236d5        oOOOOOOOOOOO*  *OOOOOOOOOOOOOO*  *OOOOOOOOOOOOo
| * 8c5f2ca       oOOOOOOOOOOO      OOOOOOOOOOOO      OOOOOOOOOOOOo
| * f769333       oOOOOOOOOOOOOo  oOOOOOOOOOOOOOOo  oOOOOOOOOOOOOOo
| * 974ea79      oOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOo
| * f5e751b      oOOOO     OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO     OOOOo
|/               oOOOOOO OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO OOOOOOo
*   78433d7       *OOOOO  OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO  OOOOO*
|\                *OOOOOO  *OOOOOOOOOOOOOOOOOOOOOOOOOOOOO*  OOOOOO*
| * 54946cb        *OOOOOO  *OOOOOOOOOOOOOOOOOOOOOOOOOOO*  OOOOOO*
|/                  *OOOOOOo  *OOOOOOOOOOOOOOOOOOOOOOO*  oOOOOOO*
*   599af99           *OOOOOOOo  *OOOOOOOOOOOOOOOOO*  oOOOOOOO*
                        *OOOOOOOOo  *OOOOOOOOOOO*  oOOOOOOOO*
                           *OOOOOOOOo           oOOOOOOOO*
                               *OOOOOOOOOOOOOOOOOOOOO*
                                    ""ooooooooo""

Let's do what we can to make that happen.

YABS - And so it begins. Yet Another Blog Site.

July 22, 2019

My very first blog ran on a cheap Dell PC tower I bought used. I installed Ubuntu on it, set up SSH, and stuck it under my desk. Free web hosting! I ran a very basic Django app on Cherokee Web Server, with posts saved to SQLite. I didn't really know anything about web design, but I was pretty happy with just the basics, and I loved that it was almost entirely Python.

Years later, I discovered Flask. It was perfect, because I really didn't need much on the backend. I started using Google App Engine, which was very convenient. I was able to very quickly and easily spin up new web apps. Some apps used AngularJS, some used Angular (the sequel to AngularJS). A couple used React, including my most recent blog which I titled "Le New Blog" (it was supposed to be a temporary name, but after two years that's the name it's retiring with).

I've never really tried to advertise any blogging I've done. It was just me writing myself notes about things I had done and learned along the way. At first, I was learning things at an incredibly rapid pace. After about five years of professional web development, I began to feel some disenchantment about building web apps. I wanted to challenge myself in other ways.

So the web apps that were still running haven't been getting any updates. I didn't much like the thought of trying to upgrade npm dependencies, get my apps to be compliant with new versions of libraries, etc.

I do, however, want to keep writing. And I started to get some ideas for how I might accomplish that. After listening to the New Rustacean podcast, I was intrigued that there was this very different way to construct a website entirely in Markdown (and Rust). The author built the entire site for the show notes using rustdoc. This layout looked like it could work for a simple blog site, and I was very fond of the idea of having that Rust style on my own site. Since my sites have always just been a collection of Markdown text, a static site generator like this is perfect.

I didn't end up using rustdoc because I wasn't planning on incorporating that much Rust code into my posts - though there will definitely be some! Instead, I followed the example of the Rust Book and used mdBook. As far as style goes, it's very similar! But it's a little more lightweight to manage the site (or book) since it's entirely Markdown.

Coming soon: random musings about programming (likely often having to do with Rust)!