Instrumenting Axum projects
Fitting the pieces together for pleasant errors and logs.
Recently I was helping someone get bootstrapped on their Axum project, and they were getting a bit frustrated with the lack of context about the goings on in their application. While Axum and it's ecosystem are well instrumented, by default most of it is not very well surfaced for beginners. It can be a bit intimidating figuring out how to put the pieces together and use them.
Extending NixOS Configurations
Using Nix flakes to extend an existing NixOS configuration with additional settings or modules.
NixOS modules and configurations offer us a tantilizing way to express and share systems. My friends and I can publish our own flakes containing nixosModules
and/or nixosConfigurations
outputs which can be imported, reused, and remixed. When it comes to secret projects though, that openness and ease of sharing can be a bit of a problem.
Let's pretend my friend wants to share a secret NixOS module or package with me, they've already given me access to the GitHub repository, and now I need to add it to my flake. My flake is public and has downstream users. I can't just up and add it as an input. For one thing, it'd break everything downstream. More importantly, my friend asked me not to.
It's terribly inconvienent to add the project as an input and be careful to never commit that change to the repository. Worse, if I did screw up and commit it, my friend might be disappointed in me. We simply can't have that.
Let's explore how to create a flake which extends some existing flake, a pattern which can be combined with a git+ssh
flake URL to resolve this precarious situation.
Nix on the Steam Deck
What's Linux without the Nix?
When I first started using Linux in 2006 I remember dreaming of a Linux Console. The idea maybe wasn't so far fetched at the time, the PlayStation 3 had just been released with OtherOS support which allowed users to install Linux (or BSD). Still, it seemed that a Linux-first console would only ever be a dream. Now in 2022, Valve's Steam Deck is a hackable Linux-first portable console.
Today, we'll be putting Nix on it, because what's Linux without Nix?
Declarative GNOME configuration with NixOS
Using Home Manager to declaratively set key desktop environment knobs.
I adore tinkering with my machine, trying new tools, extensions, themes, and ideas. When I was younger, it was simply a way to learn. Now, it's a way for me to refine my workspace and bring myself small joys.
While tinkering can be fun, it can be a chore to set up a new machine, keep configurations up to date between machines, or even just remember to keep up to date backups. We've previously explored how to create Configurable Nix packages, which solves the problem for things like neovim
, but what about when a package isn't practically configurable?
What about when we want to configure a whole desktop environment? While NixOS offers configuration settings like services.gnome.gnome-keyring.enable
for systemwide features, there's a void of knobs when you want to set things like user-specific GNOME 'Favorite Apps' or extensions.
Let's explore a useful addition to your NixOS configuration: Home Manager and its dconf
module.
Xbox Controllers on Arch Linux
Some games are simply better on controller.
I fumbled a bit setting up my Xbox controller in Arch, but managed it and wanted to share.
Let's quickly cover how to use xone
to set up a Xbox Elite Wireless Controler Series 2 connecting to Arch Linux via a Xbox Wireless Adapter for Windows 10. Once we're done, it should 'just work' in steam
, wine
, lutris
, or your other games.
Visually debugging PGX extensions
Using Visual Studio Code and LLDB to debug PGX extensions
Rust spoils me, I rarely need to reach for a debugger thanks to structured logging via tracing
and errors with spantraces thanks to eyre
. When I do find myself wanting to really dig my grubby little paws into some code, configuring the debugger targets can feel intimidating.
Experience has taught me that I can conquer intimidating things though, and sometimes that is best done with a little help. While hacking on a pgx
bug, I realized I was using a newly installed system, and needed to reconfigure my debugger settings.
This made a good opportunity for me to share with you! This short article will cover how to configure Visual Studio Code with CodeLLDB so you can visually debug your pgx
extensions. This includes being able to step into the pgx
or postgres
sources. While the instructions may be Arch Linux specific, they should be able to be adapted to a different Linux.
Getting started with Xous and the Precursor
Getting familiar with this open hardware development platform
A few years ago, I saw bunnie's Precursor on Crowd Supply and instantly fell in love. The idea enamored me despite hardly knowing what FPGAs were at the time.
Embedded development has always been a fun hobby of mine, and I think it's an excellent pursuit for any developer to improve their knowledge. If you like Rust (like me), some great starting points are the nRF52840s, the ESP32-C3-DevKitC-02, or one of the TockOS boards (like the Hail).
While I love those, one thing I've wanted for a long time was a portable board with a battery, screen, keyboard, SDR, and user-servicable parts that could be hacked on top to bottom.
The Precursor offers all of the above list (except, sadly, the SDR) and is powered by a XC7S50-1L CSG324I System-on-Chip (SoC) running the open Betrusted SoC and an iCE40UP5K SG48 Embedded Controller (EC) running the open Betrusted EC on an open hardware design. It seems almost perfect for me.
Well... I finally convinced myself to order one as a reward for doing a big thing, and it has now arrived! So, let's explore!
PostgreSQL Aggregates with Rust
Writing custom Aggregates for statistical or analytical purposes.
Reaching for something like SUM(vals)
or AVG(vals)
is a common habit when using PostgreSQL. These aggregate functions offer users an easy, efficient way to compute results from a set of inputs.
How do they work? What makes them different than a function? How do we make one? What kinds of other uses exist?
We'll explore creating some basic ones using SQL, then create an extension that defines aggregates in Rust using pgx
0.3.0's new aggregate support.
Configurable Nix packages
Let your users bring their settings with them.
The vim
and neovim
packages in Nixpkgs allow users to set custom configuration, including their customRC
and any plugins they might want.
How do they accomplish it?
In this article, we'll explore how to create packages with similar behavior. We'll create a simple Rust app that consumes a configuration file, then create a Nix flake containing both an unwrapped binary package as well as a configurable package.
Forging SQL from Rust
How pgx
creates PostgreSQL extension SQL definitions that bind to Rust.
PostgreSQL offers an extension interface, and it's my belief that Rust is a fantastic language to write extensions for it. Eric Ridge thought so too, and started pgx
awhile back. I've been working with him to improve the toolkit, and wanted to share about one of our latest hacks: improving the generation of extension SQL code to interface with Rust.
This post is more on the advanced side, as it assumes knowledge of both Rust and PostgreSQL. We'll approach topics like foreign functions, dynamic linking, procedural macros, and linkers.
A Flake for your Crate
Creating a Nix Flake for your Rust crate.
Nix (PDF) provides users a way to access the massive Nixpkgs library of packages, create reproducable builds of software, roll slim containers, create declarative VMs, or run their whole machines. A new feature of Nix, Flakes, is bringing a convention to how projects like Rust crates can be accessed, integrated, and used within Nix (or NixOS.)
Let's explore how we can make our Rust crate usable as a Nix flake. At the end of this, any nix
user with Flakes enabled should be able to run your project with something like nix run github:user/project
. They'll be able to add your repository as a Nix overlay, install the package, do interactive builds, or create a portable bundle of it.
Using rust-bindgen in Nix
Getting things linking.
While building the Nix packages for pl/Rust I bumped into a curious issue: I couldn't link to stdio.h
, or stdbool.h
! They were clearly on my path, too.
It flummoxed me for quite some time, but exploring the firefox
package led to a way forward. It was rust-bindgen
not finding libraries!
Custom live media with Nix flakes
How to make live media with Nix flakes.
I've always been quite fond of booting live media. To test or install a new operating system, to recover an old one, find some privacy, or to do a myriad of other specialized tasks. LiveUSBs and liveCDs introduced to me a new way of thinking about my computer.
It improved my mental model of the separation of between the machine, the UEFI (or BIOS), any bootloaders, and the operating system itself.
As I learnt about them over 15 years ago I spent months exploring ways to use them. I used them to rescue systems for myself and others, diagnose hardware, recovery files, and quickly set up machines.
With Nix flakes, we can define a custom live system and build it with minimal steps. This NixOS live system which could be a composition of existing NixOS modules, or an entirely new configuration.
The Entanglement of Purpose & Success
Gathering the wood, sparking the embers, and tending the flame.
There are few things I remember more clearly from my post-secondary than this question from professor Rob Hancock:
How do you define success?
I was as much impacted by the question as I was by how my cohort grappled with the question over the next semester, and beyond. It's a question I've continually faced throughout my whole life.
It turns out that success to me is an elusive, ephemeral, amorphous concept. It slips from me fingers as I grasp at it, crumbles away as I try to hold onto it, and escapes me if I become too bold or complacent.
Success doesn't need to some end goal to capstone your life: some glorious retirement in a great mansion, a C-level job title, or your offspring seeking out their own success. Success can be just a step away than what you have right now, a short walk, or a long hike. Sometimes, it's exactly where you are.
If success is a destination, then purpose is the way there. Purpose is the wind in the sails of a ship, success is a port of call. Purpose guides your path through weathering storms, treacherous seas, and emotional outbursts. It steers you away from the siren's call of whirlpools and icebergs of burnout, and ushers you aginst the looming void of utter failure.
Finding direction, defining goals
Charting a course to a more hopeful future.
The simple passage of time changes us. Our personalities, identities, social circles, fears, and passions all shift to meet our needs throughout life. As we grow and mature, we must reconcile our past and resolve ourselves to the future. Time trawls up existential questions again and again: Who am I? What is my purpose? What do I want in life? What am I willing to struggle with? What happens if I fail? What could have been, but never will be? These kinds of questions can prompt deep introspection, and act as reasons for change.
My life is unrecognizable compared to even two years ago, and with these changes, new answers to those questions started to form. I found myself desiring a change.
Hierarchical Structures in PostgreSQL
Modelling hierarchical/team/categorical/tag data with arbitrary depths.
It's a common pattern: a database developer at a startup is probably on the Product subteam of the Engineering team at their company. In a department store, shoes are a subcategory of clothing, while your favorite thermos is probably in the travel department.
In any Github organization, there are teams within teams within teams. In any large department store there are categories deeply nested. In any recipe book, there are many ways to classify food.
So how can we model them?
Getting the most of WSL
Some tricks for getting more mileage from your Linux subsystem
Not long ago, Microsoft started iterating more heavily on their long-existing "Windows Subsystem for Linux" feature. The feature has existed for some time, but recently has it become much more usable for everyday Linux development work.
In this post, we'll investigate some ways to get the most out of WSL. I won't present anything new here, just providing a single place for how I or someone else could get things set up.
Common Safety Pitfalls found by Jepsen
Learning about Jepsen and doing a survey of published analyses.
When Jepsen appeared in the industry several years ago our team at PingCAP couldn't have been happier. Finally, an approachable, understandable reliability testing framework along with an enjoyable and rigorous library of analyses of popular databases.
Jepsen is an umbrella term for the software library, collection of articles, and service of the same name. Together, the project pushes distributed systems to be more accurate in their claims and more thorough in their testing.
It's been indispensable for battle-testing TiDB and TiKV, our distributed databases, as we deploy them in some of the largest technology companies in the world.
Why Benchmarking Distributed Databases Is So Hard
Exploring the complexities of benchmarking distributed systems.
If you’re an avid reader of distributed systems news like I am, you’ve probably seen your share of benchmarks. You’ve also probably stopped taking them at face value. Unfortunately, benchmarks are hard to get right, and even more unfortunately, many articles touting benchmarks are actually benchmarketing, showcasing skewed outcomes to sell products.
So why do it at all? Let’s take a look at some of the motivations of benchmarking, the common tools, and discuss a few things to keep in mind when benchmarking.
Optional Arguments in Rust
Exploring Rust UX and API design.
When designing an API for your crate one topic which can come is how to handle optional arguments. Let's explore our Option
s in Rust!
Ascend
Don't be afraid of the mountains at your doorstep.
It seems that at times of monumental change in life there is a 'before' time, and an 'after' time. For example, the time 'before' graduation, and the time 'after'. It feels appropriate for the moment. Over the other side of the summit is unknowns. Unknowns about the future, about ones place in it, and about what changes might come next.
While it is convenient at moments to make distinct 'before' and 'after' perspectives, it is not accurate. Life marches monotonically from change to change, and each before time is a composition of all previous 'before' and 'after' times. After one change, another will come, and the current ominous 'after' time, and its unknowns, will become a 'before' time, with less unknowns.
Learning to Tread the Dawn
Recent thoughts and learnings
In the time since my last writing life has been nothing short of a whirlwind. It's been good though. I've relocated, started at PingCAP, found a new apartment, bought furniture(!!!), traveled, travelled more, and been working on a few other bit things.
Moving back to Victoria from Berlin was (as predicted) quite a change. Having lived here most of my life though, I still have old friends here and know the city well. The city has been changing, some areas are completely full of construction, even minutes outside of downtown. I'm quite interested in seeing where the city is going, and how they tackle problems like homelessness as the city becomes more and more expensive.
A New Year Dawns
A look into the future
First of all, happy new year!
It's been awhile since I wrote anything, and there are many things I'd love to write more about. Sadly, time is a limited resource and I've been managing to keep quite busy. For now, I'll stick to an update about a wave of change in my life.
The Path to Rust on the Web
Recently there has been quite a bit of talk about WebAssembly, a new format for code for the web. It is a compile target for languages like C and Rust that enables us to write, and run, code from these languages in our browser.
In the interest of learning more about this technology (and to avoid writing more Javascript) let's explore together and get our hands dirty!
Disclaimer: WebAssembly is stabilized, but most implementations are not. The information contained here may become out of date or be incorrect, despite working at the time of writing.
Setting up a Rust Development Environment
In this post we'll be discussing one way to set up your machine for Rust development. There are different ways, particularly related to the text editor, so feel free to skip what is irrelevant to you. We'll focus on:
- Setting up Rust via Rustup.
- Valuable tools like
clippy
andrustfmt
. - Configuring VS Code for Rust.
- Debugging with the command line and inside of VS Code.
- Using different compile targets.
Everything we do should be functional on both Linux and MacOS, on Windows your mileage may vary. I, unfortunately, don't have a Windows instance to test on.
The Future with Futures
Recently there has been a lot of progress in the Rust language towards a robust asynchronous stack. In this article we'll take a look at what these things are, take a tour of what's available, play with some examples, and talk about how the pieces fit together.
We'll get started with the futures
crate to start, move on to futures_cpupool
, and then eventually to tokio
. We will assume you have some knowledge of programming, and have at least thought about trying Rust before.
Pretty State Machine Patterns in Rust
Lately I've been thinking a lot about the patterns and structures which we program with. It's really wonderful to start exploring a project and see familiar patterns and styles which you've already used before. It makes it easier to understand the project, and empowers you to start working on the project faster.
Sometimes you're working on a new project and realize that you need to do something in the same way as you did in another project. This thing might not be a functionality or a library, it might not be something which you can encode into some clever macro or small crate. Instead, it may be simply a pattern, or a structural concept which addresses a problem nicely.
One interesting pattern that is commonly applied to problems is that of the 'State Machine'. Let's take some time to consider what exactly we mean when we say that, and why they're interesting.
An HTTPS Terminator Box
Over the last couple days at asquera we've been on a retreat at the Landhaus Fredenwalde. It's really beautiful out here and it's given me a chance to work on a few small projects which I've been wanting to explore for awhile now.
Anyways, yesterday I set up a system that uses Ansible, Let's Encrypt, nginx, and DigitalOcean to terminate HTTP and proxy requests to arbitrary hosts. The intended use case for this is to have Github Pages sites able to be dropped onto a custom domain that is SSL enabled, but there are many other use cases which I haven't experimented with (yet).
I was, primarily, interested in exploring using Ansible and DigitalOcean. It worked out quite well http://hoverbear.org/ is running on it at the moment!
Winds of Change
Unlike most of this blog this post is not technical in the slightest, instead it's a personal update. Though, I have been wanting to write this post for a long time now!
Many things have happened in the last few months, and life is still swirling all around me. Many things seemed uncertain, other things still do. Irregardless, things are pretty darn cool.
Anyways, here's what's happened lately.
New Roots part 5, Erecting Container Infrastructure
This is the fifth part of an ongoing series on configuring a new server. In our last post we discussed and configured some basic tools. For all intensive purposes, our 'root' system is complete. What we'll be doing now is building the infrastructure to run containers the way we want to.
Before we get around to setting things up, let's describe the what we're up to.
New Roots part 4, Making it 127.0.0.1
This is the fourth of a series on taking up residence on a new server. In our last post we talked about how to set up some basic network services like ssh
and configure iptables
, our firewall. In this post we'll talk about making your server feel like home.
There are a grand number of things we can do at this point to make our time on the machine enjoyable. Take time to evaluate your choices though. This 'root' host is going to be our 'control seat' so it's not going to be doing much more than orcestrating virtualized environments. Once we configure these environments (next post) we'll be using these for any sort of development, deployment, or experiments.
With 'root' our goal is to make a simple set of good, sharp tools to do what we need to do. These are tools like nvim
which we installed earlier. If you were following the last host you may have installed mosh
as well, which makes our ssh
sessions safe from network changes and comfortable against latency.
New Roots part 3, Services & Hardening
This is the third in a series of posts about getting settled into a server. First we talked about choosing a server, then we talked about installing a base OS on a dedicated server. In this post we'll discuss configuring, securing, and hardening our server.
In our last post we left our new server in a very, very minimal state. Heck, we didn't even tell it it's own name! In this post we'll talk about configuration. Throughout this process we're going to try to keep things simple and tightly knit. Through most of this guide you'll need to be using sudo
or acting as root
.
New Roots part 2, On the Metal
This is the second post of a series on settling into new servers. The first was about choosing a server. This post is specifically targetted at newly acquired VPS and Dedicated servers. We'll talk about installing our chosen distribution, configuring its basics, and familiarizing ourselves with the new metal.
New Roots part 1, Choosing a Server
This is the first in a series of posts where we'll discuss the process of obtaining, setting up, and settling into a new headless server. Along the way we'll install Linux, configure firewalls and web servers, set up virtual machines, tinker with system knobs, explore automation tools, and generally have a great time.
In this post, We'll discuss things to keep in mind when searching for your own hosting, things to look for, and the differences between distinctions.
This series is not intended to be a comprehensive guide. These are just the notes and rambling of a hobbyist server admin who's loved tinkering for over a decade. I'm writing these both for my own future reference, and so my readers can suggest improvements! As always, comments are welcome via email.
The Menagerie of Badssl
Late last year I was given an opporunity to participate in the Mozilla Winter of Security 2016! I'm happy to report it was, and still is, super cool. Plans diverted significantly at the very start of the project as it was discovered that the "menagerie" of certificates we wanted to build already existed.
What joy! In order to avoid any "not-invented-here" syndrome problems we pivoted, like a failing startup, and I moved to become a contributor to BadSSL. One of my mentors, April King, happened to already be a contributor to BadSSL and helped me get acquainted with the repository and the maintainer, Lucas Garron.
I was glad to discover that BadSSL is a Jekyll site. I've used Jekyll a number of times for different tasks and really enjoy working with the program. BadSSL is deployed on the Google Cloud Platform and is primarily powered via extensive nginx configurations.
Site Redesign
I've begun redesigning my website with my own theme! With school finishing and new employment looming I decided to take the time to make my site truly my own.
Gathering Our Voices 2016
This year helped again with Gathering Our Voices. I worked in the same capacity as previous years and had a similarly great time.
Rather than talking at length about events I'd rather share pictures. All of these photos were taken by my friend Geoff Howe. All images are wholly owned by the BCAAFC and you may not reproduce or use them in anything without their permission.
Arch Linux on Docker Revisited
Back in 2014 when I was learning about Docker I got around to making a base image for Arch Linux. It was a really fun exploration and I got to know a lot more about how Docker worked from it. I'd highly suggest trying to make your own sometime!
Docker has matured a lot since, and I've enjoyed following it. I took some time this last week to revisit my Arch Linux image and ensure it's still functional. I wasn't surprised when the scripts continued to work just fine, even after two years.
With nothing broken, I knew I only had one choice. I had to improve it!
A Rust Heroku Buildpack
I'm happy to introduce the first stable version of my Rust Heroku Buildpack!
Just looking to get your hands dirty? Look here.
The project was a really interesting learning experience about how Heroku does things, on bash
scripting, and about multirust
, which I'd not used before but now am quite a fan of! (I previously just rustup
'd the newest nightly every few days.)
The Way Forward
In April, I will be graduating from University. It's pretty darn exciting! Once that's happened I'll officially have a Computer Science Degree with a Networking option, meaning I focused primarily on networking for my upper year studies. I'll be completing my degree with (approximately) a 7.75 (out of 9.00) GPA and I'm pretty proud of that!
I've been placing a lot of thought into what comes next for me, and the only solid decision I've come to is that it's dependent on external factors. The problem with trying to build a plan is that many of the factors I have little control over.
What choices I make, and where my next years take me, will be dependent the opportunities which I am able to seize and which present themselves to me. In the past few years I've been incredibly lucky to have had a grand number of opportunities to engage with all sorts of amazing people and organizations. I'd like to continue that.
It's a Let's Encrypt Beta!
I was privileged to recieve one of the early Let's Encrypt beta certificates for https://hoverbear.org. I had an easy and fun time setting it up this evening on my Funtoo Container and wanted to quickly jot down how to!
Understanding Over Guesswork
Evolving How We Learn Systems with Lessons from Programming in the Large
Some bugs are just that+++a one off. A wayward moth that just happens to be innocently fluttering through the wrong relay at the wrong time. But some kinds of bugs aren't like that. Instead, they have risen to superstar status, plaguing veterans and newcomers alike. But what if these aren't bugs at all? What if they are actual deficiencies in safety and robustness offered by the C programming language as a consequence of the degree to which guesswork is introduced. Here we explore a more explicit approach to systems level programming supported by Rust, which we believe will better promote understanding of design intent, and eliminate some of the guesswork. Guided by a set of classic, but still relevant, bugs identified almost 15 years ago by Engler, we consider this in the context of the new generation of students learning about systems in a typical OS course, where students often first encounter these deficiencies.
Photos: San Fransisco Trip
A selection of images from my trip that I thought I'd share with the world.
If you want to use one of these for anything at all, just fire me an email and ask!
The Raft-rs Rust Meetup
Exciting Times
The next few weeks are full of excitement and busy-ness! First up is the Meetup in San Fran (See our planning). I can't wait! It will be so fun to meet many people I've already met in IRC and through the community!
If you'd like to do something, or show me around the city, let me know! I've never been to the USA before, let alone San Fransisco.
After that I'm moving into a new place! A friend and I landed a great location for the next year and we're looking forward to the changes ahead. Once I'm settled in classes will be starting and I have a very busy term ahead!
Blog Changeover
I've moved my feed over from Ghost to Jekyll! It took several hours spread across several days to transfer all of my posts out of Ghost since it's (experimental) export functionality just dumps a bunch of junk into a JSON document.
In this move most page links should still map to the same thing as before. This does not hold for images, diagrams, etc (but you should have hosted your own copy anyways).
Trackpad Drivers for Linux on a Mac
If you're like me and have Mac hardware but run Linux you might also run into the slight annoyance with the trackpad drivers X.org defaults to.
If you haven't explored, there is a fantastic replacement called mtrack
that I've had great luck with.
Raft Progress & Examples!
We've also got couple nice working examples! Yes, they're ready to play with!
Can't wait, need code? Visit Raft and check out the
examples/
directory!
Raft: Tooling & Infra Update
In preparation for the forthcoming 0.0.1 release of Raft we've taken several forward steps to improve our (already pretty darn cool!) tooling and infrastructure. If you attempt to set up any of these on your own and have issues let me know! I'd be more than happy to help. You can find our .travis.yml
here.
Raft: New Crates!
We've been resoundingly busy with our Raft implementation after a brief period of calm in the early summer! I'll be posting a few bits over the next weeks to help people both learn Rust, and learn about Raft!
As a result of Raft development two new crates have been put together for general consumption! Both of these are macro-centric and lead mainly by James.
Reading Rust Function Signatures
In Rust, function signatures tell a story. Just from glancing at the signature of a function an experienced Rust user can tell much of the functions behaivor.
In this article we'll explore some signatures and talk about how to read them and extract information from them. While exploring, you can find many great function signature examples in the Rust API docs. You can play around on the Playpen.
This article assumes some knowledge of Rust, glossing over a bit of the book should be quite sufficient if you are lacking that but have programmed before.
Collecting Results from Collections
I've been playing around with a new project that uses a lot of collections of results, for example Vec<Result<()>>
and asked some folks on Rust if they new a good way of transforming this into a Result<Vec<()>>
.
Turns out, .collect()
can do it for you!
ACM papers in Pandoc
Based partially off my earlier article I've set up a repository here that demonstrates how to use Travis CI to build yourself ACM styled PDFs (along with fairly decent looking HTML versions) out of a simple pandoc
markdown document!
Quadcopters: Motors
We've already talked a bit about motors in our previous articles, in this article we'll talk more about motors than the quadcopter as a whole.
Quadcopters: Stabilization
In our past articles we've explored some of the basics of the mechanics of Quadcopters. In this article we'll be doing something a bit different and discussing the algorithms behind how the Quadcopter keeps itself stable.
To do this we'll actually be inspecting some of the official Bitcraze Firmware and it's stabilizer.c
implementation.
It's okay if you don't know C or understand what's going on in this file, that's part of the purpose of this article!
Quadcopters: Sensors
Most intelligent devices existing in the physical world, a quadcopter included, take input from sensors and act on them in some way, possibly producing an output.
Arduino: Motor Fun!
Lately we've been playing with Quadcopters a lot, one of the key components of any copter is it's motors! I'll be writing about motors next, but before that I wanted to share some small, short Arduino 'labs' that you can do either on your own or with your cohort!
All of these 'labs' are as inexpensive as we could manage in terms of parts, with the exception of the Arduino.
Quadcopters: Yaw
So, in our earlier article we talked about Yaw a little but, but we didn't get into details. First, let's take a look at what yaw looks like under simple circumstances, then we'll explain the motion.
Quadcopters: The Board
This post is less about the mechanics of the quadcopter and more to satisfy those curious about what exactly is on the board.
Here are links to the schematics and component placements:
On the pictures below we'll use the schematic symbols as they are shortest.
Quadcopters: Orientation
Spinnning tens of thousands times a second, four rotors cut through the air with their propellors. Their motion lifts them in unison, for they are connected together. In the center of the apparatus sits a bank of sensors, processors, and a radio acting in concert to control the great flow of air passing over it.
Some ways away, perhaps near, perhaps far, its pilot watches over it. Whether by program or controller, they hold the fate of the airborne being in their hands.
Quadcopters are fascinating marriages of physics, electronics, aviation, and computer science that have only become popular in the last couple of years. Examining them allows us to explore a number of interesting topics.
But first, a bit of an introduction...
A Journey into Iterators
One of my favorite features of Rust is iterators. They are a fast, safe, 'lazy' way of working with data structures, streams, and other more creative applications.
You can play along on http://play.rust-lang.org/ and by browsing here. This article is not a subtitute for the documentation or experience.
Reflections on a Quarter Century
Twenty five years is a fairly long time, it turns out.
Stepping over the boundary for another year of life brings tends to bring about it a whirl of emotions, at least temporarily. Simultaneously the past, present, and future being about a poignant picture.
I find myself deeply in such a situation, and I have been for a couple months now. I've taken time to reflect on what has happened, what is happening, and what is to come. A few things seemed worth sharing. They may not be relevant to your situation, be the best advice, or make sense to you, but they are what I have experienced.
Don't take everyone seriously.
Photography from Last Decade
I did a fair amount of photography and digital imaging in high school. Please, enjoy.
Raft: Status Update
In order to celebrate the new beta release of Rust, and as part of reporting requirements for my class, I'm happy to write to you regarding hoverbear/raft!
Gathering Our Voices 2015
Over the past two years I helped the BC Association of Aboriginal Friendship Centres host Gathering Our Voices, the largest annual Youth Conference in BC.
This year, I helped out again! I also teamed up with Science Venture again to host 4 workshops!
Learning Cap'n Proto RPC
Awhile ago, I wrote a First Look at Cap'n Proto. Unfortunately I didn't cover how to utilize it's RPC capabilities. In Rust, this is via the capnp-rpc-rust
crate.
Let's do that!
Rust, Travis, and Github Pages
Rust is probably my favorite programming language (as the contents of this site suggests). My interests in networks, development operations, and systems programming align strongly with Rust. They're pursuing the trifecta: Safe, Fast, Concurrent. It's been a great learning journey for me so far and I learnt to write better code in all languages.
Github Pages is Github's very generous offering of web hosting to your projects. Basically, you create a branch gh-pages
and push it to Github, then browse to https://$USERNAME.github.io/$REPOSITORY
.
Getting Acquainted with MIO
One of my next goals in my Raft project is to tame the tick()
with mio
. In this post, we'll explore what it is, what it can do, and why it matters. First things first: What is MIO?
MIO is a lightweight IO library for Rust with a focus on adding as little overhead as possible over the OS abstractions.
On C-51
I've been following the story about the proposed bill C-51 for a couple weeks now. When the Conservatives introduced the bill I was quite unimpressed with it's lack of oversight, among other things I heard and read about it. I expected it would change due to opposition, so I did not look much further into it.
When the Liberals decided they would support the bill I was suprised, but was busy with midterms, so did not get a chance to look more deeply into the content of the bill. But I couldn't help but be reminded of what Trudeau said about the attack on Parliment months ago:
We are a proud democracy, a welcoming and peaceful nation and a country of open arms and open hearts. We are a nation of fairness, of justice and of the rule of law. We will not be intimidated into changing that. If anything, these are the values and principles to which we must hold on even tighter.
Our dedication to democracy and to the institutions we have built is the foundation of our society and a continued belief in both will guide us correctly into the future. Staying true to our values in a time of crisis will make us an example to the world.
Criminals cannot and will not dictate to us how we act as a nation, how we govern ourselves or how we treat each other. They cannot and will not dictate our values, and they do not get to decide how we use our shared public spaces.
Raft: A First Prototype
As you may have previously read, I've been working on implementing the Raft Distributed Consensus Algorithm in Rust for my classes (and fun!).
Raft: Progress on the Log
I've spent the majority of my free time this last week working on Raft. As you can see from this issue I've been mostly working on the idea of transactions.
A lot has taken shape in the codebase. It still will panic at unimplemented!()
about 300ms after you start a node, but "That's a feature!"
Note: All interfaces and code are not final, and are for educational and interest purposes.
First look at Cap'n Proto in Rust
So, I found this (this, and this) Cap'n Proto library for Rust and wanted to explore it a bit. Unfortunately the documentation is sparse for the library, and I haven't played with Cap'n Proto before.
Cap'n Proto is what's billed as a 'Data Interchange Format' and 'Capability-Based RPC System'. Or a 'Cerealization Protocol'.
Raft Experiences and Repo Made Public!
TL;DR: I've made Hoverbear/raft publicly available (It's still not ready)! Feel encouraged to contribute feedback or code via Github or email!
Raft So Far
I've been working quite a bit on my Raft implementation over the last few days and I must say it's been quite a pleasure to work in Rust, which recently released it's 1.0.0-alpha.
It takes the work out of needing to serialize and deserialize data, it enforces data safety, and it's type checking is incredible. I'm very quickly able to make a change, build it, fix the errors the compiler catches for me, and build again, then test. So far the only times my tests have failed after a successful build was when I was accidently disgarding one of my channels.
Raft in Rust
I'm currently working on implementing the Raft Distributed Consensus protocol in Rust.
This is a project I'm doing for my CSC466 class at the University of Victoria, and I'm hoping to use it as part of one of my projects for CSC462.
At the end of the project I plan to release it as MIT licensed code. Please stay tuned...
OpenWRT QoS
Using the setup from my last two posts, let's play with some QoS parameters and observe the results.
The documentation for OpenWRT's QoS is rather lacking, please feel encouraged to improve it as you go!
Vagrant Clients for OpenWRT VMs
In my last post, we explored how to set up a OpenWRT VM in VirtualBox.
Let's give it some clients! Since the router is using a intnet
s we need to configure some machines to use that as their primary way to connect to the internet.
Since Vagrant has support for multi-machine configurations and gives us minimal builds of machines, we'll use this for our clients.
OpenWRT in Virtualbox
For my CSC 467 project I'm studying the configuration and performance of various QoS parameters in OpenWRT.
The plan is to set up an OpenWRT router in VirtualBox and orcestrate some Vagrant boxes to create a VM network.
A First Look at DACS
Parsing Arguments in Rust
I was reading about the Docopt project the other day and really liked the standardized approach to --help
prompts and argument parsing that it offers.
Luckly, there is a Rust Package to play with!
Longest Common Increasing Sequence
A really 'cute' problem popped up in our Data Structures & Algorithms homework and I wanted to write about it.
If you're in the class this will obviously ruin the fun, so read it later, after you're done!
Init - Configuration
An initialization system's main interface with the user is through it's configuration scripts and service files. Let's take a look at what this looks like for systemd
and OpenRC.
With systemd, daemon configuration is handled by .toml
configuration files. With OpenRC, this task is performed by shell scripts.
A Ghost OpenRC Script
Init - Runlevels and Targets
In this post, we'll look at how runlevels work in two major init systems, systemd and OpenRC. If you're interested in trying out systemd
, I'd suggest using an Arch Linux Live ISO. For those interested in trying OpenRC, check out Funtoo. Both of these will work great in your favorite virtualization solution.
Command Execution in Rust
One of my projects over the upcoming semester is to explore the Linux boot process and the job of the init (initialization) system. The init system is considered PID 1 and is responsible for a significant portion of userspace functionality. Common init systems on Linux are OpenRC and systemd. One of the key parts of an init system is to spawn new processes, like in command shell.
Executing child processes may be useful to any number of programs, but common applications include:
- Shells
- Init systems
- Launchers
- Interfacing with command line applications
Option Monads in Rust
One common monadic structure is the Option
(or Maybe
in Haskell and other languages) type. This can be seen as an encapsulation type. Consider a function which may fail to produce a meaningful value for certain inputs. For example,
Back Channel Keycloak Requests
Integrating Immutant and Keycloak
In my past two posts, we took a surface level look at Immutant then took a look at deploying immutant applications to the Wildfly application server. This time, we'll take a look at how to integrate Keycloak with an Immutant app, again using Docker heavily.
If you haven't read the previous few articles, it might be useful to give them a quick glance over.
Deploying Immutant to Wildfly
Currently, the tutorial on deploying to Wildfly with Immutant 2 is not written. Furthermore,the JBoss documentation isn't particularly tailored towards Clojure applications.
So, let's take a look at how to do this using the convenient dockerfiles provided by the JBoss project. We'll use Docker to simplify deployment and setup of Wildfly, but instructions for a non-docker application should be similar.
Create a basic Immutant application.
Starting from a lein new app learning
command, set up your project like so:
Arch Docker Baseimage
Problem
You need a base image of Arch Linux that you're sure is a-okay.
Starting with Immutant
Immutant is suite of Clojure libraries that is part of the JBossAS/Wildfly ecosystem. Great, fantastic! Clojure is a lisp that runs on the JVM, the CLR, and compiles to Javascript. Lets get started!
Some links we'll need:
Async Auto
I've been exploring the handy async
over the last few days in the lab. One of my projects is a MongoDB API Adapter in Node.js and I was pleased by a novel way of handling control flow.
async.auto()
is a function offered by the libary which allows you to declare a set of tasks and their dependencies, then the library determines the best way to compose the initialization.
Gathering Our Voices 2014
I played dual roles for Gathering Our Voices 2014. One role was with the B.C. Association of Aboriginal Friendship Centres, and the other was with Science Venture.
The purpose of these conferences is to unite youth from throughout the country in learning, healing and sharing and to provide tangible experiences of tools, resources and knowledge that the youth can bring back to their communities.
The conference had 1800 delegates and over 2000 participants.
Gathering Our Voices 2013
I helped plan and host the 11th Gathering Our Voices Aboriginal Youth Conference in Penticton, British Columbia. Much of my time was spent learning proper software design and refining the management and registration software I developed. This was my first 'real' software project.