avatar

Andrew Womeldorf

Software Engineer

Nomad Exec Driver on Raspberry Pi 4

I received an error while trying to launch a Nomad job with the Exec driver on my cluster of Raspberry Pi 4’s. Something about CGroups. The Pi4 does not enable the memory cgroup by default, which the exec driver requires to execute correctly. To validate this assertion: $ cat /proc/cgroups #subsys_name hierarchy num_cgroups enabled ... memory 0 99 0 ... To fix, on each node, an update to /boot/cmdline.txt needed to be made, and a reboot needed to occur.

NFS, CSI, and Nomad

After far too many days of failure, I got an NFS share working with the CSI Plugin running on Nomad (version 1.3.3)! I have a three-node pi cluster running Nomad. I created the NFS share on the pi that has an external SSD plugged into it, and was able to create new files and directories on it from all three pi’s, after mounting the NFS share. I had some odd permissioning things, where I had to use sudo on the node sharing the drive in order to make a directory, but then everyone could use it.

OSX Encrypt Directory

I’ve recently begun volunteering some of my time at church to help run the sound board for worship services. The church has a digital board, which is foreign to me, and there’s been a relatively steep learning curve to get up to speed. Given that I’m not there where I can learn and practice on the board every week, I need to take pretty good notes. Besides that, I like to keep notes about each service as we do rehearsals, so that I can keep the important aspects to highlight written down for reference.

Bash Looping JQ

#!/usr/bin/env bash FAKE_API_RESPONSE='{"name":"package1.file1.js","content":"var foo = {\n\t\"bar\" = \"banana\"\n}\nconsole.log(foo)"}' echo $FAKE_API_RESPONSE | jq -r '[.name,.content] | @tsv' | while IFS=$'\t' read -r apiname script; do dirname="./$(echo "$apiname" | cut -d '.' -f 1)" filename="$(echo $apiname | echo "$dirname/$(cut -d '.' -f 2).js")" mkdir -p $dirname echo "$script" | sed 's/\\n/\ /g' | sed 's/\\t/\ /g' > $filename done I’m working with an API that returns several JSON objects, each object as a new line.

Unmarshal JSON Field with Variable Type

The API endpoint I’m querying returns a JSON object, where some fields are either an empty string or a map[string]interface{}. The map[string]interface{} actually follows a consistent schema, but since the actual received value is not a consistent type, I’m having to put the type as interface{} and check those fields specially. package main import ( "encoding/json" "log" ) type Foo struct { Name string `json:"name"` SillyField interface{} `json:"silly_field"` } func printSillyField(f Foo) { if f.

Go Concurrency Rate Limiting

Code first, story later. package main import ( "encoding/json" "fmt" "log" "math" "math/rand" "sync" "time" ) type inputType struct { offset int batchSize int } // sleep a random amount of time, and return errors sometimes func fakeAPI(input inputType) (json.RawMessage, error) { rand.Seed(time.Now().UnixNano()) n := rand.Intn(10) time.Sleep(time.Duration(n) * time.Millisecond) mod := math.Mod(float64(input.offset), 11) if mod == 0 { return json.RawMessage{}, fmt.Errorf("error: divisible by 11: %d", input.offset) } return json.RawMessage(fmt.Sprintf(`{"offset": %d}`, input.

JSON to SQLite

I have a script that queries multiple records from an external API and prints a new JSON string per record to stdout. I want a nice way to explore those results. In the past, I’ve used jq to format records as CSV and import them into SQLite. Today, I learned about sqlite-utils from Datasette. It looks like it does lots of cool things, but for today, I can pipe a list of JSON objects to be inserted into a table on a database.

PHAR

I wonder how simiar PHAR is to Pex. Is PHAR a viable way to build and deploy PHP in the same way that Pex does for Python? The linked pex is from Twitter’s Pants build system, but is implemented in other places as well. I’ve used it in Please, and it works wonderfully. In short, pex is an executable zip file, the contents of which are all of the python files needed to run the script.

Cobra Replacement: Coral

TL;DR: Try the Coral library as a drop-in replacement for Cobra. I wanted to make a CLI wrapper for an SDK I’ve been working on. I’ve used Cobra in the past, and it’s been very handy, but I remember having seen a “soft-fork” of it on HackerNews a while ago. It took a minute to find it, since I couldn’t remember if it was a replacement of Cobra or Viper, and the README confuses the issue a bit.

Not All DLQ Messages Are Alike

I have messages being placed in a Dead Letter Queue (DLQ) that I created. The AWS Lambda Function that processed these messages failed the messages more than the allowed number of times, and were therefore put into the DLQ. According to the Lambda Docs: Lambda sends the event to the dead-letter queue as-is, with additional information in attributes. And there should be some Attributes on the message: RequestID (string) ErrorCode (Number) ErrorMessage (string) However, I don’t see those attributes.