January 3, 2016

git-watch

git watch is a simple library and CLI tool for monitoring git repositories and triggering events (restart process) when they change.

git watch [1] was designed as a way to easily build and deploy daemons in development when upstream changes. The changes can be either code (in git) or configuration.

So here is a simple example which will automatically rebuild and restart a application when you git push (https://github.com/sigmonsays/git-watch/tree/master/examples/go)

Below in git-watch.yaml we check the upstream every 5 seconds and merge the master branch in. execcmd is “my_application” which is a simple application. When a upstream change is detected, we invoke the updatecmd. If the updatecmd does not succeed, the restart process is aborted. Assuming the update command succeeds, we then invoke the install command. After the install command succeeds, the application is restarted.

February 3, 2015

convenient cli tookit using python

python is a incredibly useful language and has a powerful type system which can be pretty convenient. I’ve written python for many years now and when I need a quick tool, most the time I write it in python.

These tools have built up over time and i’ve been collecting them organically into a python startup file as a set of libraries and functions.

create a file in $HOME/.pythonstartup and cherry pick anything below that you find useful.

January 5, 2015

Google Owned

Ever think about how much information google has collected about you?

Apart from services there are additional things going on that you might not be aware of

Table of Contents

Data Collection
Location Services
Services
Photos
Phone Sensors
Degoogle Resources

Data Collection

Data collected is often uploaded to google servers

IMEI, hardware serial number, Wifi MAC address
AndroidID, Google Ad ID

November 29, 2014

go powered databases

bolt - https://github.com/boltdb/bolt
ledis - http://ledisdb.com/
tiedot - https://github.com/HouzuoGuo/tiedot
cockroachdb - https://github.com/cockroachdb/cockroach

bolt

pure go key-value datastore

ledis

redis like database written in go

attractive to embed redis like features directly into your application which will reduce operational complexity.

tiedot

JSON document database

cockrochdb

A Scalable, Geo-Replicated, Transactional Datastore

design document https://docs.google.com/document/d/11k2EmhLGSbViBvi6_zFEiKzuXxYF49ZuuDJLe6O8gBU/edit

feature summary

ACID transactional semantics
versioned values
primary design goal is consistency and survivability
aims to tolerate disk, server, rack and datacenter failures

related technologies

September 22, 2014

ban systemd

updates

Nov2014 - an exploit in systemd resolver - https://news.ycombinator.com/item?id=8595335
- Just another reason to do one thing well and not try to do too much. C’mon systemd, focus.
- Please help me understand why a init.d replacement needs to provide a resolver.
- some gems from this thread
  - “I find the design of systemd-resolved to be very strange. It uses dbus to talk to glibc, and it seems to be a new, from-scratch implementation of a DNS resolver. To be clear, I don’t really think it matters whether systemd-resolved is under the systemd umbrella, but I do think that the design has a lot of unnecessary NIH syndrome.”
  - “This is a perfect example of why the systemd approach of putting a bunch of disparate components under a single tightly-coupled umbrella is bad engineering.”
  - “It’s mind blowing to realize that because of the init system we have on our system we are now vulnerable to DNS poisoning.”
- anyways, give the thread a read and see for yourself how systemd should be abandoned
website which describes a lot of good points - http://boycottsystemd.org/

September 11, 2014

building packages with docker

This is a simple idea I had while waiting for a build to complete.

The background Its common to use dpkg-buildpackage inside of a chroot managed by pdebuild. The pdebuild script takes a minimal base image tarball and installs all dependencies in it required to build your package.

The build dependencies come from debian/control Build-Depends section. The package being built is a python application using virtualenv. A lot of the packages come from apt however. There is a bunch of python pip/easy install work performed too.

September 7, 2014

docker wishlist

things I find that I wish docker supported

dockerfile

store build context with the image
- provides ability to change original dockerfile for images you download
- saves dockerfile and other build artifacts
- similar in concept to a “source package”
support for variables

There is very little that the dockerfile actually supports beyond basic static directives.

If dockerfile supported templates and variables, we could allow repetitive lines to be reduced to the bare minimum. This would allow snippets like this to be greatly reduced. We shouldn’t have to repeat 0.1.3 everywhere, it should just be a variable!

September 5, 2014

docker networking performance

While trying to determine the performance overhead of a simple docker container setup, I discovered several issues related to network performance.

two issues I have encountered

docker-proxy handles local network traffic when using the hosts bridge IP (or localhost)
connection tracking overhead of NAT increases memory usage

my setup

I am using docker containers as an easy way to deploy backends on a host. I wish to use the hosts bridge IP as a backend to route requests from the load balancer. The load balancer is just a haproxy instance running on port 8080 and it uses 500X as backend ports. I had to doctor up the output a little but this is still the output of docker ps.

September 2, 2014