Last updated: February 23, 2018 12:22 PM (All times are UTC.)
Lots of companies want to collect data about their users. This is a good thing, generally; being data-driven is important, and it’s jolly hard to know where best to focus your efforts if you don’t know what your people are like. However, this sort of data collection also gives people a sense of disquiet; what are you going to do with that data about me? How do I get you to stop using it? What conclusions are you drawing from it? I’ve spoken about this sense of disquiet in the past, and you can watch (or read) that talk for a lot more detail about how and why people don’t like it.
So, what can we do about it? As I said, being data-driven is a good thing, and you can’t be data-driven if you haven’t got any data to be driven by. How do we enable people to collect data about you without compromising your privacy?
Well, there are some ways. Before I dive into them, though, a couple of brief asides: there are some people who believe that you shouldn’t be allowed to collect any data on your users whatsoever; that the mere act of wanting to do so is in itself a compromise of privacy. This is not addressed to those people. What I want is a way that both sides can get what they want: companies and projects can be data-driven, and users don’t get their privacy compromised. If what you want is that companies are banned from collecting anything… this is not for you. Most people are basically OK with the idea of data collection, they just don’t want to be victimised by it, now or in the future, and it’s that property that we want to protect.
Similarly, if you’re a company who wants to know everything about each individual one of your users so you can sell that data for money, or exploit it on a user-by-user basis, this isn’t for you either. Stop doing that.
The key point here is that, if you’re collecting data about a load of users, you’re usually doing so in order to look at it in aggregate; to draw conclusions about the general trends and the general distribution of your user base. And it’s possible to do that data collection in ways that maintain the aggregate properties of it while making it hard or impossible for the company to use it to target individual users. That’s what we want here: some way that the company can still draw correct conclusions from all the data when collected together, while preventing them from targeting individuals or knowing what a specific person said.
In the 1960s, Warner and Greenberg put together the randomised response technique for social science interviews. Basically, the idea here is that if you want to ask people questions about sensitive topics — have they committed a crime? what are their sexual preferences? — then you need to be able to draw aggregate conclusions about what percentages of people have done various things, but any one individual’s ballot shouldn’t be a confession that can be used against them. The technique varies a lot in exactly how it’s applied, but the basic concept is that for any question, there’s a random chance that the answerer should lie in their response. If some people lie in one direction (saying that they did a thing, when they didn’t), and the same proportion of people lie in the other direction (saying they didn’t do the thing when they did), then if you’ve got enough answerers, all the lies pretty much cancel out. So your aggregate statistics are still pretty much accurate — you know that X percent of people did the thing — but any one individual person’s response isn’t incriminating, because they might have been lying. This gives us the privacy protection we need for people, while preserving the aggregate properties that allow the survey-analysers to draw accurate conclusions.
It’s something like whether you’ll find a ticket inspector on a train. Train companies realised a long time ago that you don’t need to put a ticket inspector on every single train. Instead, you can put inspectors on enough trains that the chance of fare-dodgers being caught is high enough that they don’t want to take the risk. This randomised response is similar; if you get a ballot from someone saying that they smoked marijuana, then you can’t know whether they were one of those who were randomly selected to lie about their answer, and therefore that answer isn’t incriminating, but the overall percentage of people who say they smoked will be roughly equal to the percentage of people who actually did.
Let’s imagine you’re, say, an operating system vendor. You’d like to know what sorts of machines your users are installing on (Ubuntu are looking to do this as most other OSes already do), and so how much RAM those machines have would be a useful figure to know. (Lots of other stats would also be useful, of course, but we’ll just look at one for now while we’re explaining the process. And remember this all applies to any statistic you want to collect; it’s not particular to OS vendors, or RAM. If you want to know how often your users open your app, or what country they’re in, this process works too.)
So, we assume that the actual truth about how much RAM the users’ computers have looks something like this graph. Remember, the company does not know this. They want to know it, but they currently don’t.
So, how can they collect data to know this graph, without being able to tell how much RAM any one specific user has?
As described above, the way to do this is to randomise the responses. Let’s say that we tell 20% of users to lie about their answer, one category up or down. So if you’ve really got 8GB of RAM, then there’s an 80% chance you tell the truth, and a 20% chance you lie; 10% of users lie in a “downwards” direction, so they claim to have 4GB of RAM when they’ve actually got 8GB, and 10% of users lie in an “upwards” direction and claim to have 16GB. Obviously, we wouldn’t actually have the users lie — the software that collects this info would randomly either produce the correct information or not with the above probabilities, and people wouldn’t even know it was doing it; the deliberately incorrect data is only provided to the survey. (Your computer doesn’t lie to you about how much RAM it’s got, just the company.) What does that do to the graph data?
We show in this graph the users that gave accurate information in green, and inaccurate lies in red. And the graph looks pretty much the same! Any one given user’s answers are unreliable and can’t be trusted, but the overall shape of the graph is pretty similar to the actual truth. There are still peaks at the most popular points, and still troughs at the unpopular ones. Each bar in the graph is reasonably accurate (accuracy figures are shown below each bar, and they’ll normally be around 90-95%, although because it’s random it may fluctuate a little for you.) So our company can draw conclusions from this data, and they’ll be generally correct. They’ll have to take those conclusions with a small pinch of salt, because we’ve deliberately introduced inaccuracy into them, but the trends and the overall shape of the data will be good.
The key point here is that, although you can see in the graph which answers are truth and which are incorrect, the company can’t. They don’t get told whether an answer is truth or lies; they just get the information and no indication of how true it is. They’ll know the percentage chance that an answer is untrue, but they won’t know whether any one given answer is.
Can we be more inaccurate? Well, here’s a graph to play with. You can adjust what percentage of users’ computers lie about their survey results by dragging the slider, and see what that does to the data.
of submissions are deliberately incorrect
Even if you make every single user lie about their values, the graph shape isn’t too bad. Lying tends to “flatten out” the graph; it makes tall peaks less tall, and short troughs more tall, and every single person lying probably flattens out things so much that conclusions you draw are probably now going to be wrong. But you can see from this that it ought to be possible to run the numbers and come up with a “lie” percentage which accurately balances the company’s need for accurate information with the user’s need to not provide accuracy.
It is of course critical to this whole procedure that the lies cancel out, which means that they need to be evenly distributed. If everyone just makes up random answers then obviously this doesn’t work; answers have to start with the truth and then (maybe) lie in one direction or another.
This is a fairly simple description of this whole process of introducing noise into the data, and data scientists would be able to bring much more learning to bear on this. For example, how much does it affect accuracy if user information can lie by more than one “step” in every direction? Do we make it so instead of n% truth and 100-n% lies, we distribute the lies normally across the graph with the centrepoint being the truth? Is it possible to do this data collection without flattening out the graph to such an extent? And the state of the data art has moved on since the 1960s, too: Dwork wrote an influential 2006 paper on differential privacy which goes into this in more detail. Obviously we’ll be collecting data on more than one number — someone looking for data on computers on which their OS is installed will want for example version info, network connectivity, lots of hardware stats, device vendor, and so on. And that’s OK, because it’s safe to collect this data now… so how do our accuracy figures change when there are lots of stats and not just one? There will be better statistical ways to quantify how inaccurate the results are than my simple single-bar percentage measure, and how to tweak the percentage-of-lying to give the best results for everyone. This whole topic seems like something that data scientists in various communities could really get their teeth into and provide great suggestions and help to companies who want to collect data in a responsible way.
Of course, this applies to any data you want to collect. Do you want analytics on how often your users open your app? What times of day they do that? Which OS version they’re on? How long do they spend using it? All your data still works in aggregate, but the things you’re collecting aren’t so personally invasive, because you don’t know if a user’s records are lies. This needs careful thought — there has been plenty of research on deanonymising data and similar things, and the EFF’s Panopticlick project shows how a combination of data can be cross-referenced and that needs protecting against too, but that’s what data science is for; to tune the parameters used here so that individual privacy isn’t compromised while aggregate properties are preserved.
If a company is collecting info about you and they’re not actually interested in tying your submitted records to you (see previous point about how this doesn’t apply to companies who do want to do this, who are a whole different problem), then this in theory isn’t needed. They don’t have to collect IP addresses or usernames and record them against each submission, and indeed if they don’t want that information then they probably don’t do that. But there’s always a concern: what if they’re really doing that and lying about it? Well, this is how we alleviate that problem. Even if a company actually are trying to collect personally-identifiable data and they’re lying to us about doing that it doesn’t matter, because we protect ourselves by — with a specific probability — lying back to them. And then everyone gets what they want. There’s a certain sense of justice in that.
At some point in the past, programmers used to recommend drawing flowcharts before you start coding. Then they recommended creating CRC cards, or acting through how the turtle will behave, or writing failing tests, or getting the types to match up, or designing contracts, or writing proofs, but the point is that in each case they’re there for eliciting thought before the code gets laid down.
None of these things is mutually exclusive, none of these things is the one true way, but the fact that they all isolate some part of solving the problem from some part of coding the solution is the telling point. The problem is not having the correct type system or test coverage or diagram format, the problem is trying to work in two (or more) levels of abstraction – the problem domain and the computer – at the same time.
I’m very glad to read the news that Center Parcs pulls Daily Mail ads over Tom Daley article — of its advertisement next to a homophobic Richard Littlejohn article, the holiday organisation said “We felt this placement was completely unacceptable and therefore ceased advertising with the Daily Mail with immediate effect”. London Southbank Centre also said, “We monitor the environment in which our advertising appears, to ensure the values of a publication are compatible with our own. We have no future plans to advertise within the Daily Mail”.
Predictably, there has been a little faux-anguish about “free speech”, which is mis-placed. I’m a great believer that anyone should be able to say what they want to say (even “hate speech” as long as they’re not inciting violence). My position is “I disagree with what you say, but will defend to death your right to say it. But that doesn’t mean I want to waste my time listening to you”.
Similarly, I have a choice whether to fund your free speech. In our democratic capitalist society, I have a once-every-four-years opportunity to vote between largely-indistinguishable political parties in a General Election. But the true power I have is in my choice of goods and service that I consume.
So it’s a game of “follow the money”: Centre Parcs wants my money (that’s why they advertise); the Daily Mail wants some of Centre Parc’s money; Richard Littlejohn wants some of the Daily Mail’s money. (He’s perfectly free to set up a blog to publish his views, after all. He has every right to air them, and everyone has the right to read them, or not, as they choose.)
So if I tell a company that I won’t buy their products because they indirectly fund Littlejohn, or Breitbart, that is my right and my (only) leverage as a consumer. The key is to tell organisations why you consume their products or not. That’s why my tweeting “Good” to Centre Parcs is not “virtue signalling”; it’s letting them know that I will continue to purchase their products, and why. (The last good time I had with my dad — who was gay — three months before he died was a family holiday in Centre Parcs, which he paid for. We had a great time.)
Am I trying to close the Daily Mail down? Not at all. But right now, money I give to Co-op, Hertz, Visa etc indirectly subsidises the Daily Mail’s cover price. If those companies pull their advertising, then avid Richard Littlejohn fans can continue to pay to read it, they’ll just pay more. If they don’t want to pay more, and they stop buying it, that’s up to them. It’s called “the free market”.
There are many things to dislike about consumerism and capitalism. But the fact I can tweet to organisations and exercise financial influence is a power that I cherish, and will continue to use.
My short-lived first plan for a career was in Physics. That’s what my first degree was in, but I graduated with the career goal “do something that isn’t a D.Phil. in Physics” in mind. I’d got on quite well with computers as a hobbyist, and the computing and electronics practicals in my course labs. A job as systems administrator for those very systems (a hotchpotch of NeXTSTEP, Solaris, OpenBSD, and Mac OS X) came up at the time that I graduated so I applied, got it, and became a computerer.
It’s easy to forget two things: the first is that at some point in the past, you didn’t know what you know now, but you learnt it because you were allowed to participate and were taught. Even if you think you were a self-learner, you had access to playground materials, books, tutorials, online documentation…and someone made those for you and allowed you to use them.
The second is what it was like not to know those things. I remember not understanding how OOP was anything more than putting dots in the name of your function, but now I understand it differently, and don’t know what the thing was that changed.
The second of these things shows how easy it is to be the gatekeeper. If you don’t remember what not understanding something was like, then maybe it came naturally, and if it isn’t coming naturally to someone else well maybe they just don’t get it. But the first shows that you didn’t get it, and yet here you are.
When faced with a gatekeeper, I usually react flippantly. Because of my Physics background, I explain, I know how quantum physics works, and how that enables semiconductors, and how to build a semiconductor transistor, then a NAND gate, then a processor, and basically what I’m saying is I don’t see how anyone who doesn’t know that stuff can claim to know computers at all.
But what I say to everyone else is “this stuff is really interesting, let me show it to you“.
Be the keymaster, not the gatekeeper.
I have a love/hate relationship with Clojure tooling.
I wanted to learn functional programming (FP) in a Lisp because:
1. There’s a thing I want to do and it felt like the right way to do it
N.B. ‘right’ is not always ‘easiest’
2. Lisp once beat me up quite badly. I’m bigger now. I wanted to go back and punch it on the nose
3. I won’t really ‘get’ immutability until I do some, with no option to cheat.
3. I find homoiconic functional programming conceptually elegant…
…then you try using the Clojure development environment and discover there isn’t one that everyone agrees on and the one used by expert requires you to learn a new language of keyboard hieroglyphs first.
I’ve done just enough Lisping to think that we are being as irresponsible teaching kids only object oriented programming as the BBC were in teaching them BASIC, but it would be actual child abuse to introduce them to FP via emacs and a language dependent on the underlying Object-Oriented Java Virtual Machine for it’s connection to libraries & reality.
This morning I realised I hadn’t plugged my Raspberry Pi in for ages. If you were a child who started coding with Scratch and a few drum loops on Sonic Pi then maybe a bit of the Squeak Smalltalk (Scratch is written in Squeak) or Python in a nice IDE, imagine being handed the ancient emacs scrolls and sent into a corner for a week to learn spells, before you could even start to learn to function. FP is going to initially make soup of your flabby imperative, ‘place is state’-damaged brain anyway. There is no need to make it harder. Teaching languages don’t have to train you for a job, only to think.
If we are going to raise functional children, we need a gentle slope up to Clojure. Clojure is probably the best practical language but it’s too hard. I’ve seen a few people suggest starting with Racket, which is a version of the Scheme Lisp dialect. This morning I poured DrRacket onto my Pi.
There was a minor hiccup with DrRacket not knowing what language I wanted it to read. Kids would need to be protected from that. I wanted “Determine language from source”, not to tell it I was a ‘beginning student’.
Because DrRacket is multi-lingual, the first line of the source code tells it what language to read:
You type in code then press the run button. This is not your grandfathers emacs REPL. Children should have no natural fear of their (parens). Ooh look, cats!
(define (extract str)
(substring str 4 8))
(extract “the cats out of the bag”)
We have a whole load of practices in programming that only really work well if you’re already good at whatever the process is supposed to help with.
Scrum is a process improvement framework, but only if you already know how to do process improvement. If you don’t, then Scrum is just the baseline mini-waterfall process with a chance to air your dirty laundry every fortnight.
Agile is good at helping you embrace change, but only if you’re already good enough at managing change to understand which changes should be embraced.
#NoEstimates helps you avoid the overhead of estimates, but only if you’re already good enough at estimates to know that you always write user stories that take 0.5-2 days to implement.
TDD helps you design your APIs, but only if you’re already good enough at API design to understand things like dependency injection and loose coupling.
Microservices help you isolate modules, but only if you’re already good enough at modularity not to get swamped in HTTP calls.
This is all very well for selling consultancy (“if your [agile] isn’t working, then you aren’t [agiling] hard enough, let me [agile] you some more”) but where’s the on-ramp?
I think I found a bug in a Henry Dudeney book.
Dudeney was a really famous puzzle creator in Victorian/Edwardian times. For Americans: Sam Loyd was sort of an American knock-off of Dudeney, except that Loyd stole half his puzzles from other people and HD didn’t. Dudeney got so annoyed by this theft that he eventually ended up comparing Loyd to the Devil, which was tough talk in 1910.
Anyway, he wrote a number of puzzle books, and at least some are available on Project Gutenberg, so well done the PG people. If you like puzzles, maths or thinking sorts, then there are a few good collections (and there are nicer to read versions at the Internet Archive too). The Canterbury Puzzles is his most famous work, but I’ve been reading Amusements in Mathematics. In there he presents the following puzzle:
I have nine counters, each bearing one of the nine digits, 1, 2, 3, 4, 5, 6, 7, 8 and 9. I arranged them on the table in two groups, as shown in the illustration, so as to form two multiplication sums, and found that both sums gave the same product. You will find that 158 multiplied by 23 is 3,634, and that 79 multiplied by 46 is also 3,634. Now, the puzzle I propose is to rearrange the counters so as to get as large a product as possible. What is the best way of placing them? Remember both groups must multiply to the same amount, and there must be three counters multiplied by two in one case, and two multiplied by two counters in the other, just as at present.
In this case a certain amount of mere “trial” is unavoidable. But there are two kinds of “trials”—those that are purely haphazard, and those that are methodical. The true puzzle lover is never satisfied with mere haphazard trials. The reader will find that by just reversing the figures in 23 and 46 (making the multipliers 32 and 64) both products will be 5,056. This is an improvement, but it is not the correct answer. We can get as large a product as 5,568 if we multiply 174 by 32 and 96 by 58, but this solution is not to be found without the exercise of some judgment and patience.
But, you know what? I don’t think he’s right. Now, I appreciate that he probably had to spend hours or days trying out possibilities with a piece of paper and a fountain pen, and I just wrote the following 15 lines of Python in five minutes, but hey, he didn’t have to bear with his government trying to ban encryption, so let’s call it even.
from itertools import permutations nums = [1,2,3,4,5,6,7,8,9] values =  for p in permutations(nums, 9): one = p*100 + p*10 + p two = p*10 + p three = p*10 + p four = p*10 + p if four > three: continue # or we'll see fg*hi and hi*fg as different if one*two == three*four: expression = "%s*%s = %s*%s = %s" % ( one, two, three, four, one*two) values.append((expression, one*two)) values.sort(key=lambda x:x) print("Solution for 1-9") print("\n".join([x for x in values]))
The key point here is this: the little programme above indeed recognises his proposed solutions (
158*32 = 79*64 = 5056 and
174*32 = 96*58 = 5568) but it also finds two larger ones:
584*12 = 96*73 = 7008 and
532*14 = 98*76 = 7448. Did I miss something about the puzzle? Or am I actually in the rare position of finding an error in a Dudeney book? And all it took was seventy years of computer technology advancement to put me in that position. Maths, eh? Tch.
It’s an interesting book. There are lots of money puzzles, in which I have to carefully remember that ha’pennies and farthings are a thing (a farthing is a quarter of a penny), there are 12 pennies in a shilling, and twenty shillings in a pound. There’s some rather racist portrayals of comic-opera Chinese characters in a few of the puzzles. And my heart sank when I read a puzzle about husbands and wives crossing a river in a boat, where no man would permit his wife to be in the boat with another man without him, because I assumed that the solution would also say something like “and of course the women cannot be expected to row the boat”, and I was then pleasantly surprised to discover that this was not the case and indeed they were described as probably being capable oarswomen and it was likely their boat to begin with! Writings from another time. But still as good as any puzzle book today, if not better.
Follow the adventures of my Maze Runner robot as it makes a bid for freedom.
(This is just a quick filler post while I sort out the rest of the updated blog. Watch this space for new projects coming soon - subscribe below to keep in the loop.)
For the last few weeks, my when-I-get-to-it project has been machoo, which is sort of an object-oriented system on the HURD but mostly an excuse to learn about how Mach messaging works.
I decided to build a Smalltalk-style “give me any old selector, and I’ll do something with it” messaging system on Mach, which is already a messaging system. I don’t yet know whether this is a good idea, but I like the design principle of highly loosely-coupled objects telling each other the names of the things to do.
The substrate for this system is MIG, the Mach Interface Generator. It takes descriptions of routines in an IDL that include the names of the supported routines and the argument names, types and return types. It’d certainly be possible to build each message I want to be able to send as a MIG routine, but then the client would have to import interfaces for every type it wanted to use: no bad thing, and many OO systems work that way, but not what I want.
MIG generates a ‘demuxer’ function that examines a message sent to a port, and dispatches it to a handler to the named routine. As an aside, it is highly likely that I’ve got one to many layers of indirection; that if I wrote my own function to take the place of the demuxer I could handle dispatch there without receiving arguments from a dispatcher to do more dispatch. OK, I could, but I don’t currently know how.
So what I ended up with is a two-level system, something like Smalltalk or ObjC. A “class” is a HURD translator registered on the filesystem, you send the zero-arguments message
machoo_create_object to the class and get a send right to the new object. This is two-level in that if you need to initialise the object, you would then send an
init message to the created object. Objects are messaged using the
machoo_msg_send routine, which takes a string selector like Smalltalk.
There’s currently exactly one implementation of a class, a do-nothing null object class. It does the thing that I would extract out into a pattern for other classes: its response to a
machoo_create_object message is to spawn a new thread to back the new object. So it’s like an Erlang-style system where each object has its own execution context, except that task switching is handled by the kernel rather than some virtual machine. All of the objects of a class are in a single task, and they are each on their own thread.
I still need to solve some problems, like how to return a reference to ‘this object’ from a message. I need to build more than one class of object, and extract out the common parts of class-ness so that a new class is only defined by its response to messages. So far this is proving to be an interesting and educational project.
I think the last technical conference I attended was FOSDEM last year, and now I’m sat in the lobby of the Royal Library of Brussels working on a project that I want to take to some folks at this year’s FOSDEM, and checking the mailing lists of some projects I’m interested in to find meetups and relevant talks.
It’s been even longer since I spoke at a conference, I believe it would have been App Builders in 2016. When I left Facebook in 2015 with the view of taking a year out of software, I also ended up taking myself out of community membership and interaction. I felt like I didn’t know who I was, and wasn’t about to find out by continually defining myself in terms of other people.
I would have spoken at dotSwift in 2016, but personal life issues probably also related to not knowing who I was stopped me from doing that. In April, I went to Zurich and gave what would be my last talk as a programmer to anyone except immediate colleagues for over 18 months.
During this time, I found that I don’t mind so much what I’m working on: I have positive and negative opinions of all of it. I have lots of strong opinions, and lots of syntheses of other ideas, and need to be a member of a community that can share and critique these opinions, and extend and develop these syntheses.
Which is why, after a bit of social wilderness, I’m reflecting on this week’s first conference and planning my approach to the second.
The usual disclaimer: these reviews are written primarily for myself, so sorry if it’s a little self-indulgent!
Here’s some of my highlights from 2017:
The mastermind group I’m part of is still going strong (I’ve written about the benefits of mastermind groups). In October, we held our first mastermind retreat. We booked an Airbnb in Gloucestershire which served as a great place to work, plan, goal set, and chat about our businesses. I left the retreat feeling revitalised and with a greater clarity on my goals. I’m really excited about doing this again. I’d also love to do a solo retreat.
In October, I gave my first public talk at SWM. The talk was called “How to run a freelance business without going crazy.” It was well received and I enjoyed the experience, despite being insanely nervous before hand. Shout out to my friend Dave Redfern for asking and encouraging me to speak.
I’d thought about public speaking for years but had always put it off when I was asked. This feels like a big weight off my shoulder. Its given me confidence that I can do public speaking.
I read 23 books in 2017, just missing out on my goal of 24. Most of the books I read I really enjoyed, and I’ll write up a post of my favourites. 24 books per year – 2 per month – feels about the right pacing for me at the moment, so I’ll stick to that goal for this year. I do want to make an effort to read more fiction though.
inline-block is a Slack community I started back in 2016. It’s now grown to over 230 folks. While it’s not crazy active (I like it that way), there is some really great discussion taking place. I also started a small meetup with my friend Dave Redfern called Geek+Food. The premise is super simple: every month we pick a new restaurant in Birmingham and then talk shop while eating good food (Birmingham has many great places to eat). It’s been fun.
Over the years, I’ve developed an addiction to Twitter. Launching Tweetbot with Alfred is muscle memory at this point. A slight lapse in and concentration and BAM: Twitter is in front of me before I know it. So, in December, I went completely dark on social media and avoided Twitter, Facebook and Instagram for the month. It was easier than I expected and it felt great, like I’d kicked the addiction. I’m back on the socials now, but I don’t check-in nearly as often. I’ll be taking regular breaks from social media again this year. It has had a huge impact on my productivity, that’s for sure.
I felt more organised and in-control for most of 2017 than I had in previous years. I made a few small changes which made a big impact last year:
Weekly reviews. I had tried to do weekly reviews in the past, but I never really stuck to them. Last year I made it a goal to perform a weekly review every week. Often this takes place on a Sunday, but occasionally on a Friday or a Monday. It’s something I look forward to: get a coffee, sit down with my laptop (or iPad), and spend 30 minutes reviewing and going through things for the following week. I’ll write more about my weekly reviews, but suffice to say it has really helped me focus on what’s important and stay on top of the little things that often fall through the cracks or build up over time.
Introduced Sanebox for email. I’ve always been pretty good at email in the sense that I respond quickly and follow “inbox zero”. But it was taking up too much time. Around April time, I introduced Sanebox and it has been a game-changer. Sanebox works by learning which emails you’d like to appear in the inbox and which emails should be sent to a folder to review later. I use the SaneBox digest feature to go through my emails in the evening, while all the important stuff is sent straight to my inbox. It’s really reduced the amount of time I spend dealing with email.
In February, we redecorated both bedrooms – one of which is my office. The main new addition to the office is a standing desk from Ikea which has been fantastic. I usually stand in the morning and sit in the afternoon. My office space feels great and I love working here (which isn’t always a great thing when my commute is 10 steps).
One of the benefits of writing yearly reviews is that it allows you to look back and spot patterns or trends. There’s a few reoccurring things in the “what went badly” list in particular that I want to address this year.
I sent out 4 newsletters and published 12 blog posts in 2017. I didn’t write anywhere near as much as I’d like to. About halfway through the year I stopped getting up early which disrupted my morning routine and completely wiped out my writing time. One of my priorities for 2018 is to get back into my morning routine.
This is another recurring theme from my yearly reviews. Although I enjoy working from home and the solitude that comes with it, spending too much time at home alone can quickly drive you crazy. And to make matters worse, we had to say goodbye to our dog midway through the year which made working from home feel even more isolating.
So the goal is the same as in 2015: more coffee shops, more co-working spaces, more changes of scenery. Possibly even a hot desk, if I can find a suitable place.
I put on a little weight steadily over the year. Nothing major, but not heading in the direction I want to go. I did play plenty of squash throughout the year and my general level of fitness increased, but I want to shed a few pounds this year.
I really wanted to build and a ship a product but that didn’t materialise. No excuses really, I just didn’t do it. Yes, I was busy with client work but I could have carved out a month for building something if I had wanted to.
Although I did visit Edinburgh, Swansea, and plenty of places in England, I didn’t travel as much I wanted to this year. I didn’t go abroad and that’s something I’d like to fix this year. We’d also love to be more spontaneous and just go away for a weekend when we feel like it (we’re meticulous planners and so travel is always planned well in advance).
This isn’t something I expected to include on my list as I consider myself an avid learner. I’m always reading non-fiction books or blog posts about self-improvement or something I don’t know much about. One of the problems of being a freelancer is that you often work alone and don’t get chance to learn from others around you. And I didn’t really schedule the time to learn new things. There’s plenty of web-based things I want to get a better handle on such as CSS Grid Layout.
Launch first product. Nope, that didn’t happen last year but it’s still something I’m working towards.
Write to my mailing list every two weeks. This didn’t happen, but I did restart my newsletter in a new monthly format towards the end of the year.
Publish 12 book notes. I read 23 books but only published book notes for 3 of them.
Save 4 months salary. Done. I started saving more last year.
One movie night per week. We did okay at this. It started off well, we then started missing date nights in the summer, but then picked up throughout winter.
Normally I’d set my goals for the year, but these days I prefer to set quarterly goals (which I wrote about in a recent post). The problem with setting yearly goals is that things often change. By working in 12 week blocks, you can change course more frequently if you need to.
So with that said, these aren’t goals per se, but just things I’m thinking about for 2018:
Harry Roberts, on his website reaching 10 years of age:
“Having this website changed and shaped my career. If you don’t have a blog, I urge you, start working on one this weekend. Your own blog, with your own content, at your own domain. It might just change your life.”
It’s a sentiment I agree with. This website is my base. It attracts new clients. If I wanted to get a job, I can use this website as a portfolio. If I want to sell a product, my website is where I’ll do it. So investing in my website is a thing that will be useful for years to come.
You may have noticed that “business” was neither in my good or bad list this year. It could have been in either. It was good in that it was my most profitable year to date and I was busy for most of it, but it was bad because I haven’t been building real assets. The problem I face with my business right now is that if I take some time off, or if I’m ill, then the business makes no money. This is why I want to explore building and selling products.
I want to get back into the groove of publishing regularly. I plan on continuing to send out a monthly newsletter, along with 2 or so posts per month.
I don’t want to go whole hog and give up social media. I enjoy it and get a lot of value from it. But I also recognise the downsides: the huge amount of time it can take up, the distraction it creates and the occasional negativity that can come from it. This year I want to keep my social media usage in-check. That means giving it up for a week or a month at a time now and then, and monitoring my overall activity.
This year I’d really like to get back into my hobby of photography. And it’s a good excuse to travel more. I’d love to visit the Lake District this year and we have a few other trips planned.
I’ve tried journalling in different forms and it’s never really stuck. However, gratitude journalling is something I enjoy. So, each week, I plan to write down a couple of things that were awesome, exciting, or that I was grateful for. Every year passes by so quickly and life becomes a blur. It can be hard to remember what happened. Hopefully this gratitude journal is a small step in fixing that, and should make next year’s review easier!
That’s it from me. I hope you have a great 2018.
“Rule Forty-two. All persons more than a mile high to leave the court.”
Everybody looked at Alice.
“I”m not a mile high,” said Alice.
“You are,” said the King.
“Nearly two miles high,” added the Queen.
“Well, I shan’t go, at any rate,” said Alice: “besides, that’s not a regular rule: you invented it just now.”
“It’s the oldest rule in the book,” said the King.
“Then it ought to be Number One,” said Alice.
So far it’s been a nice day, with lots of people wishing me happy birthday from midnight last night (including a rather lovely thing from Jono). I got a cool shirt off mum and dad, which I shall be wearing for this evening’s venture to Ghetto Golf, a sort of weird crazy golf place which is all neon and skulls and graffiti, with cocktails.
Dinner with Niamh this afternoon, too, which is cool. I’m still as worried about the future and the world as I was this time last year, but I can have a day off for my birthday. And I have friends. This helps. So I can do nice things; write some code, maybe publish the talk I did at Hackference, solve a problem or two. Eat biscuits. You know. Nice things. No ironing.
Many happy returns, me.
Last time, on Structure and Interpretation of Computer Programmers, I was building an object-oriented programming system on top of the HURD, and had realised that I needed to use its
trivfs library for a sender to be able to discover an object to send messages to.
I got it working very quickly, but ended up shaving a yak due to my poor understanding of the HURD translator lifecycle which meant that I didn’t think I had got it working.
My goal was to build a translator that works like the Objective-C
nil object: it accepts any message and responds by returning itself. Given that I’m building on the Mach ports abstraction, “itself” is defined as a port over which you can send messages to the same object.
If I returned an integer from the message, everything worked on both sides. However, just because a port name is an integer doesn’t mean that sending it as an
int will work, just as opening a file in one process then sending the file descriptor as a number to another process wouldn’t let the receiving process access the file. I tried sending it as a
mach_port_t, but got a lifecycle error: the client was told that the server had died.
On doing some reading, I discovered that the port had to be sent as a
mach_port_send_t for send rights to be transferred to the client. Making that change, the message now fails with a type error.
An aside, here, on getting help with this problem. There is good documentation: the HURD source is easy to read and with helpful comments, they have good documentation including examples, the OSF documentation is very helpful, there are books from “back in the day” and videos with useful insights.
On the other hand, “help” is hard to come by. I eventually answered my own stack overflow question on the topic, having not received a reply on there, the HURD mailing list or their IRC channel. The videos described above come from FOSDEM and I’m heading out there next week, I’ll try to make some contacts in person and join their community that way.
OK, so back to the main issue, I now have a fix for my problem. Well, sort of, because now that I’m correctly sending a port with a send right I’m back to getting the lifecycle error.
My current plan is not to “fix” that, but to take it as a hint that I’m doing it wrong, and to design my system differently. Using the filesystem as a namespace to look up objects is good, but using the thing I receive as the object to message is a separate responsibility. I’m changing my approach so that the filesystem contains constructors, and they return not their own port but a port to something else that represents the instance they created.
The latest pre-release of the next version of Safari is chock-full of exciting web technologies like Service Worker, Web App Manifest, Payment Request API. Congratulations to the webkit team – this looks to be great work.
One aspect of the release hasn’t met with universal acclaim, however. That’s the ability to have a short, silent video as the source of an <img> element:
<img src="adorable-cat.mp4" alt="adorable cat tears throat out of owner and eats his eyeballs">
This is basically designed to do the same job as an animated GIF, but more performantly. As Colin Bendell writes,
Early results show mp4s in <img> tags display 20x faster and decode 7x faster than the GIF equivalent – in addition to being 1/14th the file size!
People have commented that this isn’t “standard” – but there is no “standard” for the format of images. We’ve used GIF, JPG and PNG because they’re supported everywhere. But nowhere is there a prescription of what kind of images are allowed or disallowed on the <img> element.
Animated GIFs were always a hack; the GIF specification says
The Graphics Interchange Format is not intended as a platform for animation, even though it can be done in a limited way.
The best way to serve mp4 images with fallback to GIF etc for non-supporting browsers is to use the <picture> element, which was invented by a team of legendarily gorgeous chaps, and Marcos Cáceres:
<picture> <source type=video/mp4 srcset=adorable-cat.mp4> <img src=adorable-cat.gif alt="adorable cat tears throat out of owner and eats his eyeballs"> </picture>
However, as Colin Bendell also writes
there is this nasty WebKit bug in Safari that causes the preloader to download the first <source> regardless of the mimetype declaration. The main DOM loader realizes the error and selects the correct one. However, the damage will be done. The preloader squanders its opportunity to download the image early and on top of that, downloads the wrong version wasting bytes. The good news is that I’ve patched this bug and it should land in Safari TP 45.
Given that these mp4 animations will not play sound (so make sure you strip out the audio to make it even smaller), and are more performant than GIFs, I welcome this addition from the webkit team.
All the Birmingham-flavoured tech content on this page is supplied by local bloggers:
Want your blog's content featured here?
For information on submitting your blog for inclusion on this list, visit our blog submission page on Birmingham.IO.
All content, trademarks, artwork, and associated imagery are trademarks and/or copyright material of their respective owners. All rights reserved.Back to Top