Programming is hard work. Programming, designing, deploying, scaling, debugging, and maintenance all done at the same time 12+ hours a day is even harder work — and the last 4 months have really taught me that.

During the three months from June through August spent participating in the YCombinator program I wrote a ridiculous amount of code, gained much knowledge, made a great group of friends, and brought a pretty sweet project to a quick launch. All in all it was fulfilling, rewarding and fun. After August and the program came to a close I began working on our next big feature, failing to notice that in all the excitement I had completely ran out of steam. Over the course of this month I slowed down and stopped working as much, despite being in front of the computer just as long as before. New bugs continued to delay a speedy release of concert tickets and as the need for a large refactoring came firmly into view, I began buckling further under the load. Today, a mere six days until concerts are due, I am just now realizing the full extent of these effects. Here are a few things I’ve managed to take away from the experience.

Learn to Recognize Stress

Perhaps it’s a product of age or experience, but recently I’ve started to recognize stressors much more than I used to. Injuries, deaths, significant life changes, catastrophic events, these things never got to me before. I am beginning to realize more than ever the importance of recognizing, reacting to and–where possible–heading off potential stressors. I think my default tendency to ignore these things works for short-term issues, but when I become stressed or burnt out by something I have to do every day–work that affects more than just me–I have to be more vigilant.

Learn Your Limits

We all have limits. I’ll never stake my livelihood in my ability to run a 5-minute mile; it’s just not going to happen. On the one hand, we should all strive for constant improvement and one of the best ways to do that is to put ourselves in situations where we have to exceed our expectations. On the other hand, attempt to bend too far and we will break. Maybe my breaking point is 3 months solid of near round-the-clock work; maybe yours is 3 weeks or 3 years of the same — the important thing is figuring that out and taking the hint that you’re reaching it, something I failed to do this time around.

Of course, different tasks call for different limits. The type of work you do is as important to the value of this variable as anything. For certain tasks that don’t require constant concentration and mental (and/or physical) effort, your limits will likely be higher. Regardless, it is possible to push these limits, but not indefinitely.

Separate Work from Everything Else

Today, if I told you that I “work” 14 hours a day all that really means is I’m in front of the computer for that length of time. Sadly, this figure is relatively accurate. However, the amount of actual work being done during all those hours varies wildly; it could be 10 hours, it could be two. The problem is, the only true relaxation I have on a daily basis is while I’m asleep. All other times I am working or thinking about work. I feel compelled to be working if I am in front of the computer, but can’t bring myself to do it at times because I feel too overwhelmed from being there constantly. It’s a nasty bit of recursion!

My new goal is to spend far less time in front of the “work” computer. Unless I am working, I will boot into Windows to play games, grab my laptop to surf the net, or avoid the computers entirely and read a book, play XBOX, etc. No more messenger or e-mail or other nonsense after-hours, either. I will also begin maintaining a more fixed schedule which involves going back to the gym every day. The last two I’ve already started. Maybe some day I will branch out and get one of those “life” things people are always talking about. I seem to recall a time when I had one of those and it was… nice?

This is just an example of how to tackle the issue; your mileage may vary. I recognize that like the world I am imperfect — there will be days when I wake up and simply can’t bring myself to work; there will be days when I get in the “zone” and code late into the night, ruining my sleep schedule for the immediate future. The key is to make neither of these a habit.

Finally, if you have your own methods for separating work from leisure, getting into the zone, etc. — please leave them in the comments (or email tom @ this domain). If I get enough submissions I’ll create a follow-up post with the highlights and hopefully help some fellow knowledge-workers in the process.

Conclusion

Don’t get me wrong, I love programming. I love all my work. If it weren’t for mentally stimulating work I would have thrown myself from a bridge long ago. Even so, sometimes the mind needs a break. Well, my mind anyway — I’m sure there are folks who can work 20/7/365 and thrive, but I’m not one of them! The ultimate goal here is moderation in all things work and play. It’s an attempt to curb the programmer stereotype of working non-stop for 40 hours or a week straight and then stopping for an indeterminate amount of time (I still haven’t determined it and I’m writing about it!)

As for affecting when concerts show up here on TS, well, I said they’d be up by the end of the month so that’s still my goal, granting that such an accomplishment might impress even me. I don’t know how long it will take to truly “right” myself, but hopefully however long it is it can wait until next month. And, hopefully, all these lessons and changes will lead to an even faster-paced release schedule around here in the coming months — one that doesn’t live by the whims of my errant mind.

[ WARNING: Programmer talk ahead! ]

Earlier this month I decided to start switching some of our back-end code from “chewing gum and baling wire” to Twisted. I also decided that in the course of this switch that I would write code such that it could be patched into Twisted to improve some poor APIs. Well, let me tell you, it has been a ride.

I almost threw in the towel on the patches idea at one point because I was getting really worried I would miss the end-of-month deadline for Concerts. Who would have thought learning the ins and outs of a networking engine that has been under active development for over 7 years would be so difficult?! Apparently not me. Lets see how many lines of code Twisted is comprised of, shall we?

http://cloc.sourceforge.net v 1.04  T=55.0 s (18.0 files/s, 5005.4 lines/s)
——————————————————————————-
Language          files     blank   comment      code    scale   3rd gen. equiv
——————————————————————————-
Python              935     50191     10991    195158 x   4.20 =      819663.60
HTML                 36      2475        21      9696 x   1.90 =       18422.40
C                     7       667       354      4624 x   0.77 =        3560.48
Lisp                  5        87        71       442 x   1.25 =         552.50
CSS                   3        66         0       314 x   1.00 =         314.00
SQL                   1         9         0        56 x   2.29 =         128.24
C/C++ Header          1         4        13        34 x   1.00 =          34.00
Bourne Shell          2         0         6        16 x   3.81 =          60.96
DOS Batch             1         1         0         3 x   0.63 =           1.89
——————————————————————————-
SUM:                991     53500     11456    210343 x   4.01 =      842738.07
——————————————————————————-

Look at that: 842.7 kloc. The section I’m working on is a paltry 135.4 kloc. Now, lines of code may not be a good indicator of a project’s quality, efficiency, etc. but it is a pretty good indicator of whether or not it has a lot of damn lines. As a comparison, TicketStumbler is a whole 14.3 kloc, not including Django and outside libraries. Since it took me roughly 3 months to design/write/deploy all of TicketStumbler, my initial thinking must have been, “Learn 10x the code of TicketStumbler enough to write code extending it, in 3 weeks? Pffft, easy!” Needless to say, I’ve learned a few lessons along the way…

Our Estimations Suck

I’m pretty old fashioned when it comes to my code. I like things sequential. So, when i call a() then b() I know that first a() will run and when that’s done b() will start. Simple. Well, Twisted ruins everything. Twisted is Event Driven and Asychronous meaning that things don’t necessarily happen in sequential order and code should be written to respond to events such as “the data is ready” or “his ass is grass.” Despite knowledge of over a dozen programming languages and familiarity with using threads and processes to run sequential code blocks beside one another, this was a very new concept to me and one I initially found difficult to grasp enough to write code for it (though using it was quite straight forward).

The takeaway here is that we programmers (in general) are quite poor at estimating how long a task will take, whether to ourselves or to some suit. I thought I had enough time to grok Twisted and write a whole bunch of code for it, but it turns out I only had enough time for the first one; the second will fall short of where I thought it would be (though hopefully good enough for my purposes!) It seems like I learn this lesson over and over and each time I pad my estimates a little bit more, but it never seems to be enough. I recommend over-estimating your over-estimates.

Grokking Can Pay Dividends

I had “learned” Twisted after day one.  I learned how to import some modules, make a couple function calls, and use a single-line API to setup an asynchronous HTTP request.  Okay, so maybe Twisted’s HTTP client sucked; that was okay because it had all 3 features I needed.  Then it struck me: maybe this was something actually worth learning.

I know a few programmers who still hate using 3rd-party libraries, frameworks, even sometimes the standard library, because they “didn’t write it” or they “don’t know how it works.”  In about 90% of cases I still want to slap these people for reinventing the wheel over and over again, but in this case Twisted was such a useful engine that I decided to take it a step further.  Now, I have the knowledge necessary to bend this incredibly powerful engine to my will and use much of its vast feature set.

Most of the time, it doesn’t pay to go through module by module and learn complicated libraries / frameworks you want to use. It will already let you handle the common use cases and for less common cases it’s highly likely that somebody else has already written the code for you and subsequently given it away for free.  Such is the nature of Open Source.  Good programmers write, great programmers reuse; if you’re developing code for a giant project, chances are you’re a “great” (except in my case!) so there’s a general rule that code will be quite good enough for silly n00bs to use, thank you very much.

Even still, there are times when really understanding how a library/framework functions can pay dividends, even if you don’t plan to develop for it or use all its power immediately. Additionally, you’ll be amazed at how much you can learn about a foreign concept just by reading lots of code written to take advantage of said concept.

Ask For Help

As someone with the blessing/curse of a general knowledge of many different programming/web-related topics, I have a tendency to get asked a lot of questions by YC friends and other fellow programmers.  I also ask those same people just as many questions when I need to gain some advanced knowledge of something they’re more skilled in.  I won’t waste time Googling for tutorials and articles, I’ll just get it from the horse’s mouth, so to speak.  In the case of Twisted, this took the form of many queries to the Twisted Core developers who live in #twisted.  It wasn’t until getting some example code from one of them that the light switch finally flipped and I could get down to writing working code.

There’s often a desire among programmers (and I’m guilty of this too) to make something work on our own.  We think, “I wonder if I can make X do Y…” and we just try it out.  Successful attempts such as these are liberating and quite fun jaunts at times, but sometimes it doesn’t pay to screw around.  Instead, consider just asking someone who knows all about “X” how it can do “Y” and a subsequent explanation of how/why that works.

Leave Your Comfort Zone

Never before in my programming career have I been more unsure of myself.  Tons of code, new concepts, and a looming deadline made me more frustrated and stressed than I’ve been in recent memory; this summer was a cake walk by comparison.  All this because I went way outside my comfort zone.  Ever use Twisted before?  Nope.  Event-based programming experience?  Negative.  Familiarity with Unit Testing?  Not a chance.  Serious development on Open Source projects?  No.

When was the last time you boned up on your Assembly or studied up on how Interrupt Requests in hardware work?  How about the last time you did some heavy pointer work in C?  What was the last Open Source library or framework you submitted some critical patches to?  What is your latest cross-platform GUI app?  These are the sorts of things I haven’t thought about or coded in many years in some cases.

As a web programmer, my life generally isn’t too hard.  I pick a nice framework, write a CRUD app or something reasonably simple, toss a design together in HTML/CSS, throw up a server or two, and call it a day.  That’s just fine for simple stuff, but we’re entering the “Rails Era” where writing any web app is “easy” and “fun” and none of the super-smug hipsters knows why the hell they can’t get Michael Arrington to stop ragging on them on TechCrunch because the database servers are down 50% of the time.

Don’t be those guys.  Just when you think you’re on easy street, jump outside your comfort zone and figure out how you can optimize that pesky function that’s taking too long, even though it’ll require some serious digging.  Find a way to build that great feature that’s “too difficult to implement.”  Jump into the framework and find a way to make that laggy Javascript appear smoother.  Do the hard stuff now so you’ll be better prepared when the unexpected happens… like it always does.

Conclusion

My conclusion is… there isn’t one.  I’ve made progress on Twisted, but I’ll keep learning.  Once I’m satisfied there, I’ll move onto something else that I’ve been avoiding or have let decay.  Next up on my list:

• Switching from TextMate to Vim and other CLI tools to save my right wrist and buckets of time
• Taking a closer look at Git and how it can help me maintain many different code branches
• Optimizing our parsers further (there’s a 50-60x speed boost in there…)
• Learning more about alternative database possibilities, such as Cloudant, due to excessive querying needs (~20 million/day next month)

What’s in your wallet on your list?

When some quirks started showing up in some of our code, I went looking for a replacement for some of it and found my way to Twisted.  From the site:

Twisted is an event-driven networking engine written in Python

I started asking around in their IRC channels (#twisted and #twisted.web on Freenode) and it was brought to my attention that the portion of the framework I wanted to use was lacking quite a few features and was, in the words of a couple Twisted developers, “crap.”  I was then faced with a decision:  Did I use it anyway, making quick additions and changes where necessary for my uses, or did I take the time to really do it right and contribute lots of code back to the project?

Ultimately, I decided it would be worth it to help the project even if it did delay some additions around here for a little while.  The central reason for this is quite simple:  Open Source Software is REALLY important.  Most of the websites flying through your internet tubes each day wouldn’t exist if it weren’t for high-quality, free, constantly-improving software.  In fact, I can almost guarantee you wouldn’t be reading this right now without it.

From our servers to our website down to this very blog, most everything here at TicketStumbler is based on Open Source Software.  This means two things: First, that we are very dependent on OSS, but more importantly that we’ve done a lot of “taking” without too much “giving” in return.  Of course, this holds true for the vast, vast majority of sites out there, but we decided to do something about it (and if you have a website, you should too!)  Of course, there are other ways to contribute to Open Source projects.  The main way being donations, as we’ve done with numerous projects including the awesome Django framework.  However, the best method is probably by contributing code; people have argued that money may be useless to Open Source projects.

There were other reasons for this decision, though:

1.  Everyone is so nice!

The first thing you’ll notice when visiting the Twisted IRC channels is how nice and helpful everyone is.  There are a few regulars in the channels who happen to be long-time developers of Twisted and if I didn’t know better I’d think their sole purpose in life is to hang around IRC answering questions and being ridiculously helpful to everyone who comes in.  Like a cute girl you can’t possibly say “no” to, I couldn’t help but help!  This isn’t an anomoly of Twisted, though; most Open Source projects succeed in large part because of the great communities that develop around them.

2.  Open Source projects force best practices

I’ll share a secret with you: I never got on the Test Driven Development bandwagon.  I didn’t really get it; I’m supposed to write tests for code I haven’t even written?  Why don’t I just test it manually as I go along, it has to save time, right?  After only a day attempting to adhere to the TDD requirement of Twisted development, I am nearly sold on the whole thing.  There is something very comforting (and simple!) about running a single command and seeing if any of my code changes broke existing functionality or if it fails under certain circumstances.

If that weren’t enough, one of the awesome developers mentioned earlier even offered to help me learn the ways of TDD via a personal course his company sells — for free!  On top of the perk of having access to great software I’ve been given the opportunity to get some incredibly useful training, just for doing my developer civic-duty of contributing back.

3.  It feels good!

After submitting my first patch to the project I just felt good.  It made me happy to give back to a project I’m planning to use, if even in a small way to start.  When I think about the countless man-hours dedicated to writing, testing, fixing, and improving code I use each day it makes me glad to know that I didn’t only take and take.

—–

TicketStumbler makes the lives of event-goers easier whereas Open Source projects like Twisted make the lives of developers easier.  This first project is one in what I hope will be a long list of contributions back to the software that keeps me from going crazy (or at least crazier).  I’ll be writing code for Twisted for some time to come, but down the road I plan to give back to other projects, too.  I’m thinking I owe a whole lot to Django, beyond the monetary contributions made by TicketStumbler Inc. and help provided in their IRC channel (#django on Freenode).

This is a call to action for all start-up developers out there:  Set some time aside to contribute back to the projects that run your business.  You’ll thank me later.