[ WARNING: Programmer talk ahead! ]

Earlier this month I decided to start switching some of our back-end code from “chewing gum and baling wire” to Twisted. I also decided that in the course of this switch that I would write code such that it could be patched into Twisted to improve some poor APIs. Well, let me tell you, it has been a ride.

I almost threw in the towel on the patches idea at one point because I was getting really worried I would miss the end-of-month deadline for Concerts. Who would have thought learning the ins and outs of a networking engine that has been under active development for over 7 years would be so difficult?! Apparently not me. Lets see how many lines of code Twisted is comprised of, shall we?

http://cloc.sourceforge.net v 1.04  T=55.0 s (18.0 files/s, 5005.4 lines/s)
——————————————————————————-
Language          files     blank   comment      code    scale   3rd gen. equiv
——————————————————————————-
Python              935     50191     10991    195158 x   4.20 =      819663.60
HTML                 36      2475        21      9696 x   1.90 =       18422.40
C                     7       667       354      4624 x   0.77 =        3560.48
Lisp                  5        87        71       442 x   1.25 =         552.50
CSS                   3        66         0       314 x   1.00 =         314.00
SQL                   1         9         0        56 x   2.29 =         128.24
C/C++ Header          1         4        13        34 x   1.00 =          34.00
Bourne Shell          2         0         6        16 x   3.81 =          60.96
DOS Batch             1         1         0         3 x   0.63 =           1.89
——————————————————————————-
SUM:                991     53500     11456    210343 x   4.01 =      842738.07
——————————————————————————-

Look at that: 842.7 kloc. The section I’m working on is a paltry 135.4 kloc. Now, lines of code may not be a good indicator of a project’s quality, efficiency, etc. but it is a pretty good indicator of whether or not it has a lot of damn lines. As a comparison, TicketStumbler is a whole 14.3 kloc, not including Django and outside libraries. Since it took me roughly 3 months to design/write/deploy all of TicketStumbler, my initial thinking must have been, “Learn 10x the code of TicketStumbler enough to write code extending it, in 3 weeks? Pffft, easy!” Needless to say, I’ve learned a few lessons along the way…

Our Estimations Suck

I’m pretty old fashioned when it comes to my code. I like things sequential. So, when i call a() then b() I know that first a() will run and when that’s done b() will start. Simple. Well, Twisted ruins everything. Twisted is Event Driven and Asychronous meaning that things don’t necessarily happen in sequential order and code should be written to respond to events such as “the data is ready” or “his ass is grass.” Despite knowledge of over a dozen programming languages and familiarity with using threads and processes to run sequential code blocks beside one another, this was a very new concept to me and one I initially found difficult to grasp enough to write code for it (though using it was quite straight forward).

The takeaway here is that we programmers (in general) are quite poor at estimating how long a task will take, whether to ourselves or to some suit. I thought I had enough time to grok Twisted and write a whole bunch of code for it, but it turns out I only had enough time for the first one; the second will fall short of where I thought it would be (though hopefully good enough for my purposes!) It seems like I learn this lesson over and over and each time I pad my estimates a little bit more, but it never seems to be enough. I recommend over-estimating your over-estimates.

Grokking Can Pay Dividends

I had “learned” Twisted after day one.  I learned how to import some modules, make a couple function calls, and use a single-line API to setup an asynchronous HTTP request.  Okay, so maybe Twisted’s HTTP client sucked; that was okay because it had all 3 features I needed.  Then it struck me: maybe this was something actually worth learning.

I know a few programmers who still hate using 3rd-party libraries, frameworks, even sometimes the standard library, because they “didn’t write it” or they “don’t know how it works.”  In about 90% of cases I still want to slap these people for reinventing the wheel over and over again, but in this case Twisted was such a useful engine that I decided to take it a step further.  Now, I have the knowledge necessary to bend this incredibly powerful engine to my will and use much of its vast feature set.

Most of the time, it doesn’t pay to go through module by module and learn complicated libraries / frameworks you want to use. It will already let you handle the common use cases and for less common cases it’s highly likely that somebody else has already written the code for you and subsequently given it away for free.  Such is the nature of Open Source.  Good programmers write, great programmers reuse; if you’re developing code for a giant project, chances are you’re a “great” (except in my case!) so there’s a general rule that code will be quite good enough for silly n00bs to use, thank you very much.

Even still, there are times when really understanding how a library/framework functions can pay dividends, even if you don’t plan to develop for it or use all its power immediately. Additionally, you’ll be amazed at how much you can learn about a foreign concept just by reading lots of code written to take advantage of said concept.

Ask For Help

As someone with the blessing/curse of a general knowledge of many different programming/web-related topics, I have a tendency to get asked a lot of questions by YC friends and other fellow programmers.  I also ask those same people just as many questions when I need to gain some advanced knowledge of something they’re more skilled in.  I won’t waste time Googling for tutorials and articles, I’ll just get it from the horse’s mouth, so to speak.  In the case of Twisted, this took the form of many queries to the Twisted Core developers who live in #twisted.  It wasn’t until getting some example code from one of them that the light switch finally flipped and I could get down to writing working code.

There’s often a desire among programmers (and I’m guilty of this too) to make something work on our own.  We think, “I wonder if I can make X do Y…” and we just try it out.  Successful attempts such as these are liberating and quite fun jaunts at times, but sometimes it doesn’t pay to screw around.  Instead, consider just asking someone who knows all about “X” how it can do “Y” and a subsequent explanation of how/why that works.

Leave Your Comfort Zone

Never before in my programming career have I been more unsure of myself.  Tons of code, new concepts, and a looming deadline made me more frustrated and stressed than I’ve been in recent memory; this summer was a cake walk by comparison.  All this because I went way outside my comfort zone.  Ever use Twisted before?  Nope.  Event-based programming experience?  Negative.  Familiarity with Unit Testing?  Not a chance.  Serious development on Open Source projects?  No.

When was the last time you boned up on your Assembly or studied up on how Interrupt Requests in hardware work?  How about the last time you did some heavy pointer work in C?  What was the last Open Source library or framework you submitted some critical patches to?  What is your latest cross-platform GUI app?  These are the sorts of things I haven’t thought about or coded in many years in some cases.

As a web programmer, my life generally isn’t too hard.  I pick a nice framework, write a CRUD app or something reasonably simple, toss a design together in HTML/CSS, throw up a server or two, and call it a day.  That’s just fine for simple stuff, but we’re entering the “Rails Era” where writing any web app is “easy” and “fun” and none of the super-smug hipsters knows why the hell they can’t get Michael Arrington to stop ragging on them on TechCrunch because the database servers are down 50% of the time.

Don’t be those guys.  Just when you think you’re on easy street, jump outside your comfort zone and figure out how you can optimize that pesky function that’s taking too long, even though it’ll require some serious digging.  Find a way to build that great feature that’s “too difficult to implement.”  Jump into the framework and find a way to make that laggy Javascript appear smoother.  Do the hard stuff now so you’ll be better prepared when the unexpected happens… like it always does.

Conclusion

My conclusion is… there isn’t one.  I’ve made progress on Twisted, but I’ll keep learning.  Once I’m satisfied there, I’ll move onto something else that I’ve been avoiding or have let decay.  Next up on my list:

  • Switching from TextMate to Vim and other CLI tools to save my right wrist and buckets of time
  • Taking a closer look at Git and how it can help me maintain many different code branches
  • Optimizing our parsers further (there’s a 50-60x speed boost in there…)
  • Learning more about alternative database possibilities, such as Cloudant, due to excessive querying needs (~20 million/day next month)

What’s in your wallet on your list?