Engineer Interview Triage?

A while ago (actually, any post I’ve done is now "a while ago…") I wrote about Sabermetrics for Startups. I wondered if there was data you could collect in an interview process that would allow you to more accurately determine if someone, particularly an engineer, would be successful in your company.

I don’t think I have all the answers, but over the last few months, I’ve honed in on three questions that I believe have a correlation with three key skills.

I’m not talking about technical skills. There are a lot more people far better than me to judge whether or not someone is technically qualified as a great coder.

I am talking about the intangibles. In particular, I’m talking about three key intangibles — communicating, tinkering and passion for coding.  In my experience, these things make a huge difference in someone being a great contributor to your startup.

So, here are the questions. They’re simple and they aren’t pass/fail. But, I think certain answers are more correlated with success.  So, pretend you’re in the hot seat, bright lights, uncomfortable chair… you get the idea. Here goes

1. Do you have a blog?
It was Joel Spolsky who wrote a great piece about great engineers being defined not only by their h4x0r skillz, but by their ability to communicate. Here’s what the man himself had to say

The difference between a tolerable programmer and a great programmer is
not how many programming languages they know, and it’s not whether they prefer Python or Java. It’s whether they can communicate their ideas.By persuading other people, they get leverage. By writing clear
comments and technical specs, they let other programmers understand
their code, which means other programmers can use and work with their
code instead of rewriting it. Absent this, their code is worthless.

If someone has a blog, you know that they are starting to make communications and writing part of a basic set of habits. You know they value those habits enough to make time for them. A public blog improves the odds that the person sitting across from you (who has great coding skills) can also effectively advocate their ideas both inside and outside the company.

2. What’s your home page?
Great engineers make their own homepages. When they hit the "home" icon on their browsers, you’re not likely to see My Yahoo or Amazon.  They’re disatisfied with their other choices out there and they take matters into their own hands (usually just a large list of links of favorite places to go, laid out "just right"). My friend, Marc Hedlund put it this way, "Jedi Knights make their own lightsabers and great engineers make their own homepages." How true.

I think the trait indicated by making your own home pages is that the person is a "tinkerer". Tinkerers are great inside companies. They’re curious. They’re often not quite satisfied with the status quo and doing things the way others do. They’re the ones that aren’t often satisfied with the way your company is doing something. But, rather than complaining or asking, they go ahead and just fix the problem.

It’s hard to know if the person sitting across from you is a tinkerer, but if they make their own home page, it’s more likely that they are.

3. Do you contribute to an open source project?
One thing you’re looking for in a great engineer is a person who is passionate about coding. Passionate doesn’t mean all-consumed-and-working-24-7, but it does mean curious, deeply interested and committed. Besides the obvious benefits of being able to review someone’s open source code for quality, design patters and architecture decisions, contributing to an open source project has a strong correlation to the person being passionate about code. They’re less likely to just be about code-for-cash (not that there is anything wrong about that, it’s just not usually right for a very small startup). That intangible, code-as-passion, can make a huge difference to a startup.

So, that’s what I think. It’s only been a few months of thought. If you’ve got other ideas, I’m all ears.

48 Comments

  1. Bnoopy: Three questions for recruiting programmers

  2. pFadey says:

    Does 2 outta 3 count?

  3. ben says:

    I don’t have a blog. I use google as my home page. I don’t contribute to open source projects because there are other things in my life than the computer.
    Does this mean I can’t communicate, I’m not a tinkerer, and I’m not passionate about what I do?
    I don’t think so.
    These questions are pretty narrow/specific. Having a blog, to me, almost signifies this person is not a good communicator becuase they are using the internet as their means of communcation. Get out of your chair and go talk to someone.
    I tinker with tools to help me get my job done, but it just so happens that my browser home page isn’t one of them.
    I don’t do work outside of work becuase there are a ton of way more fun things to do. I really like building apps, but at the end of the day, it’s time to go have a beer, go to the gym, be social, watch the Yankees, whatever.
    I think someone answering “yes” to these three questions has a better chance of being a heads-down-can’t-communicate-nerd than someone who can understand business requirements and turn them into a working piece of software.

  4. Sumit Chachra says:

    I think trying to come up with $n$ questions that reflect a good/bad candidate for *any* position in *any* company, where $n$ is a finite number, is a very tough task! :-)
    These 3 questions are good ones…. but am sure some very good people and majority of techies will fail this test! :-(

  5. Ben, the world is flat. Do you get the paradigm shift?
    “The playing field is being leveled, says globalization guru Thomas Friedman – from Shanghai to Silicon Valley, from al Qaeda to Wal-Mart.”
    http://www.wired.com/wired/archive/13.05/friedman.html
    I live in Bulgaria. Do you know where Bulgaria is? I track 5359 RSS feeds, 797 Orkut communities, 490
    Tribe.net groups and many other sources.
    Can you compete with me?

  6. Scott Palmer says:

    I ask one question in every interview: “Forget about this position for a moment. What do you REALLY want to do?”
    If they proceed to kiss up and say this job is really what they want to do then they’re out.
    If I get a blank stare, I know they can’t handle pressure AND they’re uncreative.
    Having an answer to that question, even one that’s broad or fuzzy is huge. I want to find people who can think around the corner. I can help them learn the rest.

  7. jim sweeney says:

    Another exemplary startup post, Joe. As an advocacy blogger, I would insert the Joel Spolsky quote in a broader context: “By presuading other people, you not only promote yourself and your cause, but give your employer leaverage as well.”

  8. Larry Rosenstein says:

    I agree with the underlying points about communication and tinkering, but a blog isn’t the only way for someone to demonstrate the ability to communicate.
    Instead, you could ask a candidate to submit a code sample from a non-work-related project, perhaps with some sort of a design document. (Choosing a non-work-related project will let you see if the person likes to tinker on his/her own time.)

  9. Anonymous says:

    I know i’m really good at coding. I’m very passionate in this field and I am a tinkerer. I think of myself as a creative problem solver.
    I don’t have a blog. I don’t have time to blog. I do read a lot of blogs though. 600+ blogs on my aggregator. Reading all of them takes a lot of time.
    I don’t have my own customized homepage. I have Yahoo as my homepage. I do have the bookmarks pane in Firefox open most of the time and I do have a routine list of websites I visit daily to get my daily information.
    No. I don’t participate in open source projects. I don’t have the time.
    I guess I’m going to flat-out fail your test. I won’t make the cut.
    Want to hire great engineers? Post a problem that needs to be solved on your website. You’ll weed out 80% of the would-be applicants. You’ll be surprised at how few of them would even make the attempt to solve the problem. You’ll get a lot of applications with cover letters and resumes but with no solution to your problem.

  10. There is only one question that matters. What have you shipped?
    I haven’t had a homepage since 1997. I don’t contribute much to Open Source because I like to get paid to write code. But I do have a blog.

  11. Anuj Agarwal says:

    This is a response to Dimitar Vesselinov’s comment.
    I don’t think tracking ’5359 RSS feeds, 797 Orkut communities, 490 Tribe.net groups’ is any kind of achievement worth harping about.
    In fact, it seems like you spend more time feeding yourself information and very little time digesting it.
    If you want to talk about your achievements, tell us how you used this information to accomplish something tangible. Till then, don’t expect to be taken seriously.
    PS: Why assume people don’t know where Bulgaria is ? Is it such a crappy, obscure place that you won’t expect people to know about it ?

  12. Passing Interview Triage

    Joe Kraus (JotSpot) write about the three questions he uses to help assess engineers during the interview process. I was glad to see that I had reasonable answers to all three. :-) Do you have a blog? I do. You’re reading it. What’s your home page? As …

  13. AM says:

    I have been fortunate enough to have worked
    with some very smart and creative people.
    They are simply not interested in sharing
    every little thing that they do with the
    rest of the world.
    May be the above questions would help an
    interviewer to psyche the candidate and
    do some pointless arguments in the name
    of a “healthy discussion” with the candiate !

  14. Sam Halliday says:

    i find it strange that you titled these 3 qualities with examples rather than the underlying concepts.
    i strongly agree with your 3 main concepts; communication, initiative and passion (well, thats what i got out of it). and also that you’re emphasising that you can’t simply *ask* someone if they have these qualities… an employer has gotta look deep for tell tale signs of their existence.
    i wouldn’t want to start up anything without being surrounded by very passionate people who make me feel lazy… no matter how hard i work :-)
    but having a blog does not mean someone has communication skills. i personally find it very difficult to communicate using only text (i like to see/hear feedback as i talk), so i set up a blog to get better at communicating to a blank wall (i can’t see the audience so i always feel like i’m writing a message and sending it into the abyss rather than publishing it for potentially the world to read). but it doesn’t mean i *am* any good at communicating to the blank wall, nor does my blog reflect my person-to-person skills in any way shape or form. text input froms are a totally different world than real life, and i prefer people who can tell me everything they need me to do over a beer/coffee, not over a LiveJournal thread.
    i respect blank home pages (faster browser startups) and lots of keyboard shortcuts. tinkering is good. but tinkering TOO much leads to lack of productivity, which leads to the dark side. (especially if someone blogs as well!). if someone has slashdot as their homepage i think to myself “hmm, they need a few years to mature” :-)
    OSS contributions, yeah thats cool. but i prefer the guys who don’t have their names plastered all over the mailing lists… i’d rather see them in the CVS activity logs. of course, OSS is not necessary. just that they are in some kind of volunteer-based project and are active.
    this article was a good read, i’ll certainly be adding this to my RSS feed. i’m about to start up my own consultant business adventure… not as grand as other people around here, but this advise is all very interesting nonetheless (and free!). thanks guys!

  15. Joel Ross says:

    How to Find People With Passion

  16. Joe Kraus says:

    Thanks for all the comments. One thing I wanted to clarify. These questions are not pass/fail. If the candidate doesn’t have a blog or doesn’t make their own home page or doesn’t contribute to open source, it’s not the end of the interview. In fact, while many engineers at JotSpot do the things mentioned above, several do not and I think everyone of them is a total rockstar.
    I was just saying that people who do these three things seem to have a higher correlation with success inside the companies I’ve been a part of or invested in.
    The nice part about the questions, in my opinion, is that they point to behaviors which are:
    1) not motivated by trying to “succeed” in the interview and are therefore more likely to be indicative of the core motivations of the candidate
    2) objectively observable to outsiders (like me)

  17. Greetings,
    As I noted on Jeremy Zawodny’s blog, I have to admit I think it’s mostly bunk. Blogs don’t tell you how they communicate, just whether they feel like sharing their thoughts openly.
    The ‘personalized home page’ is one I’d really strongly disagree with, as the efficiency of the pure google homepage as a jump-off point is nearly unmatched. Sure, I tried building the ‘rss feed homepage’ once, but when my feeds expanded past a few dozen, it became useless, and the efficiency of the search engine homepage beckons.
    Now, as for open source projects, I’d rather ask, ‘Tell me about your programming outside of work.’ If they get excited, thrilled about their own personal coding, and can expound on it, then I’m glad, and it tells me more about them as a programmer. If it’s an open source project, then you can learn the communication skills by reading their checkin comments. In fact, those are far better than a blog, for knowing how they can communicate.
    And to the person who commented about not programming otuside of work, no I wouldn’t hire a programmer for a startup someone who doesn’t intensely love to code.
    – Morgan Schweers

  18. der Mouse says:

    I don’t understand. What makes anyone thing a “great engineer” has to have *anything* to do with the Web? Must everyone capable of writing a coherent sentence necessarily feel impelled to (a) run a website at all, (b) want to put up something two-way on it, and (c) feel egotistical enough to think anyone else is interested in hir daily babblage?
    I think that thinking having a blog says diddly says more about you than about anyone you think fails the test.

  19. “What makes anyone thing a ‘great engineer’ has to have *anything* to do with the Web?”
    My answer:
    Tim O’Reilly – The Software Paradigm Shift
    http://www.itconversations.com/shows/detail50.html
    Web 2.0 Conference 2005
    http://www.web2con.com

  20. Brent Ashley says:

    My personal evolution has taken me beyond a homepage filled with categorized shortcuts from which I can make outbound stab-in-the-dark searches to find information.
    Much of the information I want now comes to me via my aggregator (I use Bloglines). What remaining outbound searches I make are now made with A9 (using Google’s engine) and technorati/feedster/del.icio.us/flickr + tags.
    Any valuable engineering tinkerer will have realized the power of aggregation and taxonomy to reduce the drudgery of iterative search. We’re all lazy buggers, after all.

  21. Bill Cole says:

    0 for 3, but with explanation:
    1. No blog. On the other hand, a quick search at Google can demonstrate that I have a long history of actually communicating (discussion, not monologues decorated with graffitti) in fora like Usenet and various mailing lists, and a lot of that discussion is on technical subjects, but it is certainly not all technical. A blog is not a bad thing, but it is really a very tilted forum. It’s like a theater (or sometimes a temple) instead of a public square or a living room. On the other hand, maybe that’s the best one can hope for from a population with a dominant pattern of social discomfort: find the guy who acted as DM for `the D&D games, not one of the quiet players…
    2. Historically my start page for web browsing has been blank, for faster browser startup. A page full of links is pretty pointless when there’s one I can pull down and have disappear as needed (i.e. a Bookmarks/Favorites/Hotlist menu, depending on which browser I’m using…) With tabbed browsing and support for the starting ‘page’ being a tab set, I use that in some situations such as the workplace, where a handful of starting points defines my normal web usage. Building one’s own starting page might say that one is a tinkerer but it also may indicate a certain egotism that insists on re-inventing the wheel just to be able to say that it is MINE, rather than building on what others have provided as a flexible toolset.
    3. I’ve contributed some bug fixes and similar limited things to open source projects that I use, but the limiting factor isn’t “passion for code” at all. A lot of people with real jobs work under terms that make it very difficult and sometimes impossible to commit any significant effort outside the job to anything similar in nature to the job. For exmple, if I were to take a significant role in any OSS project, I’d need specific permission to do so from my management and a specific disclaimer of any rights in that work. That’s not because I work for an evil company, but because I work for a company that has learned that they need to protect themselves. In addition, I have a full time job (which often means well over 40 hours per week) plus I have a family. There’s the issue of balance and priorities, and it is simply not possible for me to keep up with my existing work and family obligations and give suitable attention and focus to some other project.
    I’m not saying that these three questions completely suck, because they really don’t, but it would be a gross error to ask them as yes/no questions without digging a little deeper.

  22. There have been a couple of comments along the lines of “I don’t do work outside of work because there more fun things to do.” or “I only code for money.” The kind of person a startup needs knows that creating software isn’t work, it’s the most fun thing ever. It comes down to: Do you code because it’s a job, or do you code because it hurts if you don’t?
    I’ve been using “do you answer questions on newsgroups/mailing lists”, “do you have a blog”, “what have you coded that you weren’t required to code (open source or just personal)” and “what’s your favorite software-related book/web site” as intro questions for a while. Of course, you can’t really know how good a developer somebody is from a couple of questions, but having interesting answers to those questions has turned out to be a good indicator. (And if the interviewee has contributed to an open source project, it gives me a chance to look at their code, which is a definite bonus)
    I know it’s not fair to reduce a person to the answers to a couple of questions, but as we’re going through hiring-hell right now, I understand intimately the need for some sort of triage.

  23. tom biggs says:

    Here’s another 0 for 3.
    I have excellent communication skills, verbal and written. But I don’t blog. I’m a private person, and also I’m not egotistical enough to think that everyone cares about every trivial thought that dribbles out of my keyboard. There was a great article in The Atlantic about “Caring for your Introvert”, and it said that extroverts automatically think there’s something wrong with anyone who doesn’t constantly broadcast every single thing they’re thinking. Meanwhile, we introverts are silently thinking “shut up, already!”
    My home page is Google. Far more flexible than anything I could put together. I did make an intranet site with that kind of stuff, mostly for HTML/CSS/JS practice.
    I do my own side projects, but haven’t added anything to an open source project. I think it’s a good sign if someone does, of course. Mostly I’ve been lucky enough to have mind-expanding projects at my various jobs.
    I like the spirit behind the method – trying to find the real person and not just the surface credentials – I just disagree with the questions.

  24. tom biggs says:

    Also, to those who mention having a balanced life – house, kids, real-world activities – note that the subject here is startups.
    People who run startups often *are* looking for people who don’t have outside demands on their attention, so they can work the expected long hours, crash, then come back and do it again the next day. I’m not going to pass judgement on that. To each his own. I prefer a more balanced life, and my family is glad for that.
    But that’s all irrelevant to someone who’s hiring for a startup company. “Can you give us your entire undivided attention? If not, we’ll hire the next person who can.” One VC guy said he’d check a potential company’s parking lot on Sunday morning; if he didn’t see cars there, he wouldn’t invest.
    It’s certainly exhilarating to be a part of something like that. I’ve done it in two-month bursts. But I find that there’s more value in paying attention to the whole world, with my joy in programming being a bonus.

  25. Interviewing software engineers

    This has been making the blog rounds the last few days. Joe Kraus has three interview questions can be used to tell whether an engineer might be a good fit for a start-up software company: Do you have a blog?…

  26. Three Questions

    Joe Krauss asks three questions of engineers before hiring them. Namely "Do you have a…

  27. Louis Monier. The perfect candidate I would hire. What an inspiration!
    Louis Monier On Why He’s Going To Google
    http://battellemedia.com/archives/001653.php

  28. Ginger says:

    On a more skill-oriented and experience oriented level, one of my favorite questions is to hand a developer a copy of an invoice and ask the person to sketch out the model for me.
    It’s amazing to me how many “developers” can’t do that. There are lots of developers out there tweaking other people’s code but they aren’t really creating new solutions. I ask this question early and stop the interview immediately if they can’t give me a reasonable model.

  29. Ginger says:

    On a more skill-oriented and experience-oriented level, one of my favorite questions is to hand a developer a copy of an invoice and ask the person to sketch out the model for me.
    It’s amazing to me how many “developers” can’t do that. There are lots of developers out there tweaking other people’s code but they aren’t really creating new solutions. I ask this question early and stop the interview immediately if they can’t give me a reasonable model.

  30. Alex Russell says:

    Hey, der Mouse,
    If you’re the same der Mouse I know from Bugtraq, I don’t think you have to worry about not having a blog should you apply for a job at Jot.
    Just be sure to let us know that it’s you ;-)
    Regards

  31. Jersey says:

    A little more generic than your question 2, I always ask: What was the last little piece of software you wrote for yourself? I find it’s a great way to learn about what the candidate considers “broken” and how he or she goes about working around the problem. You can also gauge the candidate’s level of enthusiasm for solving problems. If they can’t come up with one little thing they worked on for themselves, or if there’s no spark in how they talk about it, I consider it a warning sign.

  32. Great idea – I’d like to think these three questions are pertinent when interviewing another recruiter. In fact, I think I’ll try it next time!

  33. Not sure why so many people are taking this post personally . Joe is just identifying a few common, concrete telltale signs of the intangibles that define good coders. There certainly are others; no need to get bent out of shape just because you happen to not have the ones he mentions here.

  34. QuestorTheElf says:

    Why are some people getting bent out of shape about this? It’s simply because some otherwise great candidates have lost out on offers because of some overvalued killer question(s).
    Next, why do some otherwise competent people choose specifically not to blog? They don’t want to be second-guessed and have such misinterpretations cost them an opportunity. So if they must engage in some kind of communication, they’ll do so in professionally revered trade journals and public speaking engagements. If we are talking jobs, those vehicles can much more readily show the ubiquitous “Must have excellent oral and written communication skills.”
    Ironically, at no time in my career when I’ve seen that requirement has anyone asked me to produce a written sample nor give an oral presentation. Shall I develop a blog to satisfy HR? (Oops, almost forgot, those who really want to get jobs avoid HR like the plague, just like some who purposely don’t want to write a blog.)

  35. QuestorTheElf says:

    > I ask one question in every interview: “Forget about this position for a moment. What do you REALLY want to do?”
    > If they proceed to kiss up and say this job is really what they want to do then they’re out.
    And I know some hiring managers who ask this question as well who’ll do the opposite. (This is especially true at those household name companies, where some applicants just want to add the brand to their resumes.) They want the candidate to show their commitment to the position. They want the candidate to show that they were born or meant to do that precise position.
    In terms of the original issue about collecting data that can accurately predict if a given engineer will succeed, sorry! You could have the world’s greatest candidates, and they still can’t combat market forces. They change. And get this — people change too! In a world where it’s said you may have 10 to 20 jobs and switch careers 5 to 7 times, what really needs to be evaluated is first one’s dedication to a role, then one’s ability to change as required.

  36. Don Galt says:

    I would pass on any engineer that had a blog, unless the blog covered his software releases and had code snippets. There is a strong correlation between blogging and being a social animal who is more concerned about being liked than getting the job done.
    Those who can, do. Those who can’t, but want to be liked anyway, blog.
    My startpage is blank. I don’t need to set one up because I have my tool bar finely tuned to take me where I want to go with one click.
    I normally write my own html, but I’m working on a tool that will eliminate that need going forward.
    Will I be less of an engineer in your eyes because I won’t be hand coding html?
    These are all questions designed to be substitutes for actually interviewing someone.
    But as the industry matures, I’m not surprised to see people calling for the dumbing down of engineers– to focus more on their people “skills” (which is really another word for how well they kiss ass) than their engineering skills.

  37. Samir says:

    Cool! Apparently I am only 33% good software engineer…
    1. I dont have a blog… since i am an extreme introvert.
    2. I do have a custom home page in IE, and its about:blank.
    3. I dont really like Open Source Software. Actually I have written and continue to write FREE software, that is closed source… Good Open Source stuff is hard to come by…

  38. The first thing I noticed reading the answers is that a lot of people missed “… and they aren’t pass/fail” even after you repeated it.
    They’re also, I hope, not “yes/no”. For example…
    “I don’t have a blog, I have a bunch of articles I wrote, but they didn’t have blogs when I started doing them, so I never bothered taking it further than that.”
    “I don’t work on open-source software, but I’ve written a good dozen little utilities everyone at $COMPANY uses”
    “I have the browser open to a blank page, but I’ve got my del.icio.us bookmarks in my toolbar”
    Etc etc etc…

  39. Zoli's Blog says:

    Has Blogging Peaked?

    Some predict blogging soon peaks or already has. I disagree. This article discusses why blogging for $$$ will crash and burn a’la daytrading, and how then blogging will be back t “normal”: a vehicle for self-expression, exchange of ideas, social and p…

  40. I have a blog. I have two websites. I am a former programmer. I am a civil engineer. Do I get the job?
    Communication is key especially since we engineers have been bashed for our lack of communication. Perhaps we are just too impatient to dumb everything down for non-engineers. We want to get the work done already!
    Building a website is not tinkering. Rebuilding an engine or building a deck or remodeling a room…that is tinkering. Ask about what major changes they have made to their dwelling. Those types of tinkerings require creativity…a very important part of programming.
    It is time to stop stereotyping engineers. We are analytical, but that doesn’t mean we are dorks that sit in front of the computer all day wearing our coke-bottle glasses (caused by sitting in front of a pc all day???).

  41. michael says:

    I need to interview a computer software engineer for a research project.thanks

  42. J’apprécie la façon dont vous abordez ce sujet. Cela me paraît fort intéressant. A bientôt.

  43. Sandy Smith says:

    It truly is in reality a pleasant along with helpful part of facts. I’m just delighted that you just discussed this useful details about. You need to keep us up to date similar to this. Appreciation for revealing.

  44. Wow, awesome weblog format! How lengthy have you been running a blog for? you make running a blog glance easy. The overall glance of your web site is fantastic, as well as the content!

  45. levell says:

    Desirable section of content. I simply discovered your website along with accession investment to say i always be in fact beloved accounts your site threads. In any manner My business is subscribing to the provides nourishment to or perhaps I happiness you receive the ways to access continually quickly.

  46. I absolutely love your site.. Excellent colors & theme.

    Did you make this website yourself? Please reply back as I’m looking to create my own
    personal site and would like to find out where
    you got this from or what the theme is called. Many thanks!

  47. I don’t know whether it’s just me or if everyone else encountering issues with your blog.
    It appears as though some of the text within your content are running off the screen. Can somebody else
    please provide feedback and let me know if this is happening to them too?
    This might be a problem with my browser because I’ve had this happen previously.
    Many thanks

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>