Yet Another Programmer Bitching

It’s a well known fact that being the user of your own software is helpful, but I think there are also drawbacks — that eating your own dog food is harmful in some ways:

• Understanding requirements is a valuable skill. EYODF can be harmful for your career as a software designer, because you won’t work on your ability to perceive what other people need
• When all programmers are doing it, who’s going to work on accounting systems? The world needs more than code editors, compilers and project management tools. EYODF can be harmful for the environment.
• Unless you can make an OS better than BSD or some IDE as complete as Eclipse, the competition will be intense. Everyone is doing it. And no, no one cares for yet another project management tool. EYODF is not the silver bullet from a business management perspective.

I think I finally got to truly understand and love mocks and stubs.

The motivation that got me into that is quite simple: STOP TESTING BULLSHIT. Stop testing if the framework is saving your models, if the validations are being called, if file contents are read. Stop testing the same methods over and over again. That’s a waste that makes your tests more verbose and coupled.

How can you know if you’re testing bullshit? I believe there are several ways, one being probably the simplest and more effective: when you break one method, which tests/specs fail?

To picture it: imagine a random Rails ecommerce app:

class Purchase
  before_save :calculate_total_price

  def calculate_total_price
    self.total_price = subtotal + shipping + taxes
  end

  #...

Assuming the whole system is working - and covered by both unit and functional tests - if we break that method (raising one exception, for instance), it’s expected that the test checking if this method correctly sums the shipping costs with taxes will fail. Good.

What’s bad is the test_cannot_sell_to_cuba failing because it created a new Purchase and thus called the broke method. Same for the test_discount_for_the_veterans_day, that tried to see if the total_price included the discount.

And that’s only in the purchase test file. Things get real ugly when you notice that the checkout controller test is also broke, together with the returns unit test, simply because they eventually hit this damn method.

I find this a problem because OOP is all about assigning responsibilities. When something goes wrong that’s because one object failed to fulfill it’s role, and you want to be able to point him instantly.

How? Stop testing bullshit. Enter mocks and stubs.

Now that you know what to expect and what to achieve with those guys, just run to the tutorials.

I remember once in a programming contest there was this myth about one guy that used to send his programs without even compiling first. Compile? Let the judge do that, he would say. We’re talking about implementing the Floyd’s algorithm right in the first attempt, using no tests or even compiler checks as a guide.

We all know a bunch of stories like that; 200 lines of buggy code replaced by one expression, meta programming enabling a 4 months project to be done in weeks, whatever.

And it’s great. Those stories are real and inspiring. I love it.

But UML hackers… Well, you don’t hear stories about a guy that saved some project from bad design with 7 lines. Probably because people think UML is something Java people do when they are doing their boring enterprise projects. UML and hackers sounds like opposite and even conflicting terms.

And here’s my point, they are not. They are even related, imho.

To put it simple: UML is abstraction, and hackers love it. It’s about doing more with less. Hell, you can solve hairy problems in a napkin, how badass is that? And before you think that’s not real, I remember you that in a good and popular UML book called The Object Primer all diagrams are scans of quick drawings in cheap paper.

Next time you are struggling to adapt software to fit all the crazy rules imposed by the client find yourself a napkin.

Recently we launched a Rails application that syncs some data to SalesForce using the cool ActiveSalesForce connection adapter. It certainly helped a lot in development — but there are some implementation details that didn’t work as we expected.

So, here it goes, random tips for those developing a Rails application with SF integration:

• The adapter is not quoting string attributes correctly; try inserting data with single quotes to see it crashing. That happens because ASF is relying on the standard Rails method to quote strings - that replaces a single quote for two - which is not the way Sales Force expects single quotes to be escaped. We submitted a patch to fix it, read for more details and instructions on how to fix by yourself for now.
• Be sure to check if hours are mapped correctly: save a record with some date attribute and check the saved value on SF. If there’s a problem you may convert dates manually or add another fix to the adapter (check method get_fields, for some reason they are adding one “U” to the time. removing it worked for us).
• There’s a limit of calls you can make to the SF API in a 24 hours period. ASF does an excellent job on optimizing calls since it can group similar requests (like queries or inserts) in only one call. But in order to use it you have to (1) wrap your requests in a transaction, (2) group requests by type (for instance, do a bunch of queries and then a bunch of inserts).
• But inside a transaction the Sales Force ids are lazily assigned to your models. That means you won’t be able save models in SF and print their ids within the same transaction block, for instance. Close the transaction before printing/saving SF ids.

So Alex Martelli wrote one email explaining his opinions on Python vs Ruby. It’s a interesting read, the guy is smart and know what he’s talking about — but eventually it came with this argument about how Ruby can be dangerous because it allows the programmer to change any class, in runtime.

You probably read this critic before. Maybe in another form, something like “ZOMG, ANYONE CAN CHANGE BUILT-IN CLASSES METHODS!111 SOME STUPID MAY REWRITE STRING DOT EQUALS :-OOOO“.

First of all, I believe these people never tried anything serious in Ruby. They just stopped at this potential problem because it’s a nice alibi — so nice that they can go back to Python or whatever language they like most without being judged.

If they really have tried Ruby they would know that this thing just doesn’t happen. At all. Because, well, no gem or rails plugin fucks around with built-in objects, simple like that. I’m using Rails for almost 4 years and never run into a problem related to this. Friends neither.

In fact this is such a rare event that no one comes with a real case example. The examples are always weird and distant from reality, like someone modifying strings so “Hello World” == “hello world”? Not going to happen.

Further on, if we “nail down” languages to cut features that allow people to shoot their feet — as suggested in that email — we’ll end up with another Java. And then even in Java there are several ways of doing stupid shit, like coding it all in one class or replacing business models for hashes (and this is a real example).

What saves people from shooting their feet and, most importantly, what saves you from that kind of problem is process, proper education, a decent team. Not the language.

Since this is not the “Yet Another Programmer Stating How Much He Loves His Tools”, I also want to bitch about something I found wrong in RSpec: the generators you may use to create specs for your Rails application are tied to models/controllers. Although you can create specs by hand, you are told to generate a spec together with your model or controller.

And this is a DISASTER as far as BDD goes, imho.

For a simple reason, I don’t want to design my application just to get to specs; I don’t want to think about models or controllers before having my spec.

This is part of a greater problem in the Rails community, that is: most people don’t care about software design. Not in the sense that they don’t do it — but in the sense that some Rails programmers don’t see software design as an actual development phase.

Development phases, as described by those guys, are: listen to the client, write tests, generate models/controllers, write code/views and deploy (and repeat everything, because everyone knows that waterfall sucks).

But it’s clear to me that there’s something missing, that software is designed at some point not described in there.

Before typing script/generate model you realized that you needed a model. Not a controller and not a regular class in the lib folder — you asked for a persistent model. In a few seconds you are going to define attributes for this model, and even relationships. Now, I don’t care if you wrote 10 pages before realizing you need a model, or if you drawn UML diagrams in a used napkin or if you just thought about this while the client was talking to you — that thing HAPPENED.

Wikipedia says BDD “is a software development technique that questions the behavior of an application before and during the development process”. “The purpose is to define the behavior of an application rather than the implementation“.

You see, how can you question the behavior of your Rails application before the development process? How can you define the behavior but not the architecture of your Rails application with RSpec?

Well, you CAN do it, definitely.

But you have to know that your development process started before the script/generate. And you have to be careful while learning because most RSpec+Rails tutorials are talking about the cool generators. No one will tell you to generate your own spec by hand, creating a new file somewhere in the spec folder — although, for the reasons discussed before, that’s probably the only way to start a BDD project.

That’s why I wrote a patch to change RSpec generators. Just extract to your vendor/plugins/rspec_on_rails/generators folder and enjoy the new decoupled and minimal script/generate rspec <spec name>. The previous rspec generator was renamed to rspec_base.

So when starting with a new project I normally define examples using only strings and no implementation blocks. Later, when models/controllers are created I move the examples to the related spec and write the implementation block.

Critics/suggestions are most welcome, of course.

So I’m trying Behavior Driven Development in Ruby, with RSpec. It felt a little counterintuitive in the beginning, but now it seems to me that this is a great approach on connecting a useful and well known development practice (testing) with some sort of definition on what development is trying to achieve.

It’s all about noticing that in the big picture regular testing can be misleading. Eventually you are writing tests to validate your mental model of the application, hoping that it eventually conflicts with the real user requirements. And mostly it does, because we programmers got good at this game — but, anyways, how nice it would be if we have something in our tests reminding us about the ultimate goals of the project, as stated by the client?

And what if we could see that there’s something missing in the application, not by rereading documents or emails or whatever — but just by running the tests?

Well, BDD is an attempt to give us stuff like this. And RSpec is a beautiful Ruby implementation. I’m enjoying it a lot. Maybe it’s the hype, but I really have the feeling that my tests are better stating the application goals. It makes you move with more confidence.

Further on, when a RSpec test pass it not only means that some random part of the system is working the way I expected, but also that one system requirement is met.

As a bonus, I’m finally writing tests before the code. Because, really, TDD just doesn’t work for me and I always end with crappy designs. And no, I’m not going to memorize TDD ANTI-PATTERNS (there are 22 of them!) to safely adopt the practice. I’d rather start with tests that won’t impact on design, and BDD allows me to do that.

Anyways, those are just first impressions. Although they are not all positive, in fact there’s something I found UGLY in RSpec, but this is for the next post.

Sorry, another very specific problem report. In the next post I’ll compensate with a exciting report on BDD/Rspec (as exciting as a report on BDD/Rspec can be).

Thing is: if you are using validates_date_time any attribution to Time.now won’t work (UPDATED: the plugin was fixed in revision 351, update it and you should be good). Time.now in Ruby returns the current time with microseconds, although this plugin is not dealing with microseconds yet. So the time is not saved/loaded properly; you may assign it, but after reloading the model all you have is a nil date.

Just emailed the plugin author. Meanwhile just ask for the patch if you need (Wordpress does not allow .diff uploads).

And if you are not using validates_date_time, you really should. I don’t even know how Rails programmers manage date/time input without it.

In June 2006 I was doing a Rails project with Postgres and noticed that the limit constraint was not being dumped to the schema. You could run a migration adding some field with :limit => 8, for instance, but when dumped the limit would mysteriously disappear.

And it turns out this problem was introduced by a patch that changed the PG Adapter to consider the limit and create smallint, int or bigint columns according to it.

Knowing that Mysql doesn’t work how people expect it work regarding limits (defining a field as INT(x) in Mysql means that you want to display that field with x columns, not that you want x bytes for it) I went to check how it was handling this thing. And in fact it was doing the wrong thing.

So I fixed the adapters, wrote tests and submitted a patch. One potentially annoying bug exterminated, open source saves the day again!

But what happens next is, well, nothing. After a few details and messages, after upgrading to the latest trunk because the patch got outdated, after annoying people in the rails-core mailing list to take a look at this, after having one nice guy to support sqlite and sqlite3 in the patch… it’s still not there.

So a few days ago I reminded about this issue. I guess it’s something that occurs to me every semester, I dream about ActiveRecord giving people only 4 bytes integers, no matter what they ask in their migrations for Mysql databases.

And guess what, the patch was outdated of course. Things changed so badly (what else to expect after 4k revsions) that updating it against the new trunk again was a pain. I couldn’t even run ActiveRecord tests with Mysql, it took almost 2 hours to figure it out and update the fix.

What happens now? if nothing else I’ll have a dream, in six months. About ActiveRecord mysteriously dropping some bytes from your bigint Postgres column.

This is something I noticed after having to speak a new language in a daily basis (and SUCKING at it):

At first it was usual for me to get lost in the middle of a slightly more complex phrase (especially in arguments). I would run out of words to finish what I was going to say. This looks like a sign that I was translating my thoughts from Portuguese to English, otherwise I would know all the words I wanted.

Now, what happens nowadays is that I’m hardly satisfied by my arguments. Sometimes I finish some assertion and think that it didn’t really reflect what I think. Sometimes I say something and milliseconds later think it was stupid. I even remember saying that something was “just boring” when in reality I could make a real argument out of it.

But, anyways, I think this is a sign that I’m now thinking in English.

As our compiler teacher once said (and he probably was quoting someone, I don’t know): the languages you master are the limits of your thoughts; my arguments are as good as my english, they lack elements the same way I lack english vocabulary and constructions.

At least I think I’m improving, let’s see it.