Rails testing and Emacs mode line

I’ve been diving/digging into the oh-so-excellent Rails Tutorial, by Michael Hartl, and am into chapter 3, where testing begins.

Imagine my surprise — go ahead, imagine it — when my Emacs mode line changed color.

Originally:

modeline_original

And when I ran “bundle exec rake test”, it succeeded with:

modeline_success

Then changing the test code to fail, after running “bundle exec rake test” again:

modeline_failed

Tracking down this code, I found that the guard-notiffany gem was doing this behavior, sending a change to the Emacs mode line via emacsclient, with this bit of code in lib/notiffany/notifier/emacs.rb:

        def notify(color, bgcolor)
          elisp = <<-EOF.gsub(/\s+/, " ").strip
            (set-face-attribute 'mode-line nil
                 :background "#{bgcolor}"
                 :foreground "#{color}")
          EOF
          emacs_eval(elisp)
        end

The issue there was that it just changes the mode line “permanently”. I’d prefer that the mode line colors change for a moment, then time out and revert to their previous settings.

I wrote a bit of code to do this, and modified lib/notiffany/notifier/emacs.rb locally, but there is a pending pull request that has a better implementation.

However, my hack is:

        def notify(color, bgcolor)
          elisp  = "(let ((orig-bg (face-attribute 'mode-line :background))"
          elisp << "      (orig-fg (face-attribute 'mode-line :foreground)))"
          elisp << "     (set-face-attribute 'mode-line nil"
          elisp << "                         :background \"#{bgcolor}\""
          elisp << "                         :foreground \"#{color}\")"
          elisp << "     (sit-for 3)"
          elisp << "     (set-face-attribute 'mode-line nil"
          elisp << "                         :background orig-bg"
          elisp << "                         :foreground orig-fg))"
          emacs_eval(elisp)
        end

The sit-for keeps the mode line as red or green for 3 seconds or when there is Emacs input. I prefer that, to clear the mode line more quickly, since I find the bright color distracting.

RVM, Zsh and Konsole

Today I set up RVM with Zsh (Z shell) with Konsole (the KDE equivalent to gnome-terminal).

The RVM Konsole example shows it with bash, setting the “Command” field to “/bin/bash –login”. The line for Zsh is nearly the same: “/usr/bin/zsh –login”.

If you don’t have this, you’ll get the error:

% which ruby
/usr/bin/ruby
% rvm use 2.0

RVM is not a function, selecting rubies with 'rvm use ...' will not work.

You need to change your terminal emulator preferences to allow login shell.
Sometimes it is required to use `/bin/bash --login` as the command.
Please visit https://rvm.io/integration/gnome-terminal/ for an example.

Note that you’ll need to have RVM as the first component in the PATH (“echo $PATH”).

With the “Command” field set to “/usr/bin/zsh –login” (that’s “dash dash login”), when you start a new terminal you should get:

% which ruby
/home/jpace/.rvm/rubies/ruby-1.9.3-p551/bin/ruby
% rvm use 2.0
Using /home/jpace/.rvm/gems/ruby-2.0.0-p598
% which ruby
/home/jpace/.rvm/rubies/ruby-2.0.0-p598/bin/ruby

Silencing Konsole Notification

Applications in KDE can send notifications, which result in popups on the task bar:

popup

Highly annoying this is. The culprit is that control-shift-A is the shortcut to enabling this behavior, as is shown via “Konsole -> View -> Monitor for Activity”:

view_menu_cropped

It’s fortunate that this is easy to disable. Right click on “Monitor for Activity -> Configure Shortcut”, then click on “Ctrl+Shift+A”, which will then display “Input”. Click on the left arrow graphic:

shortcut

That should result in it showing “None”, which means that there will no longer be a shortcut. You might want to go back to “Konsole > View” and make sure that “Monitor for Activity” is not checked.

I seem to do this every time I set up KDE, so perhaps this will help others.

Adding Groovy Emacs Mode

I’ve been doing a lot of Gradle/Groovy work lately, and have been using Russel’s Groovy Emacs Mode.

That page provides the documentation for the case where you’re going to put all the *.el files into ~/.emacs.d. However, to keep things nice and organized I put third-party code into a vendor directory under ~/.emacs.d/lisp.

So for the Groovy mode, I added ~/.emacs.d/lisp/vendor/groovy-mode, extracted Russell’s tarball into that, and added these lines to my jep-groovy.el file:

;;; use groovy-mode when file ends in .groovy or has #!/bin/groovy at start
(add-to-list 'load-path "~/.emacs.d/lisp/vendor/groovy-mode")
(autoload 'groovy-mode "groovy-mode" "Major mode for editing Groovy code." t)
(add-to-list 'auto-mode-alist '("\.groovy$" . groovy-mode))
(add-to-list 'auto-mode-alist '("\.gradle$" . groovy-mode))
(add-to-list 'interpreter-mode-alist '("groovy" . groovy-mode))

;;; make Groovy mode electric by default.
(add-hook 'groovy-mode-hook
          '(lambda ()
             (require 'groovy-electric)
             (groovy-electric-mode)))

zsh Event Tricks

I’ve been a Z shell (zsh) user for perhaps 15 years, being a tcsh user before that.

Unfortunately, it appears that there is no zsh book from my preferred technical books publishers, O’Reilly, Pragmatic Programmers or Manning. Thus my knowledge is from the zshall man page (all 15,000+ lines, when you can find it) and from experience.

I’d like to share a few examples of what one can do with this shell, and I’m going to go from most to least recent in what I’ve found and used.

The latest thing that I learned is how to get previous arguments. I’d known about “!!” to get the previous full command line, so if I needed the second argument, I’d do “!!<tab>”, then backspace (or search back, i.e., “<ctrl>r”) to the end of the argument, delete (“<ctrl>k”) to the end of the line, then move to the beginning of the argument “<alt>b”, then delete word-by-word “<alt><backspace>” through all the other arguments that “!!<tab>” expanded to.

What a mess, although doing that a few hundred times made it easy, although far from efficient.

So when perusing the 15,000 zshall man page (one day I’ll make that my summer reading) I saw that one can write “!!:2″ to get the second argument of the previous line. Hurrah. Note that, of course, arguments are zero-indexed.

And the leading “!!” can be of the form “!-3″, meaning “the third most recent”, and “!-1″ is the same as “!!”.

I haven’t used this much, but the word designators (the argument after the “:”) can be a range, so for the second through fourth argument of the command before last, the command would be “!-2:2-4″. If there is no starting number for the range, e.g., “!!:-2″, then that is the same as 0 through N, the first N+1 arguments from the earlier command.

In action, this looks like:

designators

Another form of syntax, as seen in the example above, is !N, where N is the number of the command line to reference. In the example, 3066 is the command line in history, so !3066:3 refers to the fourth word in that command.

In a subsequent post, I’ll discuss the prompt that you just saw, and why having the command-line number is useful.

Scrabble Tutor (Xumoqi) 1.4 Released

A new version of Xumoqi (AKA Scrabble Tutor) has been released on the Google Play Store.

With this version, the results screen is improved to show column headings of correct, invalid and missed, with a score bar above the results:

1.4-07-nn-results

Please take a look, and tell me what you think.

I’m refining it to be more game-like, and plan to add hints, such as the popular “Betsy’s Foot” for “ka” words (kab, kae, kaf, kas, kat, kay, oka, ska), coming up with my own. such as the simple “yeah” for “sh” words (sha, she, shh, shy).

Introducing XuMoQi

Ki and za! Zax and zoa! Xyst and zyme! Yurta and zowie!

As I’ve written about before, I have been developing a game to help learn Scrabble words. I’m what might be considered a reluctant Android developer, being no major fan of Java, XML, and Eclipse, but I wanted an app for my phone so that I could learn and practice words for Scrabble, my board and electronic game of choice.

Other apps, such as Syrious Scramble, seem to use a dictionary other than those used for Scrabble, so I disliked the feeling of playing a word in those games and being told it was invalid, such as ain.

So I looked, in vain (and ain, eventually) for a Scrabble-compliant word app, and finding none, I decided to write one, XuMoQi, which is barely not none. That is, it is extraordinary in its plainness, overarching in being one-dimensional. (Okay, two dimensions, but only barely.)

Some random thoughts and experiences while writing the app follow.

What up, Dawg?

The word lists are huge, and I tried to optimize them for speed instead of space. I am considering rewriting them to being a modified DAWG (http://stevehanov.ca/blog/index.php?id=115), but haven’t yet figured out how to get my searching and pattern-matching to fit with those, which presume that all characters are known before doing the search.

Field-testing the app showed that it is reasonable in its speed, and I haven’t had any out of memory errors. Then again, I’ve tested in a field of one, on my relatively newish phone.

All work and no play …

Pattern-matching is done in separate threads, usually taking less time than the duration of the user inputting their words. I found this
(http://vaibhavtolia.wordpress.com/2013/10/03/79) to be a good resource on explaining the apparent overuse of, uh, resources.

Editing the EditView

Alas, the Android library doesn’t seem to have a decent (or any) EditView to fit my needs: no auto suggestion, no word completion, and submit (or send) when the enter (angled arrow) is pressed. So I wrote my own, which is on GitHub here: https://github.com/jpace/xumoqi/blob/master/src/org/incava/xumoqi/MultiLineSendEditText.java

Iconoclast Icons

I scaled the vertical learning curve of Gimp (it is really a curve if it’s a straight line?) and developed a few icons, rolling them by hand and applying various light effects. I started by following this http://www.gimpshop.com/tutorials/how-to-create-a-logo guide, which helped me learn the basics, then I found the Filters > Logo functionality, which scripts a set of filters in preset ways. Eventually I developed, with the Chrome effect, this masterpiece:

xu7

I charged myself nothing for my work as a graphics artist, and got my money’s worth.

I wasn’t sure how to change the Icons, and found an excellent guide here: http://bigknol.com/open-blog/2013/10/change-android-launcher-icon-using-eclipse-ide/.

Screenshots

As if I needed more things to dislike about Eclipse, the version I’m using, Juno Service Release 1, would not take screenshots, and with such a beautiful app as this, it was imperative to capture such artistry. But Eclipse, and perhaps this is a feature, not a bug, would not capture my screenshots, so instead of running the emulator through Eclipse, I ran it off the command line, as described here (http://www.addthis.com/blog/2013/07/22/10-tips-for-android-emulator/#.Uwp0v1ErFG4)

% cd /opt/adt-bundle/adt-bundle-linux-x86_64-20131030/sdk
% ./tools/android avd
% ./platform-tools/adb shell screencap -p | sed ‘s/\r$//’ > ~incava/android/v1.2/screen2.png

Table with equal columns

Maddening it was to get my table to display columns with an equal width. Naturally, of course, is that the width should be set to 0:

http://androidadvice.blogspot.com/2010/10/tablelayout-columns-equal-width.html.

Name

XuMoQi comes from three two-letter words commonly used in Scrabble, xu, mo, and (wait for it) qi. I pronounce it “zoo-MOE-key”, but when she saw it, my wife immediately said “zoo monkey”. So if this app ever gets a mascot, you know what it will be.

Exporting

It wasn’t intuitive to figure out how to export my app such that Google would accept it, but this explains it: http://help.testflightapp.com/customer/portal/articles/1279844-how-to-create-an-apk-adt-bundle.

Future Features

I believe that I have adequately described XuMoQi as primitive, primitive as in the sense of the opening of 2001: A Space Odyssey, but with fewer bones. Based on feedback, and my own experience, I plan to make it more game-like, i.e., keeping scores, rating words based on difficult, and adding a timer.

And despite that the graphics are incredibly perfect, I’ll probably find some way to make them even better.

Getting the App

In the unlikely chance that anyone wants to get this app, it is available on (in?) Google Play Store.

Hello, (a Different Part of the) World

Last week I traveled to Dubai and Qatar, a vacation with my spouse and an opportunity to see the Middle East.

Dubai was luxurious, a center of wealth and commerce, a spot where my spouse wanted to indulge in some well-deserved pampering. But I got out often, mostly on foot, and explored the environs (Deira City).

I really enjoyed the Middle Eastern cuisine, and foul medames is going to be rotated into my carb-free diet. Of course, it being a Muslim country, you’re out of luck if you’re craving (pork) bacon – I wasn’t – but it was interesting to see the alternatives: had I access to a kitchen, I would have loved to try the veal bacon we found in a supermarket. And beef bacon, which I did have, could also rotate into my diet, it being much less greasy (that’s a feature) than pork bacon.

Being an aficionado of tall buildings (my when-I-grow-up goal from the age of 12 to 16 or so was to be an architect), I had to go up the Burj Khalifa, at 2717 feet the tallest building on Earth. It was almost comical how tall it is, literally towering over the 1000+ foot buildings around it. A beautiful piece of architecture, and well worth the price.

We stayed at the Park Hyatt, a western-style hotel, but one matching the design of the older Middle East, as opposed to the glass-and-steel buildings in the area that could have been in the skyline of Chicago.

A couple of matters about Dubai: I’d read online that they’re very strict about bringing things into the country, specifically alcohol, tobacco, and drugs, including prescription medications. So I spent some time beforehand, getting copies of my prescriptions, then anxiously awaited the full-body and cavity searches when I arrived.

Hah. It’s simpler to get into a Smithsonian museum than to go through customs in Dubai. In fact, I wasn’t really sure that I was going through it until I was done. That’s it? Much worrying for nothing, and I guess for the first time, there was incorrect information on the Internet.

Note: Dubai is dry, i.e., alcohol isn’t sold, except in hotels, where it is insanely expensive, at five to ten times the cost in a store over here. But at the Dubai airport, you can go through the duty-free shop on the way out of the airport and bring up to five bottles out. So, for all my readers who are going to Dubai and want to drink wine while you’re there, that’s my helpful tip. You’re welcome.

Dubai is also very orderly. I noticed that people don’t even jaywalk, even at night with no cars around, and just patiently wait for the pedestrian signal to switch to green.

On to Qatar, to the west. I traveled there alone, while the missus left in the opposite direction, back to where she is currently working. I had an overnight layout in Doha, so I took off, on foot, and walked up to Souk Waqif, an outdoor (mostly) market to the north of the city. Getting up there was no easy feat, since my own feet were frequently ensconced in a nice thicket of mud, left there by the recent rain. Evidently when you almost never get rain, you don’t build drains for the roads, so much of the water went onto the “sidewalks”, i.e., the area, possibly but not necessarily paved, alongside the roads.

And Qatar is not orderly. I had a couple of close calls with cars: there evidently is no right of way for pedestrians. More like “you can get right out of the way”. That might be one reason that I encountered nearly no one else during my five or six mile stroll around the city.

Souk Waqif was fascinating. I love being a fish out of water, and as a nearly-stereotypical North American, I didn’t fit in at all. Perfect. Nearly everyone there was Arabic, but there were enough written and spoken English that I didn’t have any difficulty.

I saw and listened to an outdoor performance, and Middle Eastern music is becoming a new favorite, especially since I tend to like music that others abhor (metal, old country and western, and disco). On the plane I listened to El Liala’s Khalina Lewahdina, a very catchy mixture of Arabic music and disco (sorry, I mean “hip-hop”), and I must have listened to it ten times.

At Souk Waqif I went to an outdoor cafe and had an excellent dinner: lamb and chicken kabobs, moutabai, fattouch, and a couple of (non-alcoholic) drinks, whose names escape me. I also enjoyed, for the first time, a shisha, of grape and mint. I need one of those. (Off Jeff goes to Amazon … ). I guess that’s what is otherwise known as a hookah. Well, consider my horizons broadened.

After I ordered my dinner (and yes, this blog entry is about to have a reference to programming, so those of you that have stuck this far will have your patience rewarded), I got out my phone and decided to do a little Scrabble practice. See, I’ve been writing an Android app (my first) to help me (and anyone else) practice Scrabble words. I’m a big-time Scrabble fan (jpace317 on Origin), and
although I’m decent at bingos, I’m trying to up my game with the three- and four-letter words. (I’m looking in your direction, zax and chay.)

So I fired up my app, XuMoQi, and the first query was: “.uq” (as in, what is the missing letter). Answer, “suq”, a variant spelling of … wait for it … “souk”. As in Souk Waqif. Holy freaking moley.

(I seem to have a bit of a weird-timing record with serendipitous moments like that, such as my car finally giving up the ghost (actually, its transmission, at 175,038 miles), while I was driving out to the dealer to buy my new car. Oh, and adding to the weirdness is that it was on my birthday.)

Anyway, that’s the story of my travels.

Oh, and back home: after fourteen hours in the air, at the airport there was no question about when I was going through the passport check and customs. I wanted to apologize to those in the non-U.S. citizens queue, given that it looked like they would be waiting for at least an hour to get through. So to all my non-U.S. citizen readers who landed at Dulles airport the afternoon of 12 January 2014, on behalf of the entire United States of America, I offer my most profound apologies.

The reading material of choice during the trip: Sway, Evolution of Useful Things, Code, and Don’t Put Me In, Coach. The last book was light reading material, and perfect for unwinding on the plane. If you ever spent time as a bench-warming basketball player (I’m looking in my own direction here), I highly recommend it.

Tests Result in Better Programs, and Programmers

It’s widely noted and accepted that tests result in better code. My conjecture is that tests result in vastly improved programmers.

Even with a good test suite, the first implementation of a project will be utilitarian, complying with the edict of “just get it working!”. This are the red and green phases of the red/green/refactor cycle: red is when the tests fail, and green is when they pass. Too often, that initial version is the end of the cycle for many programmers and projects, which is why, in general, version 1 of a program is horrible:

win101logo

In the refactoring phase the project goes from simply “working” to being well-crafted. Thus the programmer as well has changed their mindset from writing “working” code to writing well-crafted code, and the programmer has been elevated to a higher level of craftsmanship.

Refactoring code is when a programmer can develop their skills and push the boundaries of their knowledge, such as delving into advanced object-oriented programming and metaprogramming, which are often considered dauntingly complex and risky. But used properly, they can dramatically improve a project, and I believe what is more important, they can dramatically improve the programmer.

In my experience, this was first proven to me when I was working on a large C++ project, my module being our persistence layer, providing a J2EE-like (but vastly simpler) interface with PostgreSQL. Although in that era (the late 1990s) Test-Driven Development hadn’t become popular, my daily goal was to write more test code than “real” code, usually between 500 and 1500 lines per day, usually, but not always, writing my tests first.

As my test suite grew and I became more confident in its ability to catch errors, I pushed my knowledge of C++, particularly with templates and the Standard Template Library, eventually reaching a point when I really understood the magic of the STL code. Had I not had such tests, and thus confidence, I likely would have refrained from pushing myself into an area that previously was in the dark, murky area of C++ to me, and I would not have gained valuable expertise as a C++ programmer.

In fact, I’d say that it’s only because of tests that I’ve felt confident learning a new language, such as when I rewrote DiffJ in JRuby, a language I hadn’t used. Having a thorough test suite made the learning process much easier.

The bottom line is that programmers must understand that testing includes refactoring (including of the test code itself), and that the refactoring phase is where programmers, and projects, can become significantly better.

Oh, and with thorough refactoring that horrible version 1 can eventually lead to significant improvements:

macos

Shorter Directory Names in Emacs Ibuffer and Mode Line

I use Emacs and Z shell exclusively, and have been frustrated at times about the features in Z shell that are not in Emacs, one being the shortened directory names. These are used for navigating (changing directories and displayed in the prompt), and are especially useful with long/deep directory names.

In Java projects, this is beneficial, because the hierarchy tends to be so deep. For example, with the path:

   /home/me/proj/com/mycompany/projectx/trunk/src/main/java/com/mycompany/util/FooUtil.java

In Z shell the directory can be given a short name with:

   hash -d projectx=/home/me/proj/com/mycompany/projectx/trunk

So the file can be referred as:

   less ~projectx/src/main/java/com/mycompany/util/FooUtil.java

And this can be done more extensively than that, such as:

   hash -d pxutil=~projectx/src/main/java/com/mycompany/util

And used as:

   less ~pxutil/FooUtil.java

That also makes for a shorter directory name displayed in the prompt, such as this for DiffJ:

    (~diffj)-[master]-(0)%

Which without Zsh hashes would be:

    (/home/jpace/proj/org/incava/diffj)-[master]-(0)%

So I’ve become used to that in my shell, but have been frustrated with my editor, which shows the full path in the default ibuffer display, running out to over 100 columns:

    TokenList.java   1699 /home/jpace/proj/org/incava/diffj/src/main/java/org/incava/diffj/code/TokenList.java

It also seemed redundant that in the above, TokenList.java is displayed twice, so the filename-and-process column could be replaced with only the directory name.

I looked around for examples about changing the columns in ibuffer, but found few examples, mainly the same code repeated on various sites online, variations of this from http://www.emacswiki.org/emacs/IbufferMode:

;; Use human readable Size column instead of original one
(define-ibuffer-column size-h
  (:name "Size" :inline t)
  (cond
   ((> (buffer-size) 1000000) (format "%7.3fM" (/ (buffer-size) 1000000.0)))
   ((> (buffer-size) 1000) (format "%7.3fk" (/ (buffer-size) 1000.0)))
   (t (format "%8d" (buffer-size)))))

That was a starting point, but I couldn’t find anything more about revising the filename-and-process column, which shows the above long path.

The following code adds a ‘dirname’ format to ibuffer to shorten the filename and show only the directory:

(defvar jep:filename-subs
  '(("/home/jpace" . "~")
    (".*/Projects/com/mycompany/is/" . "~is/")
    ("/home/jpace/proj/org/incava/" . "~incava/")
    ("/$" . "")))

(define-ibuffer-column dirname
  (:name "Directory"
	 :inline nil)
  (if (buffer-file-name buffer)
      (str-replace-all (file-name-directory (buffer-file-name buffer)) jep:filename-subs)
    (or dired-directory
	"")))

(setq ibuffer-formats
      '((mark modified read-only " "
	      (name 30 30 :left :elide)
	      " "
	      (size 9 -1 :right)
	      " " dirname)
	(mark modified read-only " "
	      (name 30 30 :left :elide)
	      " "
	      (size 9 -1 :right)
	      " " filename-and-process)
	(mark " "
	      (name 30 30 :left :elide)
	      " " filename-and-process)))

Now my ibuffer looks like:

[ diffj ]
    TokenList.java    1699 ~incava/diffj/src/main/java/org/incava/diffj/code
    build.gradle       892 ~incava/diffj
    etc....

Much better.

Coincidentally, I had the same complaint about the modeline, which defaults to the same display as ibuffer, with the full path of the file. So I tweaked the modeline code to do the same:

(defvar jep:modeline-subs
  '(("/home/jpace/" . "~/")
    (".*/Projects/com/mycompany/is/" . "~is/")
    ("/proj/org/incava/" . "~incava/")
    ("/$" . "")
    ))

(defun jep:modeline-dir-abbrev ()
  (str-replace-all default-directory jep:modeline-subs))

(setq default-mode-line-format
      (list ""
            'mode-line-modified
            "%25b--"
            " ["
            '(:eval (jep:modeline-dir-abbrev))
            "] "
            "%[("
            'mode-name
            "%n"
            'mode-line-process
            ")%]--"
             "L%l--"
             "C%c--"
            '(-3 . "%P")
            "-%-"))

Note that the modeline and ibuffer code above uses this function, which I put in ~/.emacs.d/lisp/str.el:

(defun str-replace-all (str pats)
  (if (null pats)
      str
    (let* ((pat (car pats))
	   (lhs (car pat))
	   (rhs (cdr pat)))
      (replace-regexp-in-string lhs rhs (str-replace-all str (cdr pats))))))

So there is my contribution. Here it is in action:

emacs_dirname

And it’s included with my Emacs configuration on GitHub.